Processing device, processing method, and program

Abstract

To alleviate a problem that a customer does not notice displayed information, in a technique of displaying the information to the customer who is shopping at a store.SOLUTION: A processing device according to the present invention includes: recognition means for recognizing that a product has picked up; position specifying means for specifying a position where the picked-up product was displayed; display control means for displaying information related to the picked-up product at a position corresponding to the position where the picked-up product was displayed; and means for identifying and detecting a customer and a store clerk. The display control means displays different information between the case where the customer is detected and the store clerk is detected.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to a processing device, a processing method, and a program.

Background Art

[0002] Patent Document 1 discloses a technique for displaying information (such as selling price and freshness of fresh food) of products displayed and sold in a store on a mobile terminal.

[0003] Patent Document 2 discloses a shelf terminal that detects which product has been taken out of the shelf and which product has been returned to the shelf by reading product unique identification information from an IC tag attached to the product. The shelf terminal determines that a product has been taken out when the product unique identification information changes from readable to unreadable, and determines that a product has been returned to the shelf when the product unique identification information changes from unreadable to readable. Further, the shelf terminal transmits the reading result from the IC tag to a server, receives advertisement information from the server, and displays the advertisement information on a display.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] By providing useful information to customers while shopping in a store, an increase in sales and the like are expected. However, if the position where the information is displayed is not appropriate, there may be a problem that the customer does not notice the displayed information. Neither Patent Document 1 nor 2 discloses the problem and its solution means.

[0006] The present invention aims to reduce the problem that customers shopping in a store do not notice the displayed information in the technology of displaying information to customers during shopping.

Means for Solving the Problem

[0007] According to the present invention, recognition means for recognizing that a customer has picked up a product, position specifying means for specifying the position where the product picked up by the customer was displayed, display control means for displaying provided information including information related to the product picked up by the customer at a position corresponding to the position where the product picked up by the customer was displayed, a processing device having the above is provided.

[0008] Also, according to the present invention, a computer recognizes that a customer has picked up a product, specifies the position where the product picked up by the customer was displayed, and provides a processing method for displaying provided information including information related to the product picked up by the customer at a position corresponding to the position where the product picked up by the customer was displayed.

[0009] Also, according to the present invention, a computer is functioned as recognition means for recognizing that a customer has picked up a product, position specifying means for specifying the position where the product picked up by the customer was displayed, display control means for displaying provided information including information related to the product picked up by the customer at a position corresponding to the position where the product picked up by the customer was displayed, and a program is provided.

Effects of the Invention

[0010] According to the present invention, in a technology for displaying information to customers while they are shopping in a store, the problem of customers not noticing the information being displayed can be alleviated. [Brief explanation of the drawings]

[0011]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

[0012] First Embodiment First, an overview of the processing device of this embodiment will be described. When the processing device of this embodiment recognizes that a customer has picked up a product displayed on a shelf, it identifies the location where the product was displayed and displays provided information including information related to the product at the location corresponding to the location where the product was displayed.

[0013] In this way, the processing device of this embodiment displays the provided information at "the position corresponding to the position where the product was displayed" at "the moment the customer picks up the product," thereby alleviating the problem of the customer not noticing the displayed provided information.

[0014] Next, the configuration of the processing device will be described in detail. First, an example of the hardware configuration of the processing device will be described. The functional units of the processing device of this embodiment are realized by any combination of hardware and software, centered around a CPU (Central Processing Unit) of any computer, memory, programs loaded into the memory, a storage unit such as a hard disk that stores the programs (this can store programs that are pre-loaded when the device is shipped, as well as programs downloaded from storage media such as CDs (Compact Discs) or servers on the Internet), and a network connection interface. Those skilled in the art will understand that there are many variations in the implementation methods and devices.

[0015] FIG. 1 is a block diagram illustrating an example of the hardware configuration of a processing device of this embodiment. As shown in FIG. 1, the processing device has a processor 1A, a memory 2A, an input / output interface 3A, a peripheral circuit 4A, and a bus 5A. The peripheral circuit 4A includes various modules. Note that the peripheral circuit 4A may not be included. Note that the processing device may be configured as a single device that is physically and / or logically integrated, or may be configured as multiple devices that are physically and / or logically separated. When configured as multiple devices that are physically and / or logically separated, each of the multiple devices may have the above hardware configuration.

[0016] Bus 5A is a data transmission path for the processor 1A, memory 2A, peripheral circuit 4A, and input / output interface 3A to transmit and receive data with each other. The processor 1A is an arithmetic processing device such as a CPU or a GPU (Graphics Processing Unit). The memory 2A is a memory such as a RAM (Random Access Memory) or a ROM (Read Only Memory). The input / output interface 3A includes an interface for acquiring information from an input device, an external device, an external server, an external sensor, a camera, etc., and an interface for outputting information to an output device, an external device, an external server, etc. The input device is, for example, a keyboard, a mouse, a microphone, a touch panel, a physical button, a camera, etc. The output device is, for example, a display, a speaker, a printer, a mailer, etc. The processor 1A can issue commands to each module and perform operations based on their operation results.

[0017] Next, the functional configuration of the processing device will be described. FIG. 2 shows an example of a functional block diagram of the processing device 10. As shown in the figure, the processing device 10 includes a recognition unit 11, a position identification unit 12, a provided information acquisition unit 13, a display control unit 14, and a display unit 15.

[0018] The recognition unit 11 recognizes that the customer has picked up a product displayed on the product shelf. The position identification unit 12 identifies the position where the product picked up by the customer was displayed, specifically, where in the area for displaying the product it was displayed. The recognition unit 11 and the position identification unit 12 can realize the recognition and position identification, for example, by any one of the following recognition / specific processing examples 1 to recognition / specific processing examples 3. Note that the recognition unit 11 and the position identification unit 12 may also realize the recognition and position identification by other processing examples.

[0019] "Recognition / Specific Processing Example 1" A code (such as a barcode or a two-dimensional code) is attached to the outer surface of the product. And the product is displayed on top of the display section so that the code faces downward. That is, the code attached to the product displayed on top of the display section faces the display section. The display section is an object on which the product is displayed, such as a shelf board, but is not limited thereto. The display section is made of a material that transmits electromagnetic waves (such as transparent glass or plastic). That is, the display section transmits electromagnetic waves.

[0020] The code reader continues the process of reading the code of the product displayed on top of the display section through the display section. The code reader, for example, irradiates electromagnetic waves toward the product displayed on top of the display section through the display section, receives the reflected wave, and repeats the process of analyzing the code based on the received reflected wave and converting it into information (the process of reading the code). Or, the code reader, for example, takes a picture of the product displayed on top of the display section through the display section, extracts the code attached to the product from the image, and repeats the process of analyzing the extracted code and converting it into information (the process of reading the code).

[0021] The recognition unit 11 acquires the reading result from the code reader. And based on the reading result, the recognition unit 11 recognizes that the customer has picked up the product. For example, when the code of a product that could be read until then can no longer be read, the recognition unit 11 recognizes that the customer has picked up that product.

[0022] Next, the position identification process will be described. For example, a plurality of code readers may be installed corresponding to one display section where a plurality of products are displayed. In this case, the recognition unit 11 acquires the reading results from each of the plurality of code readers, and based on each of them, recognizes that the customer has picked up a product. Then, when the code of the product that has been readable so far becomes unreadable, the position identification unit 12 identifies the position where the code reader (the first code reader) that has been reading the code irradiates electromagnetic waves as the position where the product picked up by the customer was displayed. Note that, in advance, information indicating the position where each of the plurality of code readers irradiates electromagnetic waves (the irradiated position in the area where products are displayed on the display section) (coordinates in a two-dimensional coordinate system with an arbitrary point in the display area as the origin, etc.) may be generated and stored in the processing device 10. Then, the position identification unit 12 may identify the position where the product picked up by the customer was displayed based on the said information and the information for identifying the first code reader.

[0023] Alternatively, the code reader may irradiate electromagnetic waves in sequence to a plurality of positions in the area where products are displayed on the display section while switching the electromagnetic wave irradiation direction, and sequentially read the codes of the plurality of products. Then, when the code of the product that has been readable so far becomes unreadable, the position identification unit 12 may identify the position where the product picked up by the customer was displayed based on the electromagnetic wave irradiation direction (the first electromagnetic wave irradiation direction) for reading the code. Note that, in advance, information indicating the correspondence between the electromagnetic wave irradiation direction of the code reader and the position irradiated with electromagnetic waves at each direction (the irradiated position in the area where products are displayed on the display section) may be generated and stored in the processing device 10. Then, the position identification unit 12 may identify the position where the product picked up by the customer was displayed based on the said information and the first electromagnetic wave irradiation direction.

[0024] In addition, the code reader (camera) may be installed at a position and orientation for photographing a plurality of products at once. Then, the code reader may extract a plurality of codes from the image and analyze each of the extracted plurality of codes. In this case, after the position specifying unit 12 specifies the position within the image of the code that had been readable but has become unreadable, based on the information showing the correspondence between the coordinates within the image and the coordinates of the area for displaying the products on the display section (information indicating where the first point of the area for displaying the products on the display section is located within the image), the position within the image that has been specified is converted into the position on the area for displaying the products on the display section.

[0025] "Recognition and Specifying Processing Example 2" A camera (a camera for shooting moving images) for photographing the products displayed on the display section from above the display section is installed. The recognition unit 11 recognizes the products within the image based on the feature amounts of the appearances of the products, and tracks each of the recognized products within the image. Then, the recognition unit 11 recognizes that a customer has picked up a product based on the behavior of the product. For example, the recognition unit 11 may recognize a human hand within the image, and when the position of the product changes while the human hand and the product are in contact, recognize that the customer has picked up the product.

[0026] The position specifying unit 12 first specifies the position within the image where the product picked up by the customer existed before it was picked up. For example, when tracking each of the products recognized within the image as described above, the time change of the position of each product within the image may be recorded as history data. Then, the position specifying unit 12 may specify the position within the image where the product picked up by the customer existed before it was picked up based on this information. After that, based on the information showing the correspondence between the coordinates within the image and the coordinates of the area for displaying the products on the display section (information indicating where the first point of the area for displaying the products on the display section is located within the image) prepared in advance, the position within the image where the product picked up by the customer existed before it was picked up is converted into the position on the area for displaying the products on the display section.

[0027] "Recognition and Specifying Processing Example 3" A weight sensor is installed on the display section to measure the weight of the products displayed. A plurality of weight sensors are installed corresponding to one display section where a plurality of products are displayed. That is, the weight of the products displayed on the display section is measured by the plurality of weight sensors. The recognition unit 11 recognizes that a customer has picked up a product based on the measurement value of the weight sensor. For example, when the total measurement value of the plurality of weight sensors changes to a smaller value than before, the recognition unit 11 recognizes that a customer has picked up a product.

[0028] The position identification unit 12 identifies the display position of the product picked up by the customer based on the change in the measurement value of each of the multiple weight sensors. When a display section is supported by multiple weight sensors and the weight of the products displayed on the display section is measured, the manner in which the measurement value of each of the multiple weight sensors changes will differ based on the distance relationship between the display position of the product picked up by the customer and the position of each of the multiple weight sensors. For example, if four weight sensors are installed at the four corners of the display section, a weight sensor that is relatively close to the display position of the product picked up by the customer will show a relatively large change (decrease) in the measurement value. For example, based on this relationship, it is possible to identify the display position of the product picked up by the customer.

[0029] Returning to FIG. 2, the provided information acquisition unit 13 acquires provided information including information related to the product picked up by the customer. The information related to the product may include at least one of the timing when the product picked up by the customer was put on display, the timing when the product picked up by the customer went on sale, and the timing when the product picked up by the customer was processed. In addition, the information related to the product may include advertisements for the product, campaign information for the product, how to use the product (how to cook, how to use, etc.), advertisements for products similar to the product, campaign information for products similar to the product, how to use products similar to the product, the timing of discounts for each product (the time of day when discount labels are affixed), etc.

[0030] Here, an example of the acquisition process by the provided information acquisition unit 13 will be described. The provided information acquisition unit 13 can realize the acquisition of provided information by any one of the following acquisition process examples 1 to 3. Note that the provided information acquisition unit 13 may also realize the acquisition of provided information by other process examples.

[0031] "Acquisition Process Example 1" As described in the above recognition and identification process example 1, a code is assigned to each product, and the process of reading it with a code reader is performed. And the said code indicates the information related to the product. In this case, the code reader reads the code of each product on display and acquires the information related to each product. Then, the processing device 10 stores the information related to each acquired product in the storage device in the processing device 10.

[0032] The provided information acquisition unit 13 acquires the information related to the product taken by the customer from the information related to the products stored in the storage device. For example, the provided information acquisition unit 13 may acquire the information related to the product that cannot be acquired by the code reader (because the customer has taken the product) from the information related to the products stored in the storage device. Alternatively, the processing device 10 may store the information related to each acquired product in the storage device, linked to the information indicating the position where each product is displayed. Then, the provided information acquisition unit 13 may acquire the information related to the product linked to the position where the product taken by the customer was displayed (the position specified by the position specifying unit 12).

[0033] "Acquisition Process Example 2" A product information database in which information related to products (information for each product) is registered in advance in association with product identification information (information for identifying products from each other, such as product codes) is created. Then, when the provided information acquisition unit 13 acquires the product identification information of the product taken by the customer, it refers to the product information database and acquires the information related to the product linked to the acquired product identification information. The product information database may exist in the processing device 10 or may exist in an external device configured to be communicable with the processing device 10.

[0034] The means for acquiring the product identification information is not particularly limited. In the case of the above recognition and identification processing example 1, an example where the code indicates the product identification information can be considered. In the case of the above recognition and identification processing example 2, an example where the product identification information is specified based on the feature amount of the appearance of the product included in the image can be considered. In the case of the above recognition and identification processing example 3, an example where the product identification information is specified based on the change amount of the measured value of the weight sensor and the weight of each product can be considered.

[0035] "Acquisition processing example 3" A product information database in which information related to products (information for each product individual) is registered in advance in association with the individual identification information is created. The individual identification information is information for identifying a plurality of identical products (a plurality of products with the same product code) from each other. Then, when the provision information acquisition unit 13 acquires the individual identification information of the product taken by the customer, it refers to the product information database and acquires the information related to the product associated with the acquired individual identification information. The product information database may exist in the processing device 10 or may exist in an external device configured to be communicable with the processing device 10. The means for acquiring the individual identification information is not particularly limited. In the case of the above recognition and identification processing example 1, an example where the code indicates the individual identification information can be considered. In the case of the above recognition and identification processing example 2, an example where the individual identification information is specified based on the feature amount of the appearance of the product included in the image can be considered.

[0036] Returning to FIG. 2, the display control unit 14 controls the display unit 15 to display the provision information acquired by the provision information acquisition unit 13 at a position corresponding to the position where the product taken by the customer was displayed. The display unit 15 is a display, a projection device, or the like.

[0037] In the product shelf of the present embodiment, at least one position for displaying information is provided. FIG. 3 shows an example of the product shelf 100. In the illustrated example, information can be displayed on the display part 101, the back part 102, and the display 103 (display unit 15).

[0038] The display section 101 may be a display (display unit 15), or a projection device (display unit 15) may be installed at a position and orientation that projects an image onto the display section 101. The projection device may project two-dimensional information or three-dimensional information (information that appears three-dimensional). The projection device may project from the front side of the display section 101 or from the back side of the display section 101. A projection device may be installed for each display section 101, or one projection device may be installed for multiple display sections 101.

[0039] Similarly, the spine portion 102 may be a display (display unit 15), or a projection device (display unit 15) may be installed at a position and orientation that projects an image onto that portion. The projection device may project two-dimensional information or three-dimensional information (information that appears three-dimensional). The projection device may project from the front side of the spine portion 102 or from the back side of the spine portion 102. A projection device may be installed for each spine portion 102, or one projection device may be installed for multiple spines 102. Note that the spines 102 on all multiple rows may be capable of displaying information. However, there is a possibility that the spines 102 on the lower rows may be difficult to view due to the display portions 101 on the upper rows, etc. Therefore, only the spines 102 on the upper rows (for example, only the topmost spine 102) may be capable of displaying information.

[0040] Here, an example of the relationship between the display position of the product picked up by the customer and the display position of the provided information will be described.

[0041] For example, the display control unit 14 may display the provided information at the position where the product picked up by the customer was displayed, as shown in Fig. 4. In Fig. 4(A), six products 200 are displayed on the display section 101. In Fig. 4(B), the customer picks up one of the six products 200. The provided information is then displayed at the position where the picked-up product 200 was displayed.

[0042] In addition, as shown in FIG. 3, when the merchandise shelf 100 is divided into a plurality of tiers and a display 103 is installed for each tier, the display control unit 14 may display the provided information on the display 103 corresponding to the tier including the position where the merchandise taken by the customer was displayed.

[0043] In addition, as shown in FIG. 3, when the merchandise shelf 100 is divided into a plurality of tiers and the back portions 102 of all the tiers are capable of displaying information, the display control unit 14 may display the provided information on the back portion 102 of the tier including the position where the merchandise taken by the customer was displayed.

[0044] In addition, as shown in FIG. 3, when the merchandise shelf 100 is divided into a plurality of tiers and only the back portion 102 of the topmost tier among the back portions 102 of the plurality of tiers is capable of displaying information, the display control unit 14 may display the provided information on the back portion 102 of the topmost tier of the merchandise shelf 100 including the position where the merchandise taken by the customer was displayed.

[0045] Note that the display control unit 14 displays the provided information at a position corresponding to the position where the merchandise taken by the customer was displayed in response to the recognition unit 11 recognizing that the customer has taken the merchandise. That is, the display control unit 14 displays the provided information at a position corresponding to the position where the merchandise was displayed in response to the customer taking the merchandise.

[0046] Also, the merchandise shelf 100 may have a plurality of positions for displaying information. And the display control unit 14 may display different information at a plurality of locations.

[0047] As described above, there are multiple means for realizing the recognition by the recognition unit 11 and the position identification by the position identification unit 12, and there are also multiple means for displaying the provided information. In this embodiment, any combination can be adopted. Here, for the sake of caution, as a means for realizing the recognition by the recognition unit 11 and the position identification by the position identification unit 12, the above recognition / specific processing example 1, that is, the means for reading the code through the display part 101 from the back side of the display part 101, is adopted, and an implementation example in the case of adopting a configuration in which information is projected from the back side of the display part 101 and the provided information is displayed on the display part 101 will be described. Note that this description is for explaining that the said configuration is technically feasible, and the implementation means is not limited to this configuration.

[0048] For example, as shown in FIG. 5, a mirror 303 whose orientation is controlled by a rotation controller 304, a projection device 302, and a code reader 301 are installed on the back side of the display part 101 (the side opposite to the surface on which the product 200 is displayed). The controller 305 comprehensively controls the code reader 301, the projection device 302, and the rotation controller 304.

[0049] The code reader 301 reads the code of the product 200 displayed on the display part 101 by using the reflection of the mirror 303. Then, by controlling the direction of the electromagnetic wave reflected by controlling the posture of the mirror 303, the process of sequentially reading the codes of a plurality of products 200 displayed on the display part 101 is realized.

[0050] Similarly, the projection device 302 projects information onto the display part 101 by using the reflection of the mirror 303. Then, by controlling the direction of the electromagnetic wave reflected by controlling the posture of the mirror 303, the projection of information to a desired position on the display part 101 is realized.

[0051] Next, an example of the processing flow of the processing device 10 will be described using the flowchart of FIG. 6.

[0052] First, when the recognition unit 11 recognizes that "the customer has picked up the product" (Yes in S10), the position identification unit 12 identifies the position where the product was displayed (Sll). Also, the provided information acquisition unit 13 acquires provided information including information related to the product (S12). Note that the processing order of S11 and S12 is not limited to the illustrated example. Next, the display control unit 14 causes the provided information acquired in S12 to be displayed at a position corresponding to "the position where the product was displayed" identified in S11 (S13). Thereafter, the same processing is repeated.

[0053] According to the processing apparatus 10 of the present embodiment described above, the provided information can be displayed at "the position corresponding to the position where the product was displayed" at the timing when "the customer picks up the product". According to such a processing apparatus 10, the problem that the customer does not notice the displayed provided information is reduced.

[0054] Further, according to the processing apparatus 10 of the present embodiment, as shown in FIG. 4, the provided information can be displayed at "the position where the product picked up by the customer was displayed". According to such a processing apparatus 10, the problem that the customer does not notice the displayed provided information is further reduced.

[0055] Further, according to the processing apparatus 10 of the present embodiment, a plurality of identical products on display (a plurality of products having the same product code) can be identified from each other, and information related to each (the timing of display, the timing when sales started, the timing when processing was performed, etc.) can be provided to the customer. In this way, the processing apparatus 10 can provide more useful information to the customer.

[0056] <Second Embodiment> The processing apparatus 10 of the present embodiment is different from the first embodiment in that it acquires identification information of a customer who has picked up a product (information for identifying customers from each other) and displays provided information generated for that customer.

[0057] FIG. 7 shows an example of a functional block diagram of the processing apparatus 10 of the present embodiment. It is different from an example of the functional block diagram of the processing apparatus 10 of the first embodiment in that it has a customer identification information acquisition unit 16.

[0058] The customer identification information acquisition unit 16 acquires customer identification information of a customer. For example, the customer identification information acquisition unit 16 may acquire the customer identification information of the customer by biometric authentication processing. In the biometric authentication processing, biometric information is acquired from a customer present in the store. Then, based on the acquired biometric information and a collation process with a customer database in which biometric information of each customer is registered in advance in association with customer identification information, information that matches the acquired biometric information is searched for in the customer database.

[0059] Examples of the biometric information include, but are not limited to, face information, fingerprint information, voiceprint information, iris information, etc. The biometric information may be acquired without the customer being aware by a camera installed in the store, or may be acquired by the customer operating an input device such as a fingerprint sensor, a microphone, or a camera installed in the store. These input devices may be installed for each product shelf.

[0060] Alternatively, the customer identification information acquisition unit 16 may communicate with a customer device (smartphone, smartwatch, mobile phone, tablet terminal, IC (integrated circuit) tag, IC card, etc.) by short-range wireless communication and acquire customer identification information from a reader that acquires the customer identification information stored in the customer device. The reader may be installed for each product shelf. In this case, the customer performs an operation such as holding his or her customer device at a predetermined position (antenna position of the reader).

[0061] Alternatively, the customer identification information acquisition unit 16 may acquire customer identification information from a code reader that reads a code (barcode, two-dimensional code, etc.) indicating the customer identification information displayed on a customer device (smartphone, smartwatch, mobile phone, tablet terminal, etc.). The code reader may be installed for each product shelf. In this case, the customer performs an operation of displaying the code on his or her customer device and holding the code in front of the code reader to be read.

[0062] The provided information acquisition unit 13 acquires provided information based on customer registration information registered in association with customer identification information. The customer registration information is stored in an arbitrary server. The customer registration information is information registered by each customer himself / herself. Examples of the registration means include, but are not limited to, using an app, using a homepage, etc. Hereinafter, an example of customer registration information will be described.

[0063] "Example 1 of Customer Registration Information" The customer registration information may include desired information indicating the information that the customer wishes to display. The information that can be displayed when picking up a product includes multiple types of information. For example, the timing when the product is displayed, the timing when the sale of the product starts, the timing when the product is processed, advertisements of the product, campaign information of the product, usage methods of the product (cooking methods, ways of use, etc.), advertisements of similar products, campaign information of similar products, usage methods of similar products, discount timing of each product (time zone when a discount sticker is attached), etc. are exemplified.

[0064] The customer registers in advance the information (desired information) that he / she wishes to be displayed at the timing of picking up the product from among the multiple types of displayable information as described above.

[0065] In this case, the provided information acquisition unit 13 acquires, as provided information, the "information that the customer wishes to display" indicated by the desired information of the customer registration information from among the multiple types of displayable information related to the product picked up by the customer as described above. Then, the display control unit 14 displays only the information that the customer wishes to display.

[0066] "Example 2 of Customer Registration Information" The customer registration information may include a shopping list (a list of products to be purchased). The customer creates and registers a shopping list in advance before going shopping. In this case, the provided information acquisition unit 13 acquires the shopping list as provided information. Then, the display control unit 14 displays the shopping list as provided information.

[0067] Note that the processing device 10 may recognize the products placed in the shopping basket by each customer using any means such as image recognition processing. Then, the display control unit 14 may display in an identifiable manner whether the products included in the shopping list are either "products placed in the shopping basket" or "products not placed in the shopping basket". For example, the display control unit 14 may highlight the "products not placed in the shopping basket" or display only a list of the "products not placed in the shopping basket".

[0068] Note that in this case, instead of or in addition to the process of "displaying the provided information at a position corresponding to the position where the product taken by the customer was displayed in response to the recognition that the customer has taken the product", the display control unit 14 may perform the process of "displaying the shopping list at a predetermined position (display section 101, back section 102, display 103, etc.) of the product shelf in response to the acquisition of the customer identification information". Alternatively, the display control unit 14 may perform the process of "displaying the shopping list at a predetermined position (display section 101, back section 102, display 103, etc.) of the product shelf in response to the recognition that the customer has taken the product and the detection of the approach of the customer to the product shelf by the human sensor".

[0069] Here, a modification example of the processing device 10 of the present embodiment will be described. The processing device 10 may further include a store employee identification information acquisition unit. The store employee identification information acquisition unit acquires store employee identification information for identifying store employees from each other. The store employee identification information acquisition unit can acquire the store employee identification information by the same process as the customer identification information acquisition unit 16.

[0070] When the store employee identification information is acquired, the display control unit 14 displays, at a predetermined position (display section 101, back section 102, display 103, etc.) of the product shelf, information related to the product that should be notified to the store employee. The information to be notified to the store employee is, for example, information identifying products among the displayed products whose remaining time until the expiration date or consumption date has passed the reference time.

[0071] For example, as described in the above recognition and identification processing example 1, when codes are assigned to each product and the codes are read by a code reader, the codes may indicate the expiration date or shelf life of each product. Then, based on this information, the processing device 10 may identify products among the displayed products whose remaining time until the expiration date or shelf life has passed the reference time.

[0072] The display control unit 14 can display information for highlighting the product at a position corresponding to, for example, the "product whose remaining time until the expiration date or shelf life has passed the reference time" on the display section 101 (the product's display position, the surroundings of the product, etc.).

[0073] In the case of this modification example, in response to the acquisition of the store employee identification information, the display control unit 14 can cause information to be displayed at a predetermined position on the product shelf (the display section 101, the back section 102, the display 103, etc.) that should be notified to the store employee as described.

[0074] Other configurations of the processing device 10 in the present embodiment are the same as those in the first embodiment.

[0075] According to the processing device 10 of the present embodiment described above, the same operational effects as those in the first embodiment are realized. Further, according to the processing device 10, it is possible to acquire the identification information of a customer who has picked up a product and display the provided information generated for that customer. According to such a processing device 10, it is possible to display information suitable for each customer customized for each customer. Therefore, the processing device 10 can provide more useful information to the customer.

[0076] In addition, the processing device 10 can identify a customer and a store employee, and when a store employee is present (when the store employee identification information is acquired), it is possible to display information for the store employee on the product shelf. In this way, the technology of displaying information on the product shelf can be used not only for customers but also for store employees.

[0077] <Modification Example> A modified example applicable to all the embodiments will be described. The display control unit 14 can erase (end display) the provided information displayed in a predetermined position on the product shelf (display section 101, spine section 102, display 103, etc.) at a predetermined end timing. Examples of the end timing include "when a predetermined time has elapsed since the start of display," "when a customer who picked up a product leaves the area," and "when a predetermined input is made to a predetermined input device (touch panel, physical button, microphone, etc.) installed on the product shelf." The fact that a customer has left the area can be detected by any means, such as analyzing an image captured of the area or using the sensing results of a human presence sensor.

[0078] Another modification will be described. The display control unit 14 may display the provided information on a customer device (smartphone, smartwatch, mobile phone, tablet terminal, etc.). For example, the display control unit 14 may transmit the provided information to the customer device via short-range wireless communication. After operating the customer device to view the provided information, the customer can save or delete the information in the customer device.

[0079] Another modified example will be described. In the above embodiment, an example was described in which products were displayed on display section 101 such as a shelf board, but other display methods are also possible. For example, a display method in which products are hung from hooks is exemplified. In this case, at least one position (such as a display installed in the price tag area) is provided to display information in association with the display location, and similar processing is performed.

[0080] In this specification, "acquisition" means, based on user input or based on program instructions, "the self-device goes to obtain data stored in other devices or storage media (active acquisition)", for example, making a request or inquiry to another device and receiving the response, accessing another device or storage media and reading the data, etc. Further, "acquisition" means, based on user input or based on program instructions, "the self-device inputs data output from another device (passive acquisition)", for example, receiving data distributed (or transmitted, push-notified, etc.). Further, "acquisition" may include selecting and acquiring from the received data or information, and "generating new data by editing (textualizing, rearranging data, extracting partial data, changing file format, etc.) the data and acquiring the new data".

[0081] Some or all of the above embodiments may be described as follows in the appended claims, but are not limited thereto. 1. A recognition means for recognizing that a customer has picked up a product, A position specifying means for specifying the position where the product picked up by the customer was displayed, A display control means for displaying, at a position corresponding to the position where the product picked up by the customer was displayed, provision information including information related to the product picked up by the customer. A processing device having the above. 2. The processing device according to 1, wherein the display control means displays the provision information at the position where the product picked up by the customer was displayed. 3. The processing device according to 1 or 2, wherein the display control means displays the provision information at a position corresponding to the position where the product picked up by the customer was displayed in response to the recognition that the customer has picked up the product. 4. The information related to the product includes at least one of the timing when the product picked up by the customer was displayed, the timing when the sale of the product picked up by the customer was started, and the timing when the product picked up by the customer was processed, and is the processing device according to any one of 1 to 3. 5. The processing device further includes customer identification information acquisition means for acquiring the customer identification information of the customer, The display control means is the processing device according to any one of 1 to 4, which displays the provided information acquired based on the customer registration information registered in association with the customer identification information. 6. The customer registration information includes desired information indicating information that the customer desires to be displayed, The display control means is the processing device according to 5, which displays, as the provided information, the information that the customer desires to be displayed indicated by the desired information among the information related to the product picked up by the customer. 7. The customer registration information includes a shopping list, The display control means is the processing device according to 5 or 6, which displays the shopping list as the provided information. 8. A code is provided on the outer surface of the product, The product is displayed on the display member so that the code faces downward, The recognition means recognizes that the customer has picked up the product based on the reading result of a code reader that reads the code through the display member, The position specifying means is the processing device according to any one of 1 to 7, which specifies the position where the product picked up by the customer was displayed based on the reading result. 9. A computer recognizes that the customer has picked up the product, specifies the position where the product picked up by the customer was displayed, and a processing method of displaying the provided information including the information related to the product picked up by the customer at a position corresponding to the position where the product picked up by the customer was displayed. 10. A computer is provided with recognition means for recognizing that the customer has picked up the product, Position specifying means for specifying the position where the product picked up by the customer was displayed Display control means for displaying the provided information including the information related to the product picked up by the customer at a position corresponding to the position where the product picked up by the customer was displayed A program that functions as

[0082] Although the present invention has been described with reference to the embodiments (and examples) above, the present invention is not limited to the above embodiments (and examples). Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

Explanation of Signs

[0083] 1A Processor 2A Memory 3A Input / Output I / F 4A Peripheral Circuit 5A Bus 10 Processing Device 11 Recognition Unit 12 Position Specifying Unit 13 Provided Information Acquisition Unit 14 Display Control Unit 15 Display Unit 16 Customer Identification Information Acquisition Unit 100 Product Shelf 101 Display Section 102 Back Section 103 Display 200 Product 301 Code Reader 302 Projection Device 303 Mirror 304 Rotation Controller 305 Controller

Claims

1. recognition means for recognizing that an item has been picked up; position identification means for identifying the position where the item picked up was displayed; display control means for causing information related to the item picked up to be displayed at a position corresponding to the position where the item picked up was displayed; means for detecting both the customer and the store clerk by identifying each other; and has, the display control means, a processing device that causes the information to be displayed to differ depending on whether the customer is detected or the store clerk is detected.

2. the display control means, when the customer is detected, the timing when the item picked up was displayed, the timing when the sale of the item picked up was started, the timing when the item picked up was processed, the advertisement of the item picked up, the campaign information of the item picked up, the usage method of the item picked up, the advertisement of similar items of the item picked up, the campaign information of similar items of the item picked up, the usage method of similar items of the item picked up, or the discount timing of the item picked up is displayed, when the store clerk is detected, the processing device according to claim 1, which causes information based on the expiration date or shelf life of the item to be displayed.

3. the display control means, the processing device according to claim 1 or 2, which causes information to be displayed at the position where the item picked up was displayed.

4. the display control means, the processing device according to any one of claims 1 to 3, which causes information to be displayed at a position corresponding to the position where the item picked up was displayed in response to the recognition that the item has been picked up.

5. further having customer identification information acquisition means for acquiring the customer identification information of the customer, the display control means, the processing device according to any one of claims 1 to 4, which causes information acquired based on customer registration information registered in association with the customer identification information to be displayed.

6. the customer registration information indicates information that the customer wishes to be displayed, the display control means, the processing device according to claim 5, which causes the information that the customer wishes to be displayed among the information related to the item picked up to be displayed.

7. the customer registration information includes a shopping list, the display control means, the processing device according to claim 5 or 6, which causes the shopping list to be displayed.

8. a code is assigned to the outer surface of the item, the item is displayed on a display member so that the code faces downward, Based on the reading result of a code reader that reads the code through the display member, the recognition means recognizes the product taken in hand, The position specifying means specifies the position where the product taken in hand was displayed based on the reading result. The processing device according to any one of claims 1 to 7. **Claim 9** A computer recognizes that a product has been taken in hand, specifies the position where the product taken in hand was displayed, displays information related to the product taken in hand at a position corresponding to the position where the product taken in hand was displayed, detects both the customer and the store clerk by identifying each other, In the process of displaying the information, A processing method of making the displayed information different between the case where the customer is detected and the case where the store clerk is detected. **Claim 10** A computer is used as a recognition means for recognizing that a product has been taken in hand, a position specifying means for specifying the position where the product taken in hand was displayed, a display control means for displaying information related to the product taken in hand at a position corresponding to the position where the product taken in hand was displayed, a means for detecting both the customer and the store clerk by identifying each other, functions as, The display control means is a program that makes the displayed information different between the case where the customer is detected and the case where the store clerk is detected.

Images