Display device, display method, and program

Abstract

To display more detailed information of actions of a customer.SOLUTION: An image display device includes: an acquisition unit for acquiring information of a position which a customer reaches for in a shelf on which a plurality of items are displayed; a tallying unit for tallying the cumulative total frequency with which the client reached for the item in the position on the basis of information of the item and the information of the position; and a display unit for displaying the tallied cumulative total frequency in association with the position of the item in the shelf.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to ,table display device ,table display method and program.

Background Art

[0002] Customer behavior in stores is sometimes analyzed for purposes such as understanding customers' purchasing characteristics. For example, Patent Document 1 describes that the customer's staying time in the surrounding area and the number of times products are picked up are aggregated and displayed for each display shelf.

Prior Art Literature

Patent Literature

[0003]

Patent Document 1

Summary of the Invention

Problem to be Solved by the Invention

[0004] From the perspective of performing more advanced customer behavior analysis, more detailed display is desired than aggregating and displaying the customer's staying time in the surrounding area and the number of times products are picked up for each display shelf.

[0005] The present invention is capable of solving the above-mentioned problems Table display device ,table display method and program.

Means for Solving the Problem

[0006] According to a first aspect of the present invention, the display device includes a detection unit that detects that a customer has picked up a product on a shelf where a plurality of products are displayed, and detects, based on time-series images of the shelf where the plurality of products are displayed, the time from when the customer picks up the product until the customer returns the product to the shelf, or the time from when the customer picks up the product until the customer leaves the front of the shelf ,beforeThe time from when the customer picks up the item until they return it to the shelf, or the time from when the customer picks up the item until they leave the shelf. On the shelf where the aforementioned products are displayed In the format of a heatmap In addition It comprises a display unit for displaying information.

[0007] According to a second aspect of the present invention, the display method includes a computer that detects when a customer picks up an item from a shelf where multiple items are displayed, and based on a time-series image of the shelf where the multiple items are displayed, it detects the time from when the customer picks up the item until the item is returned to the shelf, or the time from when the customer picks up the item until the customer leaves the shelf. ,before The time from when the customer picks up the item until they return it to the shelf, or the time from when the customer picks up the item until they leave the shelf. On the shelf where the aforementioned products are displayed In the format of a heatmap In addition Display.

[0008] According to a third aspect of the present invention, the program provides a computer with the following processes: detecting that a customer has picked up an item from a shelf where multiple items are displayed; and detecting, based on a time-series image of the shelf where the multiple items are displayed, the time from when the customer picked up the item until they returned it to the shelf, or the time from when the customer picked up the item until they left the shelf. ,before The time from when the customer picks up the item until they return it to the shelf, or the time from when the customer picks up the item until they leave the shelf. On the shelf where the aforementioned products are displayed In the format of a heatmap In addition This is a program that performs the display process and executes the necessary actions. [Effects of the Invention]

[0009] According to this invention, information indicating customer behavior can be displayed in more detail. [Brief explanation of the drawing]

[0010] [Figure 1]It is a schematic block diagram showing the functional configuration of an image display system according to an embodiment of the present invention. [Figure 2] It is an explanatory diagram showing an arrangement example of the pre-shelf behavior measurement sensor and the shelf condition imaging device when the shelf is viewed from the side in the same embodiment. [Figure 3] It is an explanatory diagram showing an arrangement example of the pre-shelf behavior measurement sensor when the shelf is viewed from the front in the same embodiment. [Figure 4] It is an explanatory diagram showing a display example of customer behavior index values on a display unit according to the same embodiment. [Figure 5] It is an explanatory diagram showing a display example, on a display unit, of customer behavior index values indicating the correlation between a behavior that a customer performs on a product specified by a user of the image display system according to the same embodiment and a behavior that the same customer performs on another product. [Figure 6] It is an explanatory diagram showing an example of a data structure of information stored in a storage unit according to the same embodiment, in which sensing times are associated with product identification information. [Figure 7] It is a flowchart showing an example of a processing procedure in which an image display device according to the same embodiment generates post-association information. [Figure 8] It is a flowchart showing a first example of a processing procedure in which an image display device according to the same embodiment displays customer behavior index values. [Figure 9] It is a flowchart showing a second example of a processing procedure in which an image display device according to the same embodiment displays customer behavior index values. [Figure 10] It is an explanatory diagram showing the minimum configuration of an image display device according to the present invention. [Figure 11] It is an explanatory diagram showing the minimum configuration of an image display system according to the present invention. MODE FOR CARRYING OUT THE INVENTION

[0011] Hereinafter, embodiments of the present invention will be described, but the following embodiments do not limit the invention according to the claims. In addition, not all combinations of features described in the embodiments are necessarily essential to the solution means of the invention. FIG. 1 is a schematic block diagram showing the functional configuration of an image display system according to an embodiment of the present invention. As shown in the figure, an image display system 1 includes a pre-shelf behavior measurement sensor 110, a shelf status imaging device 120, and an image display device 200. The image display device 200 includes a communication unit 210, a display unit 220, an operation input unit 230, a storage unit 280, and a control unit 290. The control unit 290 includes an approach status information acquisition unit 291, an association unit 292, and an index value calculation unit 293.

[0012] The image display system 1 analyzes a customer's pre-shelf behavior and displays the analysis result superimposed on an image of a shelf. The term "in front of the shelf" as used herein refers to the area in front of (particularly near the front surface of) a product display shelf, and the pre-shelf behavior as used herein refers to behavior performed by a customer in front of the product display shelf. Hereinafter, the product display shelf is simply referred to as a shelf. The pre-shelf behavior measurement sensor 110 is a sensor provided to detect pre-shelf behavior. As the pre-shelf behavior measurement sensor 110, for example, a distance sensor provided from above to below the front surface of the shelf can be used. When a customer reaches a hand toward the shelf, the pre-shelf behavior measurement sensor 110 detects the horizontal position and vertical position of the customer's hand in front of the shelf, whereby the image display device 200 can estimate the product that the customer has reached a hand for.

[0013] The shelf status imaging device 120 includes a camera provided in front of the shelf for capturing moving images or still images, and captures the state in front of the shelf and the state of the shelf continuously or at predetermined time intervals. Based on the image captured by the shelf status imaging device 120, the image display device 200 can detect the timing from when the customer stops in front of the shelf to when the customer leaves the front of the shelf. Thereby, the image display device 200 can detect reaching for the shelf performed after the customer stops in front of the shelf and before leaving as a series of reaching operations by the same customer. Further, the image display device 200 can detect that the customer has picked up a product and that the customer has returned the picked-up product to the shelf by comparing images before and after the customer reaches a hand toward the shelf.

[0014] Here, an example of the arrangement of the shelf-front behavior measurement sensor 110 and the shelf condition imaging device 120 will be described with reference to Figures 2 and 3. Figure 2 is an explanatory diagram showing an example of the arrangement of the shelf-front behavior measurement sensor 110 and the shelf status imaging device 120 when viewed from the side of the shelf. In the example shown in the figure, the shelf-front behavior measurement sensor 110 is installed facing downwards from the ceiling 930 above the front of the shelf 910. The shelf 910 is a multi-tiered shelf, with products 920 placed on each tier. When a customer reaches for the shelf 910, the shelf-front behavior measurement sensor 110 measures the distance between itself and the customer's hand. Using the measured distance, the image display device 200 can determine which tier of the shelf 910 the customer reached for.

[0015] The camera of the shelf condition imaging device 120 is positioned and oriented to photograph the shelf 910 from the ceiling 930 in front of the shelf 910. For example, the shelf condition imaging device 120 may have one camera, positioned and oriented to photograph the entire front of the shelf 910. Alternatively, if one camera cannot photograph the entire front of the shelf 910, the shelf condition imaging device 120 may have multiple cameras.

[0016] Figure 3 is an explanatory diagram showing an example of the arrangement of shelf-front behavior measurement sensors 110 when viewed from the front of the shelf. In this example, multiple shelf-front behavior measurement sensors 110 are arranged on the ceiling 930 along the width direction of the shelf 910. When a customer reaches for the shelf 910, the image display device 200 can determine the position of the hand in the horizontal direction of the shelf 910 by identifying the shelf-front behavior measurement sensor 110 that detected the hand.

[0017] However, the shelf-front behavior measurement sensor 110 only needs to be able to detect the position of the hand in the vertical direction of the shelf 910 and the position of the hand in the horizontal direction of the shelf 910 when a customer reaches for the shelf 910. For example, a set of 3D cameras may be installed above the front of the shelf 910 as the shelf-front behavior measurement sensor 110 to capture images of the front of the shelf in the vertical direction. The installation location of the shelf-front behavior measurement sensor 110 is not limited to above the shelf 910. For example, the shelf-front behavior measurement sensor 110 may be installed to the side of the shelf 910 to sense the front of the shelf 910 from a lateral direction.

[0018] The image display device 200 analyzes the behavior in front of the shelves based on sensing data from the shelf-front behavior measurement sensor 110. Specifically, as explained with reference to Figures 2 and 3, when a customer reaches for shelf 910, the image display device 200 detects the vertical position of the hand on shelf 910 and the horizontal position of the hand on shelf 910 based on sensing data from the shelf-front behavior measurement sensor 110. The image display device 200 also stores the arrangement of products 920 on shelf 910 in advance and estimates which product the customer reached for based on the position of the customer's hand. The image display device 200 counts, for example, the number of times the customer reached for each product and displays the counting result (for example, the total value for all customers within a predetermined period) overlaid on the image of shelf 910. The image display device 200 is configured, for example, using a computer.

[0019] The communication unit 210 communicates with other devices. The communication unit 210 receives sensing data from the shelf-front behavior measurement sensor 110. The communication unit 210 also receives images captured by the shelf condition imaging device 120 as image data. The display unit 220 has a display screen such as an LCD panel or an LED (Light-Emitting Diode) panel, and displays various images according to the control of the control unit 290. The display unit 220 displays customer behavior indicator values, such as the count result of the number of times a customer reached for a product, overlaid on the image of the shelf 910. The customer behavior indicator values ​​referred to here are indicator values ​​calculated by the indicator value calculation unit 293 regarding customer behavior.

[0020] Figure 4 is an explanatory diagram showing an example of customer behavior indicator values ​​displayed by the display unit 220. In this example, the display unit 220 displays an image of the shelf 910 on which the products 920 are arranged in area A11. The display unit 220 also displays the product images with colors corresponding to the number of times the customer has reached for the product. The display unit 220 also displays a legend in area A12 showing the correspondence between the number of times the customer reached for the product and the color. The display unit 220 displays the number of times the customer reached for the product in a heatmap-like manner. A heatmap, in this context, is a display method that shows the values ​​of each element of a matrix using color.

[0021] The number of times a customer reaches for a product is an example of a customer behavior indicator value. The display unit 220 displays the customer behavior indicator value at the position of the image of product 920 in the image of shelf 910, using the color assigned to the image of that product. In this respect, the display unit 220 displays the customer behavior indicator value together with the image of the product. However, the method by which the display unit 220 displays the customer behavior indicator value is not limited to coloring the product image. For example, the display unit 220 may display the customer behavior indicator value by applying a predetermined pattern such as stripes or dots to the product image instead of color. The display unit 220 may also display a number indicating the customer behavior indicator value superimposed on the product image.

[0022] Furthermore, the display unit 220 displays an index value indicating the correlation between the actions a customer takes towards a product specified by the user of the image display system 1 and the actions the same customer takes towards other products, at the position of the image of product 920 in the image of shelf 910. Hereafter, the user of the image display system 1 will be simply referred to as the user.

[0023] Figure 5 is an explanatory diagram showing an example of how the display unit 220 displays customer behavior indicator values, which indicate the correlation between customer actions towards a product specified by the user of the image display system 1 and actions taken by the same customer towards other products. Similar to the example in Figure 4, in the example in Figure 5, the display unit 220 displays an image of the shelf 910 on which the products 920 are arranged in area A11. In addition, a legend showing the correspondence between the number of times a customer reaches for a product and its color is displayed in area A12.

[0024] Unlike in Figure 4, in Figure 5, the display unit 220 displays the product 920 specified (selected) by the user. In the example in Figure 5, the user has specified product 920 shown in area A21, and the display unit 220 displays the specified product 920 by displaying a stripe in area A21. Hereafter, the product 920 specified by the user will be referred to as the specified product.

[0025] In the example in Figure 5, the display unit 220 shows, in a heatmap-like format, the count of the number of times customers who reached for a designated product (at least once) also reached for other products 920 (for example, the total value for all customers within a predetermined period). Specifically, the display unit 220 displays each product 920 image with a color corresponding to the number of times customers who reached for a designated product (at least once) also reached for other products 920. In the example in Figure 5, since customers who reached for the product displayed in area A21 (the designated product) also reached for the product displayed in area A22 many times, the display unit 220 displays the image of the product 920 in area A22 with a color indicating that it was reached for many times.

[0026] The operation input unit 230 is equipped with input devices such as a keyboard and mouse, and receives user input. The operation input unit 230 is an example of a product selection receiving unit and receives product selection. For example, the user selects product 920 displayed on the display unit 220 by clicking on it with the mouse.

[0027] The memory unit 280 stores various types of information. The memory unit 280 stores sensing data obtained by the shelf-front behavior measurement sensor 110, or data processed by the control unit 290 from said sensing data. For example, the memory unit 280 stores information (described later as "associated information") that associates product identification information indicating the product 920 that the customer reached for with the time when the shelf-front behavior measurement sensor 110 detected the customer's hand. Furthermore, the memory unit 280 stores the information that associates product identification information indicating the product 920 that the customer reached for with the time when the shelf-front behavior measurement sensor 110 detected the customer's hand as information on a series of reaching by the same customer for the period from when the customer approaches the shelf until when they leave the shelf. The product identification information referred to here is information that identifies each product 920 placed on shelf 910. In this embodiment, products with the same specifications (for example, products with the same name) are considered to be the same product. Pre-defined information such as the product name may be used as the product identification information, or identification information specific to the image display system 1 may be defined.

[0028] Figure 6 is an explanatory diagram showing an example of the data structure of the information (post-association information, described later) that associates the sensing time with product identification information and is stored in the storage unit 280. As shown in the figure, the storage unit 280 stores the group number, the sensing time, and the product identification information in association. In the example shown in Figure 6, the product identification information indicates the product 920 that the customer reached for, which was identified based on sensing data from the shelf-front behavior measurement sensor 110.

[0029] The sensing time indicates the time when the shelf-front behavior measurement sensor 110 detected the object indicated in the sensing data. For example, the time when the communication unit 210 received the sensing data may be used as the sensing time. The group number is used as information to group together a series of reaching actions detected from the time a customer approaches the shelf until they leave. The image display device 200 detects the timing from when a customer approaches the shelf until they leave based on the image captured by the shelf status camera 120, and assigns the same group number to the information (combination of sensing time and product identification information) based on the sensing data transmitted by the shelf behavior measurement sensor 110 during this period. This makes it possible to detect that the same customer reached for the shelf 910 multiple times, as shown in the example in Figure 5.

[0030] However, the information stored by the storage unit 280 is not limited to the information that associates sensing time with product identification information as shown in the example in Figure 6. For example, the storage unit 280 may store sensing data from the shelf-front behavior measurement sensor 110 in association with time information indicating the sensing time. Alternatively, the storage unit 280 may store information that converts the sensing data from the shelf-front behavior measurement sensor 110 into vertical and horizontal positions on the shelf 910 in the form of coordinate values. Alternatively, the storage unit 280 may store information for each product 920 indicating the number of times a customer has reached for that product.

[0031] The memory unit 280 stores the images taken by the shelf condition imaging device 120 in chronological order, corresponding to the time of capture. The memory unit 280 is implemented using a memory device provided by the image display device 200. The memory device provided by the image display device 200 may be a memory device built into the image display device 200, or it may be a memory device attached externally to the image display device 200.

[0032] The control unit 290 controls various parts of the image display device 200 to perform various processes. The control unit 290 is realized, for example, by the CPU (Central Processing Unit) of the image display device 200 reading a program from the storage unit 280 and executing it. The proximity information acquisition unit 291 acquires proximity information. This proximity information indicates whether at least a part of a customer's body has approached the shelf 910 and, if so, the position of the customer on the shelf 910 at each given time.

[0033] Specifically, when the shelf-front behavior measurement sensor 110 detects an object (especially a customer's hand), the proximity information acquisition unit 291 converts the sensing data into coordinate values ​​on the front of the shelf 910. For example, regarding the horizontal direction of the shelf 910, the storage unit 280 pre-stores information indicating the position where each shelf-front behavior measurement sensor 110 detects an object. The proximity status information acquisition unit 291 reads the information of the position where the shelf-front behavior measurement sensor 110 detects an object from the storage unit 280, according to the shelf-front behavior measurement sensor 110 that has detected an object.

[0034] Furthermore, the proximity information acquisition unit 291 reads the distance between the object detected by the shelf-front behavior measurement sensor 110 and the shelf-front behavior measurement sensor 110 itself from the sensing data. The proximity information acquisition unit 291 converts the read distance into the vertical position of the shelf 910. The proximity information acquisition unit 291 generates proximity information by associating the reception time information of the sensing data with the coordinate values ​​as time information indicating the sensing time. The presence of this proximity information indicates that the shelf-front behavior measurement sensor 110 has detected an object.

[0035] When the shelf-front behavior measurement sensor 110 detects an object, it is considered that at least a part of the customer's body (especially the customer's hand) has approached the shelf 910. In this regard, the proximity information indicates whether or not at least a part of the customer's body has approached the shelf 910. The proximity information indicates, in coordinate values, whether or not at least a part of the customer's body has approached and, if so, its position on the shelf 910. The position indicated by the coordinates is the position on the shelf 910 where at least a part of the customer's body approached.

[0036] The mapping unit 292 associates the proximity information generated by the proximity information acquisition unit 291 with one of the products 920 according to the location indicated by the proximity information. For example, the storage unit 280 pre-stores information that associates the location information detected by the proximity information acquisition unit 291 from the sensing data with one of the products 920. Furthermore, the storage unit 280 pre-stores the position of each product 920 placed on the shelf 910 in two-dimensional coordinate values ​​in the horizontal and vertical directions of the shelf 910. The mapping unit 292 associates the proximity information with the product 920 by reading the product 920 that is associated with the coordinate values ​​indicated by the proximity information generated by the proximity information acquisition unit 291 from the storage unit 280.

[0037] The mapping unit 292 generates information that associates product identification information identifying the associated product 920 with time information indicating the sensing time, and stores it in the storage unit 280. Hereinafter, the information generated by the mapping unit 292 that associates product identification information with time information indicating the sensing time will be referred to as the "post-mapping information." The post-mapping information is an example of information indicating a product associated with proximity information.

[0038] The indicator value calculation unit 293 calculates customer behavior indicator values ​​based on the mapped information generated by the mapping unit 292. For example, as shown in the example in Figure 4, the indicator value calculation unit 293 aggregates the number of times a customer reached for product 920 within a predetermined period for each product 920, and calculates a customer behavior indicator value showing the aggregated result. Furthermore, the index value calculation unit 293 calculates a customer behavior index value that shows the correlation between the actions a customer takes towards a designated product and the actions they take towards other products. For example, as shown in the example in Figure 5, the index value calculation unit 293 calculates a customer behavior index value for each of the other products 920 that a customer who reached for a designated product also reached for other products 920.

[0039] The operation of the image display system 1 will be explained with reference to Figures 7 to 9. Figure 7 is a flowchart showing an example of the processing procedure by which the image display device 200 generates the associated information. The image display device 200 starts the processing shown in the figure when its own power supply is connected (ON) and it enters an operating state. In the process shown in Figure 7, the proximity information acquisition unit 291 determines whether or not it has detected a customer approaching the shelf (step S101). Specifically, the proximity information acquisition unit 291 determines that it has detected a customer approaching the shelf if it detects in the image taken by the shelf condition camera 120 that there is a customer in front of the shelf who was not there before. If multiple customers are visible in the image taken by the shelf condition camera 120, the proximity information acquisition unit 291 may determine whether or not it has detected each of the multiple customers approaching the shelf.

[0040] If the system determines that it has not detected a customer approaching the shelf (step S101: NO), the proximity status information acquisition unit 291 returns to step S101. In other words, the image display device 200 waits for a customer to approach the shelf. If it is determined in step S101 that a customer has approached the shelf (step S101: YES), the proximity information acquisition unit 291 determines a group number for that customer's approach (step S102). For example, the initial value of the group number may be set to 0, and each time a customer approaches the shelf, the proximity information acquisition unit 291 may add 1 to the group number.

[0041] The proximity information acquisition unit 291 determines whether or not it has detected a customer leaving the shelf (step S103). Specifically, the proximity information acquisition unit 291 determines that it has detected a customer leaving the shelf if it detects from the image taken by the shelf condition camera 120 that the customer who was previously in front of the shelf is no longer there. If multiple customers are visible in the image taken by the shelf condition camera 120, the proximity information acquisition unit 291 may determine whether or not it has detected the departure of each of the multiple customers. If the unit determines that it has detected a customer approaching from in front of the shelf (step S103: YES), the proximity information acquisition unit 291 returns to step S101.

[0042] If step S103 determines that no customer approaching from in front of the shelf has been detected (step S103: NO), the proximity status information acquisition unit 291 determines whether or not sensing data has been acquired from the shelf-front behavior measurement sensor 110 (step S104). Specifically, the proximity status information acquisition unit 291 determines whether or not the communication unit 210 has received the sensing data.

[0043] If it is determined that sensing data has not been acquired (step S104: NO), the approach status information acquisition unit 291 returns to step S104. That is, the image display device 200 waits for sensing data from the shelf-front behavior measurement sensor 110. If it is determined in step S104 that sensing data has been acquired (step S104: YES), the proximity information acquisition unit 291 generates proximity information (step S105). Specifically, the proximity information acquisition unit 291 converts the sensing data received by the communication unit 210 into two-dimensional coordinate values ​​on the front of the shelf 910, and generates proximity information based on the obtained two-dimensional coordinate values.

[0044] The mapping unit 292 generates mapped information based on the two-dimensional coordinate values ​​obtained in step S105 (step S106). Specifically, the mapping unit 292 generates mapped information consisting of one row, as shown in the example data structure in Figure 6, by combining the group number determined in step S102, the sensing time, and the product identification information obtained from the two-dimensional coordinate values. The mapping unit 292 stores the mapped information (information for one line) obtained in step S106 in the storage unit 280 (step S107). The matching unit 292 returns to step S103 after step S107.

[0045] Figure 8 is a flowchart showing a first example of the processing procedure for displaying customer behavior indicator values ​​by the image display device 200. As explained with reference to Figure 4, Figure 8 shows an example where the display unit 220 displays the number of times a customer has reached for each product 920 in color. The image display device 200 starts the process shown in Figure 8 when the operation input unit 230 receives a user operation, for example, instructing the display of customer behavior indicator values.

[0046] In the process shown in Figure 8, the index value calculation unit 293 counts the number of times a customer reaches for each product 920 based on the associated information obtained in the process shown in Figure 7 (step S201). The index value calculation unit 293 colors each of the product 920 images in the shelf 910 image based on the counting result obtained in step S201 (step S202). The indicator value calculation unit 293 displays an image of the shelf 910 with the product 920 colored in step S202 on the display unit 220 (step S203). The index value calculation unit 293 terminates the process shown in Figure 8 after step S203.

[0047] Figure 9 is a flowchart showing a second example of the processing procedure for displaying customer behavior indicator values ​​by the image display device 200. As explained with reference to Figure 5, Figure 9 shows an example where the display unit 220 displays, in different colors, the number of times a customer who reached for a designated product also reached for other products 920. When the operation input unit 230 receives a user operation instructing the image display device 200 to display customer behavior indicator values ​​that show the correlation between products, the processing shown in Figure 9 begins.

[0048] In the process shown in Figure 9, the operation input unit 230 receives a designation of product 920 (step S301). For example, while the display unit 220 is displaying an image of the shelf 910, the operation input unit 230 receives a mouse click to select one of the images of product 920 included in the image of the shelf 910. The designation of product 920 in step S301 is the selection of a designated product. The index value calculation unit 293 selects all of the mapped information obtained in the process shown in Figure 7 that has the same group number as the mapped information indicating reaching for the designated product (step S302). In other words, the index value calculation unit 293 obtains mapped information indicating the reaching made by the customer who reached for the designated product.

[0049] The index value calculation unit 293 counts the number of times the customer reached for each product 920 other than the designated product, based on the associated information obtained in step S302 (step S303). Based on the counting results obtained in step S303, the index value calculation unit 293 colors each of the images of products 920 other than the specified product in the image of the shelf 910 (step S304). The indicator value calculation unit 293 displays an image of the shelf 910 with the product 920 colored on it on the display unit 220 in step S304 (step S305). The index value calculation unit 293 terminates the process shown in Figure 9 after step S305.

[0050] As described above, the proximity information acquisition unit 291 acquires proximity information that indicates whether at least a part of the customer's body has approached the shelf 910 on which the product 920 is displayed, and, if so, the position of the customer on the shelf 910 at each given time. The matching unit 292 then associates the proximity information with one of the products 920 according to the position indicated by the proximity information. This allows the matching unit 292 to identify the product 920 that the customer reached for. The index value calculation unit 293 calculates an index value related to the customer's behavior based on the information indicating the product 920 associated with the proximity information. The display unit 220 displays the index value calculated by the index value calculation unit 293 at the position of the image of product 920 in the image of the shelf 910, based on the matching performed by the matching unit 292. This allows the image display system 1 to display information indicating customer behavior in more detail. In particular, the image display system 1 can display information indicating customer behavior for each product in more detail than for each display shelf. Users can refer to this display to perform more advanced analysis of customer behavior.

[0051] The operation input unit 230 receives product selection via user input. The index value calculation unit 293 calculates an index value that shows the correlation between the actions taken by the same customer for the selected product and other products. This allows the display unit 220 to visually display the correlation between products. Therefore, users can refer to this display and analyze the correlation between products.

[0052] The user can use the image display system 1 to see where the customer reached for on the shelf. This allows the user to understand where on the shelf the customer is most likely to reach. Furthermore, the user can use the image display system 1 to obtain information such as which products the customer reached for and which products they compared. The user can then use this information for marketing individual products.

[0053] In addition to displaying the number of times a customer reaches for a product, as explained with reference to Figures 4 and 5, or alternatively, the display unit 220 may display the number of times a customer picks up a product. For example, the image display system 1 may be equipped with a camera that takes pictures from above the shelf 910 downwards, and the index value calculation unit 293 may analyze the image from this camera to determine whether or not a customer has picked up a product.

[0054] The display unit 220 may also display information regarding whether or not a product was purchased, such as whether the customer purchased the product or returned it to the shelf. For example, the index value calculation unit 293 compares an image taken by the shelf status camera 120 before the customer approaches the shelf with an image taken by the shelf status camera 120 after the customer leaves the shelf. If the index value calculation unit 293 detects that the number of products has decreased, it determines that the customer purchased the product. If the index value calculation unit 293 detects that the number of products has not decreased, it determines that the customer returned the product they picked up to the shelf.

[0055] The display unit 220 may also display the time a customer has taken hold of an item. For example, the display unit 220 may display the average time a customer has taken hold of an item, such as the time from when the customer takes hold of an item until they return it, and the time from when the customer takes hold of an item until they leave the shelf. This allows users to understand whether an individual product is something that customers will carefully consider before purchasing, or something that they have already decided to buy, and can use this information to inform their product marketing.

[0056] The image display system 1 may calculate the time a customer spends in front of a shelf and display it along with the shelf-in-front behavior indicator values ​​mentioned above. The image display system 1 may include a camera that photographs customers in front of shelves from the shelf side, and may acquire information on at least one of the customer's face orientation and the direction of their gaze, and display it together with the display of the shelf-front behavior indicator values ​​described above. The image display system 1 may also display information other than shelf-front behavior in addition to the display of the shelf-front behavior indicator values ​​described above. For example, the memory unit 280 may store information in advance about whether a customer is a member or not, and the image display system 1 may extract and display information such as "members often pick up this product" as a shelf-front behavior indicator value.

[0057] The minimum configuration of the present invention will be described with reference to Figures 10 and 11. Figure 10 is an explanatory diagram showing the minimum configuration of the image display device according to the present invention. The image display device 10 shown in the figure comprises an approach situation information acquisition unit 11, a correspondence unit 12, an index value calculation unit 13, and a display unit 14. In this configuration, the proximity information acquisition unit 11 acquires proximity information indicating whether at least a part of a customer's body is approaching the shelf where the products are displayed, and, if so, the position on the shelf at each time. The matching unit 12 associates the proximity information with one of the products according to the position indicated by the proximity information. The index value calculation unit 13 calculates an index value related to the customer's behavior based on the information indicating the product associated with the proximity information. The display unit 14 displays the index value calculated by the index value calculation unit 13 at the position of the product image in the shelf image, based on the matching by the matching unit 12. This allows the image display device 10 to display information indicating customer behavior in more detail. In particular, the image display device 10 can display information indicating customer behavior for each product in more detail than for each display shelf. Users can refer to this display to perform more advanced analysis of customer behavior.

[0058] Figure 11 is an explanatory diagram showing the minimum configuration of the image display system according to the present invention. The image display system 20 shown in the figure comprises a shelf-front behavior measurement sensor 21, an approach status information acquisition unit 22, a correspondence unit 23, an index value calculation unit 24, and a display unit 25. In this configuration, the shelf-front behavior measurement sensor 21 detects the approach of at least a part of a customer's body to a shelf where products are displayed and generates sensing data indicating the position of the approach point on the shelf. The approach status information acquisition unit 22 generates approach status information based on the sensing data, indicating whether or not an approach occurred and, if so, the position on the shelf for each time period. The mapping unit 23 maps the approach status information to a product according to the position indicated by the approach status information. The index value calculation unit 24 calculates an index value related to the customer's behavior based on the information indicating the product mapped to the approach status information. The display unit 25 displays the index value calculated by the index value calculation unit 24 at the position of the product image in the shelf image based on the mapping by the mapping unit 23. This allows the image display system 20 to display information indicating customer behavior in more detail. In particular, the image display system 20 can display information indicating customer behavior for each product in more detail than for each display shelf. Users can refer to this display to perform more advanced analysis of customer behavior.

[0059] The processing of each unit may be performed by recording a program for realizing all or part of the functions of the control unit 290, the proximity information acquisition unit 11, the mapping unit 12, the index value calculation unit 13, the proximity information acquisition unit 22, the mapping unit 23, and the index value calculation unit 24 on a computer-readable recording medium, and then having a computer system read and execute the program recorded on this recording medium. The term "computer system" here includes hardware such as the OS and peripheral devices. "Computer-readable recording media" refers to portable media such as flexible disks, magneto-optical disks, ROMs, and CD-ROMs, as well as storage devices such as hard disks built into computer systems. Furthermore, the above-mentioned program may only be for the purpose of implementing some of the functions described above, and may also be able to implement the above-mentioned functions in combination with programs already recorded in the computer system.

[0060] While embodiments of this invention have been described in detail above with reference to the drawings, the specific configuration is not limited to these embodiments and includes designs and the like that do not depart from the spirit of this invention.

[0061] This application claims priority based on Japanese Patent Application No. 2016-057537, filed in Japan on March 22, 2016, and the contents of that application are incorporated herein by reference. [Industrial applicability]

[0062] This invention can be applied to applications where it is essential to display more detailed information about customer behavior. [Explanation of symbols]

[0063] 1. 20 Image Display System 10,200 Image Display Devices 11, 22, 291 Approach Situation Information Acquisition Unit 12, 23, 292 Correspondence section 13, 24, 293 Indicator Value Calculation Section 14, 25, 220 display section 21, 110 Sensors for measuring behavior in front of shelves 120 Shelf condition photographing device 210 Communications Department 230 Operation Input Section 280 Storage section 290 Control Unit

Claims

1. A detection unit that detects when a customer picks up an item from a shelf where multiple items are displayed, and detects, based on a time-series image of the shelf where the multiple items are displayed, the time from when the customer picks up the item until they return it to the shelf, or the time from when the customer picks up the item until they leave the shelf, A display unit that overlays the time from when the customer picks up the product until they return it to the shelf, or from when the customer picks up the product until they leave the shelf, onto the shelf where the product is displayed in the form of a heat map, A display device equipped with the following features.

2. The detection unit detects the average time from when the customer picks up the product until they return it to the shelf, or the average time from when the customer picks up the product until they leave the shelf. The display device according to claim 1.

3. Computers The system detects when a customer picks up an item from a shelf where multiple items are displayed, and based on a time-series image of the shelf where the multiple items are displayed, it detects the time from when the customer picks up the item until they return it to the shelf, or the time from when the customer picks up the item until they leave the shelf. The time from when the customer picks up the product until they return it to the shelf, or the time from when the customer picks up the product until they leave the shelf, is displayed on the shelf where the product is arranged in the form of a heat map. Display method.

4. On the computer, A process that detects when a customer picks up an item from a shelf where multiple items are displayed, and detects, based on a time-series image of the shelf where the multiple items are displayed, the time from when the customer picks up the item until they return it to the shelf, or the time from when the customer picks up the item until they leave the shelf. A process of overlaying the time from when the customer picks up the product until they return it to the shelf, or from when the customer picks up the product until they leave the shelf, onto the shelf where the product is displayed in the form of a heat map, A program to execute.

Images