Display system and display method

Abstract

To provide a display system capable of efficiently displaying commodities.SOLUTION: In a display system 100 including a store server 30 and a smart glass 50, the store server 30 associates and stores a display position of a commodity 70 displayed on a shelf 80 with a status of the commodity 70 based on a captured image acquired from an imaging device 60 that captures images inside a store 200, and the smart glass 50 superimposes and displays a virtual image of the status of the commodity 70 associated with the display position of the commodity 70 on a real image of the commodity 70, in accordance with the display position of the commodity 70 transmitted from the store server 30.SELECTED DRAWING: Figure 13

Description

[Technical Field]

[0001] The present invention relates to a display system and a display method. [Background technology]

[0002] Traditionally, retailers such as convenience stores have rearranged or removed products in order to keep their shelf life close to a certain date by visually checking the expiration dates printed on labels attached to the products.

[0003] For example, a shelf allocation information creation device has been disclosed that arranges products in descending order of priority for each product set by a priority setting unit in a product placement area that is deemed easy to see based on the weight information set by a weight information setting unit (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 10-228565 Summary of the Invention [Problem to be solved by the invention]

[0005] However, because product expiration dates were checked visually, mistakes were made in rearranging products and removing products by mistake. As a result, sales were lost due to expired products or deterioration in quality, and sales were also lost due to rearranging or removing errors.

[0006] In view of the above-mentioned problems, an object of the present invention is to provide a display system that can efficiently display products. [Means for solving the problem]

[0007] In order to achieve the above-mentioned object, the display system of the present invention is a display system comprising a store server and smart glasses, wherein the store server associates and stores the display position of a product displayed on a shelf with the status of the product based on an image acquired from an imaging device that captures images inside the store, and the smart glasses display a virtual image of the status of the product associated with the display position of the product, superimposed on a real image of the product, in accordance with the display position of the product transmitted from the store server.

[0008] Furthermore, the display method of the present invention is a display method executed by a display system including a store server and smart glasses, and is characterized in that, based on an image acquired from an imaging device displayed in the store, the display position of a product displayed on a shelf and the status of the product are associated and stored in the store server, and the smart glasses display a virtual image of the status of the product associated with the display position of the product, superimposed on a real image of the product, in accordance with the display position of the product transmitted from the store server. [Effects of the Invention]

[0009] According to the present invention, merchandise can be efficiently displayed. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a diagram showing a state in which the display system according to the present embodiment is used; [Figure 2] FIG. 2 is a diagram illustrating an example of a hardware configuration of a product learning information DB. [Figure 3] FIG. 2 is a diagram illustrating an example of a hardware configuration of a viewing terminal. [Figure 4] FIG. 2 is a diagram illustrating an example of a hardware configuration of a store server. [Figure 5] FIG. 2 is a diagram illustrating an example of a hardware configuration of a mobile terminal. [Figure 6] FIG. 1 is a diagram illustrating an example of a hardware configuration of smart glasses. [Figure 7] FIG. 2 is a diagram illustrating an example of a hardware configuration of an imaging apparatus. [Figure 8] FIG. 2 is a diagram showing an example of product learning information of a product stored in a product learning information DB. [Figure 9] 10 is a diagram showing an example in which product display position information indicating the display positions of products displayed on shelves is acquired by each imaging device displayed in a store. FIG. [Figure 10] 10 is a diagram showing an example in which product display position information indicating the display positions of products displayed on shelves is acquired by each imaging device displayed in a store. FIG. [Figure 11] FIG. 2 is a diagram showing an example of a display management DB stored in a storage unit of a store server. [Figure 12] FIG. 10 is a diagram showing an example of changing the product management status and product display position information stored in the display management DB based on the display position of the product displayed on the shelf. [Figure 13] FIG. 10 is a diagram showing an example of an image viewed through smart glasses. DETAILED DESCRIPTION OF THE INVENTION

[0011] The present embodiment will be described below with reference to the drawings. Note that each embodiment can be implemented in combination with a plurality of other embodiments within a range that does not contradict each other.

[0012] A display system 100 according to this embodiment will be described. Fig. 1 is a diagram showing a state in which the display system 100 according to this embodiment is in use. The display system 100 includes a product learning information database (DB) 10, a viewing terminal 20, a store server 30, a mobile terminal 40, smart glasses 50, and an imaging device 60. The display system 100 may include components other than those shown in Fig. 1.

[0013] 2 is a diagram showing an example of the hardware configuration of the product learning information DB 10. The product learning information DB 10 stores product learning information in advance through machine learning. The product learning information DB 10 is configured by a server such as a PC equipped with a control unit 11, a storage unit 12, a display unit 13, an input unit 14, and a communication unit 15. One product learning information DB 10 is sufficient within the display system 100, but this is not limited thereto; each company or store 200 may have multiple product learning information DBs 10.

[0014] The control unit 11 controls the entire product learning information DB 10 and has a CPU (Central Processing Unit) that reads programs and executes control processes. The control unit controls the operation of the entire product learning information DB 10 by loading programs stored in the RAM (Random Access Memory), ROM (Read Only Memory), HDD (Hard Disk Drive), SSD (Solid State Drive), etc. of the storage unit 12 into the RAM of the storage unit 12 and sequentially executing the programs. The control unit 11 controls the operation of the entire product learning information DB 10 by executing various programs such as OS (Operating System) programs and application programs.

[0015] The storage unit 12 is a RAM, ROM, HDD, SSD, etc., and stores the program executed by the control unit 11, product learning information, information input to the product learning information DB 10, feature information of the product 70, various setting information, and the like.

[0016] The display unit 13 is, for example, a liquid crystal display (LCD) or an organic electroluminescence (EL) display, and displays an operation screen or the like to a user who operates the product learning information DB 10 under the control of the control unit 11.

[0017] The input unit 14 receives data input based on user operation of the product learning information DB 10. The input unit 14 is configured with a keyboard, a mouse, a touch panel, etc. The input information is notified to the control unit 11.

[0018] The communication unit 15 communicates with other terminals via a network N such as the Internet under the control of the control unit 11. The communication with other terminals is performed via wired communication or wireless communication such as Wi-Fi (registered trademark) or Bluetooth (registered trademark). For example, the communication unit 15 communicates with the viewing terminal 20 and the store server 30 via the network N under the control of the control unit 11. The communication unit 15 is, for example, a network adapter.

[0019] 3 is a diagram showing an example of the hardware configuration of viewing terminal 20. Viewing terminal 20 is configured by a server such as a PC equipped with control unit 21, storage unit 22, display unit 23, input unit 24, and communication unit 25. One viewing terminal 20 is sufficient in display system 100, but this is not limited thereto, and each company or store 200 may have multiple viewing terminals 20.

[0020] Control unit 21 comprehensively controls the entire viewing terminal 20, and has a CPU that reads programs and executes control processes. The control unit controls the operation of the entire viewing terminal 20 by expanding programs stored in the RAM, ROM, HDD, SSD, etc. of storage unit 22 into the RAM of storage unit 22 and sequentially executing them. Control unit 21 controls the overall operation of viewing terminal 20 by executing various programs such as OS programs and application programs.

[0021] The storage unit 22 is a RAM, ROM, HDD, SSD, or the like, and stores the programs executed by the control unit 21, information input at the viewing terminal 20, a display management database (DB) described below, various setting information, and the like.

[0022] Display unit 23 is, for example, a liquid crystal display or an organic EL display, and displays an operation screen or the like to a user who operates viewing terminal 20 under the control of control unit 21.

[0023] The input unit 24 accepts data input based on the operation of the user of the viewing terminal 20. The input unit 24 is configured with a keyboard, a mouse, a touch panel, etc. The input information is notified to the control unit 21.

[0024] The communication unit 25 communicates with other terminals via a network N such as the Internet under the control of the control unit 21. The communication with other terminals is performed via wired communication or wireless communication. For example, under the control of the control unit 21, the communication unit 25 performs wired communication with the product learning information DB 10 and the store server 30 via the network N. The communication unit 25 is, for example, a network adapter.

[0025] 4 is a diagram showing an example of the hardware configuration of the store server 30. The store server 30 is configured by a server such as a PC that includes a control unit 31, a storage unit 32, a display unit 33, an input unit 34, and a communication unit 35. One store server 30 is sufficient in the store 200, but this is not limited thereto; one store server 30 may be provided in the display system 100, or multiple store servers may be provided in each store 200.

[0026] The control unit 31 provides overall control over the entire store server 30, and includes a CPU that reads programs and executes control processes. The control unit controls the operation of the entire store server 30 by expanding programs stored in the RAM, ROM, HDD, SSD, etc. of the storage unit 32 into the RAM of the storage unit 32 and sequentially executing them. The control unit 31 controls the operation of the entire store server 30 by executing various programs such as OS programs and application programs.

[0027] The storage unit 32 is a RAM, ROM, HDD, SSD, or the like, and stores the programs executed by the control unit 31, information input to the store server 30, various setting information, and the like.

[0028] The display unit 33 is, for example, a liquid crystal display or an organic EL display, and displays an operation screen or the like to a user who operates the store server 30 under the control of the control unit 31.

[0029] The input unit 34 accepts data input based on the operation of the user of the store server 30. The input unit 34 is composed of a keyboard, a mouse, a touch panel, etc. The input information is notified to the control unit 31.

[0030] The communication unit 35 communicates with other terminals via a network N such as the Internet under the control of the control unit 31. The communication with other terminals is performed via wired or wireless communication. For example, under the control of the control unit 31, the communication unit 35 performs wired communication with the product learning information DB 10 and the viewing terminal 20 via the network N, and wireless communication with the mobile terminal 40 and the imaging device 60. The communication unit 35 may perform wireless communication with the product learning information DB 10 and the viewing terminal 20, and may also perform wired communication with the mobile terminal 40 and the imaging device 60. The communication unit 35 is, for example, a network adapter.

[0031] 5 is a diagram showing an example of the hardware configuration of the mobile terminal 40. The mobile terminal 40 is configured as a tablet terminal including a control unit 41, a storage unit 42, a display unit 43, an input unit 44, and a communication unit 45. The mobile terminal 40 is not limited to a tablet terminal, and may be a terminal such as a smartphone or a laptop computer that can be carried by a user. At least one mobile terminal 40 is sufficient in the store 200, but this is not limited thereto, and multiple mobile terminals 40 may be provided in the store 200.

[0032] The control unit 41 has overall control over the entire mobile terminal 40, and includes a CPU that reads programs and executes control processes. The control unit controls the overall operation of the mobile terminal 40 by expanding programs stored in the RAM, ROM, HDD, SSD, etc. of the storage unit 42 into the RAM of the storage unit 42 and sequentially executing them. The control unit 41 controls the overall operation of the mobile terminal 40 by executing various programs such as OS programs and application programs.

[0033] The storage unit 42 is a RAM, a ROM, an HDD, an SSD, or the like, and stores the programs executed by the control unit 41, information input to the mobile terminal 40, various setting information, and the like.

[0034] The display unit 43 is, for example, a liquid crystal display or an organic EL display, and displays an operation screen or the like to a user who operates the mobile terminal 40 under the control of the control unit 41. The display unit 43 is an example of a notification unit.

[0035] The input unit 44 is a touch panel integrated with the display unit 43, and receives input instructions from users such as store clerks at the store 200. The input information is notified to the control unit 41. The input unit 44 receives inputs such as sell-by dates and best-before dates based on input instructions from the store clerk.

[0036] The communication unit 45 communicates with other terminals through wireless communication such as Wi-Fi (registered trademark) or Bluetooth (registered trademark) under the control of the control unit 41. For example, under the control of the control unit 41, the communication unit 45 performs wireless communication with the store server 30 and the smart glasses 50 through a network within the store 200. The communication unit 45 is, for example, a network adapter.

[0037] FIG. 6 is a diagram illustrating an example of the hardware configuration of smart glasses 50. The smart glasses 50 include a control unit 51, a storage unit 52, a virtual image display unit 53, an input unit 54, an imaging unit 55, and a communication unit 56. The smart glasses 50 are configured as a head-mounted display (HMD) type augmented reality (AR) wearable terminal. The smart glasses 50 are an example of a goggle camera. The smart glasses 50 are not limited to a head-mounted display (HMD) type augmented reality wearable terminal, but may also be a mixed reality (MR) type wearable terminal or a head-up display (HUD) type wearable terminal. At least one smart glasses 50 is sufficient in the store 200, but this is not a limitation; multiple smart glasses 50 can be provided in the store 200.

[0038] The control unit 51 comprehensively controls the entire smart glasses 50 and has a CPU that reads programs and executes control processing. The control unit controls the overall operation of the smart glasses 50 by expanding programs stored in the RAM, ROM, HDD, SSD, etc. of the storage unit 52 into the RAM of the storage unit 52 and sequentially executing them. The control unit 51 controls the overall operation of the smart glasses 50 by executing various programs such as OS programs and application programs.

[0039] The storage unit 52 is a RAM, a ROM, an HDD, an SSD, or the like, and stores the program executed by the control unit 51, information input to the smart glasses 50, various setting information, and the like.

[0040] The virtual image display unit 53 is a display such as an LCD display or an organic EL display, and displays a virtual image to the user operating the smart glasses 50 under the control of the control unit 51, and also displays a real image that physically exists in the real world superimposed thereon. The virtual image is detailed product information or product management status for checking details of the product 70. The real image is an image of the product 70. Specifically, a virtual image of the detailed product information can be superimposed on the real image of the product 70. The detailed product information can display the sell-by date of the product 70. The best-before date of the product 70 may also be displayed as the detailed product information. The sell-by date and best-before date are examples of deadlines subject to management.

[0041] The input unit 54 accepts data input based on the operation of the user of the mobile terminal 40. The input unit 34 is composed of a gyro sensor (angular velocity sensor), a voice reception microphone, a mouse, a touch panel, buttons, etc. The input information is notified to the control unit 51.

[0042] The imaging unit 55 captures a real image present in the line of sight of the user to obtain a captured image under the control of the control unit 51. The imaging unit 55 notifies the control unit 51 of the obtained captured image.

[0043] The communication unit 56 communicates with other terminals through wireless communication such as Wi-Fi (registered trademark) or Bluetooth (registered trademark) under the control of the control unit 51. For example, under the control of the control unit 51, the communication unit 56 performs wireless communication with the store server 30 and the smart glasses 50 through a network within the store 200. The communication unit 56 is, for example, a network adapter.

[0044] 7 is a diagram showing an example of the hardware configuration of the imaging device 60. The imaging device 60 includes a control unit 61, a storage unit 62, an imaging unit 63, and a communication unit 64. The imaging device 60 is an example of an AI (artificial intelligence) camera. A plurality of imaging devices 60 may be provided in the store 200, but this is not a limitation, and it is sufficient that at least one imaging device is provided.

[0045] The control unit 61 comprehensively controls the entire imaging device 60, and has a CPU that reads programs and executes control processing. The control unit controls the operation of the entire imaging device 60 by expanding programs stored in the RAM, ROM, HDD, SSD, etc. of the storage unit 62 into the RAM of the storage unit 62 and sequentially executing them. The control unit 61 controls the operation of the entire imaging device 60 by executing various programs such as OS programs and application programs.

[0046] The storage unit 62 is a RAM, ROM, HDD, SSD, or the like, and stores the programs executed by the control unit 61, information input via the store server 30, various setting information, and the like.

[0047] The imaging unit 63 captures a real image present in the line of sight of the user to obtain a captured image under the control of the control unit 61. The imaging unit 63 notifies the control unit 61 of the obtained captured image.

[0048] The communication unit 64 communicates with other terminals via wireless communication such as Wi-Fi (registered trademark) or Bluetooth (registered trademark) under the control of the control unit 61. For example, the communication unit 64 communicates wirelessly with the store server 30 via a network within the store 200 under the control of the control unit 61. The communication unit 64 is, for example, a network adapter. The imaging device 60 transmits captured images of products 70 displayed on shelves 80 within the store 200 to the store server 30 via the communication unit 64 at a predetermined timing. The predetermined timing may be transmission in real time or transmission at an arbitrarily set time interval.

[0049] FIG. 8 is a diagram showing an example of product learning information for a product 70 stored in the product learning information DB10. The product learning information is an example of feature information. The product learning information for the product 70 is stored in the storage unit 12 of the product learning information DB10. The product learning information for the product 70 may be stored in the store server 30 or the mobile terminal 40. The product learning information stored in the product learning information DB10 associates product feature information and display position information for each product ID for arbitrarily identifying the type of product 70. The product feature information includes shape information, label / design information, and JAN code of the product 70. The display position information includes information such as the JAN code display position and the sell-by date display position. The display position information may also include information on the best-before date display position.

[0050] The store server 30 refers to the product learning information stored in the product learning information DB10, and based on the captured images acquired from each imaging device 60 displayed in the store 200, associates product display position information of the product 70 displayed on the shelf 80 in the store 200 with the product management status of the product 70 and product detail information, and stores the associated information in the display management DB321.

[0051] 9 and 10 are diagrams showing an example in which product display position information indicating the display positions of products 70 displayed on shelves 80 is acquired by each imaging device 60 displayed in store 200. As shown in FIG. 9, shelves 80 are made up of multiple shelves 80A, 80B, ..., 80L. In FIG. 10, of the multiple shelves 80A, 80B, ..., 80L, shelf 80A is described. When the shelves 80A, 80B, ..., 80L are not particularly distinguished from one another, they will be described as shelves 80. In some cases, shelves 80A, 80B, ..., 80L may be simply referred to as shelf A, shelf B, and shelf L, respectively. Although FIG. 9 shows L shelves 80, this is not limited thereto, and multiple shelves 80 can be recognized individually.

[0052] The store server 30 recognizes the display positions of the products 70 displayed on each shelf 80A, 80B, ..., 80L in the horizontal direction using coordinates formed by an X-axis extending in the left-right direction and a Y-axis extending in the up-down direction, based on the captured images acquired by the imaging device 60. Although not shown, the store server 30 can also recognize the display positions of the products 70 displayed on each shelf 80A, 80B, ..., 80L in the vertical direction using coordinates formed by a Z-axis extending in the vertical direction, based on the captured images acquired by the imaging device 60.

[0053] As shown in Fig. 10, the store server 30 recognizes the display position of a product corresponding to each product management ID based on the captured image acquired by the imaging device 60. The product management ID is an identification symbol for uniquely identifying each product 70 displayed on the shelf 80. In this embodiment, the product management ID is displayed using numbers, but this is not limited to this and any uppercase or lowercase alphabet, rounded letters, Greek letters, symbols, numbers, hiragana, katakana, or a combination including at least one of these can be set.

[0054] For example, in FIG. 10, the store server 30 acquires information of "A:X1,X2,Y1,Y2" as the display position of the product 70 corresponding to the product management ID "001." The acquired display position information is associated with the product management ID and stored in the memory unit 32. By acquiring the information of "A:X1,X2,Y1,Y2" as the display position, it is possible to recognize that the product corresponding to the product management ID "001" is displayed at the display position of the X1,X1,Y1,Y2 coordinates on the shelf 80A. The store server 30 stores the recognized display position of the product 70 as product display position information in the display management DB 321 of the memory unit 32.

[0055] 11 is a diagram showing an example of the display management DB 321 stored in the storage unit 32 of the store server 30. The display management DB 321 stores product management status, product detail information, and product display position information in association with each product management ID.

[0056] The product management status is information indicating the management status of the product 70 displayed on the shelf 80. In the embodiment shown in FIG. 11 , one of the flags for product registration, unregistered product, display position management in progress, and management completed is set to "1" as the product management status of the product 70 corresponding to the product management ID. If none of the flags is set to "1," the corresponding error is processed. For example, if the product 70 displayed on the shelf 80 has been registered, the "product registration" flag is set to "1." If the product 70 displayed on the shelf 80 has not been registered, the "product unregistered" flag is set to "1." If the display position of the product 70 displayed on the shelf 80 is being managed, the "display position management in progress" flag is set to "1." If the management of the product 70 displayed on the shelf 80 has been completed, the "management completed" flag is set to "1." By checking the product management status information stored in the display management DB 321, the store server 30 can instantly recognize the state in which the status of the product 70 corresponding to the product management ID is being managed.

[0057] The display management DB 321 may store information on the status change status associated with each product management ID. If the status of the product 70 corresponding to the product management ID has changed, the status change status is set to "1" (yes), and if the status of the product 70 corresponding to the product management ID has not changed, the status change status is set to "0" (no) or blank.

[0058] The product detail information is information that indicates detailed information about the product 70 displayed on the shelf 80. In the embodiment shown in FIG. 11, the product detail information for the product 70 corresponding to the product management ID includes the product name, product ID, JAN code, display start date information, and sell-by date information. The product name stores the name of the product 70, the product ID stores an identification symbol for arbitrarily identifying the type of product 70, the JAN code stores a product identification code corresponding to the product 70, the display start date information stores information about the date when the product 70 started to be displayed on the shelf 80, and the sell-by date information stores information about the sell-by date of the product 70. The store server 30 can immediately recognize the state of the detailed information for the product 70 corresponding to the product management ID by checking the product detail information stored in the display management DB 321.

[0059] The product display position information is information that indicates the display position of the product 70 displayed on the shelf 80. In the embodiment shown in FIG. 11 , the product display position information of the product 70 corresponding to the product management ID includes shelf position information, stacking information, horizontal position information, and vertical position information. The shelf position information stores information that identifies which shelf 80 out of shelves 80A, 80B, ..., 80L the product 70 is displayed on. The stacking information stores the number of shelves 80 on which the product 70 is displayed. The horizontal position information stores information on the coordinates of the shelf 80 on which the product 70 is displayed in the X-axis direction. The vertical position information stores information on the coordinates of the shelf 80 on which the product 70 is displayed in the Y-axis direction. The store server 30 can immediately recognize the display position of the product 70 corresponding to the product management ID by checking the product display position information stored in the display management DB 321.

[0060] FIG. 12 is a diagram showing an example of changing the product management status and product display position information stored in the display management DB 321 based on the display position of the product 70 displayed on the shelf 80A.

[0061] As shown in Figure 12 (2), when a new product 70 is displayed on shelf 80A by a user, the store server 30 recognizes that the product 70 has entered the display monitoring field of view V of the imaging device 60 based on the captured image captured by the imaging device 60, and sets the status of "product registration" in the product management status of the display management DB 321 to "1".

[0062] The store server 30 performs image processing based on the captured image captured by the imaging device 60 to extract feature information corresponding to the shape information, label, and design information of the product 70. The store server 30 then references the product learning information DB 10, identifies the JAN code and product name based on the product ID of the product feature information that matches the extracted feature information, and records these in the display management DB 321. The store server 30 also records information on the date on which the product 70 corresponding to the product ID began to be displayed as display start date information and information on the sell-by date of the product 70 corresponding to the product ID as sell-by date information. The date recorded in the sell-by date information can be recorded by adding a predetermined number of days to the date in the display start date information. For example, if the date recorded in the display start date information is "4 / 1 / 2024," the store server 30 can record "5 / 31 / 2024," two months after the display start date information, as the sell-by date information. The sell-by date information can be set arbitrarily for each type of product 70. For example, if the product 70 is a fresh food, the store server 30 can record the date one week after the display start date information as the sell-by date information, and if the product 70 is a processed food, the store server 30 can record the date six months after the display start date information as the sell-by date information. This makes it easy to manage the sell-by dates for each type of product 70. The display start date and sell-by date are examples of deadlines to be managed.

[0063] Furthermore, as shown in Figure 12 (3), when a user rearranges the displayed products 70, the store server 30 stores the display positions after rearrangement based on the captured images captured by the imaging device 60 as product display position information in the display management DB 321 of the memory unit 32.

[0064] Also, as shown in Figure 12 (4), when a user removes a displayed product 70, the store server 30 sets the "management ended" flag to "1" as the product management status based on the captured image taken by the imaging device 60, and stores it in the display management DB 321 of the memory unit 32.

[0065] Also, as shown in Figure 12 (6), when a product 70 is moved within the display monitoring field of view V of the imaging device 60, the store server 30 stores the display position after the movement as product display position information in the display management DB 321 of the memory unit 32 based on the captured image captured by the imaging device 60.

[0066] Also, as shown in Figure 12 (7), when a product 70 appears within the display monitoring field of view V of the imaging device 60 from outside the field of view V, the store server 30 sets the "product registration" flag to "1" as the status of the product 70 that has appeared based on the captured image captured by the imaging device 60, and stores it in the display management DB 321 of the memory unit 32.

[0067] Also, as shown in Figure 12 (8), when a product 70 moves from within the display monitoring field of view V of the imaging device 60 to outside the display monitoring field of view V, the store server 30 sets the "management ended" flag to "1" as the product management status based on the captured image captured by the imaging device 60, and stores it in the display management DB 321 of the memory unit 32.

[0068] Fig. 13 is a diagram showing an example of an image visually recognized by the smart glasses 50. The smart glasses 50 display a virtual image of the product management status of the product 70 associated with the display position of the product 70, superimposed on the real image of the product 70, in accordance with the display position of the product 70 transmitted from the store server 30. In the embodiment shown in Fig. 13, the smart glasses 50 display icons 91 and 92 as virtual images of the product management status, superimposed on the real image of the product 70. The icons 91 and 92 are examples of virtual images of the product management status.

[0069] The icon 91 is an example of a virtual image indicating a product to be rearranged. In the embodiment of FIG. 13 , the virtual image icon 91 is displayed as a black circle icon. A user of the smart glasses 50 can easily understand that the product 70 must be moved due to product display rearrangement by checking the product 70 with the virtual image icon 91 attached. The virtual image icon 91 is attached to the product 70 corresponding to the product management ID whose product management status stored in the display management DB 321 is "display position under management". As a result, a user of the smart glasses 50 can easily understand the product 70 to be rearranged by visually checking the product 70 with the icon 91 attached, thereby efficiently displaying the products 70.

[0070] The icon 92 is an example of a virtual image indicating a product that is not subject to management. In the embodiment of FIG. 13 , the virtual image icon 92 is displayed as a black triangle icon. A user of the smart glasses 50 can easily understand that the product 70 that has the virtual image icon 92 attached to it must be registered in the product display by checking the product 70 that has the virtual image icon 92 attached to it. The virtual image icon 92 is attached to a product 70 that has a product management ID that has a product management status of "product not registered" stored in the display management DB 321 and is displayed. This allows a user of the smart glasses 50 to easily understand the product 70 that is not subject to management by visually checking the product 70 that has the icon 92 attached to it, thereby enabling the product 70 to be displayed efficiently.

[0071] The present invention is not limited to the above-described embodiments, and the components can be modified and embodied without departing from the spirit of the invention. Furthermore, various inventions can be formed by appropriately combining multiple components disclosed in the above-described embodiments. For example, all components shown in the embodiments can be appropriately combined. Furthermore, components from different embodiments can be appropriately combined. Naturally, various modifications and applications are possible without departing from the spirit of the invention. [Explanation of symbols]

[0072] 10: Product learning information DB 20: Viewing terminal 30: Store server 40: Mobile devices 50: Smart glasses 60: Imaging device 70: Product 80: Shelf 100: Display system 200: Store

Claims

1. A display system including a store server and smart glasses, The store server storing the display positions of products displayed on shelves and the statuses of the products in association with each other based on captured images acquired from an imaging device that captures images of the inside of the store; The smart glasses display a virtual image of the status of the product associated with the display position of the product by superimposing the virtual image of the status on a real image of the product, in accordance with the display position of the product transmitted from the store server. A display system characterized by:

2. The status includes a management deadline for the product. The display system according to claim 1 .

3. The management deadline includes at least one of a sell-by date and a best-before date of the product. The display system according to claim 2 .

4. The smart glasses display the virtual image of the status of the product by superimposing it on the real image of the product, in accordance with the display position of the product transmitted from the store server via the mobile terminal. The display system according to claim 3 .

5. The smart glasses identify a reordering target product and a non-management target product associated with the management target deadline as the virtual image of the status of the product, and display the virtual image of the status of the product by superimposing it on the real image of the product.

5. The display system according to claim 4.

6. The smart glasses are a head-mounted display (HMD) type augmented reality (AR) wearable device. The display system according to claim 5 .

7. The store server storing a display management database in which the display positions of the products displayed on the shelves acquired from the imaging device are associated with the management deadlines; The display position of the product is tracked by artificial intelligence based on the captured image captured by the imaging device, and display position information of the product is updated. The display system according to claim 6 .

8. Further, a product learning information database is provided to store product learning information of the product. The store server associates and stores the product learning information of the product stored in the product learning information database with the display position of the product displayed on the shelf and the status of the product based on the captured image acquired from the imaging device. The display system according to claim 7 .

9. The product learning information of the product is acquired in advance by machine learning from the product learning information database. The display system according to claim 8 .

10. A display method executed by a display system including a store server and smart glasses, storing in the store server the display positions of the products displayed on the shelves and the statuses of the products in association with each other based on captured images acquired from imaging devices displayed in the store; The smart glasses display a virtual image of the status of the product associated with the display position of the product, superimposed on a real image of the product, in accordance with the display position of the product transmitted from the store server. A display method characterized by:

Images