Information processing device, information processing method, program, and management system

Abstract

Provided is an information processing device (10) including an acquisition unit (11) for acquiring an image in which a display space for merchandise is image-captured, an estimation unit (12) for estimating a stocking rate of the merchandise in the image in a display space, on the basis of the image, and a control unit (13) that performs notification of information relating to the merchandise in a case in which the stocking rate is no greater than a threshold value in accordance with whether the merchandise was stocked to the display space or rearranged to the front in work performed the previous time.

Description

Information processing device, information processing method, computer-readable medium, and management system

[0001] The present disclosure relates to an information processing device, an information processing method, a non-transitory computer-readable medium storing a program, and a management system.

[0002] Patent Document 1 discloses a technology that makes it possible to determine whether or not a product is displayed on a product shelf, thereby making it easier to manage the display status of the products.

[0003] Japanese Patent Application Laid-Open No. 2016-042307

[0004] However, the technology described in Patent Document 1 has a problem in that, for example, it may not be possible to appropriately encourage the display of products.

[0005] In view of the above-mentioned problems, an object of the present disclosure is to provide a technology that can appropriately encourage product display.

[0006] In a first aspect of the present disclosure, an information processing device is provided that has an acquisition unit that acquires an image of a product display space, an estimation unit that estimates the degree of filling of the product in the display space captured in the image based on the image, and a control unit that notifies information about the product if the degree of filling is below a threshold value depending on whether the product was put out or brought forward in the previous operation.

[0007] In addition, in a second aspect of the present disclosure, an information processing method is provided that acquires an image of a product display space, estimates the degree of filling of the display space for the product shown in the image based on the image, and notifies information about the product if the degree of filling is below a threshold value depending on whether the product was put out or brought forward in the previous operation.

[0008] In addition, in a third aspect of the present disclosure, a non-transitory computer-readable medium is provided that stores a program that causes a computer to execute a process that acquires an image of a product display space, estimates the degree of filling of the display space for the product shown in the image based on the image, and notifies information about the product if the degree of filling is below a threshold value that depends on whether the product was put out or brought forward in the previous operation.

[0009] In addition, in a fourth aspect of the present disclosure, there is provided a management system having a photographing device, an information processing device, and a terminal, wherein the information processing device has an acquisition unit that acquires an image of a product display space photographed by the photographing device, an estimation unit that estimates the degree of filling of the display space for the product captured in the image based on the image, and a control unit that notifies the terminal of information related to the product if the degree of filling is below a threshold value depending on whether the product was put out or brought forward in the previous operation.

[0010] According to one aspect, it is possible to appropriately encourage the display of merchandise.

[0011] Fig. 1 is a diagram showing an example of the configuration of an information processing device according to an embodiment; Fig. 2 is a diagram showing an example of the configuration of a management system according to an embodiment; Fig. 3 is a diagram showing an example of the hardware configuration of an information processing device according to an embodiment; Fig. 4 is a flowchart showing an example of processing of an information processing device according to an embodiment; Fig. 5 is a diagram showing an example of a display work DB (database) according to an embodiment; Fig. 6 is a diagram showing an example of a display screen according to an embodiment.

[0012] The principles of the present disclosure will be described with reference to some exemplary embodiments. It should be understood that these embodiments are set forth for illustrative purposes only, to aid those skilled in the art in understanding and practicing the present disclosure, without implying any limitation on the scope of the disclosure. The disclosure described herein may be implemented in various ways other than those described below.

[0013] In the following description and claims, unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs.

[0014] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. (Embodiment 1) <Configuration> The configuration of an information processing device 10 according to an embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of the configuration of the information processing device 10 according to an embodiment. The information processing device 10 has an acquisition unit 11, an estimation unit 12, and a control unit 13. Each of these units may be realized by cooperation between one or more programs installed in the information processing device 10 and hardware such as a processor 101 and a memory 102 of the information processing device 10.

[0015] The acquisition unit 11 acquires an image of a product display space. The estimation unit 12 estimates the degree of filling of the product in the display space based on the image acquired by the acquisition unit 11. If the degree of filling estimated by the estimation unit 12 is equal to or less than a threshold value depending on whether the product was put out or brought forward in the previous operation, the control unit 13 notifies information about the product.

[0016] (Embodiment 2) Next, the configuration of a management system 1 according to an embodiment will be described with reference to Fig. 2. <System Configuration> Fig. 2 is a diagram showing an example of the configuration of a management system 1 according to an embodiment. In the example of Fig. 2, the management system 1 has an information processing device 10 and an image capturing device 20. The management system 1 also has a terminal 30A and a terminal 30B (hereinafter, when there is no need to distinguish between them, they will also be simply referred to as "terminals 30"). Note that the numbers of information processing devices 10, image capturing devices 20, and terminals 30 are not limited to the example of Fig. 2.

[0017] 2 , the information processing device 10, the photographing device 20, and the terminal 30 are connected to each other so as to be able to communicate with each other via a network N. Examples of the network N include the Internet, a mobile communication system, a wireless local area network (LAN), a short-range wireless communication such as BLE, a LAN, and a bus. Examples of the mobile communication system include a fifth-generation mobile communication system (5G), a fourth-generation mobile communication system (4G), a third-generation mobile communication system (3G), and the like.

[0018] The information processing device 10 may be, for example, a server, a cloud server, a personal computer, a smartphone, etc. The information processing device 10 transmits a notification regarding the display of products to the terminal 30 based on an image captured by the imaging device 20.

[0019] The image capturing device 20 is, for example, a camera installed on the ceiling of a store, etc. The image capturing device 20 captures images of the display space of each product, for example.

[0020] The terminal 30 is, for example, a smartphone, a tablet, a laptop computer, or the like. The terminal 30 may be carried by a store clerk or the like, or may be installed in a space of the store where customers cannot enter (a backyard, a back room, or the like). The terminal 30 notifies the store clerk or the like of a notification (information) received from the information processing device 10.

[0021] <Hardware Configuration> Fig. 3 is a diagram showing an example of the hardware configuration of the information processing device 10 according to the embodiment. In the example of Fig. 3, the information processing device 10 (computer 100) includes a processor 101, a memory 102, and a communication interface 103. These components may be connected via a bus or the like. The memory 102 stores at least a part of a program 104. The communication interface 103 includes an interface required for communication with other network elements.

[0022] When the program 104 is executed by the processor 101, memory 102, and other components in cooperation with each other, the computer 100 performs at least some of the processing of the embodiments of the present disclosure. The memory 102 may be of any type suitable for a local technology network. The memory 102 may be, by way of non-limiting example, a non-transitory computer-readable storage medium. The memory 102 may also be implemented using any suitable data storage technology, such as semiconductor-based memory devices, magnetic memory devices and systems, optical memory devices and systems, fixed memory, and removable memory. While only one memory 102 is shown in the computer 100, several physically distinct memory modules may be present in the computer 100. The processor 101 may be of any type. The processor 101 may include one or more of a general-purpose computer, a special-purpose computer, a microprocessor, a digital signal processor (DSP), and, by way of non-limiting example, a processor based on a multi-core processor architecture. The computer 100 may have multiple processors, such as application-specific integrated circuit chips time-slaved to a clock that synchronizes the main processor.

[0023] Embodiments of the present disclosure may be implemented in hardware or special purpose circuits, software, logic, or any combination thereof. Some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device.

[0024] The present disclosure also provides at least one computer program product tangibly stored on a non-transitory computer-readable storage medium. The computer program product includes computer-executable instructions, such as instructions included in program modules, that execute on a target real or virtual processor or device to perform the processes or methods of the present disclosure. Program modules include routines, programs, libraries, objects, classes, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The functionality of the program modules may be combined or divided among program modules as desired in various embodiments. The machine-executable instructions of the program modules may be executed in local or distributed devices. In a distributed device, the program modules may be located in both local and remote storage media.

[0025] The program code for executing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus. When the program code is executed by the processor or controller, the functions / acts in the flowcharts and / or implementing block diagrams are performed. The program code may be executed entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine, or entirely on a remote machine or server.

[0026] The program can be stored and supplied to a computer using various types of non-transitory computer-readable media. Non-transitory computer-readable media include various types of tangible recording media. Examples of non-transitory computer-readable media include magnetic recording media, magneto-optical recording media, optical disk media, and semiconductor memory. Magnetic recording media include, for example, flexible disks, magnetic tapes, and hard disk drives. Magneto-optical recording media include, for example, magneto-optical disks. Optical disk media include, for example, Blu-ray discs, CD (Compact Disc)-ROM (Read Only Memory), CD-R (Recordable), and CD-RW (Rewritable). Semiconductor memory includes, for example, solid-state drives, mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, and RAM (Random Access Memory). The program may also be supplied to a computer by various types of temporary computer-readable media. Examples of temporary computer-readable media include electrical signals, optical signals, and electromagnetic waves. The temporary computer-readable medium can supply the program to the computer via a wired communication path such as an electric wire or an optical fiber, or via a wireless communication path.

[0027] <Processing> Next, an example of processing of the information processing device 10 according to the embodiment will be described with reference to Fig. 4 to Fig. 6. Fig. 4 is a flowchart showing an example of processing of the information processing device 10 according to the embodiment. Fig. 5 is a diagram showing an example of a display work DB (database) 501 according to the embodiment. Fig. 6 is a diagram showing an example of a display screen according to the embodiment. Note that the processing of Fig. 4 may be executed periodically (e.g., every minute), for example.

[0028] In step S101 , the acquisition unit 11 acquires, from the photographing device 20 , an image of the display space of each product.

[0029] Next, the estimation unit 12 estimates the degree of filling of each product relative to the display space based on the images acquired by the acquisition unit 11 (step S102). This reduces the effort required for a store clerk or the like to input information such as the quantity of each product restocked from a backroom or the like to the sales floor in the store into the information processing device 10.

[0030] Here, the product filling level may be, for example, the percentage of the product present in the display space of the product photographed by the camera device 20. For example, when a product shelf (a shelf on which products are displayed) having multiple tiers is photographed from the camera device 20 installed on the ceiling or the like of a store, part of the display space for a certain product may be in a blind spot due to other products displayed on a tier above the tier on which the product is displayed. In this case, if the products are displayed appropriately (e.g., aligned without gaps) in a part of the display space that is visible to the camera device 20 (not in a blind spot), the product filling level is 1 (100%) regardless of whether the product is present in a part that is in a blind spot (not visible) from the camera device 20.

[0031] The estimation unit 12 may use, for example, AI (Artificial Intelligence) to recognize the size of the display space for each product, the size of each product, and the number of each product, and calculate each filling level. In this case, for example, if the display space for a certain product visible to the imaging device 20 can display 12 units of the product, and six units of the product are displayed in the display space, the estimation unit 12 may calculate the filling level of the product to be 0.5 (= 6 / 12).

[0032] Furthermore, the estimation unit 12 may calculate, as the value of the fill level of the product, the ratio between the number of pixels in which the product is captured in an area on the image captured by the camera device 20 and the number of pixels in the area. In this case, if the number of pixels in the area is 100 and the number of pixels in which the product is captured in the area is 50, the fill level of the product is calculated to be 0.5 (= 50 / 100). Note that information indicating the area on the image captured by the camera device 20 that corresponds to the display space of the product may be set (registered) in the information processing device 10 by an administrator or the like, for example, when the camera device 20 is installed in a store or the like.

[0033] Next, the control unit 13 refers to the display work DB 501 and notifies information about the specific product based on the filling level of the specific product estimated by the estimation unit 12 and whether the specific product was put out or brought forward in the previous (most recent; closest to the present) work (step S103), and then ends the processing.

[0034] In the example of FIG. 5 , the display work DB 501 stores, in association with a product ID, the previous work type, the date and time of the previous work, the image before the previous work, the work type before the previous work, the date and time of the work before the previous work, and the display space group ID. The product ID is identification information for each product. The product ID may be, for example, a product code such as the product name, model number, or JAN (Japanese Article Number) code. The work type is information indicating whether the product associated with the product ID was put out or brought forward in the work. The work type may include "putting out" and "advancing."

[0035] The work date and time is the date and time when the store clerk or other person performed the work of stocking or moving items forward. The image before the work is an image captured by the image capture device 20 immediately before the store clerk or other person performed the work of stocking or moving items forward. The display space group ID is, for example, information indicating the location of the product display space in the store. The display space group may be formed, for example, by one or more adjacent product shelves or the like.

[0036] Stocking (replenishing) is the task of, for example, displaying products stored in the back of a store in the display space of the store's sales floor (front of the store). When stocking, the store clerk may, for example, first bring out the products to the front, and then display the products brought in from the back of the store at the back of the display space. This allows, for example, the oldest products to be displayed at the front.

[0037] Furthermore, "front-loading" refers to the process of moving products displayed at the back of the display space to the front after a customer has removed a product displayed at the front of the display space. By moving products forward, customers can easily see and pick up the products.

[0038] The product ID may be set in advance by an administrator or the like of the information processing device 10. The previous work type and the work type before that may be automatically determined and recorded by the information processing device 10, or may be recorded in response to an operation on the terminal 30 by a worker or the like.

[0039] (Regarding Determination of Work Type) The control unit 13 may estimate whether each product was put out or brought forward in each work by the store clerk or the like, based on the image acquired by the acquisition unit 11. This can reduce the effort required for the store clerk or the like to input the work type into the information processing device 10, for example.

[0040] In this case, the control unit 13 may determine that a specific product has been put out or moved forward when, for example, it detects that the specific product has been moved to the front of the display space based on an image captured at a certain point in time.The control unit 13 may then estimate whether the specific product has been put out or moved forward, for example, based on the length of time that a store clerk or other person continuously stayed in front of the display space of the specific product as seen from the image capture device 20 before that point in time.In this case, the control unit 13 may, for example, estimate that the specific product has been moved forward if the length of time is less than a threshold, and that the specific product has been put out if the length of time is equal to or greater than the threshold.This is because putting out a product takes longer than moving forward.

[0041] In addition, when the control unit 13 detects, for example, based on an image taken at a certain point in time, that a specific product has been moved to the front of the display space, it may assume that the specific product has been moved forward if the filling level of the specific product is not 1 (100%).

[0042] (Regarding Notification) For example, if a specific product was forwarded in the previous operation and the filling level of the specific product is equal to or lower than a first threshold, the control unit 13 may notify information urging the specific product to be put out. Furthermore, for example, if a specific product was put out in the previous operation and the filling level of the specific product is equal to or lower than a second threshold that is lower than the first threshold, the control unit 13 may notify information urging the specific product to be put out. This allows the store clerk or the like to be prompted to put out the product more quickly, for example, if it is considered that there are relatively few products in a blind spot from the camera device 20 due to the previous forwarding. This is because if a product was put out in the previous operation, it is considered that there are relatively many products in a blind spot from the camera device 20. Furthermore, if a product was forwarded in the previous operation, it is considered that there are relatively few products in a blind spot from the camera device 20.

[0043] Furthermore, the control unit 13 may notify information urging the specific product to be brought forward if, for example, a specific product was put out in the previous operation and the filling level of the specific product is equal to or less than a third threshold higher than the second threshold. As a result, for example, if it is considered that there are relatively many products in a blind spot as seen from the camera device 20 due to the previous stocking operation, the control unit 13 can urge the store clerk or the like to bring the products forward or put them out depending on the number of products in the blind spot as seen from the camera device 20.

[0044] Furthermore, the control unit 13 may notify information urging the store clerk to put out the specific product if, for example, the specific product was brought forward in the previous work and the work before last, and the filling level of the specific product is equal to or lower than a fourth threshold value that is higher than the first threshold value. This allows the store clerk to be urged to put out the product more quickly, for example, if it is considered that there are relatively few products in a blind spot as seen from the image capture device 20 due to the previous work and the work before last.

[0045] (Example of Notification for Each Display Space Group) The control unit 13 may send a notification prompting stocking, etc., for each specific display space group. This allows, for example, a store clerk or the like to work on multiple products in a specific sales corner at the same time, thereby improving work efficiency.

[0046] In this case, the acquisition unit 11 may acquire images of the display space for each product. The estimation unit 12 may then estimate the fill level of each product relative to the display space based on the images. The control unit 13 may then notify information about a product when the total fill level is equal to or less than a specific threshold and the fill level of a product is equal to or less than a threshold corresponding to whether the product was put out or put forward in the previous operation. This allows, for example, when the fill level of the entire sales area is equal to or less than a threshold corresponding to the sales area, to provide a notification about each product in the sales area according to the fill level of each product.

[0047] In this case, the acquisition unit 11 may acquire images of the display space for the first product and the display space for the second product. Then, the estimation unit 12 may estimate a first filling level for the display space for the first product and a second filling level for the display space for the second product based on the images. Then, the control unit 13 may notify information about the first product when the sum of the first filling level and the second filling level is equal to or less than a specific threshold and the first filling level is equal to or less than a threshold depending on whether the first product was put out or put forward in the previous operation.

[0048] ((An example of prompting the stocking of other products while prompting the stocking of a specific product)) The control unit 13 may issue a notification prompting the stocking of other products while prompting the stocking of a specific product. This allows, for example, a store clerk to work on multiple products in a specific sales corner at the same time, thereby improving work efficiency.

[0049] In this case, the acquisition unit 11 may acquire the images of the display space for the first product and the display space for the second product, and the estimation unit 12 may estimate a first filling degree for the display space for the first product and a second filling degree for the display space for the second product based on the images.

[0050] The control unit 13 may then notify information about the first product when the first filling level is equal to or less than a threshold value corresponding to whether the first product was put out or brought forward in the previous operation. The control unit 13 may also notify information about the second product when the second filling level is equal to or less than a threshold value corresponding to whether the second product was put out or brought forward in the previous operation and notify information about the first product. This allows, for example, when prompting a store clerk to put out the first product, to also prompt a store clerk to put out the second product, even if the filling level of the second product is relatively high.

[0051] (Example of notification display) The control unit 13 may display information indicating whether a specific product was put out or brought forward in the previous operation, in association with an area in which the specific product is shown in the image acquired by the acquisition unit 11. This allows, for example, a store clerk or the like to easily estimate whether a product is displayed at the back of the display space even when the back of the display space is difficult to see because it is a blind spot due to other products or the like.

[0052] In the example of Figure 6, the control unit 13 of the information processing device 10 displays, on the display screen 601 of the terminal 30, information (display objects) indicating whether each product was put out or brought forward in the previous operation, information prompting the user to put out each product, and information indicating the degree of filling of each product, superimposed on the image. In the example of Figure 6, the information indicating whether a product was put out or brought forward in the previous operation is displayed by the thickness of the line of the frame surrounding the area in which the product is shown. If the line is not thick (it is a thin line), it indicates that the product was put out in the previous operation. On the other hand, if the line is thick, it indicates that the product was brought forward in the previous operation.

[0053] In the example of Fig. 6, information prompting the stocking of a certain product is displayed by a frame surrounding the area in which the product is shown. When prompting the stocking of the product, the control unit 13 displays the frame with a solid line, and when prompting the forward movement of the product, the control unit 13 displays the frame with a dashed line. In the example of Fig. 6, the filling level of a certain product is displayed as a percentage within the area in which the product is shown.

[0054] 6, information 612 indicating that the first product was put out in the previous operation and urging the user to put out the first product, and information 613 indicating the filling level of the first product, are displayed in association with the area in which the first product is shown. Information 612 is displayed in a thin line to indicate that the first product was put out in the previous operation, and in a solid line to urge the user to put out the first product.

[0055] Additionally, information 622 indicating that the second product was put out in the previous operation and urging the second product to be brought forward, and information 623 indicating the degree of filling of the second product, are displayed in association with the area in which the second product is shown. Information 622 is displayed in thin lines to indicate that the second product was put out in the previous operation, and in dashed lines to urge the product to be brought forward.

[0056] Additionally, information 632 indicating that the third product was brought forward in the previous operation and urging the user to put the third product out, and information 633 indicating the degree of filling of the third product, are displayed in association with the area in which the third product is shown. Information 632 is displayed in a thick line to indicate that the third product was brought forward in the previous operation, and in a solid line to urge the user to put the product out.

[0057] Additionally, information 642 indicating that the fourth product was brought forward in the previous operation and urging the fourth product to be brought forward, and information 643 indicating the filling level of the fourth product, are displayed in association with the area in which the fourth product is shown. Information 642 is displayed in a thick line to indicate that the fourth product was brought forward in the previous operation, and in a dashed line to urge the fourth product to be brought forward.

[0058] Furthermore, the control unit 13 may display an image of the display space of the specific product taken at a time before the previous operation when a specific operation is performed on an area in which a specific product is captured in an image acquired by the acquisition unit 11. In this case, for example, when a store clerk or the like double-tap (tap twice with a fingertip or the like) information 612 in FIG. 6 on the terminal 30, the control unit 13 may display an image of the display space of the specific product taken at a time before the previous operation. The image of the display space of the specific product taken at a time before the previous operation may be, for example, an image recorded in the display operation DB 501 and taken before the previous operation.

[0059] ((Example of notifying a notification destination according to the previous work type)) The control unit 13 may notify a notification destination according to the previous work type. In this case, if the filling level of a specific product is below a threshold value according to whether the specific product was put out or brought forward in the previous work, the control unit 13 may notify information about the specific product to a notification destination according to whether the specific product was put out or brought forward in the previous work. As a result, for example, if the specific product was brought forward in the previous work and the filling level of the specific product is relatively low, not only a store clerk, etc. but also a store manager, etc. may be notified. This, for example, can further reduce the occurrence of a specific product being out of stock (out of stock) in the store.

[0060] <Modifications> The information processing device 10 may be a device contained in a single housing, but the information processing device 10 of the present disclosure is not limited to this. Each unit of the information processing device 10 may be realized, for example, by cloud computing configured with one or more computers. Furthermore, at least a portion of the processing of the information processing device 10 may be realized, for example, by the terminal 30. Furthermore, the information processing device 10 may be, for example, a device integrated with at least one of the terminal 30 and the imaging device 20 (for example, housed in the same housing). Such information processing devices 10 are also included as examples of the "information processing device" of the present disclosure.

[0061] The present disclosure is not limited to the above-described embodiment, and can be modified as appropriate within the scope of the present disclosure.

[0062] Some or all of the above embodiments can be described as, but are not limited to, the following supplementary notes. (Supplementary Note 1) An information processing device comprising: an acquisition unit that acquires an image of a product display space; an estimation unit that estimates the degree of filling of the product in the display space captured in the image based on the image; and a control unit that notifies information related to the product if the degree of filling is equal to or less than a threshold value corresponding to whether the product was put out or brought forward in a previous operation. (Supplementary Note 2) The information processing device according to Supplementary Note 1, wherein the control unit notifies information urging the product to be put out if the product was brought forward in a previous operation and the degree of filling is equal to or less than a first threshold, and notifies information urging the product to be put out if the product was put out in a previous operation and the degree of filling is equal to or less than a second threshold that is lower than the first threshold. (Supplementary Note 3) The information processing device according to Supplementary Note 2, wherein the control unit notifies information urging the product to be brought forward if the product was put out in a previous operation and the degree of filling is equal to or less than a third threshold that is higher than the second threshold. (Supplementary Note 4) The information processing device of Supplementary Note 1, wherein the control unit notifies information urging the product to be put out when the product was brought forward in the operation before last and the previous operation, and the filling level is equal to or lower than a fourth threshold higher than the first threshold. (Supplementary Note 5) The information processing device of Supplementary Note 1, wherein the control unit estimates whether the product was put out or brought forward in the previous operation, based on the image. (Supplementary Note 6) The information processing device of Supplementary Note 1, wherein the acquisition unit acquires the images of a display space for a first product and a display space for a second product, the estimation unit estimates a first filling level for the display space for the first product and a second filling level for the display space for the second product, based on the images, and the control unit notifies information related to the first product when a sum of the first filling level and the second filling level is equal to or lower than a specific threshold and the first filling level is equal to or lower than a threshold depending on whether the first product was put out or brought forward in the previous operation.(Supplementary Note 7) The information processing device according to Supplementary Note 1, wherein the acquisition unit acquires the images captured of a display space for a first product and a display space for a second product, the estimation unit estimates a first filling degree for the display space for the first product and a second filling degree for the display space for the second product based on the images, and the control unit notifies information about the first product and the second product when the first filling degree is equal to or less than a threshold value corresponding to whether the first product was put out or brought forward in a previous operation and when the second filling degree is equal to or less than a threshold value corresponding to notifying information about whether the second product was put out or brought forward in a previous operation and about the first product. (Supplementary Note 8) The information processing device according to Supplementary Note 1, wherein the control unit displays information indicating whether the product was put out or brought forward in a previous operation in association with an area in the image in which the product is depicted. (Supplementary Note 9) The information processing device according to Supplementary Note 8, wherein the control unit displays an image of the display space for the products taken at a time prior to a previous operation when a specific operation is performed on an area in the image in which the product is shown. (Supplementary Note 10) The information processing device according to Supplementary Note 1, wherein the control unit notifies a notification destination according to whether the product was put out or brought forward in the previous operation of information related to the product when the filling level is equal to or lower than a threshold value corresponding to whether the product was put out or brought forward in the previous operation. (Supplementary Note 11) An information processing method comprising: acquiring an image of a display space for products; estimating a filling level of the display space for the products shown in the image based on the image; and notifying information related to the product when the filling level is equal to or lower than a threshold value corresponding to whether the product was put out or brought forward in the previous operation. (Supplementary Note 12) A non-transitory computer-readable medium storing a program that causes a computer to execute the following process: acquiring an image of a product display space; estimating, based on the image, the degree of filling of the display space for the product captured in the image; and notifying information about the product if the degree of filling is equal to or less than a threshold value that depends on whether the product was put out or brought forward in the previous operation.(Supplementary Note 13) A management system having a photographing device, an information processing device, and a terminal, wherein the information processing device has: an acquisition unit that acquires an image of a product display space photographed by the photographing device; an estimation unit that estimates the degree of filling of the product in the display space photographed in the image based on the image; and a control unit that notifies the terminal of information related to the product when the degree of filling is equal to or less than a threshold value depending on whether the product was put out or brought forward in the previous operation.

[0063] REFERENCE SIGNS LIST 1 Management system 10 Information processing device 11 Acquisition unit 12 Estimation unit 13 Control unit 20 Imaging device 30 Terminal

Claims

1. an acquisition unit that acquires an image of a product display space; an estimation unit that estimates a filling level of a display space for the products shown in the image based on the image; a control unit that notifies information about the product when the filling level is equal to or less than a threshold value that corresponds to whether the product was put out or put forward in the previous work; An information processing device having the above.

2. The control unit If the product was brought forward in the previous operation, and the filling level is equal to or less than a first threshold, notify the user of information urging the product to be brought forward; If the product was put out in the previous work, and the filling level is equal to or less than a second threshold value that is lower than the first threshold value, notify the customer of information urging the customer to put out the specific product. The information processing device according to claim 1 .

3. The control unit If the product was put out in the previous operation, and the filling level is equal to or less than a third threshold value that is higher than the second threshold value, information urging the product to be put out is notified. The information processing device according to claim 2 .

4. The control unit If the product was brought forward in the previous work and the work before that, and if the filling level is equal to or less than a fourth threshold value that is higher than the first threshold value, information urging the product to be brought forward is notified. The information processing device according to claim 2 .

5. The control unit Based on the image, it is estimated whether the product was put out or brought forward in the previous work. The information processing device according to claim 1 .

6. the acquisition unit acquires the images of the display spaces for the first product and the second product; the estimation unit estimates a first filling degree of the display space for the first product and a second filling degree of the display space for the second product based on the image; the control unit notifies information about the first product when the sum of the first filling level and the second filling level is equal to or less than a specific threshold and the first filling level is equal to or less than a threshold depending on whether the first product was put out or put forward in the previous work. The information processing device according to claim 1 .

7. the acquisition unit acquires the images of the display spaces for the first product and the second product; the estimation unit estimates a first filling degree of the display space for the first product and a second filling degree of the display space for the second product based on the image; the control unit notifies information about the first product and the second product when the first filling level is equal to or less than a threshold value corresponding to whether the first product was put out or brought forward in the previous operation, and when the second filling level is equal to or less than a threshold value corresponding to whether the second product was put out or brought forward in the previous operation and notifies information about the first product; The information processing device according to claim 1 .

8. Acquire an image of the product display space, Based on the image, an estimation is made of the degree of filling of the display space of the product shown in the image; If the filling level is equal to or less than a threshold value depending on whether the product was put out or put forward in the previous work, notify information about the product. Information processing methods.

9. Acquire an image of the product display space, Based on the image, an estimation is made of the degree of filling of the display space of the product shown in the image; If the filling level is equal to or less than a threshold value depending on whether the product was put out or put forward in the previous work, notify information about the product. Have the computer perform the process, program.

10. The system includes an imaging device, an information processing device, and a terminal, The information processing device includes: an acquisition unit that acquires an image of the product display space photographed by the photographing device; an estimation unit that estimates a filling level of a display space for the products shown in the image based on the image; a control unit that notifies the terminal of information about the product when the filling level is equal to or less than a threshold value that corresponds to whether the product was put out or put forward in the previous work; A management system having: