Information processing device, information processing method, recording medium, and stocking assistance system

Abstract

An information processing device comprises: an image acquisition unit that acquires an image including a product display region imaged by a camera; an identification unit that identifies the size of the product display region by processing the image; a determination unit that determines the extent of a tendency for the quantity of at least one type of product within the product display region to decrease by processing the image; and an output unit that identifies and outputs the number of faces for each product on the basis of the identification result produced by the identification unit and the determination result produced by the determination unit.

Description

Information processing device, information processing method, recording medium, and stocking support system

[0001] The present invention relates to an information processing device, an information processing method, a recording medium, and a stocking support system.

[0002] An example of technology for preventing lost sales opportunities due to stockouts is described in Patent Document 1. The product management device in Patent Document 1 detects changes in the number of displayed products for each product, determines the degree of decrease in the number of products for which a change in the number of displayed products has been detected, and outputs a replenishment alert for products for which the determination means determines that the degree of decrease in the number of products is greater than a specified value.

[0003] Japanese Patent Application Laid-Open No. 2019-79322

[0004] The technology described in Patent Document 1 above issues a replenishment instruction to prevent stockouts of fast-selling products, and is based on the premise that the products are replenished during business hours. However, the work of stocking products to replenish them places a burden on store employees. Therefore, retail stores desire to minimize the number of times that stocking is performed during business hours while preventing products from disappearing from shelves.

[0005] The information processing device of the present disclosure comprises: an image acquisition means for acquiring an image including a product display area captured by an imaging means; an identification means for processing the image to identify the size of the product display area; a determination means for processing the image to determine the degree of a reduction trend of at least one type of product in the product display area; and an output means for determining and outputting the number of faces for each product based on the identification result by the identification means and the determination result by the determination means.

[0006] The stocking support system of the present disclosure comprises an imaging means for capturing an image of a product display area within a store and generating an image, an information processing device, and a communication device, wherein the information processing device comprises: an acquisition means for capturing an image including the product display area captured by the imaging means; an identification means for processing the image to identify the size of the product display area; a discrimination means for processing the image to determine the degree of a reduction trend of at least one type of product within the product display area; and an output means for identifying and outputting the number of faces for each product based on the identification result by the identification means and the discrimination result by the discrimination means.

[0007] The information processing method disclosed herein involves one or more computers acquiring an image including a product display area captured by an imaging means, processing the image to identify the size of the product display area, processing the image to determine the degree of a reduction trend of at least one type of product in the product display area, and identifying and outputting the number of faces for each product based on the identification and determination results.

[0008] The recording medium in the present disclosure is a computer-readable recording medium that stores a program for causing a computer to execute the following: an acquisition process for acquiring an image of a product display area captured by an imaging means; a determination process for determining the size of the product display area by processing the image; a discrimination process for determining the degree of a reduction trend of at least one type of product in the product display area by processing the image; and an output process for determining and outputting the number of faces for each product displayed in the product display area based on the identification result from the identification process and the discrimination result from the discrimination process.

[0009] According to the present disclosure, it is possible to provide an information processing device, an information processing method, a recording medium, and a stocking support system that reduce the number of stocking operations in a store.

[0010] 1 is a block diagram showing a configuration of an information processing device according to the present disclosure. FIG. 2 is a flowchart showing processing operations of an information processing device according to the present disclosure. FIG. 3 is a diagram conceptually showing a system configuration of a shelving support system according to the present disclosure. FIG. 4 is a diagram showing an example of the configuration of a computer that realizes an information processing device according to the present disclosure. FIG. 5 is a diagram for explaining the size of a product display area. FIG. 6 is a diagram showing an example of the data structure of image information. FIG. 7 is a diagram for explaining the position and number of faces of products displayed in a product display area. FIG. 8 is a diagram showing an example of the data structure of display information. FIG. 9 is a diagram showing an example of the data structure of display area information. FIG. 10 is a diagram for explaining changes in the display quantity of products displayed in a product display area over time. FIG. 11 is a diagram showing an example of the data structure of display quantity change information. FIG. 12 is a diagram showing an example of the data structure of product information. FIG. 13 is a diagram for explaining a method of specifying the number of faces using product size. FIG. 14 is a flowchart showing processing operations of an information processing device according to the present disclosure. FIG. 15 is a block diagram showing a configuration of an information processing device according to the present disclosure. FIG. 16 is a diagram for explaining adjustment of a product identification area. FIG. 17 is a flowchart showing processing operations of an information processing device according to the present disclosure. FIG. 17 is a block diagram showing a configuration of an information processing device according to the present disclosure. FIG. 18 is a flowchart showing processing operations of an information processing device according to the present disclosure. 1 is a flowchart illustrating a processing operation of an information processing device according to the present disclosure.

[0011] Hereinafter, in this disclosure, the drawings relate to one or more embodiments. In all drawings, similar components are denoted by similar reference numerals, and descriptions thereof will be omitted as appropriate. In each of the following drawings, configurations of parts that are not related to the essence of this disclosure are omitted and are not shown.

[0012] In the present disclosure, "acquisition" includes at least one of the following: a device going to retrieve data or information stored in another device or storage medium (active acquisition); and a device inputting data or information output from another device (passive acquisition). Examples of active acquisition include making a request or inquiry to another device and receiving a response, and accessing and reading information from another device or storage medium. Examples of passive acquisition include receiving information that is distributed (or transmitted, push notification, etc.). Furthermore, "acquisition" may also mean selecting and acquiring data or information from received data or information, or selecting and receiving distributed data or information.

[0013] <Example of Functional Configuration> As shown in Fig. 1, the information processing device 100 includes an image acquisition unit 102, an identification unit 104, a discrimination unit 106, and an output unit 108. The image acquisition unit 102 acquires an image including a product display area captured by an imaging unit (e.g., camera 5 in Fig. 3 described later). The identification unit 104 processes the image to identify the size of the product display area. The discrimination unit 106 processes the image to determine the degree of a reduction trend of at least one type of product in the product display area. The output unit 108 identifies and outputs the number of faces for each product based on the identification result by the identification unit 104 and the discrimination result by the discrimination unit 106.

[0014] Here, the product display area includes at least a portion of the display shelf on which the products are displayed, and is the area above the shelves of the display shelf, as described below. The size of the product display area includes width, height, and depth. However, the size of the product display area is at least the width. The product display area may also include multiple areas above multiple shelves, in which case the size of the product display area may include the width and the number of shelves. The number of faces is a value indicating how many products will be displayed when lined up horizontally on the display shelf. Generally, in the layout of a display shelf, products that sell well are given a higher number of faces than other products.

[0015] 2, the image acquisition unit 102 first acquires an image including a product display area captured by the imaging unit (camera 5) (step S101). The identification unit 104 processes the image to identify the size of the product display area (step S103). The discrimination unit 106 processes the image to determine the degree of a decrease in at least one type of product in the product display area (step S105). The output unit 108 identifies and outputs the number of faces for each product based on the identification result by the identification unit 104 and the discrimination result by the discrimination unit 106 (step S107).

[0016] In the information processing device 100 according to the present disclosure, the image acquisition unit 102 acquires an image including a product display area captured by an imaging unit, and the identification unit 104 processes the image to identify the size of the product display area. Furthermore, the determination unit 106 processes the image to determine the degree of a decrease in the number of at least one type of product in the product display area. The output unit 108 then determines the number of faces of products that can be displayed in the product display area based on the size of the product display area identified by the determination unit 104. Furthermore, the output unit 108 determines the allocation of the number of faces of products to be displayed in the product display area based on the degree of a decrease in the number of products identified by the determination unit 106.

[0017] For example, with this information processing device 100, the output unit 108 can specify and output the ratio of the number of faces for each type of product in the product display area so that products with a large degree of decline in stock have a larger number of faces than the current planogram, and products with a small degree of decline in stock have a smaller number of faces than the current planogram. As the number of faces increases, the number of displays also increases. In other words, by increasing the number of faces allocated to products with a large degree of decline in stock, the number of displays for those products in the product display area can be increased. This reduces the likelihood of those products running out of stock, eliminating the need to replenish those products. In this way, with this information processing device 100, it is possible to obtain an information processing device, information processing method, recording medium, and stocking support system that reduce the number of stocking operations in a store.

[0018] A detailed example of the information processing device 100 will be described below.

[0019] First Embodiment System Overview A stocking support system 1 supports the replenishing of merchandise on display in a store. Stores include various types of retail stores such as convenience stores, supermarkets, drug stores, and home improvement stores.

[0020] 3, the stocking support system 1 includes an information processing device 100, a camera 5, and a communication device 200. The information processing device 100 is connected to at least one camera 5 and at least one communication device 200 via a communication network 3.

[0021] The information processing device 100 is a computer such as a personal computer or a server computer. The information processing device 100 may be a computer (standalone) installed in a store. Alternatively, the information processing device 100 may be a server computer installed separately from the store and connected via a communication network 3 to a communication device 200 used by a store clerk 30. Alternatively, the information processing device 100 may be a web server that cooperates with the server computer to provide a website that provides the services of the stocking support system 1.

[0022] When the information processing device 100 is a stand-alone computer, an application program that realizes the information processing device 100 of the stocking support system 1 is installed on the computer. Then, the computer can be made to execute the application program, thereby realizing the functions of the information processing device 100.

[0023] The information processing device 100 includes a storage device 120. The storage device 120 may be provided inside or outside the information processing device 100. In other words, the storage device 120 may be hardware that is integrated with the information processing device 100, or may be hardware that is separate from the information processing device 100.

[0024] The communication device 200 is an operation terminal device used by the store clerk 30, employees, managers, etc., and is, for example, a computer such as a smartphone, tablet terminal, or personal computer. A predetermined application program is installed on the communication device 200. Then, by having the communication device 200 execute the application program, it is possible to use the services provided by the stocking support system 1. Alternatively, a browser may be launched on the communication device 200 to access the website of the stocking support system 1. Then, by using the communication device 200 to log in to the stocking support system 1 using a predetermined account, it is possible to use the services of the stocking support system 1.

[0025] The stocking support system 1 may further include a POS (Point Of Sales) terminal 300. The POS terminal 300 is connected to the information processing device 100 via the communication network 3. The POS terminal 300 may have the functions of at least one of a product registration device and a payment device operated by a store clerk or a customer.

[0026] In addition, the communication network 3 to which the camera 5 is connected, the communication network 3 to which the communication device 200 is connected, the communication network 3 to which the POS terminal 300 is connected, and the communication network 3 to which the information processing device 100 is connected may each be a combination of different networks.

[0027] The camera 5 photographs the display shelf 10 on which the products 20 are displayed. When the display shelf 10 has depth and a plurality of products 20 are displayed in the depth direction, the camera 5 is installed so that the products 20 displayed at the back are also included in the imaging range. By installing the camera 5 in this manner, the camera 5 can photograph the products 20 even at the back of the display shelf 10.

[0028] The camera 5 includes an imaging element such as a lens and a CCD (Charge Coupled Device) image sensor, and is, for example, a network camera such as an IP (Internet Protocol) camera. The network camera has, for example, a wireless LAN (Local Area Network) communication function and is connected to the information processing device 100 via the communication network 3, i.e., a relay device (not shown) such as a router. These cameras 5 may be so-called surveillance cameras, with multiple cameras installed in a store.

[0029] The images generated by the camera 5 are taken periodically and transmitted to the information processing device 100. However, the images generated by the camera 5 may also be taken in real time and transmitted to the information processing device 100. Furthermore, the images transmitted to the information processing device 100 do not have to be transmitted directly from the camera 5. The images taken by the camera 5 may be temporarily stored in another storage device, and the information processing device 100 may read them out from the storage device sequentially or at predetermined intervals. Furthermore, the images transmitted to the information processing device 100 may be still images, moving images, or frame images transmitted at predetermined intervals.

[0030] In the present disclosure, the camera 5 generates an image every hour during the store's business hours and transmits it to the information processing device 100 .

[0031] The information processing device 100 also includes an image processing device (not shown). However, the image processing device may be provided inside or outside the information processing device 100. In other words, the image processing device may be hardware that is integrated with the information processing device 100, or may be hardware that is separate from the information processing device 100.

[0032] When the image processing device is provided outside the information processing device 100, the image generated by the camera 5 may be transmitted to the image processing device. Then, the result of image processing by the image processing device may be transmitted from the image processing device to the information processing device 100. However, the image generated by the camera 5 may be transmitted to the information processing device 100, and the image may be transmitted from the information processing device 100 to the image processing device. Then, the result of image processing by the image processing device may be transmitted to the information processing device 100.

[0033] The image processing device processes the image captured by the camera 5 to identify the size of the product display area in the image. Furthermore, the image processing device processes the image to identify the area where the product 20 is present and determine the area occupied by the product 20. Furthermore, the image processing device detects the presence of the product 20 using a pre-registered product identification area of ​​the product 20 and extracts feature quantities within the product identification area. The image processing device may identify the product 20 by matching the extracted feature quantities with pre-registered product feature quantities. The product identification area is pre-stored in the display information 140 (see FIG. 8 , described later) as information indicating the outer shape of the product 20 displayed in the product display area, along with its position information, in association with the identification information of the product 20.

[0034] 4 is a block diagram illustrating an example of the hardware configuration of a computer 1000 that realizes the information processing device 100 according to the present disclosure. The communication device 200 and the POS terminal 300 are also realized by the computer 1000. The camera 5 may also be realized by the computer 1000 provided in the form of at least one chip.

[0035] The computer 1000 has a bus 1010 , a processor 1020 , a memory 1030 , a storage device 1040 , an input / output interface 1050 , and a network interface 1060 .

[0036] The bus 1010 is a data transmission path for transmitting and receiving data among the processor 1020, memory 1030, storage device 1040, input / output interface 1050, and network interface 1060. However, the method of connecting the processor 1020 and the like to each other is not limited to bus connection.

[0037] The processor 1020 is implemented by a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or the like.

[0038] The memory 1030 is a main storage device realized by a RAM (Random Access Memory) or the like.

[0039] The storage device 1040 is an auxiliary storage device realized by a hard disk drive (HDD), a solid state drive (SSD), a memory card, a read-only memory (ROM), or the like. The storage device 1040 stores program modules that realize each function of the information processing device 100 (e.g., the image acquisition unit 102, the identification unit 104, the discrimination unit 106, the output unit 108, the information acquisition unit 110 described below, the notification unit 112, the order processing unit 114, etc.). The processor 1020 loads each of these program modules into the memory 1030 and executes them, thereby realizing each function corresponding to the program module. The storage device 1040 may also store each data of the storage device 120 of the information processing device 100.

[0040] The program module may be recorded on a recording medium. The recording medium on which the program module is recorded may include a non-transitory, tangible medium usable by the computer 1000, and the program code readable by the computer 1000 (processor 1020) may be embedded in the medium.

[0041] The input / output interface 1050 is an interface for connecting the computer 1000 with various input / output devices. The input / output interface 1050 also functions as a communication interface for performing short-range wireless communication such as Bluetooth (registered trademark) and NFC (Near Field Communication).

[0042] The network interface 1060 is an interface for connecting the computer 1000 to the communication network 3. This communication network 3 is, for example, a local area network (LAN) or a wide area network (WAN). The network interface 1060 may be connected to the communication network 3 wirelessly or by wire.

[0043] The computer 1000 is then connected to necessary devices (e.g., a display, touch panel, operation buttons, touchpad, keyboard, mouse, speaker, microphone, camera, printer, etc. of the information processing device 100; a display, touch panel, touchpad, operation buttons, keyboard, mouse, speaker, microphone, camera, printer, etc. of the communication device 200; a display, touch panel, touchpad, operation buttons, keyboard, mouse, speaker, microphone, camera, printer, code reader, etc. of the POS terminal 300) via the input / output interface 1050 or the network interface 1060.

[0044] Each component of the information processing device 100 according to the present disclosure in each figure is realized by any combination of hardware and software of the computer 1000 in Figure 4. Those skilled in the art will understand that there are various variations in the realization method and device. The functional block diagrams showing the information processing device 100 according to the present disclosure in each figure show blocks of logical functional units, rather than a configuration of hardware units.

[0045] <Example of Functional Configuration> The following will be described with reference to Fig. 1. The information processing device 100 includes an image acquisition unit 102, an identification unit 104, a discrimination unit 106, and an output unit 108. <Regarding Image Acquisition Processing> The image acquisition unit 102 acquires an image including a product display area captured by a camera 5 (imaging means). As shown in Fig. 5, a display shelf 10 on which products 20 are displayed has at least one shelf 12. In the example of Fig. 5, the areas above the three shelves 12 (S001, S002, S003) are respectively shown as product display areas 14a, 14b, and 14c (however, hereinafter, when there is no particular need to distinguish between them, they will simply be referred to as product display areas 14).

[0046] The camera 5 captures an image of at least one product display area 14a, 14b, or 14c and generates an image. In the example of FIG. 5 , the camera 5 generates an image including the three product display areas 14a, 14b, and 14c of the display shelf 10 (R001). The image acquisition unit 102 acquires the image generated by the camera 5. For example, the image acquisition unit 102 periodically acquires images at predetermined time intervals, such as every hour or every 30 minutes, during the store's business hours. The image acquisition unit 102 acquires the image generated by the camera 5 and stores it as image information 130 ( FIG. 6 ) in the storage device 120 or the memory 1030 or storage device 1040 of the computer 1000.

[0047] Note that various pieces of information described below may also be stored in the storage device 120, or in the memory 1030 or storage device 1040 of the computer 1000. In the following description, "stored in the storage device 120" means "stored in the storage device 120, or in the memory 1030 or storage device 1040 of the computer 1000."

[0048] The image information 130 includes information indicating the shooting range, information indicating the shooting time, and information that can identify the image for each camera 5. However, if the image data includes the time (timestamp) at which the image was generated, the shooting time does not need to be included in the image information 130.

[0049] Specifically, as shown in Figure 6, the image information 130 includes identification information (shown as camera ID in the figure) that can identify the camera 5, as well as identification information (shown as rack ID in the figure) that can identify the display shelf 10 (furniture) as information that can identify the product display area 14 that is the subject of the image captured by the camera 5, information indicating the time the image was captured, and identification information that can identify the image (which may include the image file name or a path indicating the storage location in the storage device where the image file is stored).

[0050] For example, if the product display area 14 to be photographed is determined for each camera 5, the information for identifying the product display area 14 may be identification information for identifying the camera 5 (e.g., identification information for the camera 5 or an IP (Internet Protocol) address). Alternatively, the information for identifying the product display area 14 may include identification information for identifying the display shelf 10 (rack ID) and identification information for identifying the shelf 12 (e.g., shelf ID). In the example of FIG. 6 , the product display area 14 photographed by the camera 5 with a camera ID of C001 includes all of the product display areas 14a, 14b, and 14c with a rack ID of R001. Therefore, the information for identifying the product display area 14 includes only identification information for identifying the display shelf 10 (rack ID), and does not include identification information for identifying the shelf 12.

[0051] Furthermore, by controlling the shooting direction, etc., the camera 5 may use one camera 5 to change the direction for each of the multiple product display areas 14 and take images in sequence, thereby generating an image for each product display area 14. In this case, the information that can identify the product display area 14 preferably includes identification information (rack ID) that can identify the display shelf 10 and identification information (e.g., shelf ID) that can identify the shelf 12. However, as will be described later, separate identification information (e.g., area ID) that can identify the product display area 14 may be assigned to each product display area 14 and used.

[0052] Alternatively, one camera 5 may be used over multiple days to move the installation position of the camera 5 with respect to multiple display shelves 10 and take images of one display shelf 10 each day. In this case, the image information 130 includes, for each camera 5 and for each date, information indicating the shooting range, information indicating the shooting time, and information that allows the image to be identified.

[0053] The time interval for capturing images is preferably an interval that allows at least the degree of the decrease in the number of products 20 in each product display area 14 to be detected. For example, if the rate at which the number of products 20 is decreasing in a product display area 14 is faster than a standard, a first time interval that is shorter than the standard may be set as the time interval, and if the rate at which the number of products 20 is decreasing in a product display area 14 is slower than the standard, a second time interval that is longer than the standard may be set as the time interval. The time interval may be set by a store clerk or may be automatically set according to the above standard. The time interval may also be set according to the season, day of the week, or other attributes.

[0054] <Display of Products 20> In a store, a shelf allocation indicating which products 20 are to be displayed on which display shelves 10 and in what manner is determined in advance. As shown in Fig. 7 , multiple types of products 20 (three types, product A, product B, and product C, in the illustrated example) are displayed in the product display area 14, each with a predetermined number of faces. In this example, product A is displayed in one face, product B in two faces, and product C in two faces in the product display area 14.

[0055] The shelf allocation, which indicates which products 20 are to be displayed on which display shelf 10 and how, is stored in the storage device 120 as display information 140. The display information 140 in Fig. 8 includes the products 20 displayed on each shelf 12 of the display shelf 10, the display position of each product 20, and the number of faces of the product 20, all associated with each other. Since the types of products 20 that sell vary depending on the season and the day of the week, different display information 140 may be prepared for each season, day of the week, etc.

[0056] Specifically, as shown in Figure 8, the display information 140 includes, in association with each other, identification information (rack ID) that can identify the display shelf 10, identification information that can identify the shelf 12 (shown as shelf ID in the figure), location information that indicates the display position of the product 20, identification information that can identify the product 20 (shown as product ID in the figure), and the number of faces.

[0057] There are various possible position information indicating the display position of the product 20, and although two examples are shown in Fig. 7, the information is not limited to these. In the example of Fig. 7(a), the display position information of the product 20 can be indicated by the distance (for example, x coordinate value) from the reference position (x0) of the shelf 12 to the left end (which may be the center or center of gravity) of the product 20. However, for a product 20 with multiple faces, the display position of the product 20 is the position of the leftmost product 20 when viewed from the front.

[0058] In the example of FIG. 7( b), the display position information of the product 20 can be indicated by position information indicating the position of the product identification area 50, which indicates the area occupied by each product 20 displayed within the product display area 14. The example shown in the figure includes a product identification area 50a for product A, a product identification area 50b for product B, and a product identification area 50c for product C. The position of each product identification area 50 is indicated by a predetermined position of the product identification area 50 (for example, the coordinate value of the bottom left vertex of a rectangle). However, for a product 20 with multiple faces, the display position of the product 20 is determined to be the position information of the product identification area 50 of the product 20 on the left side when viewed from the front. In addition, information indicating the size and shape of the product identification area 50 of each product 20 may also be stored separately in the storage device 120 as product information (not shown) for each product 20.

[0059] <Regarding Identification of the Size of the Product Display Area 14> The identification unit 104 processes the image generated by the camera 5 to identify the product display area 14 and further identify the size (height, width, depth) of the product display area 14. However, if only one row of products 20 is displayed in the product display area 14, the depth information of the size of the product display area 14 does not need to be identified.

[0060] The identification unit 104 identifies the camera 5 (C001) that captured the processed image by referencing the image information 130 in FIG. 6 , and identifies the display shelf 10 (R001) that the camera 5 is capturing. The identification unit 104 then processes the image to identify multiple product display areas 14 on the display shelf 10, three product display areas 14a, 14b, and 14c in this example. The identification unit 104 assigns identification information (e.g., R1, R2, and R3) that can be used to identify each of the identified product display areas 14. The identification unit 104 then identifies the size of each detected product display area 14. The sizes of the product display areas 14 identified by the identification unit 104 are stored in the display area information 132 in FIG. 9 .

[0061] 9 , the display area information 132 includes, in association with each other, information capable of identifying image data (in this example, the file name of the image data), identification information (camera ID) capable of identifying the camera 5 that captured the image, identification information (rack ID) capable of identifying the display shelf 10, identification information (R1, R2, R3) capable of identifying the product display areas 14, and information indicating the size of each product display area 14 (width W, heights H1, H2, H3, depth D). The display area information 132 is stored in the storage device 120.

[0062] <Regarding the degree of decrease in the number of products 20> The determination unit 106 processes the image to determine the degree of decrease in the number of at least one type of product 20 in the product display area. First, the determination unit 106 reads out the image acquired by the image acquisition unit 102 from the storage device 120, and causes the image processing device to process the image.

[0063] As shown in FIG. 10 , the number of products 20 in the product display area 14 decreases over time (times t1, t2, and t3). As described above, the images used by the discrimination unit 106 are captured at predetermined time intervals (e.g., at regular intervals). The discrimination unit 106 performs image processing using the images captured at time t1 and time t2, for example, to identify the occupied area of ​​each type of product 20 for each image at each time. The discrimination unit 106 then determines the difference between the occupied areas of the products 20 in the images taken at time t1 and time t2 as the reduction rate for that time interval. The occupied area and reduction rate identified by the discrimination unit 106 are stored in the display quantity change information 150 of FIG. 11 .

[0064] The display quantity change information 150 includes a calculated reduction rate of the products 20 for each time period, in association with each product 20 displayed in each product display area 14 of each display shelf 10. Specifically, as shown in Fig. 11 , the display quantity change information 150 includes identification information (rack ID) for identifying the display shelf 10, identification information (area ID) for identifying the product display area 14, identification information (product ID) for identifying the product 20, position information indicating the display position of the product 20, information indicating the date, information indicating the time the image was captured, information indicating the area occupied by the product 20 within the product display area 14, and information indicating the reduction rate of the products 20 for a predetermined time interval.

[0065] As described above, the image acquisition unit 102 acquires images at predetermined time intervals (one hour), for example, starting from store opening (10:00 AM). The determination unit 106 processes the image each time it is acquired to calculate the area occupied by the product 20 in the product display area 14. The determination unit 106 then calculates the reduction rate of the product 20 for each predetermined time interval using the area occupied by the product 20 calculated for each predetermined time interval. In this example, the determination unit 106 calculates the reduction rate ((S2-S1) / S1, (S3-S2) / S1, ...) as the ratio of the area occupied by the product 20 at the time of store opening (S1) to the area occupied by the product 20 (S2-S1, S3-S2, ...) that has decreased from the previous predetermined time interval for each predetermined time interval. In another example, the discrimination unit 106 may calculate the ratio of the occupied area of ​​the product 20 (S2-S1, S3-S1, ...) that has decreased since the store opened at predetermined time intervals to the occupied area of ​​the product 20 (S1) at the time of store opening as a reduction rate ((S2-S1) / S1, (S3-S1) / S1, ...).

[0066] In the above example, the rate of decrease was calculated using the area occupied by the products 20 in the product display area 14 to determine the degree of decrease trend for each type of product 20. In another example, the amount of free space in the product display area 14 relative to the area where the products 20 should be displayed may be calculated, and the rate of increase of the free space at each predetermined time interval may be calculated. In this case, the discrimination unit 106 may identify the area where the products 20 should be displayed based on the product identification area 50 in the display information 140 described in FIG. 7. Alternatively, the discrimination unit 106 may process the image to recognize the shelf tag 16 ( FIG. 10 ) of the product 20 placed on the shelf 12 corresponding to the product display area 14, and identify, for example, the area above the shelf tag 16 of the product 20 as the area where the product 20 should be displayed.

[0067] <Regarding the Face Number Identification Process> The output unit 108 identifies and outputs the number of faces for each product 20 based on the discrimination result by the discrimination unit 106 and the identification result by the identification unit 104.

[0068] The output face number can be used for initial stocking of merchandise at the opening of the store on a day with similar conditions to the day on which the face number was specified, such as the next day or the same day of the week in the following week. Similar conditions may include, in addition to the day of the week, a campaign period, the season, a weekday / holiday division, or a specific event being held near the store (such as a sports day).

[0069] There are various methods for specifying the number of faces, including, but not limited to, the following examples.

[0070] <<Example Identification Method 1>> For example, the output unit 108 assigns a larger number of faces to a product 20 with a greater degree of decline trend than to other products. Various methods for determining the degree of decline trend are possible, including, but not limited to, the following examples. (Example 1) When all products 20 displayed in the product display area 14 have the same width, the output unit 108 first calculates the number of faces (e.g., 5) of products that can be displayed in the product display area 14 using the width of the products 20. Then, the output unit 108 specifies the number of faces (2 faces, 1 face, 2 faces) for each of the three types of products 20 displayed in the product display area 14 according to the ratio of the degree of decline trend (e.g., 2:1:2).

[0071] (Example 2) The magnitude of the downward trend is determined using the ranking among multiple types of products 20, which will be described later. As one example, the output unit 108 determines that the higher the ranking of the downward trend, the greater the degree of the downward trend. As another example, the output unit 108 may determine that the degree of the downward trend is large for products 20 whose ranking of the downward trend is equal to or greater than a first standard. The output unit 108 may determine that the degree of the downward trend is small for products 20 whose ranking of the downward trend is less than a second standard (second standard<first standard).

[0072] <<Example Identification Method 2>> The output unit 108 uses the determination result by the determination unit 106 to determine a ranking among multiple types of products 20 based on a decreasing trend in the product display area 14. For example, the output unit 108 first determines a ranking based on the degree of a decreasing trend of the products. Then, the output unit 108 determines the number of faces in descending order of the ranking. For example, the output unit 108 may set the number of faces to 3 for the products 20 ranked in the top three places, set the number of faces to 2 for the products 20 ranked in the fourth and fifth places, and set the number of faces to 1 for the other products 20. The range of the ranking and the number of faces are predetermined, and the output unit 108 determines the number of faces by referring to the settings.

[0073] <<Method of determining ranking based on decreasing trend>> There are various possible methods for the output unit 108 to determine a ranking based on a decreasing trend, including but not limited to the following examples. First, the output unit 108 determines the ranking of the decreasing trend for each type of product 20. (Example 1) The maximum absolute value of the rate of decrease for each predetermined time interval in one day is used. Here, the rate of decrease is a negative value. The ranking of the decreasing trend for each type of product 20 is determined in descending order of the maximum absolute value of the rate of decrease. (Example 2) The maximum value of the slope (differential Δd) of the change in the absolute value of the rate of decrease for each predetermined time interval in one day is used. The ranking of the decreasing trend for each type of product 20 is determined in descending order of the maximum slope (differential Δd) of the change in the absolute value of the rate of decrease. (Example 3) In one day, the area occupied by the product 20 (or the ratio of the area occupied by the product 20 to the area in which the product 20 should be displayed) is greater than or equal to a threshold (for example, 0 cm). ２ The order of decreasing tendency is determined for each type of product 20 in ascending order of elapsed time.

[0074] In the above explanation, the rate of decrease is described as a negative value, but the discrimination unit 106 may obtain the absolute value when calculating the rate of decrease and store the rate of decrease as a positive value in the display quantity change information 150.

[0075] Furthermore, if the restocking of products 20 is performed during business hours, the occupied area of ​​the products 20 that had been reduced will increase. Therefore, when the restocking of products 20 is performed during business hours, the discrimination unit 106 preferably acquires images after the restocking, processes the images to identify the occupied area of ​​the products 20, and further stores the image together with the time of the restocking in the display quantity change information 150. The discrimination unit 106 then calculates the time from the time of the restocking to the time when the images were acquired at a predetermined time interval, and calculates a conversion coefficient to convert this time to a predetermined time interval (e.g., one hour). For example, if the time from the time of the restocking to the time when the images were acquired at a predetermined time interval is 20 minutes, the conversion coefficient is set to 60 minutes / 20 minutes = 3. The discrimination unit 106 then converts the difference between the occupied area identified using the images after the restocking and the occupied area identified using images acquired at a predetermined time interval after the restocking into a reduction rate for the predetermined time interval using the calculated conversion coefficient.

[0076] The conversion coefficient can also be used when the time interval between the image capture times differs from the predetermined time interval, even when the work is not being replenished. The determination unit 106 calculates a conversion coefficient for converting the time interval between the capture times of two images, which are used as the basis for calculating the occupied area used to calculate the reduction rate, into a predetermined time interval, based on the capture times of the two images. The determination unit 106 can then correct the reduction rate calculated from the difference in occupied area to a value corresponding to the predetermined time interval using the conversion coefficient.

[0077] <<Identification Method Example 3>> The identification unit 104 further processes the image to identify the size of the product 20. The output unit 108 further uses the size of the product 20 to identify the number of faces.

[0078] The information processing device 100 stores information indicating the size of each type of product 20 in the storage device 120 as product information 160. Specifically, as shown in Fig. 12 , the product information 160 includes identification information (product ID) that can identify the product 20 and information indicating the size of the product 20, including the width, height, and depth, in association with each other.

[0079] For example, as shown in Fig. 13 , if the products 20 are displayed using the number of faces determined using the degree of decreasing trend, there is a possibility that the product display area 14 will not be able to accommodate all the products 20, or that empty space will be created in the product display area 14. For this reason, the output unit 108 adjusts the number of faces using the width of each type of product 20. In the example of Fig. 13 , the number of faces for product B is reduced from 2 to 1. The output unit 108 may also calculate the display quantity using the depth of each type of product 20, and adjust the number of faces taking the display quantity into consideration.

[0080] Various adjustment methods by the output unit 108 are possible, and examples thereof include, but are not limited to, the following. (Example 1) When the product display area 14 is too large for all the products 20 to fit, the number of faces is subtracted by one from the products 20 that are ranked lower in order according to the ranking based on the decreasing trend described below, until the sum of the widths and the number of faces of each product 20 displayed in the product display area 14 becomes equal to or less than the width of the product display area 14. (Example 2) When free space is created in the product display area 14, the number of faces is added by one from the products 20 that are ranked higher in order according to the ranking based on the decreasing trend described below, until the free space becomes smaller than the width of any of the products 20 displayed in the product display area 14. (Example 3) When the product display area 14 is too large for all the products 20 to fit, the difference between the sum of the widths and the number of faces of each product 20 displayed in the product display area 14 and the width of the product display area 14 is calculated. Then, the width of the product 20 is subtracted from the product 20 with a lower ranking in accordance with the ranking based on the decreasing trend (described later) (step S1), and the number of faces of the product 20 after the subtraction is subtracted by 1 (step S2). Then, steps S1 and S2 are repeated until the difference becomes 0.

[0081] <Regarding the output process of the face number> There are various possible methods for outputting the face number, including but not limited to the following examples. A combination of the following methods is also possible. (Example 1) Send to communication device 200. For example, send to a predetermined destination by email or SMS (Short Message Service). (Example 2) Notify communication device 200. For example, a predetermined application is installed in advance on communication device 200 used by store clerk 30. Furthermore, the application is started on communication device 200, and the clerk logs in to a service provided by stocking support system 1 using a predetermined account. Output unit 108 notifies the account of the face number. The notified face number is displayed on the display of communication device 200. (Example 3) If information processing device 100 is installed in a store, the face number is printed out on a printer connected to information processing device 100.

[0082] <Operation Example> The following will be described with reference to FIG. 14. The flow of FIG. 14 includes the same steps S101 to S107 as the flow of FIG. 2, and further includes step S201 between steps S105 and S107. This process is executed, for example, after business hours or once a day at a predetermined time. However, this process does not have to be executed every day; it is preferable to execute it at least before and after a change (replacement) of the product lineup 20 (for example, seasonally, during a campaign, or when a predetermined event is held). It is also preferable to execute it appropriately, for example, when the store clerk 30 feels the need to improve the stocking process. Therefore, this process may be executed automatically at a predetermined timing or when an operator's operation is accepted, and at least one of these may be selectable.

[0083] Furthermore, the processing of each step does not necessarily have to be executed simultaneously as a series of processes, and the information processing device 100 may execute each process at different times. Furthermore, images captured by the camera 5 are stored in the storage device 120, and the information processing device 100 accepts the designation of a predetermined date and time (period). Then, in step S105, the discrimination unit 106 generates display quantity change information 150 using the images from the designated predetermined date and time (period). Then, in step S107, the output unit 108 may identify the number of faces and perform output processing.

[0084] First, the image acquisition unit 102 acquires an image including the product display area 14 captured by the camera 5 (step S101). The identification unit 104 processes the image to identify the size of the product display area 14 (step S103). The discrimination unit 106 processes the image to determine the degree of a decrease in at least one type of product 20 in the product display area 14 (step S105). Furthermore, the identification unit 104 processes the image to identify the size of each type of product 20 in the product display area 14 (step S201). The output unit 108 identifies and outputs the number of faces for each product based on the identification results by the identification unit 104, the discrimination results by the discrimination unit 106, and the size of the product 20 (step S107).

[0085] In the information processing device 100 according to the present disclosure, the image acquisition unit 102 acquires an image including a product display area captured by an imaging unit, and the identification unit 104 processes the image to identify the size of the product display area. Furthermore, the determination unit 106 processes the image to determine the degree of a decrease in the number of at least one type of product in the product display area. The output unit 108 then determines the number of faces of products that can be displayed in the product display area based on the size of the product display area identified by the determination unit 104. Furthermore, the output unit 108 determines the allocation of the number of faces of products to be displayed in the product display area based on the degree of a decrease in the number of products identified by the determination unit 106.

[0086] For example, according to this information processing device 100, the output unit 108 can specify and output the ratio of the number of faces for each type of product in the product display area so that products with a large degree of declining trend have a larger number of faces than the current planogram, and products with a small degree of declining trend have a smaller number of faces than the current planogram. As the number of faces increases, the display quantity also increases. In other words, by increasing the number of faces allocated to products with a large degree of declining trend, the display quantity of those products in the product display area can be increased. As a result, those products are less likely to run out of stock, and restocking work for those products can be avoided.

[0087] In particular, in a retail store where only one employee is responsible for both customer service and stocking, the burden on the employee increases if stocking occurs frequently. Thus, the information processing device 100 provides an information processing device, information processing method, recording medium, and stocking support system that reduce the number of stocking operations in a store.

[0088] Second Embodiment The information processing device 100 in Fig. 15 is the same as the information processing device 100 in Fig. 1 except that it further includes an information acquisition unit 110. Note that each element in Fig. 15 may be combined with each element in one or more embodiments to the extent that no contradiction occurs.

[0089] <Example of Functional Configuration> The information processing device 100 has the same image acquisition unit 102, identification unit 104, discrimination unit 106, and output unit 108 as in Fig. 1, and further has an information acquisition unit 110. The information acquisition unit 110 acquires information indicating at least one of the location of the product display area 14 in the store, the type of product, the sales history of the product, information indicating product fashion trends, and product attributes (seasonality, target event, etc.). The output unit 108 further uses the information acquired by the information acquisition unit 110 to identify the number of faces.

[0090] The information indicating the position of the product display area 14 in the store may be information indicating the installation position (layout) of the display shelves 10 including the product display area 14. The information indicating the position of the product display area 14 in the store includes at least one of information that can identify the floor or sales floor of the store on which the product display area 14 is installed, information indicating its relative position with respect to the entrance to the store or sales floor, information indicating whether the display shelves 10 are multiplexed along an aisle or are in an end corner, information indicating whether the area is a sales floor for products that are subject to sales promotion, and information indicating the installation position or number of display shelves 10 by type (room temperature / refrigerated / frozen).

[0091] The information indicating the type of product is information indicating how many types of products 20 are in the same category. For example, if the product 20 is milk, there are multiple types such as manufacturer, size, ingredients (low-fat, fat-free, modified / unmodified, processed milk / raw milk / dairy drink), place of origin, and package (paper carton, bottle, plastic bottle).

[0092] For example, the information indicating the sales history of the product is acquired from the sales history information held by the POS terminal 300. The sales history information is used to identify the past sales of the product 20. The information indicating the sales can be used in the same way as the degree of a declining trend of the product 20. In other words, the output unit 108 increases the proportion of the number of faces of products 20 that have sold well in the past sales results. At this time, the output unit 108 acquires seasonal information indicating the season, month, day of the week, etc. of the sales history, and uses the sales history for the same period of the seasonal information to identify the number of faces for that period.

[0093] Examples of information indicating the trend of a product include, but are not limited to, information indicating a product 20 that is trending on a social networking service (SNS), information indicating a product 20 for which the number of articles related to the product 20 posted on the SNS is greater than a threshold, etc. The information indicating the trend of a product can be used in the same way as the degree of a declining trend of the product 20. In other words, the output unit 108 uses the information indicating the trend of a product to identify the degree to which the product 20 is popular. The output unit 108 increases the proportion of the number of faces for products 20 that are highly popular.

[0094] The information indicating the product attributes includes, for example, information indicating the seasonality of the product 20, information indicating the storage method (room temperature, refrigerated, frozen, etc.), information indicating the target event for which the product 20 sells well, and the like.

[0095] An example of combining the above information to determine the number of faces will be described. The output unit 108 determines the number of types and faces of seasonal products 20 to increase the number of types of seasonal products 20 (e.g., ice cream, which sells well in the summer) during the summer (July to August). In this case, the allocation of frozen product display shelves 10 (product display areas 14) may be greater than in other seasons. The output unit 108 allocates fewer frozen product display shelves 10 (product display areas 14) to frozen foods other than ice cream than to ice cream.

[0096] The information acquisition unit 110 may acquire weather information corresponding to the address of the store having the product display area 14 .

[0097] The weather information includes at least one of a future weather forecast and information indicating past weather history. The weather information can be obtained, for example, from a website that provides predetermined weather information. Alternatively, past weather history information may be stored in the storage device 120. The weather information includes, for example, at least one of the following: weather (sunny, cloudy, rainy, snowy), temperature (maximum, minimum, average, difference from the previous day), humidity (maximum, minimum, average, difference from the previous day), probability of precipitation, amount of precipitation, wind speed, presence or absence of lightning, presence or absence of gusts of wind, presence or absence of hail, and information about typhoons.

[0098] Various methods for using the acquired weather information are possible, including but not limited to the following examples. (Example 1) Adjusting the number of faces for the next day using information indicating the weather forecast for the next day or past weather information for the same period. For example, assume that information regarding weather that will increase sales is predetermined for each product 20. The output unit 108 determines whether the information indicating the weather forecast for the next day or past weather information for the same period corresponds to information regarding weather that will increase sales. If the output unit 108 determines that the above condition is met, it multiplies the degree of the declining trend of the product 20 determined by the discrimination unit 106 (the absolute value of the rate of decline) by a weighting coefficient k1 (where k1 > 1). Alternatively, if the output unit 108 determines that the above condition is met, it may raise the ranking of the product 20 by a predetermined value (e.g., 2) based on the degree of the declining trend of the product 20 determined by the discrimination unit 106 (e.g., raising it from fifth place to third place, two places up).

[0099] Furthermore, information regarding weather that will result in a decrease in sales may be predetermined for each product 20. The output unit 108 determines whether information indicating the weather forecast for the next day or weather information for the same period in the past corresponds to information regarding weather that will result in a decrease in sales. If it is determined that the above condition is met, the output unit 108 multiplies the degree of the decrease trend of the product 20 determined by the discrimination unit 106 (the absolute value of the decrease rate) by a weighting coefficient k2 (where k2<1). Alternatively, if it is determined that the above condition is met, the output unit 108 may lower the ranking of the product 20 based on the degree of the decrease trend of the product 20 determined by the discrimination unit 106 by a predetermined value (e.g., 2) (e.g., lowering the ranking from fourth place to sixth place, two places down).

[0100] (Example 2) The result of the determination of the degree of a product's declining trend determined by the output unit 108 is corrected by taking into account the influence of the weather on that day. As in (Example 1) above, it is assumed that information regarding the weather that increases or decreases sales is predetermined for each product 20. The output unit 108 then determines whether the information indicating the weather for the period when the degree of the product's declining trend was determined corresponds to weather information related to an increase in sales. If the output unit 108 determines that the above condition is satisfied, it multiplies the degree of the product's declining trend (the absolute value of the rate of decline) determined by the determination unit 106 by a weighting coefficient k2 (where k2<1). Alternatively, if the output unit 108 determines that the above condition is satisfied, it may lower the ranking of the product 20 based on the degree of the product's declining trend determined by the determination unit 106 by a predetermined value (e.g., 2) (e.g., lowering the ranking from fourth place to sixth place, two places down).

[0101] Furthermore, the output unit 108 determines whether the information indicating the weather during the period in which the degree of the product's decreasing trend was identified corresponds to information regarding weather related to a decrease in sales. If it is determined that the above condition is satisfied, the output unit 108 multiplies the degree of the product's decreasing trend (the absolute value of the rate of decrease) determined by the determination unit 106 by a weighting coefficient k1 (where k1 > 1). Alternatively, if it is determined that the above condition is satisfied, the output unit 108 may raise the ranking of the product 20 based on the degree of the product's decreasing trend determined by the determination unit 106 by a predetermined value (e.g., 2) (e.g., raising it from 5th place to 3rd place, two places up).

[0102] <Operation Example> An operation example of the information processing device 100 will be described below with reference to Fig. 16. The flow of Fig. 16 includes the same steps S101 to S107 as the flow of Fig. 2, and further includes step S301 between step S105 and step S107. In step S301, the information acquisition unit 110 acquires supplemental information indicating at least one of the location of the product display area 14 in the store, the type of product, the sales history of the product, information indicating product fashion trends, and product attributes (such as seasonality), or weather information corresponding to the address of the store having the product display area 14. Then, in step S107, the output unit 108 identifies and outputs the number of faces using the supplemental information or weather information acquired by the information acquisition unit 110, along with the identification result of the identification unit 104 and the determination result of the determination unit 106.

[0103] As described above, in the information processing device 100 of this embodiment, the information acquisition unit 110 acquires supplemental information or weather information. Then, the output unit 108 identifies and outputs the face number using the supplemental information or weather information acquired by the information acquisition unit 110, along with the identification result of the identification unit 104 and the determination result of the determination unit 106. In this way, the information processing device 100 of this embodiment can correct the determination result of the determination unit 106 using information on factors that affect sales for each product 20, and identify the face number using the corrected determination result. In this way, the information processing device 100 of this embodiment can achieve the same effects as the above-mentioned embodiments, and can further improve the accuracy of the face number.

[0104] Third Embodiment The information processing device 100 in Fig. 17 is the same as the information processing device 100 in Fig. 1 except that it further includes a notification unit 112. Note that each element in Fig. 17 may be combined with each element in one or more embodiments to the extent that no contradiction occurs.

[0105] 17 , the information processing device 100 includes the same image acquisition unit 102, identification unit 104, discrimination unit 106, and output unit 108 as the information processing device 100 in FIG. 1 , and further includes a notification unit 112. When the number of faces output by the output unit 108 has changed from the current number of faces, the notification unit 112 notifies the user to prompt adjustment of product area identification information (bounding box indicating product identification area 50) indicating the size of the product whose number of faces has been changed in the image. The product area identification information is information indicating the bounding box (product identification area 50) used to specify the arrangement of the products 20 to be displayed in each product display area 14 of the display shelf 10 on the operation screen when generating the display information 140.

[0106] Here, the current number of faces corresponds to the number of faces stored in the display information 140. The notification unit 112 can determine whether or not there has been a change by comparing the number of faces output by the output unit 108 with the number of faces stored in the display information 140.

[0107] The notification unit 112, for example, displays a notification on the display of the communication device 200 urging the user to adjust the product identification area 50. Various notification methods are possible, including displaying a notification on the application icon, displaying a notification on the home screen or standby screen, or displaying a pop-up. Upon receiving the notification, the operator of the communication device 200 can display a screen for making the adjustment by selecting an operation menu for adjusting the product identification area 50. The adjustment screen includes, for example, drawing elements (in this example, rectangular bounding boxes indicating the product areas) that indicate the product identification areas 50 corresponding to the multiple products 20 included in the product display area 14.

[0108] As shown in FIG. 18 , when generating display information 140, the information processing device 100 can set the arrangement of product identification areas 50 (in the figure, each type of product 20 is designated as 50a to 50c) indicating the display areas of the products 20 displayed within the product display area 14 by accepting an operator's operation on an operation screen. The operation screen is displayed on the display of the communication device 200. In the figure, images of each product are included along with the product identification areas 50, but this is for illustrative purposes only, and product images do not need to be included on the operation screen. However, information capable of identifying the product 20 corresponding to the product identification area 50 (e.g., product name, product code, etc.) may be displayed in association with a drawing element (bounding box) indicating the product identification area 50.

[0109] 18 shows an example in which the number of faces of product B displayed in the product display area 14 is changed from 2 to 3, and the number of faces of product C is changed from 2 to 1. Here, the sizes of product B and product C are different. Because the product displayed in the product identification area 50c corresponding to product C has been changed to product B, an operation to change the size of the product identification area 50c from the size of product C to the size of product B is required. Therefore, the notification unit 112 notifies the user on the operation screen to prompt the user to change the product identification area 50c' of product 20. The notification unit 112 displays a message such as, for example, "The number of faces of product B and product C has changed. Please change the bounding box." on the operation screen of the display of the communication device 200.

[0110] The notification unit 112 may also highlight a drawing element (bounding box) corresponding to the product identification area 50c′ to be changed in the product display area 14 displayed on the operation screen. The highlighting may be performed in various ways, such as highlighting, color change display, shading display, blinking display, etc., but is not limited to these.

[0111] The information processing device 100 reflects the content of the product area specification information changed on the operation screen in the display information 140 and updates the display information 140 .

[0112] The discrimination unit 106 identifies products whose face counts have been changed using the product area identification information. In particular, the discrimination unit 106 identifies products whose face counts have been changed in the product display area 14 using the updated product area identification information (product identification area 50) by referring to the updated display information 140. Then, the discrimination unit 106 determines the degree of a decreasing trend for each product 20 after the change.

[0113] <Operation Example> An operation example of the information processing device 100 will be described below with reference to Fig. 19. The flow in Fig. 19 is executed after step S107 of the flow in Fig. 2 or Fig. 16 is executed. However, the flow in Fig. 19 does not have to be executed immediately after step S107 of the flow in Fig. 2 or Fig. 16, and may be executed at a predetermined timing.

[0114] After the output unit 108 outputs the number of faces for each product 20 in step S107, the notification unit 112 compares the output number of faces with the current number of faces (stored in the display information 140) to determine whether there has been a change (step S401). If the notification unit 112 determines that the number of faces output by the output unit 108 has changed from the current number of faces (YES in step S401), it notifies the user to adjust the product area identification information (bounding box indicating the product identification area 50) that indicates the size of the product whose number of faces has been changed in the image (step S403).

[0115] Upon receiving the notification, the operator changes the bounding box on the operation screen as described above, thereby changing the product area identification information, thereby updating the display information 140 of the information processing device 100.

[0116] As described above, in the information processing device 100 of this embodiment, if the number of faces output by the output unit 108 has changed from the current number of faces, the notification unit 112 notifies the operator to adjust the product area identification information (bounding box of the product identification area 50) indicating the size of the product in the image whose number of faces has been changed. In this way, the information processing device 100 of this embodiment not only achieves the same effects as the above-mentioned embodiment, but also, if the number of faces identified based on the degree of a decrease in the number of products 20 differs from the number of faces in the current display information 140, it can change the display information 140 and prompt the operator to change the product area identification information (bounding box of the product identification area 50) used for identification by the discrimination unit 106. This allows the information processing device 100 of this embodiment to prevent a decrease in the accuracy of image recognition of each product 20 by the discrimination unit 106.

[0117] (Fourth embodiment) The information processing device 100 in Fig. 20 is the same as the information processing device 100 in Fig. 1 except that it further includes an order processing unit 114. Note that each element in Fig. 20 may be combined with each element in one or more embodiments to the extent that no contradiction occurs.

[0118] <Example of functional configuration> As shown in Fig. 20, the information processing device 100 has the same image acquisition unit 102, identification unit 104, discrimination unit 106, and output unit 108 as the information processing device 100 in Fig. 1, and further has an order processing unit 114. The order processing unit 114 performs order processing based on the order quantity for each product 20 in the store. The order processing includes, for example, a process of generating order information including the order quantity for each product 20 for each supplier, and a process of transmitting the order information to a predetermined destination of the supplier. If the face number output by the output unit 108 has changed from the current face number, the order processing unit 114 calculates the order quantity for the product using the changed face number, and performs order processing using the order quantity.

[0119] <Operation Example> An operation example of the information processing device 100 will be described below with reference to Fig. 21. The flow in Fig. 21 is executed after step S107 of the flow in Fig. 2 or Fig. 16 has been executed. However, the flow in Fig. 21 does not have to be executed immediately after step S107 of the flow in Fig. 2 or Fig. 16, and may be executed at a predetermined timing.

[0120] After the output unit 108 outputs the number of faces for each product 20 in step S107, the order processing unit 114 compares the output number of faces with the current number of faces (stored in the display information 140) to determine whether there has been a change (step S501). If the order processing unit 114 determines that the number of faces output by the output unit 108 has changed from the current number of faces (YES in step S501), it calculates the order quantity for the product using the changed number of faces (step S503). Then, the order processing unit 114 performs order processing using the calculated order quantity for the product (step S505).

[0121] As described above, in the information processing device 100 of this embodiment, if the face number output by the output unit 108 has changed from the current face number, the order processing unit 114 calculates the order quantity for the product using the changed face number and performs order processing using this order quantity. In this way, the information processing device 100 of this embodiment achieves the same effects as the above embodiment, and further, if the face number determined based on the degree of the product 20's decrease trend differs from the face number in the current display information 140, the changed face number can be reflected in the order quantity and order processing can be performed. In this way, the information processing device 100 of this embodiment can prevent product inventory shortages or surpluses by increasing or decreasing the display quantity for each product in response to a change in the face number.

[0122] Fifth Embodiment The information processing device 100 of this embodiment is the same as the information processing device 100 of Fig. 1 except that it has a configuration that identifies and outputs the reason why the ranking based on the decreasing trend satisfies the criteria. Note that each element of the information processing device 100 of this embodiment may be combined with each element of one or more embodiments to the extent that no contradiction occurs.

[0123] <Example of Functional Configuration> The information processing device 100 of this embodiment has the same configuration as the information processing device 100 in Fig. 1, and will be described using Fig. 1. The output unit 108 identifies a product whose ranking based on the identified downward trend meets a criterion (e.g., a product ranked first or higher). The output unit 108 then identifies the reason why the product meets the criterion from at least one of the number of faces, the number of purchases, and the presence or absence of a sales promotion method (e.g., POP, etc.) for the product. The output unit 108 then outputs information indicating the reason why the product meets the criterion.

[0124] The ranking based on the decline trend indicates the order of the rate at which the products 20 are removed from the product display area 14. The criteria for ranking based on the decline trend include, for example, being first or in the top three, and being included in a predetermined ranking. The reason why a product 20 satisfies the criteria is a condition related to the display of the product 20 that correlates with the product 20 being removed from the product display area 14 quickly. Examples of the reason include, but are not limited to, the following. (Example 1) Small Number of Faces For example, a possible reason for the rapid removal of a product 20 is that the product 20 is in stock but not replenished in time. In this case, increasing the number of faces of the product 20 may lower the ranking based on the decline trend of the product 20. Therefore, the output unit 108 uses the decline trend of a high-ranked product 20 and the change in the inventory quantity of the product 20 over time to determine whether there is an inventory quantity or whether the inventory quantity is above a threshold within a predetermined period (the period required for replenishment) from the time when the product 20 is removed from the product display area 14 at a predetermined percentage or less. If it is determined that the stock quantity is present or is equal to or greater than the threshold, the output unit 108 sets information indicating that the number of faces for the product 20 is small as the reason.

[0125] (Example 2) Small Purchase Quantity For example, one possible reason why the product 20 is quickly sold out is that the product 20 is out of stock and cannot be replenished. In this case, by increasing the purchase quantity of the product 20, it may be possible to lower the ranking of the product 20 based on its decreasing trend. When the output unit 108 determines that there is no stock or that the stock quantity is less than the threshold, it sets information indicating that the purchase quantity of the product 20 is small as the reason.

[0126] (Example 3) Sales promotion measures, such as POP or discount stickers, are present. Furthermore, whether or not the product 20 is a promotional item may be a factor in the speed at which the product 20 is sold out. Therefore, the output unit 108 processes the image acquired by the image acquisition unit 102 to determine whether or not the product 20 is a sales promotion measure, such as POP or discount stickers, is present. If the output unit 108 determines that the product 20 is a promotional item, it sets information indicating the presence of the sales promotion measure as the reason for the product 20.

[0127] 22 , a POP 60 is installed as a sales promotion device on a specific product 20 (product B in the figure) in the product display area 14. For example, the determination unit 106 processes the image to determine whether or not a sales promotion device is installed in the product display area 14, or to identify the product 20 on which the sales promotion device is installed. The information identified by the determination unit 106 is stored in the storage device 120 as display status information 170.

[0128] The display status information 170 includes, in association with each other, the display position, the number of faces, and the presence or absence of a sales promotion means for the displayed products 20 for each product display area 14 of the display shelf 10. Specifically, as shown in Fig. 23, the display status information 170 includes, in association with each other, identification information (rack ID) for identifying the display shelf 10, identification information (area ID) for identifying the product display area 14, position information indicating the display position of the product within the product display area 14, identification information (product ID) for identifying the product displayed at that position, the number of faces of the product, and information indicating the presence or absence of a sales promotion means (POP 60).

[0129] The output unit 108 also determines a ranking based on the identified decreasing trend. At this time, the output unit 108 may accept a designation of a period for determining the ranking. The output unit 108 stores the ranking based on the identified decreasing trend as ranking information 180 in the storage device 120.

[0130] 24 , the ranking information 180 indicates the ranking of the products 20 in descending order of the absolute value of the rate of decrease, which indicates the degree of the trend of decrease of the product 20 identified by the discrimination unit 106. The output unit 108 uses identification information (product ID) that can identify the product 20 included in the ranking information 180 to refer to the display information 140 or the display state information 170, and can acquire information such as the display position, number of faces, and presence or absence of a POP 60 of the product 20 corresponding to the product ID.

[0131] <Operation Example> An operation example of the information processing device 100 will be described below with reference to FIG. 25 . The process of FIG. 25 is executed at any timing (e.g., once a week, once a month, etc.). First, the output unit 108 refers to the ranking information 180 and identifies a product whose ranking based on a decreasing trend meets a criterion (step S601). For example, the output unit 108 identifies the product ranked first in the ranking information 180. Then, the output unit 108 determines whether the identified product is in stock (step S603). For example, the output unit 108 refers to inventory information (not shown) of the product 20 in the store and determines whether the inventory of the identified product is equal to or greater than a threshold. If it is determined that the inventory of the identified product is equal to or greater than the threshold, it is determined that the product is in stock (YES in step S603), and the output unit 108 sets information indicating that the number of faces is low as the reason (step S605).

[0132] On the other hand, if it is determined that the inventory of the identified product is not above the threshold, it is determined that the product is out of stock (NO in step S603), and the output unit 108 sets information indicating that the purchase quantity is low as the reason (step S607).

[0133] Furthermore, the output unit 108 references the display status information 170 and determines whether or not the product has a sales promotion means (such as a POP 60) (step S609). If it is determined that a sales promotion means (such as a POP 60) is present (YES in step S609), the output unit 108 sets information indicating the presence of a sales promotion means (such as a POP 60) as the reason (step S611). If it is determined that a sales promotion means (such as a POP 60) is not present (NO in step S609), the output unit 108 bypasses step S611 and proceeds to step S613. Then, the output unit 108 outputs information indicating the set reason (step S613).

[0134] In step S601, the criterion may be the lowest ranking, and the product ranked lowest based on the declining trend may be identified. In this case, if it is determined in step S609 that there is no sales promotion means (such as POP 60) (NO in step S609), the output unit 108 sets information indicating that there is no sales promotion means (such as POP 60) as the reason (not shown).

[0135] As described above, in the information processing device 100 of this embodiment, the output unit 108 identifies a product whose ranking based on the identified decline trend meets a criterion (e.g., ranked first or higher). The output unit 108 then identifies the reason why the product meets the criterion from at least one of the product's face count, purchase quantity, and the presence or absence of a sales promotion method (e.g., POP, etc.). The output unit 108 then outputs information indicating the reason why the product meets the criterion. Thus, the information processing device 100 of this embodiment not only achieves the same effects as the above-described embodiment, but also identifies products whose ranking is based on the degree of decline trend, such as products that sell out quickly, and outputs the reason for this. This allows the store staff 30 or manager to take appropriate measures according to the reason.

[0136] For example, for a product 20 with a small number of faces, the shelf allocation is changed to increase the number of faces in the display information 140, and the display information 140 is updated. The information processing device 100 may accept an operation to change the shelf allocation and update the display information 140. Furthermore, for a product 20 with a small purchase quantity, the order quantity can be increased to increase the purchase quantity. The information processing device 100 accepts an operation to change the order quantity and causes the order processing unit 114 to execute order processing. Furthermore, for a product 20 that is ranked lowest and has no sales promotion means (such as a POP 60), the store clerk 30 can take measures such as installing a sales promotion means (such as a POP 60).

[0137] Although the embodiments of the present invention have been described above with reference to the drawings, these are merely illustrative examples of the present invention, and various other configurations may also be employed. (Another Embodiment 1) For example, in the above-described embodiment, a configuration was described in which the number of faces is determined by further using the size of the product 20. When the number of faces of multiple types of products 20 displayed in the product display area 14 is determined based on the size of the product 20 and the size of the product display area 14, a situation may arise in which the product 20 does not fit into the product display area 14 even if the number of faces of the product 20 is adjusted depending on the size of the product 20. Furthermore, there may be cases in which it is more appropriate to reflect the progress determined by the store clerk 30 rather than automatically adjusting the number of faces of the product 20. Therefore, in another embodiment, the notification unit 112 of the information processing device 100 may further notify the user that the product 20 will not fit into the product display area 14 when displayed using the number of faces output by the output unit 108.

[0138] (Another embodiment 2) In the above embodiment, a configuration has been described in which the number of faces of at least one type of product 20 is identified for each product display area 14 corresponding to a predetermined shelf 12. In another embodiment, the output unit 108 identifies, for each type of product 20, a shelf 12 on which the product 20 is to be displayed and the number of faces among a plurality of product display areas 14 corresponding to a plurality of shelf boards 12.

[0139] In the multiple product display areas 14 corresponding to the multiple shelves 12, the attributes of the products 20 displayed in each product display area 14 are predetermined. For example, the attributes of the products 20 displayed in the product display area 14 corresponding to the shelf 12 located on the third shelf of the display shelf 10 (corresponding to the customer's eye level) include at least one of the following: high sales, trending, declining in popularity, and promotional products. The attributes of the products 20 displayed in the product display area 14 corresponding to the shelf 12 located on the second shelf of the display shelf 10 include being regular products.

[0140] The output unit 108 uses the identification result of the identification unit 104, the discrimination result of the discrimination unit 106, and the information acquired by the information acquisition unit 110 to identify the product display area 14 to be displayed and the number of faces to be displayed in the product display area 14.

[0141] In this configuration, if the product display area 14 and number of faces of the display target output by the output unit 108 are changed from the current number of faces, the notification unit 112 notifies the user to prompt adjustment of the product area identification information (product identification area 50) in the image, which indicates the product display area 14 of the display target and the size of the product whose number of faces has been changed.

[0142] (Other Embodiment 3) In another embodiment, the notification unit 112 may identify a location where products 20 ranked in a predetermined order based on a decreasing trend (for example, first place or top three) are displayed, and may issue a notification suggesting that products 20 that are desired to be sold quickly be displayed in that location. A location where products 20 ranked high based on a decreasing trend are displayed can be said to be a location where sales of the products 20 are high and the products 20 are likely to sell out.

[0143] The notification unit 112 may identify products 20 that are desired to be sold quickly and suggest a display location for the products 20 by specifying the products 20. Examples of products 20 that are desired to be sold quickly include older versions of products for which new products have been released, and products that have been stored in inventory for longer than a predetermined period of time. The notification unit 112 may receive information about older products and notify the user of a display location for the products in which products 20 that have a predetermined ranking (e.g., first place or top three) based on a declining trend are displayed. Alternatively, the notification unit 112 may refer to inventory information (not shown) for the products 20, identify products that have been stored in inventory for longer than a predetermined period of time, and suggest the identified location as a display location for the products.

[0144] Another example of the present disclosure may be a program that causes at least one computer to execute the above-disclosed method, or a computer-readable recording medium on which such a program is recorded. This recording medium includes a non-transitory tangible medium. This computer program includes computer program code that, when executed by a computer, causes the computer to perform the information processing method on an information processing device.

[0145] Any combination of the above components, and conversion of the present disclosure into a method, device, system, recording medium, computer program, etc., are also valid aspects of the present disclosure.

[0146] Furthermore, the various components of the present disclosure do not necessarily have to be independent entities, but may be formed as a single member by multiple components, one component may be formed from multiple components, one component may be a part of another component, or part of one component may overlap with part of another component, etc.

[0147] Furthermore, although the methods and computer programs disclosed herein describe a number of steps in a sequential order, the order in which the steps are performed does not limit the order in which the steps are performed. Therefore, when implementing the methods and computer programs disclosed herein, the order of the steps can be changed as long as it does not cause any problems in terms of the content.

[0148] Furthermore, the steps of the method and computer program disclosed herein are not limited to being executed at different times, and may be executed while one step is being executed, or may be executed partially or entirely overlapping with another step, etc.

[0149] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0150] In addition, although the flowcharts used in the above description show a sequence of steps (processes), the order of steps executed in each embodiment is not limited to the sequence shown in the flowcharts. In each embodiment, the order of steps shown in the diagrams can be changed as long as it does not cause any problems in terms of the content.

[0151] Some or all of the above embodiments can be described as, but are not limited to, the following notes. 1. An information processing device comprising: an image acquisition means for acquiring an image including a product display area captured by an imaging means; an identification means for identifying the size of the product display area by processing the image; a determination means for determining the degree of a reduction trend of at least one type of product in the product display area by processing the image; and an output means for identifying and outputting the number of faces for each product based on the identification result by the identification means and the determination result by the determination means. 2. An information processing device described in 1., wherein the identification means further identifies the size of the product by processing the image, and the output means identifies the number of faces further using the size of the product. 3. An information processing device described in 1. or 2., wherein the output means assigns a higher number of faces to products with a greater degree of a reduction trend than to other products. 4. An information processing device described in 1. to 3. 5. The information processing device described in any one of items 1. to 4., further comprising an information acquisition means for acquiring information indicating at least one of the position of the product display area in a store, the type of the product, the sales history of the product, information indicating product fashion trends, and attributes of the product, wherein the output means specifies the number of faces by further using the information acquired by the information acquisition means. 6. The information processing device described in any one of items 1. to 5., further comprising a notification means for, when the number of faces output by the output means has changed from the current number of faces, notifying a user to prompt adjustment of product area identification information indicating the size of the product whose number of faces has been changed in the image.7. An information processing device described in any one of 1. to 6., further comprising an order processing means that, when the number of faces output by the output means has changed from the current number of faces, calculates an order quantity for the product using the changed number of faces and performs order processing using the order quantity. 8. An information processing device described in any one of 1. to 7., wherein the output means uses the determination result by the determination means to identify a ranking among multiple types of products that is based on a decreasing trend in the product display area. 9. An information processing device described in 8., wherein the output means identifies products whose identified ranking based on the decreasing trend meets a criterion, identifies a reason why the product meets the criterion from at least one of the number of faces of the product, the number purchased, and the presence or absence of a sales promotion measure, and outputs information indicating the reason why the product meets the criterion. 10. An information processing device described in 6., wherein the determination means uses the product area identification information to identify the product whose number of faces has been changed.

[0152] 11. A stocking support system comprising: an imaging means for imaging a product display area in a store and generating an image; an information processing device; and a communication device, wherein the information processing device comprises: an acquisition means for acquiring an image including the product display area imaged by the imaging means; an identification means for processing the image to identify the size of the product display area; a determination means for processing the image to determine the degree of a reduction trend of at least one type of product in the product display area; and an output means for identifying and outputting the number of faces for each product based on the identification result by the identification means and the determination result by the determination means. 12. An information processing method in which one or more computers acquire an image including the product display area imaged by the imaging means, process the image to identify the size of the product display area, process the image to determine the degree of a reduction trend of at least one type of product in the product display area, and identify and output the number of faces for each product based on the identification result and the determination result. 13. 14. A computer-readable recording medium storing a program for causing a computer to execute: an acquisition process for acquiring an image of a product display area captured by an imaging means; a specification process for specifying the size of the product display area by processing the image; a determination process for determining the degree of a reduction trend of at least one type of product in the product display area by processing the image; and an output process for specifying and outputting the number of faces for each of the products displayed in the product display area based on the specification result of the specification process and the determination result of the determination process. 14. A computer-readable recording medium storing a program for causing a computer to execute: an acquisition process for acquiring an image of a product display area captured by an imaging means; a specification process for specifying the size of the product display area by processing the image acquired by the acquisition means; a determination process for determining the degree of a reduction trend of each product in the product display area by processing the image acquired by the acquisition means; and an output process for specifying and outputting the number of faces for each of the products displayed in the product display area based on the specification result of the specification process and the determination result of the determination process.

[0153] Furthermore, some or all of the configurations described in Supplementary Notes 2 to 10 that are subordinate to Supplementary Note 1 (information processing device) may also be subordinate to Supplementary Note 11 (information processing method), Supplementary Note 12 (stocking support system), Supplementary Note 13 (recording medium), and Supplementary Note 14 (program) in the same subordinate relationship as Supplementary Note 1 to 10. Furthermore, not limited to Supplementary Note 14, some or all of the configurations described as Supplements may also be subordinate to various hardware, software, various recording means for recording software, or systems, within the scope of each of the above-described embodiments.

[0154] 1 Stocking support system 3 Communication network 5 Camera 10 Display shelf 12 Shelf board 14, 14a, 14b, 14c Product display area 16 Shelf tag 20 Product 30 Store clerk 50, 50a, 50b, 50c Product identification area 60 POP 100 Information processing device 102 Image acquisition unit 104 Identification unit 106 Discrimination unit 108 Output unit 110 Information acquisition unit 112 Notification unit 114 Order processing unit 120 Storage device 130 Image information 132 Display area information 140 Display information 150 Display quantity change information 160 Product information 170 Display status information 180 Ranking information 200 Communication device 1000 Computer 1010 Bus 1020 Processor 1030 Memory 1040 Storage device 1050 Input / output interface 1060 Network interface

Claims

1. An information processing device comprising: an image acquisition means for acquiring an image including a product display area captured by an imaging means; an identification means for identifying the size of the product display area by processing the image; a discrimination means for discriminating the degree of a reduction trend of at least one type of product in the product display area by processing the image; and an output means for identifying and outputting the number of faces for each product based on the identification result by the identification means and the discrimination result by the discrimination means.

2. An information processing device according to claim 1, wherein the identification means further identifies a size of the product by processing the image, and the output means further identifies the number of faces using the size of the product.

3. An information processing device according to claim 1 or 2, wherein said output means causes the number of faces of a product that is showing a greater degree of decline in its stock to be greater than those of other products.

4. An information processing device as claimed in any one of claims 1 to 3, further comprising an information acquisition means for acquiring information indicating at least one of the position of the product display area in a store, the type of the product, the sales history of the product, information indicating trends in trends of the product, and attributes of the product, and the output means further uses the information acquired by the information acquisition means to identify the number of faces.

5. An information processing device according to any one of claims 1 to 4, further comprising an information acquisition means for acquiring weather information corresponding to an address of a store having the product display area, and the output means further uses the weather information to identify the number of faces.

6. An information processing device according to any one of claims 1 to 5, further comprising a notification means for notifying, when the number of faces output by the output means has changed from the current number of faces, that a request is made to adjust product area identification information indicating the size of the product in the image whose number of faces has been changed.

7. An information processing device according to any one of claims 1 to 6, further comprising an order processing means for, when the number of faces output by the output means has changed from the current number of faces, calculating an order quantity for the product using the changed number of faces and performing order processing using the order quantity.

8. An information processing device according to any one of claims 1 to 7, wherein the output means uses the discrimination result by the discrimination means to determine a ranking among the multiple types of products, the ranking being based on a decreasing trend in the product display area.

9. An information processing device according to claim 8, wherein the output means identifies products whose ranking based on the identified downward trend meets a criterion, identifies the reason why the products meet the criterion from at least one of the number of faces, the number purchased, and the presence or absence of sales promotion measures for the products, and outputs information indicating the reason why the products meet the criterion.

10. An information processing device according to claim 6, wherein the discrimination means identifies the product whose number of faces has been changed using the product area identification information.

11. A stocking support system comprising: an imaging means for imaging a product display area within a store and generating an image; an information processing device; and a communication device, wherein the information processing device comprises: an acquisition means for acquiring an image including the product display area imaged by the imaging means; an identification means for identifying the size of the product display area by processing the image; a discrimination means for discriminating the degree of a reduction trend of at least one type of product in the product display area by processing the image; and an output means for identifying and outputting the number of faces for each product based on the identification result by the identification means and the discrimination result by the discrimination means.

12. An information processing method in which one or more computers acquire an image including a product display area captured by an imaging means, process the image to identify the size of the product display area, process the image to determine the degree of a reduction trend of at least one type of product in the product display area, and identify and output the number of faces for each product based on the identification and determination results.

13. A computer-readable recording medium storing a program for causing a computer to execute the following processes: an acquisition process for acquiring an image of a product display area captured by an imaging means; a determination process for determining the size of the product display area by processing the image; a discrimination process for determining the degree of a reduction trend of at least one type of product in the product display area by processing the image; and an output process for determining and outputting the number of faces for each product displayed in the product display area based on the determination result from the determination process and the discrimination result from the discrimination process.

Images