Commodity management system and commodity display fixture

Abstract

To provide a commodity management system using a simple additional facility and a simple detection evaluation control system.SOLUTION: A commodity management system capable of outputting a position where a commodity displayed in a commodity display fixture 1 is removed includes a placement surface index HM provided on a placement surface 13 of the commodity display fixture 1 on which the commodity is displayed and hidden by the commodity, a detection device 2 capable of detecting the placement surface index HM when the commodity is removed from the placement surface 13, and an index processing module 3 capable of specifying a pickup position which is a position where the commodity is removed in a store based on the placement surface index HM detected by the detection device 2 and outputting the pickup position.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a product management system capable of outputting the position where a displayed product has been removed, and a product display fixture used in this system. [Background technology]

[0002] Patent Document 1 discloses a product management device that includes a load change detection means for detecting changes in the load placed on the shelves of a display shelf and a position detection means for detecting the position at which a hand approaches the shelf, and that manages products on the display shelf based on the detection results of the load change detection means and the position detection means. In this case, a product taken out of the display shelf is identified by using a product table that links unit load information for each product with display position information for each product on each shelf.

[0003] Conventionally, there has been known a management technology for managing items by attaching a non-contact electromagnetic wave tag to the item and reading the information stored in the electronic tag using a tag reader. As an application example of such a product management technology using an electromagnetic wave tag, Patent Document 2 discloses product management using an electromagnetic wave tag in which an antenna unit for transmitting and receiving electromagnetic waves and a jamming unit for jamming the electromagnetic waves are movable between a jamming position where they are close to each other and a non-jamming position where they are spaced apart. When an item is placed on the jamming unit, the jamming unit swings and moves to cover the antenna unit, blocking electromagnetic wave communication. When the item is removed from above the jamming unit, the jamming unit becomes free and moves away from the antenna unit, allowing electromagnetic wave communication to take place. This is utilized for product management.

[0004] Patent Document 3 discloses a mobile inventory robot system. In this system, a mobile inventory robot is manually guided around a store to identify items on shelves, the position of each item on the shelf, and the barcode of each item, thereby generating an inventory map and an item database for the store. More specifically, an item image is searched for in a product database using a barcode decoded from a photographed shelf image. If the searched item image does not match the item image extracted from the shelf image, the item is recognized as out of stock, and item inventory management is performed. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 2024-27681 [Patent Document 2] Japanese Patent Application Laid-Open No. 2009-123200 [Patent Document 3] US Patent Application Publication No. 2008 / 0077511 Summary of the Invention [Problem to be solved by the invention]

[0006] The product management device disclosed in Patent Document 1 requires two detection methods—a weight sensor that measures changes in shelf weight and a position detection sensor that detects the position of a hand entering the shelf—and two detection control systems with different evaluation methods for the detection results, resulting in high control costs. The product management technology disclosed in Patent Document 2 distinguishes between the presence and absence of products and the type of product based on the blocking and propagation states of electromagnetic waves, resulting in a relatively simple control system. However, because it requires a mechanical structure that selectively creates the blocking and propagation states of electromagnetic waves, the total product management costs increase as the number of products to be managed increases. The mobile inventory robot system disclosed in Patent Document 3 is unsuitable for small stores because the barcode scanner (camera) used for product identification is mounted on the guided robot. When small barcodes are used, a high-resolution camera is required, resulting in high detection control system costs.

[0007] In view of the above circumstances, an object of the present invention is to provide a merchandise management system using simple additional equipment and a simple detection and evaluation control system, and a merchandise display fixture used in this system. [Means for solving the problem]

[0008] The product management system according to the present invention is capable of outputting the position where a product displayed on a product display fixture has been removed, and comprises: a display surface indicator that is provided on the display surface of the product display fixture on which the product is displayed and is hidden by the product; a detection device that is capable of detecting the display surface indicator when the product is removed from the display surface; and an indicator processing module that is capable of identifying a pickup position in the store where the product was removed based on the display surface indicator detected by the detection device, and outputting the pickup position.

[0009] According to this configuration, the placement surface indicator provided on the placement surface on which the products are displayed is detected by the detection device as long as no products are placed on the placement surface indicator. In other words, if the placement surface indicator is hidden by a product, the detection device will detect the placement surface indicator that is exposed when the product is removed, and from this detection result, the position of the picked-up product (pick-up position) can be identified, and the identified pick-up position is output. Product management can be performed using this pick-up position.

[0010] In the present invention, it is proposed that the detection device is a camera and the placement surface index is an ArUco marker. Compared to QR codes (registered trademark) and barcodes, ArUco markers are easier and faster to detect with a camera, and their shape and configuration can also be used to estimate the camera's posture. For this reason, by configuring the placement surface index with an ArUco marker and the detection device with a camera, even if the placement surface index is small and placed in a narrow space in a product display fixture, the placement surface index that is exposed when the product is removed can be reliably detected.

[0011] Since an identification code (ID code) can be assigned to each placement surface indicator such as an ArUco marker, a QR code (registered trademark), or a barcode, if the identification code is linked to the ID of the fixture or the ID of the placement surface, it is possible to identify the fixture or the placement surface from which the product was picked up. For this reason, the present invention proposes providing a fixture identification unit that can identify the fixture ID linked to the product display fixture or the placement surface ID linked to the placement surface based on the placement surface indicator detected by the detection device.

[0012]

[0013] When the product pickup positions are identified, new products can be replenished according to the number of picked up products, so it is important to quickly notify the pickup positions to those involved in product replenishment or replenishment-related equipment. For this reason, the present invention proposes a product display fixture that includes a notification unit that notifies the status of the products displayed on the product display fixture, wherein the plurality of storage surface indicators are arranged in a row in the depth direction of the storage surface, and the notification unit notifies that the products need to be replenished when the detection device detects some of the storage surface indicators in one row, and notifies that the products are out of stock when the detection device detects all of the storage surface indicators in one row.

[0013] Since an identification code (ID code) can be assigned to each placement surface indicator, by linking the identification code with product information (product name, stock status, etc.), it is possible to identify the pickup position of the product picked up from the placement surface as well as the product information of the picked up product. For this reason, the present invention proposes providing a product identification unit that identifies the product linked to the pickup position based on the placement surface indicator detected by the detection device.

[0014] Supermarkets and other retailers handle a wide variety of products, including confectionery, fresh fish, meat, and beverages. Among these diverse products, some require prompt notification of their pickup, while others do not. This varies depending on the store's sales policy. Furthermore, shortages or out-of-stock of specific products, such as featured items or discounted items, must be avoided as much as possible from a business perspective. Therefore, picking up specific products from the display surface of the specific product requires prompt notification and replenishment. Since shortages or out-of-stock of specific products are considered more important than those of other regular products, it is preferable to notify the shortage or out-of-stock of specific products more prominently. Therefore, the present invention proposes a notification unit that notifies the status of the products displayed on the product display fixture, and the notification unit changes its notification mode when the product identified by the product identification unit is a specific product compared to when the product is not the specific product. Examples of changes in the notification mode include changing the volume and frequency of the notification sound, the intensity and flashing frequency of the notification light, etc. The notification message changes the font color and font size.

[0015] As described above, by detecting the exposed placement surface indicator, it is possible to know that a product displayed on the placement surface of a product display fixture has been removed, and it is also possible to identify the pickup position. Separately, the vacancy of the placement row space, in which products are arranged in a row from the front (the front where customers pick up products) to the rear (rear) of the product display fixture, can be confirmed by detecting the exposed cross-plane indicator (an indicator positioned at the rear of the product placement row in a direction intersecting the placement surface). This is because the cross-plane indicator becomes exposed when all products are removed from the placement row space. Therefore, the detection of such cross-plane indicators can also be used in the product management system of the present invention. For this reason, the present invention proposes that the product display fixture is provided with a cross-plane indicator that is located behind the product and is provided on a cross-plane that extends in a direction that crosses the storage surface, and is hidden by the product, and that the detection device is capable of detecting at least one of the storage surface indicator and the cross-plane indicator when the product is removed from the storage surface, and that the indicator processing module is capable of identifying the pickup position on the product display fixture based on at least one of the storage surface indicator and the cross-plane indicator, and outputting the pickup position.

[0016] In a product display fixture with a placement surface indicator and an intersection surface indicator, the following two conditions can be grasped as the product status of the products displayed on the placement surface. One is a decrease in the number of displayed products detected by the exposure (detection) of the placement surface indicator. The other is a vacancy (out-of-stock) of at least one row of displayed products detected by the exposure (detection) of the intersection surface indicator (which results in the exposure of a row of placement surface indicators). Utilizing this, product replenishment that should be carried out when the number of displayed products is decreasing and measures to address product shortages that should be carried out when the number of displayed products is vacant can be carried out in a timely manner. For this reason, the present invention proposes a notification unit that notifies the status of the products displayed on the product display fixture, wherein the notification unit notifies that the products need to be replenished when the detection device detects only the placement surface indicator, and notifies that the products are out of stock when the detection device detects both the placement surface indicator and the intersection surface indicator.

[0017] Through the detection of the placement surface indicator, it is possible to derive not only the pickup location where the product was picked up, but also the map location on the placement surface where the placement surface indicator was placed, and even the location within the store of the product display fixture that includes the placement surface (map information). For example, if the index IDs assigned to the placement surface indicator and the intersection surface indicator are linked to the map locations of the product display fixtures in the store and the map locations of the placement positions on each placement surface of each product display fixture, the pickup location can be indicated on the store map through map matching processing. This allows the product manager to easily grasp the pickup location. For this reason, the present invention proposes including a map generation unit capable of generating a map image indicating the pickup location, and a display device that displays the map image generated by the map generation unit.

[0018] When a camera is used to detect the pickup position, the captured image used to detect the pickup position is useful for the merchandise manager to confirm the pickup position. For this reason, the present invention proposes that the detection device is a camera, and that the map image shows an image of the store around the pickup position captured by the detection device.

[0019] If the detection device that detects the placement surface index is fixed, only placement surface indexes located within the detection range of the detection device can be detected. Therefore, in order to detect all placement surface indexes, it is necessary to install multiple detection devices to expand the detection range. If the detection device is configured to be able to scan space, the detection range that can be covered by a single detection device can be expanded. Furthermore, if the detection device is installed in an autonomous mobile device, it is also possible to detect all placement surface indexes with a single detection device. Therefore, the present invention proposes an autonomous mobile device that is capable of autonomous travel and has a travel device and the detection device, wherein the detection device is capable of detecting the placement surface index while the autonomous mobile device is traveling autonomously, and the index processing module is capable of identifying the pickup position based on the placement surface index detected by the detection device and outputting the pickup position.

[0020] This application covers not only the above-described product management system, but also the product display fixture used in this product management system. The product display fixture of the present invention includes the placement surface indicator and detection device described in the above-described product management system, and the detection device is configured to be able to detect placement surface indicators provided on product display fixtures other than the product display fixture equipped with the detection device. In other words, the detection devices provided on each product display fixture can be configured to detect each other's placement surface indicators, eliminating the need for construction work such as attaching each detection device to a store's ceiling or other structure. [Brief explanation of the drawings]

[0021] [Figure 1] FIG. [Figure 2] FIG. [Figure 3] 1 is a schematic diagram illustrating a placement surface indicator and a cross surface indicator provided on a product display fixture. [Figure 4] FIG. 2 is a schematic functional block diagram illustrating an example of a functional configuration of a product management system. [Figure 5] FIG. 2 is a schematic diagram illustrating the relationship between a placement surface indicator, which is an ArUco marker, and a product. [Figure 6] FIG. 1 is a schematic diagram illustrating the relationship between an ArUco marker, which is a cross-sectional indicator, and a product. [Figure 7] FIG. 10 is a screen diagram showing an example of notification information displayed on the screen of the mobile communication terminal. [Figure 8] 1 is an information flow diagram for explaining information flowing through major functional blocks. [Figure 9] FIG. 10 is a schematic diagram showing another embodiment in which the detection device is mounted on an autonomous mobile device. [Figure 10] 10 is a schematic diagram showing the relationship between uneven placement sections that divide the placement surface and placement surface indicators. FIG. [Figure 11] FIG. 10 is a schematic diagram showing an example of an indicator whose posture is changed. [Figure 12] 10A and 10B are schematic diagrams showing other examples of indicators whose postures are changed. DETAILED DESCRIPTION OF THE INVENTION

[0022] Next, an example of a product management system according to the present invention will be described with reference to the drawings.

[0023] This product management system manages products displayed on a product display fixture 1 (hereinafter simply referred to as fixture 1) as shown in Figure 1. Although arrows are used in Figure 1, in this specification, the front side (anterior side) of fixture 1 from which customers take out products indicates the "front," and the opposite side, i.e., the rear side (rear side), indicates the "rear." The left and right directions as viewed from the front of fixture 1 indicate the "left" and "right." Hereinafter, the "left and right direction" may be referred to as the "horizontal" or "horizontal direction." The "front and rear direction" may be referred to as the "vertical" or "vertical direction."

[0024] The fixture 1 is a product display shelf that is installed primarily in stores such as supermarkets and convenience stores to display and sell products, and multiple fixtures 1 are arranged in parallel or individually in the sales floor. The fixture 1 shown in FIG. 1 is mainly composed of support columns 10 arranged spaced apart on the left and right, a rear panel 11 that is a rear panel provided between the left and right support columns 10, and multiple shelves 12 extending forward from the left and right support columns 10. Note that while the shelf columns 12 are arranged only on one side of the left and right support columns 10 in FIG. 1, they may also be arranged on the other side as shown in FIG. 2.

[0025] The upper surface of the shelf board 12 is used as a product placement surface 13, and products are placed on this placement surface 13. The placement surface 13 is divided by partitions or the like according to the type (size) of the product, and products are placed in each section (placement section). Sections lined up in the left-right direction are referred to as placement section rows, and sections lined up in the front-to-back direction are referred to as placement section columns. For simplicity's sake, in the following explanation, unless there are special circumstances, it is assumed that each section is the same size, and that the placement section rows and placement section columns are arranged in a matrix so that they have the same placement sections.

[0026] As shown in Fig. 1, the placing surface 13 has placing surface indicators HM provided in the set placing sections. That is, the placing surface indicators HM are set in rows of placing sections in the depth direction on the placing surface 13, and in rows of placing sections in the horizontal direction. In Fig. 1, the placing surface indicators HM are only shown on the placing surface 13 of the top shelf 12, and only a portion of them, but in reality, the placing surface indicators HM are provided on the placing surfaces 13 of all the shelves 12. Also, in Fig. 1, for the purpose of easy viewing, products are placed only on the placing surface 13 of the second shelf 12 from the top, but of course products are placed on all the shelves 12.

[0027] In order to easily set the placement surface indicator HM on the placement surface 13, as shown in FIG. 3, a matrix of placement surface indicators HM laid out in accordance with the display plan is printed out on a print sheet, and this print sheet is then attached to the placement surface 13.

[0028] The cross plane indicator VM is located behind the products and is provided on a cross plane that extends in a direction intersecting with the placement surface 13, for example, along a plane perpendicular to the placement surface 13 (which does not have to be a vertical plane in the strict sense and may be a plane inclined from the vertical). In FIG. 1, the cross plane indicator VM is provided on the rear wall panel 11. If necessary, a special cross plane indicator VM for identifying the fixture 1 is provided on the front surface of the base shelf, which is the shelf panel 12 located at the lowest end of the fixture 1. Basically, the cross plane indicator VM is provided behind a vertical row of compartments set on the placement surface 13, so that the number of cross plane indicators VM corresponding to the number of rows of compartments is provided on the rear wall panel 11. These cross plane indicators VM may also be provided in the form of a printed sheet on which the cross plane indicators VM are arranged horizontally and attached to the rear wall panel 11, as shown in FIG. 3.

[0029] The placement surface index HM and the crossing surface index VM are figures with information embedded therein, such as an ArUco marker, a QR code (registered trademark), or a barcode. A barcode is a one-dimensional code that records information horizontally, while an ArUco marker and a QR code (registered trademark) are two-dimensional codes that record information in two directions, vertically and horizontally. In this embodiment, an ArUco marker is used as the placement surface index HM and the crossing surface index VM. In either case, to read the information embedded in the figure, a detection device that photographs (including scanning) the figure and a computing device that generates information from the image acquired by the detection device are required. The detection device and the computing device can be integrated, but in this embodiment they are configured separately.

[0030] In this embodiment of the product management system, both the placement surface indicator HM and the cross surface indicator VM are used, but of course only the placement surface indicator HM may be used. In this case, the fixture 1 does not need to have a rear wall panel 11. The product management system of the present invention can also be applied to a store that has a mixture of fixtures 1 equipped with only the cross surface indicator VM, fixtures 1 equipped with only the placement surface indicator HM, and fixtures 1 equipped with both the placement surface indicator HM and the cross surface indicator VM.

[0031] As shown in FIGS. 1 and 2, in this embodiment, the detection device is a camera 2. The camera 2 is positioned above the front of the fixture 1 so as to be able to photograph the placement surface indices HM and crossing surface indices VM that are exposed and not hidden by the products. The camera 2 is positioned above the front of the fixture 1. If it is not possible for one camera 2 to photograph all of the placement surface indices HM and crossing surface indices VM on one fixture 1, multiple cameras 2 may be used, or a movable camera 2 may be used. Furthermore, to make it easier to capture the placement surface indices HM and crossing surface indices VM, an optical refractor such as a mirror 21, which is a mirror member, may be placed between the camera 2 and the placement surface indices HM and crossing surface indices VM. Furthermore, a camera 2 dedicated to photographing the placement surface indices HM and a camera 2 dedicated to photographing the crossing surface indices VM may be separately placed.

[0032] Figure 4 shows an outline of a functional block diagram illustrating the functional configuration of a merchandise management system. Images captured by cameras 2 directed toward the placement surface index HM and crossing surface index VM provided on fixtures 1 are recorded in a captured image recording unit 22 provided in the relay device of camera 2. Since many cameras 2 are installed in the store, images captured from each camera 2 are sent to and recorded in the captured image recording unit 22. In the following explanation, when there is no need to particularly distinguish between the placement surface index HM and the crossing surface index VM, they will be collectively referred to simply as index M.

[0033] An edge computer 4 is provided as a central control device for controlling the product management system. The edge computer 4 has a memory unit and a CPU (Central Processing Unit) that executes programs. The memory unit is composed of, for example, a HDD, a ROM (Read Only Memory), or a non-volatile memory. The edge computer 4 has an index processing module 3, a data output unit 41, and a real-time control unit 42 as functional units particularly related to the present invention. In other words, the functional units are realized by the CPU of the edge computer 4 executing a program.

[0034] The index processing module 3 acquires a captured image from the captured image recording unit 22, and if the captured image contains an index M, it performs information on the product removed from the fixture 1 (picked-up product) based on the index M, such as identifying the position of the removed product (pick-up position), the type of the removed product, and the fixture 1 and placement surface 13 on which the removed product was placed. For this purpose, the index processing module 3 has a captured image acquisition unit 31, an extraction unit 32, an index information reading unit 33, a position detection unit 34, a fixture identification unit 35, and a product identification unit 36.

[0035] The captured image acquisition unit 31 acquires the captured images recorded in the captured image recording unit 22 based on the set product management schedule and provides them to the extraction unit 32. The captured images include the index M exposed toward the camera 2 and products that block the index M from the camera 2. The extraction unit 32 determines whether the captured captured image includes the index M, and if the index M is included, extracts the area of ​​the index M from the captured image. The presence of the index M in the captured image means that a product has been removed from the position or area corresponding to this index M.

[0036] The index information reading unit 33 reads the index ID (placing surface index ID or cross surface index ID) assigned to the index M extracted by the extraction unit 32, and provides it to the position detection unit 34, the fixture identification unit 35, and the product identification unit 36.

[0037] The position detection unit 34, fixture identification unit 35, and product identification unit 36 ​​derive information (data) assigned to the read index ID from a table. For example, the fixture identification unit 35 derives the fixture ID, placement surface ID, and placement section ID from the index ID, and thereby identifies the fixture 1, placement surface 13, and placement section from which the product was removed (the product was picked up). The position detection unit 34 determines the position (pickup position) of the product removed from the identified placement surface 13 or placement section. The product identification unit 36 ​​identifies the type of product removed from the identified placement surface 13 or placement section. In other words, the functions of the position detection unit 34, fixture identification unit 35, and product identification unit 36 ​​are related to each other, so they may be integrated into a single pickup position detection unit.

[0038] An example of a configuration for deriving data assigned to a read index ID from the index ID will now be described with reference to Figures 5 and 6. Figure 5 shows a mechanism for identifying a picked-up product (a picked-up product) from the detection of an exposed placement surface index HM, and Figure 6 shows a mechanism for identifying a row of picked-up products (out-of-stock products) from the detection of an exposed cross surface index VM.

[0039] FIG. 5 shows a plan view of some of the placement surface indicators HM (ArUco markers) arranged in a grid pattern on the placement surface 13 set on a specific shelf 12 of the fixture 1. Here, let's assume that a state in which products are placed on each placement surface indicator HM changes when the leftmost product is removed, exposing the placement surface indicator HM that was hidden by that product. The exposed placement surface indicator HM is captured in an image captured by the camera 2 and extracted by the extraction unit 32. The placement surface indicator ID (simply indicated as ID 31 in FIG. 5) assigned to the extracted placement surface indicator HM is read from the extracted placement surface indicator HM. Once the placement surface indicator ID is known, the fixture ID, fixture position, placement surface ID, placement surface position, placement section ID, and placement section position associated with it are derived. In other words, the pickup position is identified. Finally, the product type linked to the placement section ID is derived, and the type of the removed product (product name, etc.) is obtained. In this embodiment, every time the exposed placement surface index HM is photographed by the camera 2, the placement surface index HM is extracted by the extraction unit 32. However, the present invention is not limited to this, and the placement surface index HM may be photographed by the camera 2, and then extracted by the extraction unit 32 after a certain period of time.

[0040] Similar to FIG. 5, FIG. 6 illustrates how products removed across a row of storage compartments (i.e., missing products) are identified from the exposed cross-plane indicator VM. FIG. 6 illustrates a plan view of some of the cross-plane indicators VM (ArUco markers) arranged horizontally along the product storage row on the rear wall panel 11 of the fixture 1. When all products in a specific product storage row are removed, the cross-plane indicator VM becomes exposed. The exposed cross-plane indicator VM is captured in an image captured by camera 2 and extracted by extraction unit 32. The cross-plane indicator ID (simply shown as ID1001 in FIG. 6) assigned to the extracted cross-plane indicator VM is read from the extracted cross-plane indicator VM. Once the cross-plane indicator ID is known, the fixture ID, fixture position, storage surface ID, storage surface position, storage section row ID, and storage section row position associated with it are derived. In other words, the location of the missing item is identified. Finally, the product type linked to the placement section row ID is derived, and the type of out-of-stock product (product name, etc.) is obtained.

[0041] The product removal information (product pickup information) generated by the index processing module 3 is output from the data output unit 41 for product management, preferably together with the photographed image and the photographing time on which the information is based. This makes it possible to grasp product removal information that is difficult to grasp from POS data (for example, information indicating that a product has been removed from the placement surface 13 at the pickup position and then returned to the placement surface 13).

[0042] As shown in Figure 4, when the real-time control unit 42 detects that a product has been removed from the placement surface 13 of the fixture 1, it issues an alarm using an alarm (for example, a lamp such as a PATLITE (registered trademark) 51) placed near the fixture, to alert store staff and others to take care of product management, such as replenishing products.

[0043] At least one edge computer 4 is installed in a store, and the processing results (product management information) by the index processing module 3 output from the edge computer 4 are transferred to a host computer 6 via a data communication network such as a private line, the Internet, or a public line. The host computer 6 is installed within the store or in a management center that manages multiple stores. In this embodiment, the processing results by the index processing module 3 are transferred to the host computer 6 approximately every five minutes.

[0044] The host computer 6 processes product management information (mainly based on pick-up position data over time for each fixture 1) sent from the store's edge computer 4, creates product analysis information and product display evaluation information, and reports and manages storage of the information. The host computer 6 includes a calculation unit 6A, a reporting unit 6B, a database 6C, and a map generation unit 6D.

[0045] The calculation unit 6A performs statistical calculations and inferential calculations using the received product management information and past product management information stored in the database 6C. The map generation unit 6D accesses the database 6C and outputs the requested fixture 1, preferably a store image around the pickup location, or a store map, or both, and provides them to the notification unit 6B. The notification unit 6B includes a notification information generation unit 61, a display unit 62, and a printing unit 63. The notification information generation unit 61 generates product analysis information and product display evaluation information using the calculation results of the calculation unit 6A. The product analysis information here is, for example, information indicating when, where, and how each product was sold, and the product display evaluation information is information indicating best-selling display positions, empty product sections, and sections with missing products over time. The display unit 62 converts the information into display information (e.g., HTML data) and provides it to a user accessing the host computer 6 or the mobile communication terminal 8. The display unit 62 can also directly display notification information such as product analysis information and product display evaluation information on the screen of the display device 62a of the host computer 6 or the screen of a display device (e.g., an attached display) of the mobile communication terminal 8. The notification unit 6B can also send product analysis information, product display evaluation information, or a message simply regarding the pickup location (e.g., "Product A was picked up from the placement surface ID of the fixture ID") to the mobile communication terminal 8, such as a registered user's smartphone. FIG. 7 shows an example of notification information displayed on the screen of the mobile communication terminal 8. This screen example displays a fixture map within the store showing the number of shortages by type (category) and the locations of the shortages (pickup locations). The locations of shortages are highlighted on the fixture map with colors or arrows. The printing unit 63 converts the information generated by the notification information generation unit 61 into print data and prints it out from the printer 63a. The printed information is sent to the responsible person.

[0046] Figure 8 is an information flow diagram showing the information (data) that flows through the main functional blocks of the functional blocks described in Figure 4. Camera 2, which is a detection device, captures a picture of fixture 1. If the captured image is a still image, it is used as is, but if the captured image is a video, a still image extracted from the video is used as the captured image to be processed by edge computer 4. The captured image, to which the shooting time and camera ID are assigned as attribute information, is provided to index processing module 3 of edge computer 4 at a predetermined interval. If one camera 2 cannot cover fixture 1, fixture 1 is captured by multiple cameras 2, so multiple captured images of one fixture 1 become the processing target.

[0047] The captured image provided to the index processing module 3 is transferred to the extraction unit 32, which extracts one or more placement surface index images from the captured image. At this time, the extraction unit 32 may extract the images by dividing one captured image into a plurality of regions. For example, the image region corresponding to the placement surface 13 of each stage in the captured image is determined in advance from the field of view of the camera 2, and therefore the captured image is divided into regions for each stage (each placement surface 13).

[0048] When a placement surface index image is extracted, the index information reading unit 33 reads a placement surface index ID from the extracted placement surface index image. The read placement surface index ID is provided to a fixture identification unit 35 to derive the fixture ID or fixture position from which the product was picked up, the read placement surface index ID is provided to a position detection unit 34 to derive the placement surface position, placement section ID, and placement section position, which are the product pickup position, and the read placement surface index ID is provided to a product identification unit 36 ​​to derive the product name of the picked up product.

[0049] When some of the placing surface indicators HM (for example, a row of placing surface indicators HM) on one placing surface 13 are detected as exposed, the notification unit 6B notifies a request for replenishment of the corresponding products. When all of the placing surface indicators HM in a row of placing surface indicators HM are detected as exposed, the notification unit 6B notifies that all of the corresponding products have been removed, that is, that the products are out of stock.

[0050] When an intersecting plane index image is extracted, the index information reading unit 33 reads an intersecting plane index ID from the extracted intersecting plane index image. The read intersecting plane index ID is provided to the fixture identification unit 35, which derives the fixture ID or fixture position where a product is missing (all products in one vertical row have been removed). The read intersecting plane index ID is provided to the position detection unit 34, which derives the placement surface position, placement section row ID, and placement section row position, which are the missing product positions. The read intersecting plane index ID is provided to the product identification unit 36, which derives the product name of the missing product. The information derived by the position detection unit 34, fixture identification unit 35, and product identification unit 36 ​​is collectively referred to as product removal information. In any case, when an intersecting plane index ID is detected, it means that all products in one row on one placement surface 13 have been removed, and the notification unit 6B notifies the user that the product is missing. Detection of one row of placement surface indices HM also determines that all of the products in one row have been removed, so when both are detected, the notification is made in common.

[0051] The edge computer 4 adds the product removal information, the product removal time, a map image of the fixture 1 from which the product was removed, and the photographed image used for the product removal information, and transmits the resulting information to the notification information generation unit 61 of the host computer 6. The notification information generation unit 61 displays, prints, and transmits the product management information and chronological product management information generated from the received information. The product management information includes information about the removed product, specific information about the fixture 1 or display surface 13 from which the product removal occurred, the time of removal, etc. This allows supervisors and area managers in charge of multiple stores to grasp the product management information of each store without actually visiting each store. The generated product management information is stored in the database 6C as product management history information. The notification information generation unit 61 periodically statistically processes the product management history information and outputs it as chronological product management information including statistical data on product removal.

[0052] When the generated product management information and time-dependent product management information are displayed on the display device 62a or the mobile communication terminal 8, it is advantageous for product management if a predetermined specific product, for example, fresh fish, is displayed in a font, font color, font size, etc. that is different from other products such as meat or prepared foods, as this makes the product type that you particularly want to pay attention to stand out.

[0053] As shown in Figure 8, when the product pickup position is output, the real-time control unit 42 outputs a product pickup signal to notify in real time that the product has been removed, and for example, turns on a patrol light 51 or a specified lamp in the fixture management room for a certain period of time.

[0054] In the above description, the support form of the camera 2 is not particularly limited. As can be seen from FIG. 2 , the camera 2 is positioned above the front of the fixture 1 so as to be able to capture the placement surface index HM, the crossing surface index VM, or both, without interfering with shoppers' passage. For this reason, the camera 2 is preferably suspended from the ceiling or attached via a mounting bracket to the ceiling or support 10 of the opposite fixture 1. In this configuration, the cameras 2 attached to two opposing fixtures 1 can capture each other's placement surface index HM, the crossing surface index VM, or both. If a single camera 2 cannot capture all placement surface indexes HM and crossing surface index VM on the front side of a single fixture 1, additional cameras 2 with different capture angles can be added or attached via a swivel mount. Furthermore, a camera 2 dedicated to capturing placement surface index HM and a camera 2 dedicated to capturing crossing surface index VM may be used.

[0055] [Another embodiment] (1) The form of the fixture 1 shown in the above-described embodiment is an example, and the fixture 1 may be one that can be frozen or refrigerated, or may be one that is laid flat.

[0056] (2) In the above-described embodiment, the camera 2 serving as the detection device for detecting the index M was fixed (including a swiveling type and a scanning type). However, as schematically shown in FIG. 9 , a detection device (camera 2) for detecting the crossing plane index VM may be mounted on the autonomous mobile device 100. In this case, when at least one detection device is aligned vertically with the shelf 12 of the furniture 1, the crossing plane index VM can be detected (photographed) from approximately the front. Alternatively, a detection device capable of scanning vertically may be used as the detection device. Furthermore, the autonomous mobile device 100 is also capable of detecting (photographing) the placement surface index HM. In this case, a common camera 2 may be used as the detection device for the placement surface index HM and the crossing plane index VM, or dedicated cameras 2 may be used for each. The autonomous mobile device 100 may be configured so that the height position of the part of the autonomous mobile device 100 where the camera 2 is installed is adjustable. In this embodiment, the placing surface index HM and the crossing surface index VM are ArUco markers, so even when the camera 2 photographs the placing surface index HM and the crossing surface index VM while moving, the placing surface index HM and the crossing surface index VM are more likely to be detected accurately compared to a configuration in which the placing surface index HM and the crossing surface index VM are QR codes (registered trademark) or barcodes.

[0057] (3) In the above-described embodiment, when the product pickup location is output, the real-time control unit 42 turns on the patrol light 51 or a specific lamp in the furniture management room for a certain period of time to notify in real time that the product has been removed. As a variation of this, if the product identification unit 36 ​​identifies a specific product (such as a featured product, a discounted product, or a promotional product) as a pre-defined product, the notification may be different from that for regular products. For example, the patrol light 51 or a specific lamp in the furniture management room may flash at different intervals or light up in different colors. Furthermore, when audio notification is provided using a buzzer or speaker, the tone and volume may be different for specific products and regular products. Furthermore, a message informing the staff member that a product, particularly a specific product, has been picked up may be sent to the staff member's mobile communication terminal 8 using a font color or font size different from that for regular products.

[0058] (4) In the above-described embodiment, for simplicity, the bottom sizes of the products overlapping the placement surface indicators HM are assumed to be substantially the same, and the placement surface indicators HM are arranged in rows and columns with equal widths. However, similar product management processing is possible even if the products have different bottom sizes. In FIG. 10, products with different bottom sizes are placed on the placement surface 13, and a placement surface indicator HM is provided at the center position of each product (it does not have to be at the center). In FIG. 10, placement surface indicators HM of the same size are unevenly arranged in rows and columns according to the bottom sizes of the products. In other words, the placement sections are uneven. Note that the size of the placement surface indicators HM may be similar to the bottom sizes of the products. In any case, even for such a group of uneven placement surface indicators HM, an indicator figure can be easily created and printed out on a sheet, as shown in FIG. 3, using an ArUco marker.

[0059] (5) In the above-described embodiment, the indices M (the placement surface indices HM and the intersecting surface indices VM) are fixedly positioned. Alternatively, the indices M themselves may be configured to change position. That is, when a product is removed, the indices M corresponding to the product change position from a position hidden by the product to a position where they are easily detected by the detection device (camera 2). One such embodiment is shown in FIG. 11. In this embodiment, a rising member capable of moving downward and upward is used as the movable body 15. When substantially a row of products is placed on the rising member, the rising member is in a horizontal position. When the row of products is removed, at least a portion of the rising member rises from the placement surface 13 due to a biasing force (biasing member). The biasing force may be provided by the elastic properties of the movable body 15, such as a highly elastic resin or a shape-memory material, or may be provided by a biasing member added to the movable body 15, such as a spring or rubber. Therefore, the terms "biasing force" and "biasing member" are used synonymously herein. By providing an index M on the moving surface 15a of the rising member serving as the moving body 15 that faces forward when rising, the index M functions as a crossing surface index VM that changes its position from a first position hidden by the product to a second position exposed to the front. Furthermore, when the rising member is placed on the placement surface 13 corresponding to the bottom surface of each product, the index M provided on the moving surface 15a that faces forward functions as a placement surface index HM.

[0060] FIG. 12 shows an embodiment using a moving body 15 that behaves like a so-called noren curtain. In this embodiment, a flexible sheet-like member is used as the moving body 15. When products are placed in the depth direction of the corresponding placement section, a portion of the sheet-like member (the bent portion) serves as moving surface 15a, lying face down on top of the products lined up in the depth direction. When a row of products is removed, gravity and an elastic restoring force act as a biasing force, causing moving surface 15a to transition from a face down state to a hanging state. By providing an index M on moving surface 15a, which is the bent portion of this sheet-like moving body 15, this index M functions as a cross-plane index VM that changes position from a first position where it is hidden by the products to a second position where it is exposed to the front.

[0061] (6) The configurations of the edge computer 4, the host computer 6, and the mobile communication terminal 8 are merely examples, and may be configured as an integrated whole, or may be configured with many additional computer devices.

[0062] (7) The functional units in the functional block diagram shown in FIG. 4 may be integrated or further divided.

[0063] The configurations disclosed in the above embodiments (including other embodiments, the same applies below) can be applied in combination with configurations disclosed in other embodiments, as long as no contradiction arises. Furthermore, the embodiments disclosed in this specification are examples, and the embodiments of the present invention are not limited to these, and can be modified as appropriate within the scope that does not deviate from the purpose of the present invention. [Industrial Applicability]

[0064] The present invention is applicable to a merchandise management system that detects the position where a displayed merchandise has been removed, and to merchandise display fixtures used in such a system. [Explanation of symbols]

[0065] 1: Fixtures (product display fixtures) 2: Camera (detection device) 3: Indicator processing module 4: Edge computers 6: Host computer 6A: Arithmetic unit 6B: Notification Department 6C: Database 6E: Map image generation unit 10: Strut 11: Rear wall plate (rear plate) 12: Shelf 13: Placement surface 21: Mirror 22: Photographed image recording unit 31: Image capture unit 32:Extraction part 33: Index information reading unit 34: Position detection unit 35: Fixtures and Fixtures Department 36:Product Specification Department 41: Data output section 42: Real-time control section 51: Patlite 61: Notification information generation unit 62:Display section 63: Printing Department 100:Autonomous driving device HM: Placement surface index (index) VM: Cross plane index (index)

Claims

1. A product management system capable of outputting the position where a product displayed on a product display fixture has been removed, a display surface indicator provided on a display surface on which the product is displayed and hidden by the product; a detection device capable of detecting the placement surface indicator when the product is removed from the placement surface; A product management system comprising: an index processing module that is capable of identifying a pickup position, which is the position in the store where the product was removed, based on the placement surface index detected by the detection device, and outputting the pickup position.

2. the detection device is a camera; The product management system according to claim 1 , wherein the placement surface indicator is an ArUco marker.

3. A product management system as described in claim 1, which is equipped with a fixture identification unit that can identify a fixture ID associated with the product display fixture or a placement surface ID associated with the placement surface based on the placement surface indicator detected by the detection device.

4. a notification unit that notifies the state of the products displayed on the product display fixture; The plurality of placement surface indicators are arranged in a row in a depth direction of the placement surface, The product management system described in claim 1, wherein the notification unit notifies that the product needs to be replenished when the detection device detects some of the storage surface indicators in a row, and notifies that the product is out of stock when the detection device detects all of the storage surface indicators in a row.

5. The product management system according to claim 1 , further comprising a product identification unit that identifies the product linked to the pickup position based on the placement surface index detected by the detection device.

6. a notification unit that notifies the state of the products displayed on the product display fixture; The product management system according to claim 5 , wherein the notification unit changes the notification mode when the product identified by the product identification unit is a specific product from the mode when the product is not the specific product.

7. The product display fixture is provided with an intersecting surface indicator that is located behind the product and extends in a direction intersecting with the placement surface, and is hidden by the product; the detection device is capable of detecting at least one of the placement surface indicator and the crossing surface indicator when the product is removed from the placement surface; The product management system according to claim 1 , wherein the indicator processing module is capable of identifying the pickup position on the product display fixture based on at least one of the placement surface indicator and the cross surface indicator, and outputting the pickup position.

8. a notification unit that notifies the state of the products displayed on the product display fixture; The product management system described in claim 7, wherein the notification unit notifies that the product needs to be replenished when only the placement surface indicator is detected by the detection device, and notifies that the product is out of stock when the detection device detects both the placement surface indicator and the cross surface indicator.

9. a map generation unit capable of generating a map image showing the pickup position; The product management system according to claim 1 , further comprising: a display device that displays the map image generated by the map generation unit.

10. the detection device is a camera; The merchandise management system according to claim 9 , wherein the map image shows an image of a store around the pickup location captured by the detection device.

11. an autonomous driving device that has a driving device and the detection device and is capable of autonomous driving; the detection device is capable of detecting the placement surface index while the autonomous driving device is autonomously driving, The product management system according to claim 1 , wherein the indicator processing module is capable of identifying the pickup position based on the placement surface indicator detected by the detection device and outputting the pickup position.

12. A product display fixture used in the product management system according to any one of claims 1 to 11, The placement surface index and the detection device are provided, The detection device is configured to be able to detect the display surface indicator provided on another product display fixture.

Images