Management device and commodity replenishment system

JPWO2024237239A1Undetermined Publication Date: 2024-11-21
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
JP2025520588
Authority / Receiving Office
JP · JP
Patent Type
Applications
Priority Date
2024-04-25
Filing Date
2024-05-13
Publication Date
2024-11-21

AI Technical Summary

Technical Problem

Current product replenishment systems in convenience stores require manual intervention by store clerks to replenish inventory shelves, which is time-consuming, labor-intensive, and inefficient, especially in low-temperature environments, and limits the operational rate of the product movement system.

Method used

A management device and product replenishment robot system that uses machine learning to identify replenishment needs based on sales data, autonomously moves products from inventory to display shelves, and vice versa, using imaging and gripping mechanisms to optimize product placement and reduce human intervention.

Benefits of technology

The system significantly reduces the burden on store staff by automating the replenishment process, improving efficiency and maintaining the operational rate of the product movement system, even in challenging environments.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

[Problem] To make it possible to reduce the burden of work to replenish commodities on an inventory shelf for a store salesperson. [Solution] One embodiment of the present disclosure provides a management device 300 that manages commodity replenishment by means of a commodity replenishment robot 100 that performs an operation of gripping a commodity placed on an inventory shelf and moving the commodity to a display shelf, and an operation of gripping a commodity placed on a commodity replenishment shelf and moving the commodity to the inventory shelf. The management device 300 includes a control unit 310 and a storage unit 320. The storage unit 320 stores either sales data including past sales numbers and sales dates and times of commodities, or a trained model generated by executing machine learning by means of a learning device using the sales data. The control unit 310 is configured to execute, on the basis of the sales data or the trained model, the generation of commodity replenishment information relating to commodities to be replenished on the inventory shelf, the commodity replenishment information including placement layout information for placing the commodities to be replenished on the inventory shelf in each region on the commodity replenishment shelf.
Need to check novelty before this filing date? Find Prior Art

Description

Management device and product replenishment system

[0001] The present disclosure relates to a management device and a product replenishment system.

[0002] Patent Document 1 discloses a product moving system including a product moving device that moves products placed on inventory shelves in a store such as a convenience store to a display shelf different from the inventory shelf. This product moving device includes a gripping unit that grips the product, an arm unit that moves the gripping unit to the inventory shelf and the display shelf, a first imaging unit that photographs the display shelf, a second imaging unit that photographs the inventory shelf, and a control unit that controls the operation of the arm unit, gripping unit, and first and second imaging units.

[0003] The control unit of the product moving device identifies replenishment target products, which are products that need to be replenished on the display shelves, based on the display shelf image data generated by the first imaging unit by photographing the display shelves, and identifies the positions of inventory products in the inventory shelf image data that correspond to the replenishment target products, based on the inventory shelf image data generated by the second imaging unit by photographing the inventory shelves. The control unit further causes the gripper to grip the replenishment target products based on the inventory shelf image data, and after the gripper has gripped the replenishment target products, moves the gripper toward the display shelf and places the replenishment target products in product placement positions on the display shelves based on the display shelf image data.

[0004] In this way, the product moving device disclosed in Patent Document 1 performs the operation of moving products from inventory shelves to display shelves to replenish them.

[0005] International Publication No. 2023 / 022214

[0006] According to the system disclosed in Patent Document 1, it is possible to replenish products on inventory shelves by moving products from inventory shelves to display shelves, but the replenishment of products on inventory shelves still requires the store clerk to do so.

[0007] Each shelf of a stock shelf has multiple rows of products separated by partitions, and each row is stocked with a specific product. The system recognizes which products are stocked in each row on each shelf of the stock shelf. For example, a label listing the product name and other information to be replenished is placed in front of each row on each shelf of the stock shelf, and store clerks repeatedly check the labels and replenish the rows with the products they need. This task of replenishing products on stock shelves while searching for the appropriate row takes a lot of time and effort. Furthermore, stock shelves that handle beverages at convenience stores and other stores are located in back-of-house areas where low temperatures are controlled, and working in such a low-temperature environment places a great physical strain on store clerks.

[0008] Furthermore, while a store clerk is replenishing products on the shelves, the product moving device must be moved to a position where it will not interfere with the clerk's work and must be stopped for safety reasons, so the product moving device cannot be operated during that time. Therefore, the replenishing of products on the shelves by a store clerk is one factor that reduces the operating rate of the product movement system.

[0009] An object of the present disclosure is to provide a means for reducing the burden on store staff of replenishing inventory shelves.

[0010] According to one aspect of the present disclosure, there is provided a management device that manages product replenishment by a product replenishment robot that performs operations of grasping products placed on inventory shelves and moving them to display shelves and grasping products placed on replenishment shelves and moving them to inventory shelves. The management device has a control unit and a memory unit. The memory unit stores sales data including past sales quantities and sales dates and times of the products, or a trained model generated by performing machine learning with a learning device using the sales data. The control unit is configured to execute, based on the sales data or the trained model, generating product replenishment information regarding products to be replenished on inventory shelves, the product replenishment information including placement layout information for placing the products to be replenished on the inventory shelves in each area on the replenishment shelves.

[0011] Other features and advantages of the present disclosure can be seen from the following description and the accompanying drawings, which are given by way of example and are not exhaustive.

[0012] According to the present disclosure, a means is provided that can reduce the burden on store staff of replenishing inventory shelves.

[0013] 1 is a diagram showing the overall configuration of a product replenishment system according to a first embodiment of the present disclosure. FIG. 1 is a plan view schematically showing the arrangement of shelves in a store and a product replenishment robot arranged in the store. FIG. 1A is a view of a display shelf as seen from the front side, and FIG. 1B is a view of the display shelf as seen from the rear side. FIG. 1B is a schematic diagram for explaining the functions of an inventory shelf. FIG. 2 is a side view schematically showing the configuration of a product replenishment robot. FIG. 3 is a perspective view showing the peripheral structure of the tip of the arm of the product replenishment robot. FIG. 4 is a block diagram showing the configuration of the product replenishment robot. FIG. 5 is an image of a display shelf photographed by a first camera (left side) of the product replenishment robot. FIG. 6 is a flowchart showing the product replenishment operation by the product replenishment robot. FIG. 7 is a perspective view showing the overall configuration of a product image acquisition device. FIG. 8 is a diagram showing a series of operations for acquiring product images using the product image acquisition device and identifying the products. FIG. 9 is a block diagram showing the configuration of a management device. FIG. 10 is a block diagram showing the configuration of a terminal device. FIG. 11 is a flowchart for explaining an example operation of the product replenishment system. FIG. 12 is a diagram showing an example of a display screen for product replenishment information displayed on the display operation unit of the terminal device. FIG. 13 is a diagram showing a state in which identification information of a product replenishment shelf photographed by the camera of the terminal device is displayed on the display operation unit of the terminal device. FIG. 14 is a diagram showing an example of a display screen for placement layout information displayed on the display operation unit of the terminal device. 21. A diagram showing an example of work start request information displayed on the display operation unit of the terminal device. A diagram showing an example of completion confirmation request information displayed on the display operation unit of the terminal device. A diagram showing an example of a warning message displayed on the display operation unit of the terminal device. A perspective view showing the overall configuration of a product replenishment shelf in a second embodiment of the present disclosure. A block diagram of a control unit provided in the product replenishment shelf shown in FIG. 21. A flowchart for explaining an example of operation of the product replenishment system of the second embodiment. A diagram showing an example of product replenishment information displayed on the display operation unit of the terminal device. A diagram showing an example of product placement position table information in the second embodiment. A perspective view showing the overall configuration of an inventory shelf in a third embodiment. A flowchart for explaining an example of operation of the product replenishment system of the third embodiment. A diagram showing an example of product placement position table information in the third embodiment.

[0014] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0015] First Embodiment Overall Configuration of Product Replenishment System

[0016] First, the overall configuration of the merchandise replenishment system according to the first embodiment of the present disclosure will be described. Fig. 1 is a diagram showing the overall configuration of the merchandise replenishment system according to the first embodiment of the present disclosure.

[0017] As shown in FIG. 1, the merchandise replenishment system 1 according to the first embodiment of the present disclosure includes a merchandise replenishment robot 100 that moves merchandise, a merchandise image acquisition device 200 provided on the merchandise replenishment robot 100, a management device 300 that manages the schedule of merchandise replenishment work performed by the merchandise replenishment robot 100, and a terminal device 400 that is operated by a store clerk or the like involved in merchandise replenishment.

[0018] Merchandise replenishment robot 100 is placed in a store such as a convenience store, and replenishes merchandise onto inventory shelves SL20 and from inventory shelves SL20 to display shelves SL10. A merchandise image acquisition device 200 provided on merchandise replenishment robot 100 captures images of merchandise placed on merchandise image acquisition device 200 by merchandise replenishment robot 100. Based on the captured merchandise images, a control device 150 of merchandise replenishment robot 100 identifies the type of merchandise placed on merchandise image acquisition device 200.

[0019] The management device 300 transmits to the terminal device 400 product replenishment information generated by scheduling the types and quantities of products to be replenished on the inventory shelves SL20 and the timing of replenishment. The product replenishment information includes stacking layout information for placing various products on the product replenishment shelves SL30 for transporting the products to be replenished to the inventory shelves SL20. The management device 300 also causes the product replenishment robot 100 to perform an operation to replenish products from the product replenishment shelves SL30 to the inventory shelves SL20 based on the product replenishment schedule.

[0020] The terminal device 400 displays the product replenishment information transmitted from the management device 300 on the display screen of the terminal device 400, and presents the product replenishment information, such as the timing for replenishing products on the inventory shelf SL20 and the type and number of products to be replenished on the inventory shelf SL20, to the store clerk using the terminal device 400. The store clerk places designated quantities of various designated products in predetermined positions on the product replenishment shelf SL30 in accordance with the loading layout information displayed on the screen of the terminal device 400.

[0021] The store clerk moves the replenishment shelf SL30 on which the replenishment items are placed to a position accessible by the replenishment robot 100 at the timing indicated in the replenishment information. The replenishment robot 100 moves the items on the replenishment shelf SL30 to predetermined positions on the inventory shelf SL20, replenishing the items on the inventory shelf SL20. At this time, the replenishment robot 100 may, as necessary, place the items on the replenishment image acquisition device 200 to identify the type of item to be replenished. Between replenishment operations on the inventory shelf SL20, or after the replenishment operations on the inventory shelf SL20 have been completed, the replenishment robot 100 moves the items from the inventory shelf SL20 to the display shelf SL10, replenishing the items on the display shelf.

[0022] In this way, the product replenishment system 1 of the present disclosure replenishes products from the product replenishment shelf SL30 to the inventory shelf SL20 in accordance with product replenishment information that indicates the type and quantity of products to be replenished on the inventory shelf SL20 and the timing of replenishment, and further replenishes products from the inventory shelf SL20 to the display shelf SL10.

[0023] Next, the layout of the store will be described. Fig. 2 is a plan view showing the shelves and product replenishment robots arranged in the store. Fig. 3(a) is a view of the display shelves from the front side, and Fig. 3(b) is a view of the display shelves from the rear side. Fig. 4 is a schematic diagram for explaining the functions of the inventory shelves.

[0024] As shown in Figure 2, the store is divided into an in-store space SH1 and a backyard space SH2. The in-store space SH1 is a space where customers select and purchase products T. The backyard space SH2 is a space where inventory of products T is stored. The store is equipped with display shelves SL10, inventory shelves SL20, product replenishment shelves SL30, a product replenishment robot 100, and a product image acquisition device 200 on the product replenishment robot 100. A track consisting of two rails R is laid on the floor between the display shelves SL10 and the inventory shelves SL20 in the store, and the product replenishment robot 100 is configured to move on these rails R in the left-right direction as shown in the figure.

[0025] The product T is, for example, a beverage container or a cylindrical beverage can having a main body Ta and a cap Tb attached to the upper end of the main body Ta. The product T may be made of various materials, such as PET, glass, aluminum, steel, or other metals.

[0026] As shown in Figures 3(a) and (b), the display shelf SL10 has a plurality of shelf boards SL11 (also referred to as display shelf tiers). A plurality of types of products T are arranged on the shelf boards SL11. For example, products T of the same type are arranged in two or three rows. The products T may also be arranged in only one row. A plurality of partition boards SL12 are provided on the upper surface of each shelf board SL11 to separate the products T in each row. Figure 3 shows an example in which products T of the same type are arranged in two rows, and in this example, a partition board SL12 is provided every two rows of products T. The arrangement of the partition boards SL12 is not limited to this, and the partition boards SL12 can be arranged at intervals of one row or two or more rows.

[0027] The front of the display shelf SL10 faces the in-store space SH1, allowing customers to pick up products T from the front side of the display shelf SL10. The shelf board SL11 is inclined so that the front side of the display shelf SL10 is relatively lower than the rear side. As a result, when a customer picks up a product T, the other products T lined up behind that product T slide along the shelf board SL11 and move to the front side.

[0028] The rear of the display shelf SL10 faces the backyard space SH2, and a store clerk or a product replenishment robot 100 replenishes products T onto the display shelf SL10 from the rear side of the display shelf SL10. Although not shown, doors may be provided on the front and rear of the display shelf SL10. Although only one display shelf SL10 is shown in Fig. 3 for the sake of simplicity, multiple display shelves SL10 may be arranged in the store.

[0029] The inventory shelf SL20 is arranged in the backyard space SH2 facing the display shelf SL10, with the front surface of the inventory shelf SL20 facing the rear surface of the display shelf SL10. Similar to the display shelf 10, the inventory shelf SL20 has multiple shelves (tiers) arranged in the vertical direction. Products to be replenished on the display shelf SL10 are arranged on the shelves of the inventory shelf SL20. A store clerk or a product replenishment robot 100 replenishes products T onto the inventory shelf SL20 from the front side of the inventory shelf SL20.

[0030] The product replenishment shelf SL30 has multiple shelves (tiers) arranged in the vertical direction. The bottom of the product replenishment shelf SL30 is provided with multiple casters that allow the product replenishment shelf SL30 to be moved in any direction. The product replenishment shelf SL30 may also be affixed with a label bearing characters, a two-dimensional code, or the like as identification information for identifying the product replenishment shelf SL30. The identification information written on the label can be read using a camera provided in the terminal device 400.

[0031] The configuration and function of the inventory shelf SL20 in this embodiment will now be described with reference to FIG. 4 . Each shelf 21 of the inventory shelf SL20 is configured to be able to switch between a first position, in which the front edge (right side in the figure) is higher than the rear edge (left side in the figure), as shown by the solid line in FIG. 4 , and a second position, in which the front edge is lower than the rear edge, as shown by the dotted line in FIG. 4 . The inventory shelf SL20 is equipped with a drive mechanism (not shown) for switching each shelf SL21 between the first position and the second position. The drive mechanism includes, for example, an electric motor as a drive source. The drive mechanism switches the position of each shelf SL21 between the first position and the second position in response to a control signal transmitted, for example, from a communication unit 195 (see FIG. 7 ) equipped in the product replenishment robot 100. The drive mechanism may be provided individually for each shelf SL21 of the inventory shelf SL20 so as to drive each shelf SL21 independently, or may be provided as a single mechanism so as to drive multiple shelves SL21 together via a connecting mechanism or the like.

[0032] When the product replenishment robot 100 replenishes products on each shelf SL21 of the inventory shelf SL20, the shelf SL21 is placed in a first position in which the front edge is higher than the rear edge. In the first position, the shelf SL21 is inclined downward from the front to the rear. As a result, products placed on the front side of a specific product row on the shelf SL21 for replenishment slide down the inclination from the front to the rear toward the rear of the shelf SL21. Therefore, by repeatedly placing products on the front side of the shelf SL21, products can be sequentially replenished in the product row on the shelf SL21.

[0033] On the other hand, when the product replenishment robot 100 moves products from the inventory shelf SL20 to the display shelf SL10 to replenish the products on the display shelf SL10, the shelf SL21 of the inventory shelf SL20 is placed in a second position in which the front edge is lower than the rear edge. In the second position, the shelf SL21 slopes downward from the rear to the front. As a result, products placed in each product row on the shelf SL21 slide down the slope from the rear to the front toward the front of the shelf SL21. Each time the product replenishment robot 100 removes the product located at the front of the front side of the inventory shelf SL20 for replenishment, the products behind it slide down sequentially toward the front of the shelf SL21. This allows the product replenishment robot 100 to remove the product located at the front of the front side of the inventory shelf SL20 and move it to the display shelf SL10 for replenishment. <Configuration and Operation of the Product Replenishment Robot 100>

[0034] The product replenishment robot 100 will be described below with reference to Figures 2 and 5 to 7. Figure 5 is a side view showing a schematic configuration of the product replenishment robot 100. Figure 6 is a perspective view showing the peripheral structure of the tip of the arm of the product replenishment robot 100. Figure 7 is a block diagram showing the configuration of the product replenishment robot 100.

[0035] The product replenishment robot 100 has a gripper 10, an arm 20, a contact detection sensor 30, first cameras 50R and 50L, a second camera 60, a third camera 70, a horizontal movement mechanism 80, an elevator mechanism 90, and a control device 150. The product replenishment robot 100 is a robot that moves in the space between a display shelf SL10 and an inventory shelf SL20. The product replenishment robot 100 performs the following operations: gripping a product T on a product replenishment shelf SL30 with the gripper 10 and moving it to a storage position of that product on the inventory shelf SL20 (a lane in which that product T is stored); and gripping a product T in the inventory shelf SL20 with the gripper 10 and moving the gripped product T to a display position of that product on the display shelf SL10 (a lane in which that product T is displayed).

[0036] As shown in FIG. 6 , the gripping portion 10 has a pair of fingers 11 a, 11 b for gripping an object. The pair of fingers 11 a, 11 b are shaped to be able to grip the vicinity of the cap portion Tb of a container of the product T, such as a PET bottled beverage, and to grip the outer periphery of the product T, such as a canned beverage. The gripping portion 10 may be configured with a pair of fingers 11 a, 11 b, or may have an attraction structure for attracting and holding the object, or a structure for holding the object using adhesive force, magnetic force, or the like. It is preferable that a flexible holding member (not shown) is provided on the gripping surface of each finger 11 a, 11 b to better hold the product T when gripped. The holding member may be formed of, for example, a rubber sheet material, a urethane resin sheet material, or the like.

[0037] The arm unit 20 has a plurality of link members 21, 22, and 23. The plurality of link members 21, 22, and 23 constitute an articulated robot arm. As an example, the articulated robot arm may be a six-axis arm having degrees of freedom in linear directions along the X-axis, Y-axis, and Z-axis, and degrees of freedom around the X-axis, Y-axis, and Z-axis. The articulated robot arm may also have any other mechanism, such as a Cartesian coordinate system robot arm, a polar coordinate system robot arm, a cylindrical coordinate system robot arm, or a SCARA robot arm. One end of the arm unit 20 is fixed to the lifting mechanism 90. A gripper 10 is provided at the tip of the arm unit 20. The operation of the arm unit 20 is controlled by a control device 150.

[0038] The arm unit 20 can move the gripper 10 toward the inventory shelf SL20 or toward the display shelf SL10 by moving the respective link members 21, 22, and 23. The orientation of the arm unit 20 is not fixed to a specific direction, but for convenience of explanation, the direction in which the respective link members 21, 22, and 23 are extended will be referred to as the extension direction Ax of the arm unit 20 (see FIG. 6 ). The arm unit 20 grips the product T by moving the gripper 10 forward toward the product T. The extension direction Ax of the arm unit 20 corresponds to the forward direction of the gripper 10 in the gripping operation.

[0039] The contact detection sensor 30 is a sensor that detects that the gripping unit 10 has come into contact with the product T when gripping the product T, and that when placing the product T gripped by the gripping unit 10 on a shelf SL21 of the inventory shelf SL20 or a shelf SL11 of the display shelf SL10, the product T gripped by the gripping unit 10, the gripping unit 10, or the arm unit 20 has come into contact with an obstacle such as a wall or a support of the shelf SL10 or SL20. The contact detection sensor 30 may be, for example, a torque sensor, an acceleration sensor, an inertial measurement unit (IMU), a motor input current sensor, or the like.

[0040] The torque sensor may be, for example, a strain gauge that detects torque generated at the axis of each joint of the arm unit 20. The acceleration sensor may be any of various types of acceleration sensors, such as capacitance type or piezo-resistance type, that are installed in the gripping unit 10 or the arm unit 20.

[0041] An inertial measurement unit (IMU) is a device that detects three-dimensional inertial motion (translational and rotational motion in three orthogonal axes). It includes an acceleration sensor that detects translational motion and an angular velocity (gyro) sensor that detects rotational motion. The gyro sensor detects angular velocity, allowing the angle or angular change of an object to be acquired. For example, if a gear or a gear and a toothed belt are used as a drive transmission mechanism for the wrist portion of the gripper 10, redundancy in the movement of the wrist portion of the gripper 10 can occur due to play between the gears or stretching of the toothed belt. Therefore, for example, during the operation of placing a product T gripped by the gripper 10 on a shelf SL11 or SL21 of a shelf SL10 or SL20, if the arm 20 is further operated with the product T in contact with the shelf SL11 or SL21, applying additional force to the gripper 10, some displacement occurs in the wrist portion of the gripper 10, causing a change in the attitude (i.e., angle) of the gripper 10. Therefore, by detecting such angle changes that may occur in the gripping portion 10 during the operation of placing the product T on the shelf SL11, SL21 using an IMU installed in the gripping portion 10, it is possible to detect that the product T has come into contact with the shelf SL11, SL21.

[0042] Furthermore, in this embodiment, by using an inertial measurement unit (IMU) as the contact detection sensor 30, it is possible to detect the relative height position of the PET bottled beverage to be grasped and the gripping portion 10 by detecting the pitch angle that occurs in the gripping portion 10 when the height of the gripping portion 10 is lowered and the gripping portion 10 is brought into contact with the vicinity of the neck of the PET bottled beverage in the series of actions when the gripping portion 10 grasps the cap portion Tb of the PET bottled beverage of the product T. Then, by adjusting the height of the gripping portion 10 from the state in which the gripping portion 10 is in contact with the vicinity of the neck of the PET bottled beverage, it becomes possible for the gripping portion 10 to grasp the cap portion Tb of the PET bottled beverage at an appropriate height position.

[0043] When a servo motor or the like is used to drive each joint of the arm unit 20, when an external force occurs that causes an angle deviation from the angle at which the gripping posture is maintained, the servo motor operates to zero the angle deviation in order to maintain the original angle. When the arm unit 20 is further moved with the product T gripped by the gripper 10 in contact with the shelf SL11 or SL21 of the shelf SL10 or SL20, a current is input to the servo motor to drive the servo motor against the load. Therefore, by detecting the current input to the servo motor for such operation with a motor input current sensor, it is possible to detect that the product T is in contact with the shelf SL11 or SL21. The motor input current sensor can be configured, for example, as the control unit 151 (see FIG. 7 ), which will be described later.

[0044] The two first cameras 50R, 50L are disposed on the left and right sides of the arm unit 20, respectively. The first camera 50L, attached to the first side surface 23a, which is the left side of the arm unit 20, faces in a first direction A1, which is perpendicular to the extension direction Ax of the arm unit 20 (see FIG. 6 ). The first camera 50L is mainly used to photograph the display shelf SL10. The first camera 50R, attached to the second side surface 23b, which is the right side of the arm unit 20 and parallel to the first side surface 23a, opposite the first side surface 23a, faces in a second direction A2, which is opposite to the first direction A1. The first camera 50R is mainly used to photograph the inventory shelf SL20. By arranging the first cameras 50R and 50L facing in opposite directions in this manner, the first cameras 50R and 50L can simultaneously photograph the display shelf SL10 and the inventory shelf SL20, respectively, while keeping the arm section 20 in the same position.

[0045] The performance of the first cameras 50R, 50L may be the same or different, but for simplicity, an example in which both cameras have the same performance will be described below. However, since the purpose and conditions for photographing the display shelf SL10 are different from the purpose and conditions for photographing the inventory shelf SL20, it is of course possible to use cameras with different performance to suit the respective purposes and conditions.

[0046] The first cameras 50R and 50L may include, for example, an imaging element that generates an image (e.g., an RGB image) in which pixels are arranged two-dimensionally, and a depth sensor that is a distance detection device that generates distance data. The depth sensor is not limited to a specific type as long as it can acquire distance data to an object. For example, a stereo lens type or a LiDAR (Light Detection and Ranging) type can be used. The depth sensor may generate, for example, a depth image. In another aspect of the present invention, one or both of the first cameras 50R and 50L may acquire distance data using, for example, an ultrasonic element.

[0047] Note that the first camera 50L is directed in a first direction A1, which means that the imaging direction of the imaging element and depth sensor of the first camera 50L is direction A1. Similarly, the first camera 50R is directed in a second direction A2, which means that the imaging direction of the imaging element and depth sensor of the first camera 50R is direction A2. The directions A1 and A2 do not necessarily have to be 180° opposite, but may be directions that allow images of the product display shelf SL10 and the inventory shelf SL20 to be captured.

[0048] One or both of the first cameras 50R, 50L may be provided on the grip portion 10. The first cameras 50R, 50L do not necessarily have to be provided on the same member; for example, the first camera 50R may be attached to one of the link members 21 to 23, and the first camera 50L may be attached to another of the link members 21 to 23. However, when the first cameras 50R, 50L are provided on the same member, a common coordinate system is used, which has the advantage of simplifying the calculations for image processing compared to when the cameras 50R, 50L are provided on separate link members.

[0049] The second camera 60 is used to capture an image of the gripper 10 gripping the product T, and the relative positional relationship between the product T gripped by the gripper 10 and the shelf boards SL11, SL21 of the shelves SL10, SL20. Similar to the first cameras 50R, 50L, the second camera 60 may have, for example, an imaging element that generates an image in which pixels are arranged two-dimensionally (an RGB image, for example), and a depth sensor that generates distance data.

[0050] As an example, the second camera 60 is installed in a position close to the gripping unit 10, below the link member 23 that is closest to the gripping unit 10 among the link members 21 to 23 of the arm 20, with the imaging direction of its imaging element and depth sensor directed directly below the link member 23 and the gripping unit 10 (the −z direction in FIGS. 5 and 6) or downward and forward (slightly further toward the +x direction than the −z direction in FIGS. 5 and 6). This enables the second camera 60 to capture images of at least the lower portion of the product T gripped by the gripping unit 10 and the shelves SL11 and SL21 located in front of the gripping unit 10.

[0051] The third camera 70 is a camera that changes direction in response to operation by, for example, a remote operator to photograph a predetermined object. The third camera 70 is attached to a part of the lifting mechanism 90, for example. The third camera 70 is capable of moving horizontally and vertically in the space between the display shelf SL10 and the inventory shelf SL20 in accordance with the operation of the horizontal movement mechanism 80 and the lifting mechanism 90. The part of the lifting mechanism 90 to which the third camera 70 is attached is rotatable about a support column 95, and the third camera 70 is configured to rotate left and right about the support column 95 in accordance with the rotation of that part, so that it can photograph the display shelf SL10 or the inventory shelf SL20 as needed.

[0052] The third camera 70 may be, for example, a stereo lens type. Although not limited thereto, the third camera 70 may have a wider angle of view than the first cameras 50R and 50L and the second camera 60.

[0053] The horizontal movement mechanism 80 has a base plate 81 and a drive mechanism (not shown). The base plate 81 supports the lifting mechanism 90 and allows it to slide along rails (not shown) laid between the display shelves SL10 and the inventory shelves SL20 in the store. The drive mechanism (not shown) includes a motor, rollers, etc., and operates based on control signals from the control device 150 (see FIG. 7 ) to move the lifting mechanism 90 to a predetermined position along the rails. The lifting mechanism 90 has a support column 95, a first lifting mechanism 91, and a second lifting mechanism 92. The support column 95 is fixed to the base plate 81 and extends vertically.

[0054] The first lifting mechanism 91 has a drive mechanism (not shown). The drive mechanism (not shown) includes a motor, a linear guide, etc., and operates based on a control signal from a control device 150 (see FIG. 7). By operating the drive mechanism (not shown), the first lifting mechanism 91 moves up and down in the vertical direction along a support column 95. An upper portion of the first lifting mechanism 91, to which the third camera 70 is attached, is configured to be rotated left and right around the support column 95.

[0055] The second lifting mechanism 92 is held by the first lifting mechanism 91. One end of the arm unit 20 is attached to the second lifting mechanism 92. The second lifting mechanism 92 has a drive mechanism (not shown). The drive mechanism (not shown) includes a motor, a linear guide, etc., and operates based on a control signal from the control device 150 (see FIG. 7). By operating the drive mechanism (not shown), the second lifting mechanism 92 also moves up and down in the vertical direction.

[0056] When grasping a product T that is located at a predetermined height, the lifting mechanism 90 uses the first lifting mechanism 91 to move the arm unit 20 and the gripping unit 10 to a height near where the product T can be grasped, and then uses the second lifting mechanism 92 to fine-tune the height of the arm unit 20 and the gripping unit 10.

[0057] In this embodiment, the first lifting mechanism 91 and the second lifting mechanism 92 are provided as the lifting mechanism, but in another aspect of the present invention, a configuration in which only one lifting mechanism is provided may be used.

[0058] 7, the control device 150 includes a control unit 151, a storage unit 160, an input unit 191, an output unit 193, and a communication unit 195. Although the control device 150 is depicted as a single element in FIG. 7, the control device 150 does not necessarily have to be a single physical element, and may be composed of multiple physically separated elements.

[0059] The input unit 191 is a device for receiving input from an operator. The input unit 191 may be configured with devices for inputting to a computer, such as a keyboard, a mouse, or a touch panel. The input unit 191 may also have a voice input device, such as a microphone. The input unit 191 may also have a gesture input device that recognizes and identifies the operator's movements through image recognition.

[0060] The output unit 193 is used by the product replenishment robot 100 to output an alert to a store clerk or the like, and is configured, for example, by one or a combination of a speaker, a display, a light-emitting device, and a vibration device. The communication unit 195 has the function of receiving data from an external device and the function of transmitting data to an external device. If the product replenishment robot 100 is configured to be remotely operable, the communication unit 195 receives input from the operator via the operation unit of an external device (not shown), and the control device 150 causes the product replenishment robot 100 to perform a predetermined operation based on the input. Note that communication between the operation unit of the external device and the communication unit 195 may be wired or wireless.

[0061] When product replenishment robot 100 is configured to be remotely operable, operation unit 191 may be a device worn by the operator. This device includes a display device (not shown) and an operation device (not shown). The display device may be, for example, a head-mounted display (HMD) having a display visible to the operator. The operation device may include, for example, one or more input sensors capable of detecting the movement of a body part (e.g., a hand or arm) of the operator.

[0062] Storage unit 160 includes temporary or non-temporary storage media such as ROM (Read Only Memory), RAM (Random Access Memory), and HDD (Hard Disk Drive). Storage unit 160 stores computer programs executed by control unit 151, trained models (described below), and the like. The computer programs stored in storage unit 160 include instructions for implementing a method for controlling commodity replenishment robot 100 by control unit 151 (described below with reference to FIG. 9 and the like).

[0063] The storage unit 160 has an acquired data storage unit 160a and a reference data storage unit 160b. The acquired data storage unit 160a stores, for example, image data captured by the cameras 50R, 50L, 60, and 70. The reference data storage unit 160b stores various data necessary for the operation of the product replenishment robot 100. This various data includes, for example, data on each shelf SL10 to SL30 (shape data, position data, lane coordinate data, etc.) and data on the product T (shape data, position data, surrounding image data, etc.).

[0064] The control unit 151 is configured with, for example, one or more central processing units (CPUs). The control unit 151 executes computer programs stored in the storage unit 160 to function as an operation control unit 152, an imaging control unit 153, and an image data processing unit 155.

[0065] The operation control unit 152 generates control signals for operating the gripping unit 10, the arm unit 20, the horizontal movement mechanism 80, the lifting mechanism 90, and each unit of the control device 150. The operation control unit 152 generates the control signals by referring to input signals from the operation unit 191 and various data stored in the storage unit 160. The control signals may be generated using the processing results of the image data processing unit 155. The operation control unit 152 also transmits and receives data via the communication unit 195, and performs predetermined output via the output unit 193.

[0066] The imaging control unit 153 controls the operation of each of the cameras 50R, 50L, 60, and 70. The imaging timing of each of the cameras 50R, 50L, 60, and 70 may be determined using data stored in advance in the reference data storage unit 160b, for example.

[0067] The image data processing unit 155 performs various information processing using the captured image data and distance data (depth data) captured by each of the cameras 50R, 50L, 60, and 70. As an example, the image data processing unit 155 performs image analysis of the image data captured by the first camera 50R to identify products on the inventory shelf SL20 and the product replenishment shelf SL30. The image data processing unit 155 has a replenishment possibility determination unit 155a and a grasping target identification unit 155b. The image data processing unit 155 also performs various information processing using captured image data captured by an image acquisition unit 260 of the product image acquisition device 200 (described later). As an example, the image data processing unit 155 performs processing to identify the type of product using image data of the outer periphery of the product captured by the image acquisition unit 260 (an image including a product label, barcode, etc. surrounding the product).

[0068] The replenishment possibility determination unit 155a determines, for example, whether a space Sp (see FIG. 8 ) exists behind the last product T lined up on the display shelf SL10 and the inventory shelf SL20, where an additional product can be placed, based on at least one of the image data captured by the first camera 50L and the distance data. FIG. 8 shows an image of the display shelf SL10 captured by the first camera 50L of the product replenishment robot 100. If a space Sp exists, this product T needs to be replenished. Therefore, if a space Sp exists, the replenishment possibility determination unit 155a sends a notification to the operation control unit 151 to notify the operation control unit 151 that the product T should be replenished on the shelf SL11 below the space Sp. Upon receiving this notification, the operation control unit 151 performs a replenishment operation for the product T.

[0069] The gripping target specifying unit 155b performs at least one of the following processes based on at least one of the image data captured by the first camera 50R and the distance data: determining whether or not there are replenishment target products to be gripped on the inventory shelf SL20 and the replenishment shelf SL30; specifying the size or shape of the replenishment target products; and determining the gripping position of the replenishment target products. In the case of a product T having a cap Tb as shown in FIG. 2, the gripping target specifying unit 155b sets the gripping position, for example, near the cap Tb. On the other hand, in the case of a product T such as a canned beverage without a cap, the gripping target specifying unit 155b may set the gripping position to the side of the container. (Product Replenishment Operation by the Product Replenishment Robot 100)

[0070] Next, a description will be given of the product replenishment operation by the product replenishment robot 100. An example in which products T from the inventory shelf SL20 are replenished to the display shelf SL10 will be described below. Fig. 9 is a flowchart showing the product replenishment operation by the product replenishment robot 100.

[0071] First, in step S11, the rear surface of the display shelf SL10 is photographed by the first camera 50L attached to the arm unit 20 of the product replenishment robot 100. The product replenishment robot 100 moves the arm unit 20, horizontal movement mechanism 80, and lifting mechanism 90 so that the first camera 50L can photograph each level of the display shelf SL10. Next, the product replenishment robot 100 operates the first camera 50L to capture an image of the rear surface of the display shelf SL10 and also acquire distance data to the products T lined up on the display shelf SL10.

[0072] Next, in step S12, a determination is made as to whether or not replenishment is possible. In the "determination of whether or not replenishment is possible," the replenishment possibility determination unit 155a in the image data processing unit 155 of the product replenishment robot 100 analyzes the captured image of the rear surface of the display shelf and determines which product T can be displayed on which shelf SL11 (in other words, which product T needs to be replenished). The product replenishment robot 100 acquires an image of the rear surface of the display shelf SL10 as shown in FIG. 6.

[0073] The image data processing unit 155 of the product replenishment robot 100 identifies the lane on the shelf SL11 to which the product T is to be displayed on the display shelf SL10 based on the image data captured by the first camera 50L. Furthermore, the image data processing unit 155 of the product replenishment robot 100 can acquire distance data to the product T (depth data Dep visualized in FIG. 8 ), and therefore can recognize that there is space Sp behind the product T, such as the product T in lanes "7" and "8," where the number of products displayed has decreased. The replenishment availability determination unit 155a of the product replenishment robot 100 determines whether there is space Sp for the product T based on the distance data to the product T, and determines whether more products T can be displayed in lanes "7" and "8." Step S12 determines which product T needs to be replenished on which shelf SL11 of the display shelf SL10.

[0074] The method for determining whether a product needs to be replenished is not limited to the above method, and various other methods can be used. For example, it may be possible to determine the percentage of the product T that is allocated within a predetermined three-dimensional space, and if this value is equal to or less than a predetermined reference value, it may be determined that the product T needs to be replenished.

[0075] Next, in step S13, the inventory shelf SL20 is photographed. The product replenishment robot 100 operates the arm unit 20, the horizontal movement mechanism 80, the lifting mechanism 90, and the first camera 50R to photograph the inventory shelf SL20 from the front side. As an example, the product replenishment robot 100 photographs the inventory shelf SL20 one shelf at a time to obtain a photographed image showing the inventory status of the products T. The photograph of the inventory shelf SL20 does not need to be taken after the photograph of the display shelf SL10; the photograph of the inventory shelf SL20 may be taken before the photograph of the display shelf SL10.

[0076] Next, in step S14, the image data processing unit 155 of the product replenishment robot 100 analyzes the captured image of the inventory shelf SL20 acquired in step S13 and identifies the products lined up on the inventory shelf SL20. In addition, the grasping target identification unit 155b of the image data processing unit 155 identifies the grasping position of the product. Through the steps up to this point, the product replenishment robot 100 acquires information indicating which product T needs to be replenished at which position on which shelf SL11 of the display shelf SL10, as well as information indicating the position on the inventory shelf SL20 of the product to be replenished corresponding to that product T and the grasping position of that product to be replenished.

[0077] Next, in step S15, the product replenishment robot 100 performs a product replenishment operation (pick-and-place operation) based on the acquired information. Specifically, the operation control unit 152 of the control unit 151 (see FIG. 7 ) of the product replenishment robot 100 operates the arm unit 20, horizontal movement mechanism 80, and lifting mechanism 90 to move the gripper 10 toward a predetermined product to be replenished on the inventory shelf SL20. The gripper 10 then grips the predetermined gripping position of the product to be replenished (for example, the cap Tb of the product T if the product T is a PET bottled beverage) and lifts the product. The operation control unit 152 then operates the arm unit 20, horizontal movement mechanism 80, and lifting mechanism 90 to move the gripped product T to a predetermined placement position on the display shelf SL10 and releases the product T from the gripper 10 to place the product T in the predetermined position. Thereafter, the product replenishment robot 100 repeats the same pick-and-place operation to complete the replenishment of the product T.

[0078] While the above describes an example of replenishing the display shelf SL10 with products T from the inventory shelf SL20, the replenishing operation when replenishing the inventory shelf SL20 with products T from the replenishing shelf SL30 is also performed according to the same process. In this case, as an example, the first camera 50L captures an image of the front of the inventory shelf SL20, and the image data processing unit 155 of the product replenishment robot 100 identifies the lane on the shelf SL21 to which the products from the inventory shelf SL20 are to be replenished based on the image data captured by the first camera 50L. Furthermore, the first camera 50R captures an image of the front of the replenishing shelf SL30, and the image data processing unit 155 of the product replenishment robot 100 identifies the position of each product placed on the replenishing shelf SL30 based on the image data captured by the first camera 50R. When replenishing products T from the product replenishment shelf to the inventory shelf SL20, the shelf board SL21 of the inventory shelf SL20 is first placed in a first position in which the front end (right side in the figure) is higher than the rear end (left side in the figure), and after replenishing is completed, the shelf board SL21 is placed in a second position in which the front end is lower than the rear end. <Configuration and operation of product image acquisition device 200>

[0079] 10 is a perspective view showing the overall configuration of the commodity image acquisition device 200 included in the commodity replenishment system 1 of this embodiment. The commodity image acquisition device 200 includes a turntable 210, a rotation drive unit 220, an elevation mechanism 230, an elevation drive mechanism 240, an elevation drive unit 250, and an image acquisition unit 260.

[0080] The turntable 210 is configured to be rotatable about a central axis, and can rotate about the central axis together with a product placed thereon. The rotation drive unit 220 includes a mechanism and a drive source for rotationally driving the turntable 210, and the drive source is, for example, an electric motor. The turntable 210 may be configured to be directly coupled to the rotation shaft of an electric motor of the drive source and be rotationally driven, or may be configured to be rotationally driven by receiving the rotational driving force of the electric motor via a drive transmission means such as a gear.

[0081] The lifting mechanism 230 is a mechanism that extends and retracts the rotating table 210 to raise and lower the rotating table 210, and is formed, for example, of a scissors-type link mechanism that extends and retracts in the vertical direction. The lifting drive mechanism 240 is a drive mechanism for raising and lowering the lifting mechanism 230, and is configured, for example, to reciprocate one end of the lower side of the scissors mechanism of the lifting mechanism 230 in a direction toward and away from the other end. The lifting drive unit 250 includes a mechanism and a drive source that drives the lifting drive mechanism 240, and the drive source is, for example, an electric motor.

[0082] The main body of commodity image acquisition device 200, which is made up of turntable 210, rotation drive unit 220, lifting mechanism 230, lifting drive mechanism 240, and lifting drive unit 250, is installed, for example, on base plate 81 of commodity replenishment robot 100. Furthermore, rotation drive unit 220 and lifting drive unit 250 are connected by wired or wireless connection to communication unit 195 of control device 150 of commodity replenishment robot 100, and control device 150 controls the operation of these drive units (e.g., electric motors).

[0083] Image acquisition unit 260 has a function of acquiring an image of the outer periphery of a product placed on turntable 210, and includes, for example, an imaging element that generates a captured image in which pixels are arranged two-dimensionally (an RGB image, for example). Image acquisition unit 260 is installed, for example, on lifting mechanism 90 that forms the housing of product replenishment robot 100.

[0084] The image acquisition unit 260 captures an image of a product T held by the gripper 10 of the commodity replenishment robot 100 while the product T is positioned in front of the image acquisition unit 260 (the image acquisition position for the image acquisition unit 260), or while a product placed on the turntable 210 is being rotated by the rotation drive unit 220. The image acquisition unit 260 then captures an image of the product's side or outer periphery (including product labels, barcodes, etc., surrounding the product) and acquires the image data. The image acquisition unit 260 then transmits the acquired image data to the control device 150 of the commodity replenishment robot 100 via the communication unit 195 of the control device 150. Upon receiving the image data of the product, the image data processing unit 155 of the control device 150 performs processing to identify the type of the product using the image data of the product's side or outer periphery (including product labels, barcodes, etc., surrounding the product), as described above. In this manner, the commodity image acquisition device 200 and the control device 150 function as a commodity identification system for identifying the type of product held by the commodity replenishment robot 100. Furthermore, by mounting the product image acquisition device 200 on the product replenishment robot 100, the product can be placed at the image acquisition position by the image acquisition unit 260 simply by operating the arm unit 20 after the product replenishment robot 100 has grasped the product.

[0085] (Product Image Acquisition and Identification Operations by Product Image Acquisition Device 200) FIG. 11 is a diagram showing a series of operations for acquiring an image of a product T by the product image acquisition device 200 and identifying the product T.

[0086] In the commodity replenishment system 1, when an image of the commodity T is acquired by the commodity image acquisition device 200 to identify the commodity T, as shown in FIG. 11( a), the commodity T is grasped by the gripper 10 of the commodity replenishment robot 100 and moved above the turntable 210 of the commodity image acquisition device 200 to a height position where the image acquisition unit 260 of the commodity image acquisition device 200 can capture an image of the side of the commodity T. The image acquisition unit 260 then captures an image of the side of the commodity T that faces the image acquisition unit 260. The image data of the commodity T captured by the image acquisition unit 260 is transmitted to the control device 150 of the commodity replenishment robot 100 via the communication unit 195 of the control device 150, as described above. The control device 150 then performs a process to identify the type of the commodity T based on the image data (first commodity identification process). For example, the control device 150 can identify the type of the commodity T by comparing the image data transmitted to the control device 150 with image data of various commodity surroundings stored in the memory unit 160 of the control device 150.

[0087] In the first product identification process, an image of one side of a product T that has been gripped by the gripper 10 of the product replenishment robot 100 and placed opposite the image acquisition unit 260 is captured, and a process is performed to identify the type of product T based on the image data. Since it is arbitrary which part of the side of the product T gripped by the gripper 10 faces the image acquisition unit 260 and is imaged, in some cases a side of the product T other than the front of the label may be imaged, making it impossible for the control device 150 to identify the product T. In such cases, the second imaging operation described below is performed to acquire image data of the entire periphery of the side of the product T.

[0088] In the example shown in FIG. 11( a), the vertical length of the product T is relatively short. As a result, when the product T is placed at a height where the image acquisition unit 260 can capture an image, the bottom surface of the product T does not contact the turntable 210 of the product image acquisition device 200, and a gap is created between the bottom surface of the product T and the turntable 210. In this case, as shown in FIG. 11( b), the lifting mechanism 230, which has been retracted to its lowest position, is extended by the lifting drive mechanism 240 and the lifting drive unit 250 until the turntable 210 contacts the bottom surface of the product T. This places the bottom surface of the product T on the turntable 210. The contact of the turntable 210 with the bottom surface of the product T can be detected based on the load (e.g., overcurrent applied to the electric motor) generated in the lifting drive unit 250 when the turntable 210 contacts the bottom surface of the product T. In addition, when the product T grasped by the gripping unit 10 of the product replenishment robot 100 is moved above the turntable 210 to a height position where the image acquisition unit 260 can capture an image, if the bottom surface of the product T is placed on the turntable 210, the operation shown in Figure 11 (b) will not be executed.

[0089] After the bottom surface of the product T is placed on the turntable 210, the gripping unit 10 temporarily releases the product T, the turntable 210 is rotated by the rotation drive unit 220, and an image of the periphery of the side of the product T rotating with the turntable 210 is captured by the image acquisition unit 260. As described above, the image data of the product T captured by the image acquisition unit 260 is transmitted to the control device 150 of the product replenishment robot 100 via the communication unit 195 of the control device 150, and the control device 150 performs a process to identify the type of the product T based on the image data (second product identification process). In the second product identification process, a process to identify the type of product T is performed based on an image of the periphery of the side of the product T, resulting in higher accuracy of product identification compared to the first product identification process. <Configuration of the management device 300>

[0090] 12, the management device 300 includes a processor 310, a storage unit 320, an input unit 330, a display unit 340, and a communication unit 350. Although the management device 300 is depicted as a single element in FIG. 12, the management device 300 does not necessarily have to be a single physical element, and may be composed of multiple physically separated elements.

[0091] The input unit 330 is a device for receiving input from an operator of the management device 300. The input unit 330 may be configured with a keyboard, a mouse, and a touch panel. The display unit 340 is a display device that displays a display screen generated by the processor 310, and may be, for example, a liquid crystal display or an organic EL display device. The communication unit 350 communicates with the control device 150 and terminal device 400 of the commodity replenishment robot 100 via wired or wireless communication, and inputs and outputs information, control signals, and the like between these components.

[0092] The storage unit 320 includes a temporary or non-temporary storage medium such as a read-only memory (ROM), a random access memory (RAM), a hard disk drive (HDD), or a solid state drive (SSD). The storage unit 320 stores a computer program executed by the processor 155. The computer program stored in the storage unit 320 includes instructions for performing information processing by the processor 310, which will be described later with reference to FIG. 14 and the like. The storage unit 160 also at least temporarily stores information received from the control device 150 or the terminal device 400, and various data (including intermediate data) generated by processing operations by the processor 310.

[0093] The storage unit 320 also stores, for example, past sales data for each product at each store. The sales data includes information such as the sales date and time and the quantity of each product. This sales data can be used as statistical information for predicting which products should be replenished on which time schedule on the display shelves SL10 and inventory shelves SL20 of each store. The storage unit 320 may also store a trained model generated by performing machine learning using such sales data with a learning device. This trained model can be used as a sales prediction model for generating a schedule for replenishing products in a store.

[0094] The processor 310 is configured, for example, with one or more CPUs (Central Processing Units). The processor 310 operates to realize the processes and functions performed by the processor 310 as described with reference to Fig. 14. In particular, the processor 310 uses past sales data or a learned sales prediction model stored in the storage unit 320 to predict which products should be replenished on which time schedule on the display shelves SL10 and inventory shelves SL20 of each store, and based on the prediction, schedules the types and quantities of products to be replenished on the inventory shelves SL20 and the timing of replenishment, thereby generating product replenishment information.

[0095] The product replenishment information includes loading layout information regarding products to be replenished from the product replenishment shelf SL30 to the inventory shelf SL20 at a certain replenishment timing, including the loading position and loading quantity of each product in each area on the product replenishment shelf SL30 where the products are loaded. The loading layout information may include shelf designation information that designates the product replenishment shelf SL30 on which the products should be loaded.

[0096] The generated product replenishment information is transmitted from the communication unit 350 to the control device 150 and the terminal device 400. The terminal device 400, which is the user of the terminal device 400, displays the received loading layout information on its display screen. A store clerk places designated quantities of each type of designated product in each designated area on a specific designated product replenishment shelf SL30 in accordance with the loading layout information displayed on the terminal device 400. <Configuration of the terminal device 400>

[0097] 13, the terminal device 400 includes a processor 410, a storage unit 420, a display operation unit 430, a communication unit 440, and a camera 450. Note that this configuration is an example, and other configurations may be included.

[0098] The processor 410 controls the overall operation of the terminal device 400 by executing a program stored in the storage unit 420, and is configured from a CPU, a GPU, etc. The processor 410 operates to realize the processes and functions performed by the processor 410, as will be described with reference to FIG.

[0099] The storage unit 420 stores various control processes, programs for the processor 410 to execute each function, input data, etc., and is composed of RAM, ROM, and other storage. The storage unit 420 also temporarily stores the content of communications with other terminals. The storage unit 420 particularly stores a program for causing the processor 410 to execute the processes performed by the terminal device 400, which will be described with reference to FIG. 14.

[0100] The display operation unit 430 is a user interface used by the user of the terminal device 400 to input instructions and to display text, images, etc. in accordance with the display processing of the processor 410, and is composed of a touch panel, etc., if the terminal device 400 is a tablet terminal, etc.

[0101] The communication unit 440 is a communication interface for communicating with the control device 150 and the management device 300 via a network.

[0102] Camera 450 is primarily used to read labels affixed to product replenishment shelves SL30, which contain characters, two-dimensional codes, and the like as identification information for identifying product replenishment shelves SL30. Storage unit 420 also stores a character recognition program that recognizes characters, two-dimensional codes, and the like read by camera 450, and a program that decodes codes read by camera 450.

[0103] The processor 410 displays the product replenishment information received from the management device 300 on its display operation unit 430. A store clerk places designated quantities of various designated products in designated positions on a designated specific product replenishment shelf SL30 in accordance with the loading layout information displayed on the terminal device 400. The store clerk photographs the identification labels attached to the designated specific product replenishment shelf SL30 with the camera 450 of the terminal device 400. The processor 410 transmits image data of the identification labels photographed by the camera 450 to the management device 300 via the communication unit 440. <Operation Example>

[0104] Next, an example of the operation of the product replenishment system 1 of this embodiment will be described. Fig. 14 is a flowchart for explaining an example of the operation of the product replenishment system 1 of this embodiment.

[0105] First, in step S21, the processor 310 in the management device 300 of the product replenishment system 1 uses past sales data or a learned sales forecast model stored in the memory unit 320 of the management device 300 to predict the time schedule for replenishing each product on the display shelves SL10 and inventory shelves SL20 of each store, and based on this prediction, schedules the type and quantity of products to be replenished on the inventory shelves SL20 of the target store where the products will be replenished, as well as the timing of the replenishment, to generate product replenishment information.

[0106] The product replenishment information includes loading layout information regarding products to be replenished from the product replenishment shelf SL30 to the inventory shelf SL20 at a certain replenishment timing, including the loading position and loading quantity of each product in each area on the product replenishment shelf SL30 on which the products are loaded. The loading layout information may include information specifying the product replenishment shelf SL30 on which the products should be loaded.

[0107] The generated product replenishment information is transmitted from the communication unit 350 to the control device 150 and the terminal device 400. Therefore, the control device 150 of the product replenishment robot 100 recognizes, based on the product replenishment information including the stacking layout information, when the products T placed on the specified product replenishment shelf SL30 in accordance with the stacking layout information should be replenished to the inventory shelf SL20. Furthermore, upon receiving the product replenishment information, the terminal device 400 displays the product replenishment information on the display operation unit 430, allowing the store clerk using the terminal device 400 to recognize the product replenishment information.

[0108] Next, in step S22, the processor 410 of the terminal device 400 executes a process of displaying the product replenishment information transmitted from the management device 300 in step S21 on the display operation unit 430 of the terminal device 400 and receiving a selection input of the product replenishment work by the store clerk on the display operation unit 430. The processor 410 of the terminal device 400 generates a display screen to be displayed on the display operation unit 430 based on the product replenishment information transmitted from the management device 300.

[0109] Fig. 15 is a diagram showing an example of a display screen of product replenishment information displayed on the display operation unit 430 of the terminal device 400. In the example shown in Fig. 15, display items of product replenishment information include "RESTOKING ID," "DATE ​​& TIME," "SHELF ID," and "STATUS."

[0110] "RESTOKING ID" is information that identifies each scheduled replenishment operation. "DATE ​​& TIME" is information that indicates the date and time when the replenishment shelf SL30 loaded with the replenishment target products will be placed at the replenishment position within the store.

[0111] "SHELF ID" is information for identifying the product replenishment shelf SL30 used for the product replenishment work specified by the "RESTOKING ID." In the example shown in Fig. 15, one of two product replenishment shelves SL30 with IDs "1" and "2" is specified corresponding to each "RESTOKING ID."

[0112] "STATUS" indicates information instructing the store staff on the work to be performed on the replenished shelf SL30. In this example, "STATUS" displays four types of work instruction information: "PREPARE," "LOAD," "LOADED," and "UNLOAD." "PREPARE" is work instruction information for placing various products on the replenished shelf SL30 having the "SHELF ID" specified by the "RESTOKING ID" in accordance with the loading layout information. "LOAD" is placement work instruction information for placing the replenished shelf SL30, on which various products have been placed in response to the "PREPARE" instruction, in a predetermined position within the store. "LOADED" is information indicating the state in which the replenished shelf SL30, on which various products have been placed, has been placed in a predetermined position within the store. "UNLOAD" is removal work instruction information for removing a replenishment shelf SL30 that has become emptied of products after the replenishment of products from the replenishment shelf SL30 to the inventory shelf SL20 by the product replenishment robot 100 from a predetermined position in the back yard of the store. The display portions of "STATUS" such as "PREPARE," "LOAD," "LOADED," and "UNLOAD" can be selected and operated by the user by clicking on the display operation unit 430 when activated.

[0113] In the example shown in FIG. 15 , four product replenishment tasks are scheduled, and the "STATUS" for the two most recent replenishment tasks is an active "PREPARE" status that requests the placement of products on each replenishment shelf SL30. The display color of the active "PREPARE" status may change, for example, from blue to orange to red as the time approaches, such as more than 30 minutes, 30 minutes, and 15 minutes before the "DATE ​​& TIME" at which the replenishment shelf SL30 should be placed at a replenishment location in the store. This visually prompts the store staff to begin the task of placing products on the replenishment shelf SL30.

[0114] On the other hand, for product replenishment work for which the timing to place products on the product replenishment shelf SL30 has not yet arrived, the "STATUS" is inactive (disabled) and "PREPARE." Therefore, for product replenishment work for which the timing to place products on the product replenishment shelf SL30 has not yet arrived, the "PREPARE" display portion cannot be selected.

[0115] As shown in FIG. 15 , the store clerk performs a selection operation by clicking on the display portion M1 of "PREPARE" in the active state of "STATUS" corresponding to the "RESTOKING ID" for which the replenishment work is to be performed, from among the replenishment information displayed on the display operation unit 430 of the terminal device 400. This causes the processor 410 of the terminal device 400 to activate the camera 450. The store clerk then operates the terminal device 400 to photograph, with the camera 450 of the terminal device 400, the identification information (a label containing characters, a two-dimensional code, etc.) attached to the replenishment shelf SL30 designated by the "SHELF ID" information associated with the "RESTOKING ID." FIG. 16 shows the identification information of the replenishment shelf SL30 photographed by the camera 450 of the terminal device 400, displayed on the display operation unit 430 of the terminal device 400. The "RESTOKING ID" information associated with the selected "PREPARE" and image data of the identification information attached to the product replenishment shelf SL30 photographed by camera 450 are transmitted from terminal device 400 to management device 300.

[0116] Next, in step S23, the processor 310 of the management device 300 transmits the placement layout information associated with the "RESTOKING ID" selected in step S22 to the terminal device 400, and causes the display operation unit 430 to display it.

[0117] The processor 310 of the management device 300 performs character recognition processing or code decoding processing on image data of identification information (a label containing characters, a two-dimensional code, or the like) photographed by the camera 450. The processor 310 of the management device 300 determines whether the identification information photographed by the camera 450 matches the "SHELF ID" associated with the "RESTOKING ID" selected as described above for the product replenishment operation. If the processor 310 determines that the two match, it causes the display operation unit 430 of the terminal device 400 to display the placement layout information included in the product replenishment information associated with the "RESTOKING ID." Furthermore, the processor 310 changes the display portion M1 of "STATUS" associated with the selected "RESTOKING ID" from "PREPARE" to "LOAD" in the product replenishment information displayed on the display operation unit 430 of the terminal device 400. On the other hand, if the processor 310 determines that the two do not match, it displays a message such as "This is not the product replenishment shelf to be worked on" on the display operation unit 430 for a certain period of time, and then returns the display on the display operation unit 430 to the display screen shown in Figure 15.

[0118] 17 is a diagram showing an example of a display screen of placement layout information displayed on the display operation unit 430 of the terminal device 400. The placement layout information displayed on the display operation unit 430 indicates the types and numbers of products to be placed in each area (areas 1 to 10 in the example shown in FIG. 17 ) divided on each shelf of the product replenishment shelf SL30 on which products are to be placed.

[0119] The store clerk retrieves various products stored in the store's product storage area (usually at room temperature) and places the specified number of products in the designated areas on the product replenishment shelf SL30 according to the placement layout information displayed on the display operation unit 430 of the terminal device 400 as shown in Figure 17. As an example, when placing only one delivery package of a product in a designated area, the upper and front portions of the delivery package can be removed, and the remaining part of the delivery package, including the products inside, can be placed in the designated area on the product replenishment shelf SL30 (e.g., areas 1 to 6 and 8 shown in Figure 17). Alternatively, multiple types of products may be placed together in a designated area (e.g., area 7 shown in Figure 17).

[0120] When various products are placed on the product replenishment shelf SL30 in accordance with the placement layout information, the product placement work on the product replenishment shelf SL30 is completed, and the product replenishment shelf SL30 is ready to be placed in a predetermined position within the store.

[0121] 17 , on the display screen of the placement layout information displayed on the display operation unit 430 of the terminal device 400, the display portion of each product can be selected by clicking on the display operation unit 430. For example, if a product specified in the placement layout information is out of stock, such as when the product is not available in the product storage location in the store or when fewer than the specified number of products are stored, the store clerk selects the product by clicking on the product on the display screen of the placement layout information displayed on the display operation unit 430. The processor 410 of the terminal device 400 then displays a column for accepting changes to the number of products for that product on the display operation unit 430 and accepts numerical input on the display operation unit 430. When the store clerk inputs the numerical value of the number of products that can be placed on the display operation unit 430, the processor 410 of the terminal device 400 transmits numerical information about the product to the management device 300. When the processor 310 of the management device 300 receives the numerical information about the product, it updates the number of units of the product in the placement layout information according to the numerical value, and sends product replenishment information including the updated placement layout information to the control device 150.

[0122] Next, in step S24, the processor 310 of the management device 300 transmits placement work start information to the terminal device 400, prompting the terminal device 400 to place the product restocking shelf SL30 with the "SHELF ID" associated with the "RESTOKING ID" selected in step S22 at a predetermined position within the store, and causes the display operation unit 430 to display this information.

[0123] FIG. 18 is a diagram showing an example of work start information displayed on the display operation unit 430 of the terminal device 400. In the example shown in FIG. 18 , the work start information displays a relatively large display portion M2 on the display operation unit 430, displaying "LOAD CASE SHELF #1," a message urging the user to place the restocked shelf SL30 with a "SHELF ID" of #1 in its designated location within the store. The work start information may also include a countdown timer display indicating the remaining time until the restocked shelf SL30 is to be placed in its designated location within the store. In the example shown in FIG. 18 , "14:53" is displayed, indicating a remaining time of 14 minutes and 53 seconds. The work start information is displayed on the display operation unit 430 of the terminal device 400 a predetermined time (e.g., 15 minutes) before the time the restocked shelf SL30 is to be placed in its designated location within the store. The work start information may initially be displayed in a first color (e.g., green) and then gradually change to a second color (e.g., red) as the remaining time becomes shorter. This makes it possible to visually prompt the store clerk using the terminal device 400 to place the product replenishment shelf SL30 in the predetermined position by the specified time.

[0124] Next, in step S25, the processor 310 of the management device 300 receives information from the terminal device 400 indicating that the store clerk has confirmed the placement work start information prompting the store clerk to place the product replenishment shelf SL30 in a designated location within the store.

[0125] Display portion M2 of the work start information displayed on display operation unit 430 can be clicked by a user on display operation unit 430. When a store clerk performs an operation of clicking display portion M2 of the placement work start information displayed on display operation unit 430 of terminal device 400 as shown in Fig. 18 , processor 410 of terminal device 400 transmits information indicating that the work start information has been confirmed by the user to management device 300, and management device 300 receives this information.

[0126] Next, in step S26, upon receiving information indicating that the placement work start information has been confirmed by the user, processor 310 of management device 300 transmits an instruction to control device 150 of product stocking robot 100 to move product stocking robot 100 to its home position. Upon receiving this instruction, control device 150 interrupts the product stocking operation being performed by product stocking robot 100 at that time, for example, from inventory shelf SL20 to display shelf SL10, and moves product stocking robot 100 to its home position. The home position is, for example, a position near one end of rail R shown in FIG. 2 .

[0127] Furthermore, when processor 310 of management device 300 receives information indicating that the placement work start information has been confirmed by the user, processor 410 of terminal device 400 activates camera 450. The store clerk then operates terminal device 400 to photograph, with camera 450 of terminal device 400, the identification information (a label bearing characters, a two-dimensional code, etc.) attached to the replenishment shelf SL30 designated by the placement work start information. The reason why the store clerk is required to photograph the identification information of the replenishment shelf SL30 designated by the placement work start information is to ensure that the store clerk places products on the designated replenishment shelf SL30 and arranges the replenishment shelf SL30 in its designated position within the store without error. Image data of the identification information attached to the replenishment shelf SL30 photographed by camera 450 is transmitted from terminal device 400 to management device 300.

[0128] After performing the above operation on the terminal device 400, the store clerk moves and places the product replenishment shelf SL30 in a predetermined position within the store. The predetermined position is, for example, a position near the other end of the rail R shown in FIG. 2 , i.e., a position near the end of the rail R on the opposite side from the home position of the product replenishment robot 100. By moving to the other end of the rail R, the product replenishment robot 100 can grasp and remove products from the product replenishment shelf SL30 placed in the predetermined position.

[0129] Next, in step S27, the processor 310 of the management device 300 transmits completion confirmation request information to the terminal device 400, which requests confirmation that the product replenishment shelf SL30 with the "SHELF ID" instructed in the placement work start information has been placed in the designated position within the store, and displays the information on the display operation unit 430.

[0130] The processor 310 of the management device 300 performs character recognition or code decoding on the image data of the identification information (a label containing characters, a two-dimensional code, or the like) of the replenishment shelf SL30 captured by the camera 450 and transmitted to the management device 300 in step S26. The processor 310 determines whether the identification information captured by the camera 450 matches the "SHELF ID" associated with the "RESTOKING ID" selected for the replenishment work. If the processor 310 determines that the two match, it transmits completion confirmation request information to the terminal device 400, requesting confirmation that the replenishment shelf SL30 with the "SHELF ID" specified in the placement work start information has been placed in its designated position within the store, and displays the completion confirmation request information on the display operation unit 430. On the other hand, if the processor 310 determines that the two do not match, it displays a message such as "This is not the replenishment shelf to be worked on" on the display operation unit 430 for a certain period of time, and then returns the display on the display operation unit 430 to the display screen shown in FIG. 18 .

[0131] 19 is a diagram showing an example of completion confirmation request information displayed on the display operation unit 430 of the terminal device 400. In the example shown in Fig. 19, the completion confirmation request information is displayed on the display operation unit 430 as a relatively large display portion M3 showing "LOADED," which is a display for accepting confirmation that the product restock shelf SL30 has been placed in a predetermined position within the store. The display portion M3 of the completion confirmation request information displayed on the display operation unit 430 can be clicked by the user on the display operation unit 430.

[0132] Next, in step S28, the processor 310 of the management device 300 receives from the terminal device 400 completion confirmation information that the product replenishment shelf SL30 with the "SHELF ID" specified in the placement work start information has been placed in the specified position within the store.

[0133] When the store clerk has moved and placed the specified product replenishment shelf SL30 in the prescribed position within the store as described above, he or she clicks on the display portion M3 of the completion confirmation request information displayed on the display operation unit 430 of the terminal device 400 as shown in Figure 19. The processor 410 of the terminal device 400 then transmits completion confirmation information to the management device 300 indicating that the product replenishment shelf SL30 with the "SHELF ID" specified in the work start information has been placed in the prescribed position within the store, and the management device 300 receives this information.

[0134] Next, in step S29, processor 310 of management device 300 sends an instruction to control device 150 of product replenishment robot 100 to cause product replenishment robot 100 to replenish products on product replenishment shelves SL30 located at predetermined positions within the store onto inventory shelves SL20. Upon receiving the completion confirmation information, processor 310 of management device 300 further changes display portion M1 of "STATUS" associated with the "RESTOKING ID" performing the product replenishment work from "LOAD" to "LOADED" in the product replenishment information displayed on display operation unit 430 of terminal device 400.

[0135] When the processor 310 of the management device 300 receives the completion confirmation information, it sends an instruction to the control device 150 of the product replenishment robot 100 to cause the product replenishment robot 100 to move the product on the product replenishment shelf SL30 located at a predetermined position in the store onto the inventory shelf SL20. Upon receiving the instruction, the control device 150 operates the product replenishment robot 100, and the product replenishment robot 100 performs the operation of moving the product on the product replenishment shelf SL30 onto the inventory shelf SL20.

[0136] As described above, the product replenishment information including the placement layout information is transmitted from the management device 300 to the control device 150, so the control device 150 has information regarding how many of each product are placed in which area on the product replenishment shelf SL30. The control device 150 also has information regarding which product should be replenished in which lane on the inventory shelf SL20. Therefore, the control device 150 causes the product replenishment robot 100 to repeatedly grasp and remove a specific product from the product replenishment shelf SL30 and move that product to the corresponding lane on the corresponding inventory shelf SL20, thereby replenishing the products on the product replenishment shelf SL30 to the inventory shelf SL20.

[0137] The operation of replenishing products from the product replenishment shelf SL30 to the inventory shelf SL20 may be performed, for example, sequentially for each area in the placement layout described with reference to Fig. 14 for all products in each area, or may be performed by repeatedly replenishing one or several products in each area in the placement layout for each area. Explaining the latter more specifically, as an example, the operation of replenishing three products in each area may be performed sequentially from area 1 to area 8, and this may be repeated multiple times until all products on the product replenishment shelf SL30 are used up.

[0138] The operation of the product replenishment robot 100 to replenish products T from the replenishment shelf SL30 to the stock shelf SL20 is similar to the operation described with reference to Figure 9, and therefore a description thereof will be omitted here. However, as described above, when replenishing products T from the replenishment shelf SL30 to the stock shelf SL20, the drive mechanism of the stock shelf SL20 puts the shelf SL21 of the stock shelf SL20 into a first position in which the front end is higher than the rear end, in accordance with a control signal transmitted from the communication unit of the control device 150 of the product replenishment robot 100 when starting the operation of replenishing products T from the replenishment shelf SL30 to the stock shelf SL20. As a result, products placed on the front side of a predetermined product row on the shelf SL21 for replenishment slide down the slope from the front side to the rear side toward the rear side of the shelf SL21, and therefore, by repeating the operation of placing products on the front side of the shelf SL21, products can be sequentially replenished to the product row on the shelf SL21.

[0139] Furthermore, after the replenishing operation from the product replenishment shelf SL30 to the inventory shelf SL20 is completed, the drive mechanism of the inventory shelf SL20 puts the shelf SL21 of the inventory shelf SL20 into a second position in which the front end is lower than the rear end in accordance with a control signal transmitted from the communication unit of the control device 150 after the replenishing operation is completed. As a result, the products placed in each product row of the shelf SL21 slide down toward the front side of the shelf SL21 along the slope that slopes downward from the rear side to the front side, so that each time the product located at the front of the front side of the inventory shelf SL20 is removed from the inventory shelf SL20 by the product replenishment robot 100 for replenishment, the products behind it slide down toward the front side of the shelf SL21 in sequence, so that the product replenishment robot 100 can remove the product located at the front of the front side of the inventory shelf SL20 and move it to the display shelf SL10 for replenishment.

[0140] Furthermore, in the operation of the product replenishment robot 100 to replenish products from the product replenishment shelf SL30 to the inventory shelf SL20, an image of the product T may be acquired by the product image acquisition device 200 described with reference to Figures 10 and 11, etc., and the image data of the product T may be transmitted to the control device 150, and the control device 150 may then execute a process (the first and second product identification processes described above) to identify the type of product T based on the image data. The product identification process may be executed for all products taken out by the product replenishment robot 100 from the product replenishment shelf SL30, or may be executed for some of the products on the product replenishment shelf SL30.

[0141] To explain an example with reference to the placement layout information shown in Figure 17, since each of areas 1 to 6 and 8 of the product replenishment shelf SL30 contains a single product in a box, there is little chance that a store clerk will mistakenly place the wrong product during placement. Therefore, the product identification process may be omitted for products placed in these areas, or the product identification process may be performed only on the first product. On the other hand, since multiple types of products are mixed in area 7 of the product replenishment shelf SL30, it is preferable to perform the product identification process individually for all products placed in area 7.

[0142] Next, in step S30, when the product replenishment robot 100 completes the product replenishment operation from the product replenishment shelf SL30 to the inventory shelf SL20, the control device 150 sends information indicating that the product replenishment operation has been completed to the management device 300, and the management device 300 receives the information.

[0143] Next, in step S31, the processor 310 of the management device 300 transmits removal work start information to the terminal device 400, prompting the removal of the product restocking shelf SL30 located at a predetermined position within the store from the predetermined position, and displays it on the display operation unit 430.

[0144] When the processor 310 of the management device 300 receives information from the terminal device 400 indicating that the product restocking operation has been completed, the processor 310 changes the display portion M1 of "STATUS" associated with the "RESTOKING ID" that performed the product restocking operation from "LOADED" to "UNLOAD" in the product restocking information displayed on the display operation unit 430 of the terminal device 400. This "UNLOAD" corresponds to removal operation start information that prompts the removal of the product restocking shelf SL30 that is located in a predetermined position in the store from its predetermined position.

[0145] Next, in step S32, the processor 310 of the management device 300 receives removal confirmation information from the terminal device 400 indicating that the product restocking shelf SL30 with the "SHELF ID" associated with the "RESTOKING ID" selected as described above for the product restocking operation has been removed from its predetermined location.

[0146] When a store clerk clicks on the "UNLOAD" display portion displayed on the display operation unit 430 of the terminal device 400, the processor 410 of the terminal device 400 transmits information indicating that the user has operated "UNLOAD" to the management device 300, and the management device 300 receives the information. This information corresponds to confirmation information that the product replenishment shelf SL30 with the "SHELF ID" associated with the "RESTOKING ID" selected as described above for product replenishment work has been removed from its predetermined location.

[0147] Next, in step S33, the processor 310 of the management device 300 transmits to the control device 150 of the product replenishment robot 100 an instruction to move the product replenishment robot 100 to its home position.

[0148] Upon receiving the instruction, the control device 150 moves the product stocking robot 100 to the home position. While the product stocking robot 100 is moving to the home position, the processor 310 of the management device 300 may display a warning message M4 as shown in Fig. 20 on the display operation unit 430 of the terminal device 400. In the example shown in Fig. 20, while the product stocking robot 100 is moving to the home position, a message is displayed prohibiting store staff from entering the operating area of ​​the product stocking robot 100 between the display shelf SL10 and the inventory shelf SL20.

[0149] When product replenishment robot 100 moves to the home position, information indicating that product replenishment robot 100 has moved to the home position is sent from control device 150 to management device 300, and processor 310 of management device 300 causes this message display on display operation unit 430 of terminal device 400 to end. This allows store staff to remove product replenishment shelf SL30 from its designated location within the store where it was located, and, if necessary, enter the operating area of ​​product replenishment robot 100 between display shelf SL10 and inventory shelf SL20 (which is normally a low-temperature environment due to refrigeration of products) to perform work.

[0150] Finally, in step S34, the processor 310 of the management device 300 sends to the control device 150 an instruction to cause the product replenishment robot 100 to resume the operation of replenishing products from the inventory shelf SL20 to the display shelf SL10.

[0151] After the store clerk removes the replenishment shelf SL30 from its designated location in the store, for example, by again clicking on the "UNLOAD" display portion M1 displayed on the display operation unit 430 of the terminal device 400, the processor 410 of the terminal device 400 sends information to the management device 300 indicating that the user has again operated "UNLOAD." The processor 310 of the management device 300 then sends to the control device 150 an instruction to cause the product replenishment robot 100 to resume replenishment operations from the inventory shelf SL20 to the display shelf SL10. Upon receiving this instruction, the control device 150 causes the product replenishment robot 100 to resume replenishment operations from the inventory shelf SL20 to the display shelf SL10. The product replenishment operations by the product replenishment robot 100 from the inventory shelf SL20 to the display shelf SL10 are continued until the next scheduled replenishment operation for the inventory shelf SL20 is started.

[0152] As described above, in the product replenishment system according to this embodiment, a placement layout including the various products and the numbers of products to be placed on the replenishment shelf SL30, which holds the products to be replenished on the inventory shelf SL20, is generated by the management device 300 in accordance with a schedule for replenishment of the inventory shelf SL20, and is presented to a store clerk via the terminal device 400. The store clerk places the various products on the replenishment shelf SL30 in accordance with the placement layout displayed on the display operation unit 430 of the terminal device 400, and places the replenishment shelf SL30 in a predetermined position within the store. Thereafter, the product replenishment robot 100, controlled by the control device 150, sequentially moves the various products placed on the replenishment shelf SL30 to the corresponding product rows on the inventory shelf SL20, thereby replenishing products from the replenishment shelf SL30 to the inventory shelf SL20. After the product replenishment robot 100 has completed replenishment of products from the replenishment shelf SL30 to the stock shelf SL20, it resumes replenishment of products from the stock shelf SL20 to the display shelf SL10 until the next scheduled replenishment of products from the replenishment shelf SL30 to the stock shelf SL20 begins. In this way, according to the product replenishment system of the present disclosure, in addition to replenishing products from the stock shelf SL20 to the display shelf SL10, the product replenishment robot can also replenish products to the stock shelf SL20, thereby reducing the burden on store staff of replenishing products on stock shelves.

[0153] Furthermore, according to the product replenishment system of this embodiment, the replenishment of products on inventory shelves can also be performed by a product replenishment robot, thereby improving the operating rate of the product replenishment system compared to when store staff replenish products on inventory shelves.

[0154] Second Embodiment Next, a second embodiment of the present disclosure will be described. In the first embodiment described above, an example was shown in which a store clerk places products on a replenishment shelf SL30 according to a placement layout displayed on a terminal device 400. In this example, since there is a possibility that the store clerk may place products in the wrong placement position, a product image acquisition device 200 is provided as a means for compensating for this, and as a means for identifying products when the product replenishment robot 100 replenishes products from the replenishment shelf SL30 to the inventory shelf SL20. This embodiment provides a means for more flexible product placement on a replenishment shelf and for further improving the efficiency and accuracy of the replenishment work as a system as a whole.

[0155] The merchandise replenishment system of this embodiment may also have the same configuration as the merchandise replenishment robot 100, control device 150, management device 300, and terminal device 400 included in the merchandise replenishment system 1 of the first embodiment. However, in this embodiment, the merchandise image acquisition device 200 of the first embodiment may be omitted.

[0156] <Configuration of product replenishment shelf SL130> Here, the configuration of the product replenishment shelf SL130 in this embodiment will be described. The product replenishment shelf SL130 in this embodiment replaces the product replenishment shelf SL30 in the first embodiment. Fig. 21 is a perspective view showing the overall configuration of a product replenishment shelf in the second embodiment of the present disclosure, and Fig. 22 is a block diagram of a control unit provided in the product replenishment shelf shown in Fig. 21.

[0157] 21 , the product replenishment shelf SL130 in this embodiment has a frame structure SL131 that forms the overall framework of the shelf, and a plurality of shelf boards SL132 that are provided on the frame structure SL131. Each shelf board SL132 is installed at a slight downward incline from the front side, where products are loaded and unloaded, toward the back side, and a stopper SL132a is provided on the back side of each shelf board SL132 to prevent products placed on the front side of the shelf board SL132 from falling off the shelf board SL132 after sliding along the incline toward the back side.

[0158] Furthermore, each shelf SL132 is provided with a plurality of partition plates SL132b for separating the product lanes, and a plurality of floor plates SL132c equipped with weight sensors 540 (see FIG. 22) are individually installed for each product lane. The weight sensors 540 may be of any known configuration, such as a load cell type, a piezoelectric element type, or a capacitance type. Each weight sensor 540 is assigned identification information associated with the product lane of the shelf SL132 on which it is installed, thereby enabling measurement of the weight of products placed on the product lane of the shelf SL132 on which the weight sensor 540 is installed. The weight sensor 540 is one of the product placement sensors that detects the placement of products on the product lane of the shelf SL132. In addition to the weight sensor 540, an optical sensor or the like may also be used as the product placement sensor.

[0159] The product replenishment shelf SL130 is also provided with a code scanner 550 for scanning codes (including barcodes, two-dimensional codes, etc.) attached to the outer periphery of product containers. The code scanner 550 is configured to, for example, irradiate the code attached to the product with a laser beam and receive the reflected light to read the product information (including the product type, volume, etc.) indicated by the code.

[0160] The product replenishment shelf SL130 is provided with a plurality of casters SL133 on the underside of the frame structure SL131, which enable the product replenishment shelf SL130 to be moved on the store floor. As an example, in this embodiment, ball-shaped casters SL133 are provided on the four corners of the underside of the frame structure SL131.

[0161] 22 , the control unit 500 provided in the product replenishment shelf SL130 includes a processor 510, a storage unit 520, and a communication unit 530. Note that this configuration is an example, and other configurations may also be included. The above-mentioned multiple weight sensors 540 and code scanner 550 are connected to the control unit 500 so as to be able to communicate with each other.

[0162] The processor 510 controls the overall operation of the control device 500 by executing a program stored in the storage unit 520, and is composed of a CPU, a GPU, etc. The processor 510 operates to realize the processes and functions performed by the processor 510 as will be described with reference to FIG.

[0163] The storage unit 520 stores various control processes, programs for the processor 510 to execute each function, input data, etc., and is composed of RAM, ROM, other storage, etc. The storage unit 520 also temporarily stores the contents of communications with other terminals, the weight sensor 540, and the code scanner 550. The storage unit 520 particularly stores a program for causing the processor 510 to execute the processes performed by the control device 500, which will be described with reference to FIG. 23.

[0164] The communication unit 530 is a communication interface for communicating with the terminal device 400 via a network.

[0165] The processor 510 transmits data relating to the weight of the items placed on each product lane measured by each weight sensor 540 to the terminal device 400 via the communication unit 530. The processor 510 also transmits information on the code read by the code scanner 550 to the terminal device 400.

[0166] <Example of operation>

[0167] Next, an example of the operation of the product replenishment system 1 of this embodiment will be described. Fig. 23 is a flowchart for explaining an example of the operation of the product replenishment system of the second embodiment.

[0168] First, in step S41, the processor 310 in the management device 300 of the product replenishment system 1 uses past sales data or a learned sales forecast model stored in the memory unit 320 of the management device 300 to predict the time schedule for replenishing each product on the display shelves SL10 and inventory shelves SL20 of each store, and based on the prediction, schedules the types and quantities of products to be replenished on the inventory shelves SL20 of the target store where the product replenishment will be performed, thereby generating product replenishment information. The product replenishment information relates to products to be replenished from the product replenishment shelves SL130 to the inventory shelves SL20. The generated product replenishment information is transmitted from the communication unit 350 of the management device 300 to the terminal device 400.

[0169] Next, in step S42, the terminal device 400 that has received the product replenishment information displays the product replenishment information on the display operation unit 430, and presents the product replenishment information to the store clerk using the terminal device 400. Fig. 24 shows an example of the product replenishment information displayed on the display operation unit 430 of the terminal device 400. As shown in Fig. 24, the product replenishment information includes information regarding the type and quantity of products to be placed on the product replenishment shelf SL130.

[0170] The store clerk performs the task of placing the specified number of products of the type specified in the presented product replenishment information on the product replenishment shelf SL130. The task of placing products on the product replenishment shelf SL130 includes having the code scanner 550 read the code of the product that the worker has removed from the packaging box, and then placing the product on any product lane of any shelf SL131 of the product replenishment shelf SL130.

[0171] In step S43, the processor 501 of the control unit 500 acquires the product code information read by the code scanner 550 in a code reading process and information indicating the product lane on the shelf SL131 on which a weight change was detected by the weight sensor 540 when a product was placed thereon after the code reading process, associates these with each other, and transmits them from the communication unit 530 to the terminal device 400. This processing operation is repeated for each product. The processor 501 also transmits numerical data of the total weight of the products placed on each product lane on each shelf SL131, measured by each weight sensor 540 installed on each product lane on each shelf SL131, from the communication unit 530 to the terminal device 400 each time the code scanner 550 executes a code reading process.

[0172] Next, in step S44, the processor 410 of the terminal device 400 that has received the information adds 1 to the "number placed" and subtracts 1 from the "number remaining" for the product identified by the received code information among the products included in the product replenishment information, displays the calculated quantities on the display operation unit 430, and outputs a voice message such as "The remaining replenishment quantity of product A is X units" from the speaker (not shown) of the terminal device 400. For example, if the original remaining number of product A to be placed is 24 units, when the terminal device 400 receives code information related to product A, the processor 410 adds 1 to the "number placed" of product A in the product replenishment information and subtracts 1 from the "number remaining," displays the calculated "number placed" of 1 and "number remaining" of 23 on the display operation unit 430, and outputs a voice message such as "The remaining replenishment quantity of product A is 23 units" from the speaker (not shown) of the terminal device 400. The processor 410 may further display text information such as "23 units of product A remaining" on the display operation unit 430 as shown in Fig. 24. These processes are executed each time the terminal device 400 receives code information. The processor 410 further updates the "number placed" and "number remaining" values ​​of any product included in the product replenishment information stored in the memory unit 420 that have had changes to the "number placed" and "number remaining" values.

[0173] Next, in step S45, following the processing of step S44 above, or in parallel with step S44 above, the processor 410 of the terminal device 400 that has received the information above updates product placement position table information, which is information regarding the type and position of one or more products placed on each shelf SL131 of the product replenishment shelf SL130. In the terminal device 400 of this embodiment, this product placement position table information is stored in the memory unit 420, and the product placement position table information is updated each time the terminal device 400 receives the information above from the control unit 500.

[0174] Fig. 25 is a diagram showing an example of product placement position table information. The product placement position table information includes information regarding which type of product is placed at which position for each shelf SL131 of the product replenishment shelf SL130. Fig. 25 shows an example of detailed product placement position table information for shelf SL131 with shelf number #1.

[0175] When the processor 410 of the terminal device 400 receives from the control unit 500 the code information of the product read by the code reading process using the code scanner 550 and information indicating the product lane of the shelf SL131 on which a weight change was detected by the weight sensor 540 when the product was placed after the code reading process, the processor 410 counts the order in which the information was received for the product lane of the shelf SL131, associates the shelf SL131, the product lane, and the sequence number with the product information indicated by the code information, and updates the product placement position table information stored in the memory unit 420.

[0176] 25, for example, when the terminal device 400 receives code information indicating product C and information indicating the second product lane (Lane 2) of shelf SL131 with shelf number #1 where a weight change was detected, the processor 410 counts the sequential number of the information about that product lane (Lane 2) by referring to the existing information about that product lane (Lane 2) of that shelf SL131 stored in the product placement position table information. If this is the first information received about that product lane, the processor 410 updates the product placement position table information stored in the memory unit 420, assuming that product C has been placed in the first position of the second product lane (Lane 2) of shelf SL131 with shelf number #1. Each time the terminal device 400 receives the information transmitted from the control unit 500 resulting from a store clerk reading the code information of a product using the code scanner 550 and placing that product on any product lane of any shelf SL131 of the product replenishment shelf SL130, the processor 410 of the terminal device 400 executes the above-described process to update the product placement position table information stored in the memory unit 420. In this way, the product placement position table information, which includes the type of each product and the placement position of that product for each shelf SL131 of the product replenishment shelf SL130, is managed in a matrix data structure.

[0177] Next, in step S46, when the placement of the type and quantity of products specified in the product replenishment information on the product replenishment shelf SL130 is completed or the placement operation is interrupted midway, and the store clerk touches the "End scan" button B1 displayed on the display operation unit 430 of the terminal 400, the processor 410 of the terminal 400 transmits the product placement position table information stored in the memory unit 420 at that time to the management device 300 via the communication unit 440.

[0178] Furthermore, in step S46, after the store clerk has moved the replenishment shelf SL130 on which the products are placed as described above to a predetermined position within the store, the store clerk touches the "Start replenishment operation" button B2 displayed on the display operation unit 430 of the terminal 400, which causes the processor 410 of the terminal 400 to send command information to the management device 300 via the communication unit 440 to start a replenishment operation to sequentially replenish the multiple products on the replenishment shelf SL130 onto the inventory shelf SL20. The predetermined position within the store to which the replenishment shelf SL130 is moved is, for example, a position near the other end of the rail R shown in FIG. 2 , i.e., a position near the end of the rail R opposite the home position of the replenishment robot 100.

[0179] Finally, in step S47, after receiving the product placement position table information and replenishment operation start command information as described above, the processor 310 of the management device 300 sends an instruction to the control device 150 of the product replenishment robot 100 to cause the product replenishment robot 100 to sequentially replenish multiple products on the product replenishment shelf SL130 located at a predetermined position in the store onto the inventory shelf SL20. Upon receiving the instruction, the control device 150 operates the product replenishment robot 100, and the product replenishment robot 100 performs an operation to move the products on the product replenishment shelf SL30 onto the inventory shelf SL20.

[0180] As described above, product placement position table information including the type and placement position of each product placed on each shelf SL131 of the product replenishment shelf SL130 is transmitted from the management device 300 to the control device 150, so the control device 150 has information regarding which product is placed in which product lane and in which position on which shelf SL131 of the product replenishment shelf SL130. The control device 150 also has information regarding which product should be replenished in which lane of the inventory shelf SL20. Therefore, the control device 150 causes the product replenishment robot 100 to repeatedly perform the operation of grasping and removing a specific product from the product replenishment shelf SL130 and moving that product to the corresponding lane on the corresponding inventory shelf SL20, thereby replenishing the products on the product replenishment shelf SL130 to the inventory shelf SL20.

[0181] 9, so a description thereof will be omitted here. Furthermore, in this embodiment, as in the first embodiment, when replenishing products T from the replenishment shelf SL130 to the inventory shelf SL20, the drive mechanism of the inventory shelf SL20, in accordance with a control signal transmitted from the communication unit of the control device 150 of the product replenishment robot 100 when starting the operation of replenishing products T from the replenishment shelf SL130 to the inventory shelf SL20, places the shelf SL21 of the inventory shelf SL20 in a first position in which the front end is higher than the rear end, and sequentially replenishing products into the row of products on the shelf SL21. Furthermore, after the product replenishment operation from the product replenishment shelf SL130 to the inventory shelf SL20 is completed, the drive mechanism of the inventory shelf SL20, in accordance with a control signal transmitted from the communication unit of the control device 150 after the replenishment operation is completed, puts the shelf SL21 of the inventory shelf SL20 into a second position in which the front end is lower than the rear end, and enables the product replenishment robot 100 to take out the product located at the forefront of the front side of the inventory shelf SL20 and move it to the display shelf SL10 to replenish it.

[0182] As described above, according to this embodiment, even if a store clerk places an item on any product lane on any shelf SL131 of the replenishment shelf SL130, the product replenishment robot 100 can replenish the item on the replenishment shelf SL130 to a predetermined position on the inventory shelf SL20. This prevents a problem such as an item being replenished to an incorrect position on the inventory shelf SL20 due to an incorrect product placement position on the replenishment shelf SL130. Furthermore, because an item can be placed in any position on the replenishment shelf SL130, the store clerk can place the item on the replenishment shelf SL130 without checking the placement position, thereby reducing the burden and time required for the store clerk to place items on the replenishment shelf SL130.

[0183] Furthermore, it is no longer necessary to generate placement layout information in advance in the management device 300, and various processing steps based on the placement layout information can be omitted.

[0184] (Modification) Next, a modification of the second embodiment will be described. In the second embodiment described above, an example of operation has been described in which, in the task of placing a product on the product restocking shelf SL130, a store clerk has the code scanner 550 read the code of the product removed from the packaging box and then place the product on any product lane of any shelf SL131 on the product restocking shelf SL130, and the processor 501 of the control unit 500 associates the product code information read by the code scanner 550 in the code reading process with information indicating the product lane of the shelf SL131 on which a weight change has been detected by the weight sensor 540 as a result of the product being placed thereon after the code reading process, and transmits the associated information from the communication unit 530 to the terminal device 400. In contrast to this, this embodiment provides a means that, when the same type of product is repeatedly placed on the product replenishment shelf SL130, the code scanner 550 is required to read the product's code only once the first time, and as long as products of the same type are subsequently repeatedly placed on the product replenishment shelf SL130, the process of having the code scanner 550 read the codes for the same type of product can be omitted.

[0185] In this modification, in step S42, the store clerk may have the code scanner 550 read the code of a product removed from a packaging box, and then place two or more products of the same type in succession on any product lane of any shelf SL131 of the product replenishment shelf SL130. Then, in step S43, the processor 501 of the control unit 500 acquires code information for a certain type of product (e.g., a specific product A with a volume of 500 ml) read by the code scanner 550, as well as information about one or more product lanes on the shelf SL131 on which weight changes were detected by the weight sensor 540 as multiple products were sequentially placed after the code reading process, and information about the placement order, and associates these information with each other and stores them at least temporarily in the memory 520. Then, when the code scanner 550 subsequently reads the code of that product to place another type of product, the processor 501 transmits the associated information stored in the memory 520 to the terminal device 400 via the communication unit 530.

[0186] Thus, according to this embodiment, unless the type of product being placed is changed, even if a store clerk performs the code reading process for a certain product only once at the beginning using the code scanner 550 and then places the same type of product in succession at any position on the product replenishment shelf SL130, product placement position table information for multiple such products can be obtained, thereby making it possible to omit the task of repeatedly having the code scanner 550 read the codes for the same type of product.

[0187] It should be noted that when placing products on the product lane of the shelf SL131 of the product replenishment shelf SL130 after the code scanner 550 has read the product code, there is a possibility that the product may tip over on the product lane or that multiple products may be placed at once. In the former case, the weight sensor 540 may detect two weight changes: when the product is placed on the product lane and when the product tips over. In the latter case, the weight sensor 540 may detect only one weight change. Therefore, in either case, there is a possibility that the quantity of products placed on the product replenishment shelf SL130 may not be correctly recognized.

[0188] In contrast to this, in this modified example, as described above, the processor 501 transmits, each time the code reading process is executed by the code scanner 550, numerical data on the weight of the product on each product lane measured by each weight sensor 540 installed on each product lane of each shelf board SL131 from the communication unit 530 to the terminal device 400. This numerical data on the product weight indicates the increase in weight between the previous scanning operation and the current scanning operation. The processor 410 of the terminal device 400 then compares the product placement position table information, which may include the type and quantity of products acquired as having been placed on each product lane of each shelf SL131 after the code reading process, with information regarding the quantity of products actually placed on each product lane between the previous and current scanning operations, which is calculated by dividing the numerical data regarding the weight of the products on each product lane by the weight per unit of the product identified by the scanned code information (for example, 500 g as the weight of product A with a capacity of 500 ml), and if the two do not match, the quantity of products calculated based on the numerical data regarding the increase in the weight of the products on each product lane may be taken as positive, and the quantity information of the products included in the product placement position table information as having been placed on that product lane may be updated.

[0189] As an example, a case will be described in which, after the code scanner 550 reads the code of product A, product A is tipped over once on the first product lane in an attempt to place it on the first product lane of the first shelf SL131, and then picked up and placed back on the first product lane. In this case, after the code reading process of product A is performed, the weight sensor 540 installed on the first product lane of the first shelf SL131 detects two weight changes, and the terminal device 400 acquires product placement position table information assuming that two products A are placed on that product lane. Meanwhile, the numerical data of the weight increase of the product on the first product lane measured by the weight sensor 540 of that product lane (the data of the weight increase after the code reading of product A) is approximately 500 g, the weight of one product A, and this numerical value is transmitted to the terminal device 400. The processor 410 of the terminal device 400 compares the increased quantity of product A on the first product lane, "2," indicated in the product placement position table information, with the quantity of products actually placed on each product lane between the previous and current scanning operations (essentially 1), obtained by dividing the weight increase of 500g by the value of one product A, 500g, and since the two do not match, it updates the product placement position table information to indicate that only one product A, the latter quantity, has been placed on the first product lane of the first shelf SL131.

[0190] As another example, a case will be described in which two items A are placed simultaneously on the first item lane of the first shelf SL131 after the code scanner 550 has read the code of item A. In this case, after the code of item A has been read, the weight sensor 540 installed in the first item lane of the first shelf SL131 detects a single weight change, and the terminal device 400 acquires item placement position table information assuming that one item A has been placed on that item lane. Meanwhile, the numerical data of the weight increase of the item on the first item lane measured by the weight sensor 540 of that item lane (the data of the weight increase after the code of item A has been read) is approximately 1,000 g, which is the weight of two items A, and this numerical value is transmitted to the terminal device 400. The processor 410 of the terminal device 400 compares the increased quantity of product A on the first product lane, "1," indicated in the product placement position table information, with the quantity of products actually placed on each product lane between the previous and current scanning operations (essentially 2), obtained by dividing the weight increase of 1000g by the value of one product A, 500g, and since the two do not match, it updates the product placement position table information to indicate that the latter quantity, two products A, have been placed on the first product lane of the first shelf SL131.

[0191] Third Embodiment Next, a third embodiment of the present disclosure will be described. In the above-described first and second embodiments, an example has been described in which the product replenishment robot 100 replenishes products from the replenishment shelves SL30 and SL130 to the inventory shelf SL20. This embodiment provides a means for enabling the product replenishment system to recognize the type and location of products placed on the inventory shelf, even when products are replenished to the inventory shelf without going through the replenishment shelf.

[0192] The merchandise replenishment system of the present embodiment may also have the same configuration as the merchandise replenishment robot 100, control device 150, management device 300, and terminal device 400 included in the merchandise replenishment system 1 of the first embodiment. However, in the present embodiment, the merchandise image acquisition device 200 of the first embodiment may be omitted, and further, the merchandise replenishment shelves SL30, SL130 of the first and second embodiments may be omitted.

[0193] <Configuration of Stock Shelf SL120> The configuration of the stock shelf SL120 in this embodiment will now be described. The stock shelf SL120 in this embodiment replaces the stock shelf SL20 in the first and second embodiments. FIG. 26 is a perspective view showing the overall configuration of a stock shelf in the third embodiment. The stock shelf SL120 in this embodiment generally has a configuration in which the stock shelf SL20 described in the first embodiment is equipped with the control unit 500, weight sensor 540, and code scanner 550 that are equipped in the product replenishment shelf SL130 described in the second embodiment. However, the stock shelf SL120 in this embodiment does not necessarily have to have a function for changing the posture of each shelf SL122.

[0194] As shown in FIG. 26 , the inventory shelf SL120 of this embodiment includes a frame structure SL121 that forms the overall framework of the shelf SL120 and multiple shelf boards SL122 attached to the frame structure SL121. Each shelf board SL122 is further provided with multiple partition boards (not shown) for separating the product lanes, and multiple floor boards SL122a, each equipped with a weight sensor 540 (see FIG. 22 ), are individually installed for each product lane. The weight sensors 540 may be of any known configuration, such as a load cell type, a piezoelectric element type, or a capacitance type. Each weight sensor 540 is assigned identification information associated with the product lane of the shelf board SL122 on which it is installed, thereby enabling the weight of the product placed on the product lane of the shelf board SL122 on which the weight sensor 540 is installed to be measured. The weight sensor 540 is one of the product placement sensors that detects the placement of a product on the product lane of the shelf board SL122. As the product placement sensor, an optical sensor or the like may be used in addition to the weight sensor 540 .

[0195] The replenishment shelf SL120 is further provided with a code scanner 550 for scanning codes (including barcodes, two-dimensional codes, etc.) attached to the outer periphery of product containers. The code scanner 550 is configured to, for example, irradiate the code attached to the product with a laser beam and receive the reflected light to read the product information (including the product type, volume, etc.) indicated by the code. The replenishment shelf SL120 is also provided with a control unit 500 similar to that described with reference to FIG. 22. <Example of Operation>

[0196] Next, an example of the operation of the product replenishment system 1 of this embodiment will be described. Fig. 27 is a flowchart for explaining an example of the operation of the product replenishment system of the third embodiment.

[0197] First, in step S51, the processor 310 in the management device 300 of the product replenishment system 1 uses past sales data or a learned sales forecast model stored in the memory unit 320 of the management device 300 to predict the time schedule for replenishing each product on the display shelves SL10 and inventory shelves SL120 of each store, and based on the prediction, schedules the types and quantities of products to be replenished on the inventory shelves SL120 of the target store where the products will be replenished, thereby generating product replenishment information. The product replenishment information relates to the products to be replenished on the inventory shelves SL120. The generated product replenishment information is transmitted from the communication unit 350 of the management device 300 to the terminal device 400.

[0198] Next, in step S52, the terminal device 400 that has received the product replenishment information displays the product replenishment information on the display operation unit 430 and presents the product replenishment information to the store clerk using the terminal device 400. The product replenishment information includes information regarding the type and quantity of products to be placed on the inventory shelf SL120.

[0199] The store clerk performs the task of placing the specified number of products of the type specified in the presented product replenishment information on the inventory shelf SL120. The task of placing the products on the inventory shelf SL120 includes having the code scanner 550 read the code of the product that the worker has removed from the packaging box, and then placing the product on any product lane of any shelf SL122 of the inventory shelf SL120.

[0200] In step S53, the processor 501 of the control unit 500 acquires the product code information read by the code scanner 550 in a code reading process and information indicating the product lane on the shelf SL122 on which a weight change was detected by the weight sensor 540 due to the product being placed thereon after the code reading process, associates these with each other, and transmits them from the communication unit 530 to the terminal device 400. This operation is repeated for each product. The processor 501 also transmits numerical data of the total weight of the products placed on each product lane on each shelf SL122, measured by each weight sensor 540 installed on each product lane on each shelf SL122, from the communication unit 530 to the terminal device 400 each time the code scanner 550 executes a code reading process.

[0201] Next, in step S54, the processor 410 of the terminal device 400 that has received the information increments the "number placed" by one and decrements the "number remaining" by one for the product identified by the received code information among the products included in the product replenishment information, and displays the calculated quantities on the display operation unit 430, while outputting a voice message such as "X units of product A remain to be replenished" from the speaker (not shown) of the terminal device 400. The processor 410 may further display text information such as "23 units of product A remaining" on the display operation unit 430, as shown in FIG. 24. These processes are executed each time the terminal device 400 receives code information. The processor 410 further updates the "number placed" and "number remaining" values ​​of products included in the product replenishment information stored in the memory unit 420, for which changes have been made to the "number placed" and "number remaining" values.

[0202] Next, in step S55, following the processing of step S54 above, or in parallel with step S54 above, the processor 410 of the terminal device 400 that has received the information above updates product placement position table information, which is information regarding the type and position of one or more products placed on each shelf SL122 of the inventory shelf SL120. In the terminal device 400 of this embodiment, this product placement position table information is stored in the memory unit 420, and the product placement position table information is updated every time the terminal device 400 receives the information above from the control unit 500.

[0203] Fig. 28 is a diagram showing an example of product placement position table information. The product placement position table information includes information regarding which type of product is placed at which position for each shelf SL122 of the inventory shelf SL122. Fig. 28 shows an example of detailed product placement position table information for the shelf SL122 with shelf number #1.

[0204] When the processor 410 of the terminal device 400 receives from the control unit 500 the code information of a product read by the code scanner 550 in a code reading process and information indicating the product lane of the shelf SL122 on which a weight change was detected by the weight sensor 540 as a result of the product being placed thereon after the code reading process, the processor 410 counts the order in which the information has been received for that product lane of that shelf SL122, associates that shelf SL122, that product lane, and that sequence number with the product information indicated by the code information, and updates the product placement position table information stored in the memory unit 420. For each shelf SL122 of the inventory shelf SL120, the product placement position table information, which includes the type of product and the placement position of each product, is managed in a matrix data structure.

[0205] Next, in step S56, when the placement of the type and quantity of products specified in the product replenishment information on the product replenishment shelf SL130 is completed or the placement operation is interrupted midway, and the store clerk touches the "End scan" button B1 (see Figure 24) displayed on the display operation unit 430 of the terminal 400, the processor 410 of the terminal 400 transmits the product placement position table information stored in the memory unit 420 at that time to the management device 300 via the communication unit 440.

[0206] Finally, in step S57, when the store clerk touches the "Start Replenishment Operation" button B2 (see FIG. 24) displayed on the display operation unit 430 of the terminal 400, the processor 410 of the terminal 400 sends command information to the management device 300 via the communication unit 440 to start a replenishment operation to sequentially replenish multiple items on the inventory shelf SL120 onto the display shelf SL10. After receiving the product placement position table information and the replenishment operation start command information as described above, the processor 310 of the management device 300 sends an instruction to the control device 150 of the product replenishment robot 100 to cause the product replenishment robot 100 to sequentially replenish multiple items on the inventory shelf SL122 located at a predetermined position in the store onto the display shelf SL10. Upon receiving the instruction, the control device 150 operates the product replenishment robot 100, and the product replenishment robot 100 performs an operation to move the items on the inventory shelf SL120 onto the display shelf SL10.

[0207] As described above, product placement position table information, which includes the type and placement position of each product placed on each shelf SL122 of the inventory shelf SL120, is transmitted from the management device 300 to the control device 150. Therefore, the control device 150 has information regarding which product is placed on which shelf SL122 of the inventory shelf SL120, which product lane, and which position. The control device 150 also has information regarding which product should be replenished in which lane of the product SL10. Therefore, the control device 150 causes the product replenishment robot 100 to repeatedly grasp and remove a specific product from the inventory shelf SL120 and move that product to the corresponding lane on the corresponding product SL10, thereby replenishing the products on the inventory shelf SL120 to the display shelf SL10. Note that the product replenishment operation by the product replenishment robot 100 from the inventory shelf SL120 to the display shelf SL10 is similar to the operation described with reference to FIG. 9 , and therefore will not be described here.

[0208] In this embodiment, similar to the inventory shelf SL20 of the first embodiment, each shelf SL122 of the inventory shelf SL120 is configured to be switchable between a first position in which the front end of the shelf SL122 is higher than the rear end, and a second position in which the front end of the shelf SL122 is lower than the rear end, and when replenishing products from the inventory shelf SL120 to the display shelf SL10, the drive mechanism of the inventory shelf SL120 drives the inventory shelf SL120 in accordance with a control signal transmitted from the communication unit of the control device 150 of the product replenishment robot 100 to the control unit 500 when starting the operation of replenishing products from the inventory shelf SL120 to the display shelf SL10. In addition, after the product replenishment robot 100 has completed the replenishment operation from the inventory shelf SL120 to the display shelf SL10, the drive mechanism of the inventory shelf SL120, in accordance with a control signal transmitted from the communication section of the control device 150 to the control unit after the replenishment operation is completed, puts each shelf SL122 of the inventory shelf SL120 into a first position in which the front end is higher than the rear end, allowing the store clerk to place products on the front side of the inventory shelf SL120.

[0209] As described above, according to this embodiment, even if a store clerk places a product on any product lane on any shelf SL122 of the inventory shelf SL120, the product replenishment robot 100 can replenish the product on the inventory shelf SL122 to the specified position on the display shelf SL10. This prevents a problem such as a product being replenished to the wrong position on the display shelf SL10 due to an incorrect product placement position on the inventory shelf SL120. Furthermore, because products can be placed in any position on the inventory shelf SL120, the store clerk can place the product on the inventory shelf SL120 without checking the placement position, thereby reducing the burden and time required for the store clerk to place products on the inventory shelf SL120.

[0210] As with the modified example of the second embodiment, this embodiment may also be provided with a means for omitting the task of having the code scanner 550 read the codes for the same type of product, or a means for recognizing the quantity of products placed on the product lane even when the product is tipped over on the product lane or when multiple products are placed on the product lane at once.

[0211] Although the present disclosure has been described above through the embodiments of the invention, the above-described embodiments do not limit the invention according to the claims. Furthermore, combinations of features described in the embodiments of the present disclosure may also fall within the technical scope of the present invention. Furthermore, it will be apparent to those skilled in the art that various modifications and improvements can be made to the above-described embodiments.

Claims

1. A management device that manages product replenishment by a product replenishment robot that grasps products placed on inventory shelves and moves them to display shelves, said management device having a control unit and a memory unit, said memory unit storing sales data including past sales numbers and sales dates and times of the products, or a trained model generated by performing machine learning with a learning device using the sales data, and said control unit configured to execute generating product replenishment information for the products to be replenished on the inventory shelves based on the sales data or the trained model.

2. The management device according to claim 1, wherein the management device is communicatively connected to a terminal device used by an operator, and the control unit is further configured to transmit the product replenishment information to the terminal device and display it on the terminal device.

3. The management device of claim 2, wherein the product replenishment robot is further configured to grasp products placed on a product replenishment shelf and move them to the inventory shelf, the product replenishment information includes placement layout information for placing the products to be replenished on the inventory shelf in each area on the product replenishment shelf, and the control unit is further configured to cause the terminal device to display the placement layout information included in the product replenishment information selected by an input operation by the worker on the terminal device.

4. The management device of claim 3, wherein the terminal device is equipped with a camera that captures images and acquires image data, the product replenishment shelves are provided with identification information for uniquely identifying the product replenishment shelves, and the control unit is configured to, when the identification information of the product replenishment shelves photographed by the camera of the terminal device matches the identification information of the product replenishment shelves associated with the placement layout information included in the selected product replenishment information, cause the terminal device to display the placement layout information.

5. The management device according to claim 4, wherein the placement layout information displayed on the terminal device includes the type and number of products to be placed in each area on the product replenishment shelf.

6. The management device of claim 5, wherein the number of each of the products included in the placement layout information is changeable by an input operation by the worker on the terminal device, and the control unit is further configured to update the placement layout information to reflect the number of the products changed by the input operation by the worker on the terminal device.

7. The management device of claim 3, wherein the control unit is configured to, when it receives completion confirmation information from the terminal device indicating that the product replenishment shelf on which the products are placed in accordance with the placement layout information has been placed in a predetermined position, cause the product replenishment robot to perform an operation of moving the products placed on the product replenishment shelf in accordance with the placement layout information to the inventory shelf.

8. The management device of claim 7, wherein the terminal device is equipped with a camera that captures images and acquires image data, the product replenishment shelf is assigned identification information for uniquely identifying the product replenishment shelf, and the control unit is configured to cause the product replenishment robot to perform an operation of moving the product to the inventory shelf when the identification information of the product replenishment shelf photographed by the camera of the terminal device matches the identification information of the product replenishment shelf associated with the placement layout information included in the selected product replenishment information.

9. The management device of claim 7, wherein the control unit is configured to, when it receives removal confirmation information from the terminal device indicating that the product replenishment shelf has been removed from the specified position after replenishment of the products from the product replenishment shelf to the inventory shelf has been completed, cause the product replenishment robot to perform an operation to move the products from the inventory shelf to the display shelf.

10. The management device described in claim 7, wherein the product replenishment robot is equipped with a product identification device for identifying the type of product grasped by the product replenishment robot, and is configured to identify the type of at least one of the products placed in at least one of the areas of the product replenishment shelf using the product identification device.

11. The management device of claim 3, wherein the inventory shelf comprises one or more shelves, the shelves being configured to be able to switch between a first position in which the front end of the shelf facing the display shelf arranged opposite the inventory shelf is higher than the rear end of the shelf opposite the front side, and a second position in which the front end is lower than the rear end, and the shelf is in the first position while the product replenishment robot is performing an operation to move the products placed on the product replenishment shelf to the inventory shelf, and is in the second position while the product replenishment robot is performing an operation to move the products placed on the inventory shelf to the display shelf.

12. The management device of claim 3, wherein the inventory shelf comprises: one or more shelves each having a plurality of product lanes, each of which is provided with a product placement sensor for detecting that a product has been placed on it; a code reading unit which reads code information attached to the product; and a control unit which controls the product placement sensor and the code reading unit and is communicatively connected to the management device and the terminal device, wherein the control unit is configured to associate the code information read by the code reading unit with position information on the product lane of the shelf of the inventory shelf on which the product was placed after the code information was read.

13. The management device of claim 12, wherein the control unit is further configured to transmit the associated code information and position information to the terminal device, and the terminal device is configured to generate product placement position table information for a plurality of products identified by the code information, which associates the product type with the position information of the product on the inventory shelf, and transmit the product placement position table information to the management device, and the control unit of the management device is configured to cause the product replenishment robot to perform an operation of moving the products placed on the inventory shelf in accordance with the product placement position table information to the display shelf.

14. The management device of claim 12, wherein the control unit is configured to associate one piece of code information read by a single code reading operation by the code reading unit with the position information of at least one of the products placed on the product lane of the shelf of the inventory shelf after the code information is read.

15. The management device described in claim 13, wherein the product placement sensor is a weight sensor which detects a change in weight on the product lane and the weight of the product placed on the product lane, the control unit is configured to further transmit to the terminal device information relating to an increased weight of the product placed on each of the product lanes between two code reading operations by the code reading unit, and the terminal device is configured to compare information relating to the type and quantity of the product placed on each of the product lanes indicated in the product placement position table information with a calculated quantity obtained by dividing the increased weight of each of the product lanes by the weight per unit of the associated product, and if the calculated quantity does not match, to update the previously described quantity of the product included in the product placement position table information to the calculated quantity.

16. The product replenishment robot is further configured to perform an action of grasping products placed on the product replenishment shelf and moving them to the inventory shelf, the product replenishment shelf comprising: one or more shelves with multiple product lanes on each shelf, each product lane of the shelf being equipped with a product placement sensor for detecting that a product has been placed on it; a code reading unit that reads code information attached to the product; and a control unit that controls the product placement sensor and the code reading unit and is communicatively connected to the management device and the terminal device, the control unit being configured to associate the code information read by the code reading unit with position information on the product lane of the shelf of the product replenishment shelf on which the product was placed after the code information was read.

17. The management device of claim 16, wherein the control unit is further configured to transmit the associated code information and position information to the terminal device, and the terminal device is configured to generate product placement position table information for a plurality of products identified by the code information, which associates the product type with the position information of the product on the product replenishment shelf, and transmit the product placement position table information to the management device, and the control unit of the management device is configured to cause the product replenishment robot to perform an operation of moving the products placed on the product replenishment shelf in accordance with the product placement position table information to the inventory shelf.

18. The management device according to claim 16, wherein the control unit is configured to associate one piece of code information read by a single code reading operation by the code reading unit with the position information of at least one of the products placed on the product lane of the shelf of the product replenishment shelf after the code information is read.

19. The management device described in claim 17, wherein the product placement sensor is a weight sensor which detects weight changes on the product lane and a total weight of the products placed on the product lane, the control unit is configured to further transmit total weight information regarding the total weight of the products placed on each of the product lanes to the terminal device, and the terminal device is configured to compare information regarding the type and quantity of the products placed on each of the product lanes indicated in the product placement position table information with a calculated quantity obtained by dividing the total weight of each of the product lanes by the weight per unit of the associated product, and if the calculated quantity does not match, update the previously described quantity of the products included in the product placement position table information to the calculated quantity.

20. A product replenishment system comprising: a product replenishment robot that grasps and moves products; and the management device according to claim 1.