Estimation system, learning system, estimation method, learning method, program

Abstract

To provide an estimation system capable of grasping to which position an operation is performed in the height direction.SOLUTION: A estimation system includes: a sensor information acquisition unit for, by vibration of a vibration unit, which is installed at any position of a fixture having a plurality of shelf plates in the height direction, in response to application of vibration generated through an operation of placing or taking out an object to any of the shelf plates in the fixture, acquiring sensor information obtained from a sensor for detecting vibration in which at least one of the duration or intensity of vibration generated according to the operation is changed; and an estimation unit for obtaining position information including a position in the height direction of the fixture operated by inputting the acquired sensor information into a learned model having learned the relation between the sensor information and the position in the fixture operated.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an estimation system, a learning system, an estimation method, a learning method, and a program.

Background Art

[0002] [[ID=十二年]]In recent years, the use of smart shelf systems has been under consideration. A smart shelf system can detect the position where an operation is performed and the operation content (placement or retrieval) in response to an operation such as placing a product on a shelf board or retrieving a product in scenarios such as product management in a store or inventory management of products in a warehouse. The demand for such smart shelf systems has been increasing. Also, in the context of a smart home, there is a demand for a smart shelf that can naturally grasp living behaviors and enable appropriate information presentation.

[0003] In order to widely deploy smart shelves, a low introduction cost and applicability to a wide range of target furniture are required. A simple configuration specification that can be attached to existing furniture is required. In Non-Patent Document 1, a technique for estimating operations on a table by using a load sensor without using a camera is disclosed.

Prior Art Documents

Non-Patent Documents

[0004] [[ID=三十一年]]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, while we are able to estimate the position on a flat surface such as a shelf or desk, some shelves have multiple shelves, and in these cases, we are unable to estimate which position in the height direction the operation is being performed on.

[0006] The present invention has been made in view of these circumstances, and its purpose is to provide an estimation system, a learning system, an estimation method, a learning method, and a program that can determine at what position in the height direction an operation was performed. [Means for solving the problem]

[0007] To solve the above-mentioned problems, one aspect of the present invention is an estimation system comprising: a sensor information acquisition unit that acquires sensor information from a sensor that detects vibrations in which at least one of the duration or intensity of the vibrations generated in response to an operation in which an object is placed on or removed from any of the shelves of the fixture is changed, and the vibration unit is vibrated in response to an operation in which an object is placed on or removed from any of the shelves of the fixture, thereby changing the duration or intensity of the vibrations generated in response to the operation; and an estimation unit that inputs the acquired sensor information into a trained model that has learned the relationship between the sensor information and the position of the operated fixture to obtain position information including the position in the height direction of the operated fixture.

[0008] Furthermore, one aspect of the present invention is a learning system comprising: a vibration unit provided at any position on a fixture having multiple shelves arranged in the height direction, and a learning data acquisition unit that acquires sensor information obtained from a sensor that detects vibrations in which at least one of the duration or intensity of the vibrations generated in response to an operation of placing or removing an object from any of the shelves on the fixture, thereby acquiring position information including the position in the height direction on the operated fixture; and a learning unit that generates a trained model that learns the relationship between the sensor information and the position information using the acquired learning data.

[0009] Furthermore, one aspect of the present invention is an estimation method for obtaining position information, including the position in the height direction of the operated furniture, by providing a vibration unit at any of the positions of a furniture having multiple shelves in the height direction, and acquiring sensor information from a sensor that detects vibrations in which at least one of the duration or intensity of the vibrations generated in response to an operation in which an object is placed on or removed from any of the shelves of the furniture is changed, by inputting the acquired sensor information into a trained model that has learned the relationship between the sensor information and the position on the operated furniture.

[0010] Furthermore, one aspect of the present invention is a learning method in which a vibration unit is provided at any position on a fixture having multiple shelves arranged in the height direction, and the vibration unit vibrates in response to vibrations generated when an operation is performed on or off any of the shelves on the fixture, thereby acquiring sensor information obtained from a sensor that detects vibrations in which at least one of the duration or intensity of the vibrations generated in response to the operation has changed, and position information including the position in the height direction on the fixture that was operated on, and using the acquired learning data, a trained model is generated which has learned the relationship between the sensor information and the position information.

[0011] Furthermore, one aspect of the present invention is a program that causes a computer to perform the following actions: a vibration unit is provided at any position on a fixture having multiple shelves arranged in the height direction, and the vibration unit vibrates in response to vibrations generated when an object is placed on or removed from any of the shelves on the fixture, thereby acquiring sensor information obtained from a sensor that detects vibrations in which at least one of the duration or intensity of the vibration generated in response to the operation has changed, inputting the acquired sensor information into a trained model that has learned the relationship between the sensor information and the position on the operated fixture, thereby obtaining position information including the position on the operated fixture in the height direction.

[0012] Furthermore, one aspect of the present invention is a program that causes a computer to perform the following actions: a vibration unit is provided at any position on a fixture having multiple shelves arranged in the height direction, and the vibration unit vibrates in response to vibrations generated when an object is placed on or removed from any of the shelves on the fixture, thereby acquiring sensor information obtained from a sensor that detects vibrations in which at least one of the duration or intensity of the vibrations generated in response to the operation has changed, and position information including the position in the height direction on the operated fixture, and using the acquired training data to generate a trained model that has learned the relationship between the sensor information and the position information. [Effects of the Invention]

[0013] As explained above, this invention makes it possible to determine at what position in the height direction the operation was performed. [Brief explanation of the drawing]

[0014] [Figure 1] This is a perspective view showing an example of a fixture used in an estimation system according to one embodiment of this invention. [Figure 2A]It is a perspective view showing a case where the vibration unit 25a and the vibration unit 25b are attached to the shelf T. [Figure 2B] It is a side view showing the vibration unit 25a and the vibration unit 25b when viewed from the side. [Figure 3A] It is a plan view of the scroll member 250 in plan view. [Figure 3B] It is a side view of the scroll member 250 when viewed from the side. [Figure 4] It is a waveform diagram showing an example of the vibration characteristics of the vibration unit. [Figure 5A] It is a perspective view in another example of the vibration unit. [Figure 5B] It is a diagram for explaining the vibration possible directions of the vibration unit 35. [Figure 5C] It is a cross-sectional view for explaining the internal structure of the vibration unit 35 on the A-A line in FIG. 5A. [Figure 5D] It is a cross-sectional view for explaining the internal structure of the vibration unit 36 in a configuration different from that of FIG. 5C. [Figure 5E] It is a cross-sectional view for explaining the internal structure of the vibration unit 37 in a configuration different from that of FIG. 5C. [Figure 5F] It is a cross-sectional view showing a cross-section when a plurality of vibration units are connected. [Figure 5G] It is a cross-sectional view showing a cross-section when a plurality of vibration units are connected. [Figure 6] It is a schematic block diagram for explaining the functions of the management system S. [Figure 7] It is a block diagram showing an example of the functional configuration of the learning system 40 according to the present embodiment. [Figure 8] It is a block diagram showing an example of the functional configuration of the estimation system 50 according to the present embodiment. [Figure 9] It is a diagram showing as an image the simulation result when no vibration unit is provided on the shelf. [Figure 10] It is a diagram showing as an image the simulation result when a vibration unit is provided on the shelf. [Figure 11]This diagram illustrates the processing flow of the management system S. [Figure 12] This flowchart shows an example of the operation of the vibration generation location estimation system 4 according to this embodiment. [Figure 13] This figure shows an example of how the estimation results are displayed on the screen of the user terminal 2 or the display device 6. [Figure 14] This graph shows the detection results obtained by the sensor device. [Modes for carrying out the invention]

[0015] An estimation system according to one embodiment of the present invention will be described below with reference to the drawings. Figure 1 is a perspective view showing an example of a fixture used in an estimation system according to one embodiment of the present invention. Shelf T is an example of a fixture and has three shelves: a lower shelf, a middle shelf, and an upper shelf. Shelf board 11a is the lower shelf, shelf board 11b is the middle shelf, and shelf board 11c is the upper shelf, and objects can be placed on the top surface of each. Support posts 12a, 12b, 12c, and 12d are provided at the four corners of shelf T, and shelves boards 11a, 11b, and 11c are attached to these support posts. On the underside of the shelf board 11a, a sensor 14a is provided at a position corresponding to the support column 12a, a sensor 14b is provided at a position corresponding to the support column 12b, and a sensor 14d is provided at a position corresponding to the support column 12d. As a result, sensors 14a, 14b, and 14c are located between the support column and the floor surface, respectively. Hereinafter, when sensors 14a, 14b, and 14c are not specifically distinguished, they may simply be referred to as sensor 14.

[0016] Sensors 14a, 14b, and 14c each detect the force applied to shelf T. These sensors 14a, 14b, and 14c detect the force applied to shelf T by, for example, detecting vibrations caused by the force applied to shelf T. Sensors 14a, 14b, and 14c transmit the detected information (hereinafter also referred to as "sensor information") to the vibration generation location estimation system 4 (described later). Here, shelf T is designed so that when an object is placed on any of the shelves or an item is removed from a shelf, the vibrations caused by that operation propagate to the bottom of the shelf. Sensor 14 detects these vibrations.

[0017] Sensors 14a, 14b, and 14c are, for example, one of the following: a vibration sensor, a load sensor, a pressure sensor, or an acceleration sensor. The vibration sensor detects the generation of force applied to shelf T by a change in vibration. The load sensor detects the force applied to shelf T by a change in vibration. The pressure sensor detects the force applied to shelf T by a change in pressure. The acceleration sensor detects the force applied to shelf T by a change in acceleration. Here, the force applied to shelf T arises in response to at least one of the following operations: placing an object (e.g., object 18a, object 18b) on one of the shelves of shelf T, or removing an object that has been placed on one of the shelves of shelf T. Therefore, sensors 14a, 14b, and 14c each detect vibrations corresponding to the operation on shelf T.

[0018] Sensor information has an arbitrary number of dimensions. The number of dimensions is determined according to the type and number of sensor devices 3. For example, if two single-axis sensor devices 3 are installed, the number of dimensions of the sensor information is 2 (1 axis x 2). Also, for example, if three two-axis sensor devices 3 are installed, the number of dimensions of the sensor information is 6 (2 axes x 3).

[0019] The vibration unit 15a is attached to the lower surface of the shelf board 11b near the support column 12a, and the vibration unit 15b is attached to the lower surface of the shelf board 11b near the support column 12d. The vibration unit 15c is attached to the lower surface of the shelf board 11c near the support column 12a, and the vibration unit 15d is attached to the lower surface of the shelf board 11c near the support column 12d. The vibration units 15a, 15b, 15c, and 15d may be simply referred to as vibration unit 15 unless otherwise specified. The vibration unit 15 vibrates in response to vibrations applied to it, which result from an operation on one of the shelves of the fixture (e.g., shelf T) where an object is placed or removed. By vibrating, the vibration unit 15 can amplify the vibrations generated by the operation on shelf T. In other words, the vibration unit 15 can emphasize the vibrations generated by the operation on shelf T. Such a vibration unit 15 can be anything that can generate vibrations.

[0020] In this diagram, object 18a is shown placed on the upper surface of shelf 11b, and object 18b is shown placed on the upper surface of shelf 11c. User U can perform operations on any of shelf 11a, shelf 11b, or shelf 11c. Operations on an object include at least one of the following: placing an object on a shelf, or removing an object that has been placed on a shelf. Any object from among several types of objects may be placed on shelf boards 11a, 11b, and 11c, or the types of objects that can be placed on each shelf board may be predetermined. Furthermore, of the shelves 11a, 11b, and 11c, only one object may be placed on each shelf, or multiple objects may be placed on each shelf.

[0021] Figure 2 shows other configuration examples of the vibration unit 15. Figure 2A is a perspective view showing the case where vibration units 25a and 25b are attached to shelf T, and Figure 2B is a side view showing the vibration units 25a and 25b viewed from the side. In Figures 2A and 2B, the vibration units 25a and 25b are integrally mounted to the base member 20. The height of the base member 20 corresponds to the height from the lower shelf to the upper shelf of the shelf T. The base member 20 may be attached to the shelf T as, for example, the back plate of the shelf T.

[0022] The vibration unit 25a is provided with a vortex member 25a1. Here, the vortex member 25a1 is formed by forming a spiral groove in the base member 220. Here, the groove is formed to penetrate from one main surface (front) to the other main surface (back) of the base member 20, and when vibration is applied from the outside, the center of the vortex of the vortex member 25a1 can vibrate in the direction passing through the one main surface and the other main surface (horizontal direction), and the center of the vortex of the vortex member 25a1 can vibrate in the height direction (vertical direction) of the base member 20. In other words, the vortex member 25a1 and other vortex members can vibrate in any direction (three-dimensional direction) in three-dimensional space in response to vibration applied from the outside. Here, a weight may be attached to the radial center of the vortex member 25a1, and the vibration characteristics of the vortex member 25a1 can be changed by changing the weight of this weight. Such weights can be attached not only to the vortex member 25a1 but also to other vortex members.

[0023] Furthermore, the vibration unit 25a is provided with a connecting member 25a3 that connects the center of the vortex member 25a1 to the base member 20. One end of the connecting member 25a3 is attached to the base member 20 near the outer circumference of the vortex member 25a1, and the other end of the connecting member 25a3 is connected to the center of the vortex member 25a1 by a mounting pin 25a2. In the vibration unit 25a, the central part of the vortex member 25a1 and the base member 20 are connected by a connecting member 25a3, which makes it easier to dampen the vibration of the vortex member 25a1 compared to when they are not connected. This connecting member 25a3 may be provided as needed, and may not be provided if the vibration of the vortex member 25a1 does not need to be damped. Furthermore, by changing the material, thickness, length, etc. of the connecting member 25a3, the degree of damping may be increased (easier to dampen) or decreased (less to dampen).

[0024] The vibration unit 25b is composed of a combination of the first unit 251b and the second unit 252b. The first unit 251b has at least one of the following characteristics: an extension of the vibration duration and an increase in the vibration intensity. The second unit 252b has at least one of the following characteristics: a reduction in the duration of vibration and a decrease in the intensity of vibration. By combining the first unit 251b and the second unit 252b, it is possible to edit the characteristics that change at least one of the duration or intensity of vibrations generated in response to operations on shelf T. For example, the vibration unit 25b is composed of a combination of one type of first unit from among several types of first units with different characteristics and one type of second unit from among several types of second units with different characteristics.

[0025] For example, if the first unit has the characteristic of greatly extending the duration of vibration and increasing the intensity of vibration, and the second unit has the characteristic of slightly shortening the duration of vibration and having almost no effect on the intensity of vibration, then when these are combined, a vibration unit can be obtained that has been edited to have the characteristics of slightly shortening the duration of vibration of the first unit, but keeping the intensity of vibration the same. Furthermore, for example, if the first unit has the characteristic of slightly extending the duration of vibration and increasing the intensity of vibration, and the second unit has the characteristic of not significantly affecting the duration of vibration but slightly reducing the intensity of vibration, then when these are combined, a vibration unit can be obtained that has been modified so that the duration of vibration of the first unit does not change much, but the intensity of vibration is slightly reduced.

[0026] Here, the central parts of the vortex member 251b1 of the first unit 251b and the vortex member 252b1 of the second unit 252b are connected by a connecting member 25b3. One end of the connecting member 25b3 is connected to the central part of the vortex member 251b1 by a mounting pin 251b2, and the other end of the connecting member 25b3 is connected to the central part of the vortex member 252b1 by a mounting pin 252b2. These vortex members 25a1, 251b1, and 252b1 may be manufactured integrally with the base member 20 by machining the base member 20 itself, or they may be manufactured using a 3D printer. Therefore, using vortex members has the advantage of being easier to process and manufacture compared to using coil springs. This diagram illustrates the case where two vortex members are connected, but it is also possible to connect three or more.

[0027] Figure 3 shows the configuration of the vortex members 250 (vortex members 25a1, vortex members 251b1, and vortex members 252b1). Figure 3A is a plan view of the vortex members 250, and Figure 3B is a side view of the vortex members 250. The vortex member 250 has a starting end 255 near the center point 257, and has a spiral shape (algebraic spiral shape) in which the diameter gradually increases from the starting end 255 to the end 256. It is formed with a width t that is approximately the same from the starting end 255 to the end 256, and a thickness of W. Furthermore, in the radial direction of the vortex member 250, the distance P between the centers of adjacent vortices in the spiral shape is formed to be equal. In the width direction of the starting end 255, the distance between the side closer to the vortex center point 257 and the center point 257 is set to da / 2, and the distance between the center point 257 and the end 256 is set to do / 2.

[0028] Here, when manufacturing the vortex member 250, it can be created by directly processing a plate-shaped member to form a groove, making the processing easy. Furthermore, in the vortex member 250, the vibration characteristics can be changed by arbitrarily changing at least one of the following variables: width t, distance P, distance da / 2, distance do / 2, thickness W, the angle between the line connecting the start end 255 and the center point 257 and the line connecting the end end 256 and the center point 257, and the ease of elastic deformation depending on the material of the plate-shaped member being processed. Moreover, when changing at least one of these variables, the vibration characteristics (elasticity) of the vortex member 250 can be arbitrarily designed by simulating the frequency when the vortex member 250 vibrates. For this reason, the design of the vibration characteristics of the vortex member 250 is easy. Thus, using a vortex-shaped elastic body as an elastic component offers the advantage of easier design and fabrication.

[0029] The location where such a vibration unit using a vortex member is installed is not limited to the back panel of the shelf; it may also be installed on at least one of the side panels, support columns, or shelves of the shelf. In this case, when a vibration unit using a vortex member is installed on a shelf, it may be installed at a distance from the shelf or a cover may be provided on the vortex member so that it does not come into contact with any part of the shelf or with objects placed on the shelf when the vortex member vibrates.

[0030] Figure 4 is a waveform diagram showing an example of the vibration characteristics of a vibration unit. In Figure 4, the vertical axis represents the signal intensity, and the horizontal axis represents time. Reference numeral 400 denotes a waveform diagram for a case where the vibration intensity and vibration duration are average characteristics among several types of vibration units. Reference numeral 410 denotes a waveform diagram for one of several types of vibration units, where the vibration intensity is average, but the vibration characteristics are such that the vibration duration is shortened due to damping. Then, a vibration unit having the vibration characteristics shown in reference numeral 400 is designated as the first unit, and a vibration unit having the vibration characteristics shown in reference numeral 410 is designated as the second unit, and by connecting these together, a combined vibration unit can be obtained. Reference numeral 420 denotes the vibration characteristics of a combined vibration unit including such a first unit and a second unit. In this case, the vibrations of the first unit and the second unit are combined to form a resonance state in the interval from time t1 to time t2, which is later than the interval from time t0 to time t1. As a result, the amplitude increases, then decreases after time t2, and the vibration almost stops at time t3. Furthermore, regarding the duration of the vibration, the first unit has a longer vibration duration than the second unit, but the combined vibration unit has an even longer vibration duration than the first unit. In this way, by combining vibration units with different characteristics, it is possible to obtain a new type of vibration unit with editable vibrations.

[0031] Here, depending on the characteristics of the shelf, such as the total weight of the shelf, the number of shelves, the number of support posts, the width and thickness of the support posts, and the mounting position of the support posts on the shelves, even if an object is placed on a shelf, the vibration detection result may be so weak that it is not possible to distinguish which shelf the object is placed on. In contrast, as described above, by editing coupled vibrations according to the characteristics of vibration intensity and vibration duration, it is possible to create a combination of vibration characteristics that make it easier to detect the position in the height direction, depending on the characteristics of the shelf and the weight of the object. Furthermore, for example, even if a relatively light object is placed on a shelf, it becomes easier to detect its position in the height direction.

[0032] Furthermore, even when there are many shelves, such as 10 or 20 shelves, the first and second units can be arbitrarily combined to increase the variations in coupled vibrations in the combined vibration unit, making it easier to identify each shelf. This allows for the arbitrary creation of combined vibration units with individual coupled vibrations corresponding to each shelf level, and their attachment to any position on the shelf. When attaching the combined unit to the shelf, it can be attached by adhesive or by screws. Furthermore, for example, in a shelf with 10 shelves, 10 different types of combined vibration units may be manufactured and one unit may be installed on each of the 10 shelves. The combined vibration units may be installed on the top or bottom surface of the shelves. The combined vibration units may also be installed at any position on the surface of the shelf (front, back, right, left, or center). The combined vibration units may also be installed on the support columns instead of the shelves. In addition, the combined units may be installed on each shelf, or installed on the top 5 shelves but not on the bottom 5 shelves, or installed on every other shelf, or on several arbitrary shelves.

[0033] The optimal mounting position for the combination unit or vibration unit varies depending on the shelf structure (number of tiers, number of support columns, mounting position of support columns on the shelf boards, etc.), the materials that make up the shelf, and the properties of the objects placed on it. However, it is preferable to mount the unit in a position where vibrations occur that make it easier for the vibration generation position estimation system 4 (described later) to identify the position in the height direction. For example, the vibration unit may be placed anywhere on the shelf, but by installing it above the shelf (for example, on the top board, the top of the support columns, the top of the side panels, the top of the back, etc.), it may be possible to obtain detection results from the sensor device that make it easier to identify the position in the height direction corresponding to the operation.

[0034] Here, we have described the case where vibration units are connected and installed on a shelf, but if the vibrations generated in response to operations performed on the shelf can be changed, individual vibration units may be installed on the shelf without connecting them. In particular, when changing the vibrations generated in response to operations, it is desirable to apply changes that emphasize the characteristics of the vibrations. When emphasizing the characteristics of a vibration, for example, the amplitude may be made larger than the overall amplitude, the duration of the vibration may be extended, the amplitude of a specific frequency component included in the vibration may be increased, the amplitude of a specific frequency component included in the vibration may be decreased, or the duration of the vibration of a specific frequency component may be shortened. In this way, when an operation is performed at any position in the height direction, the intensity and duration of the vibration transmitted to the sensor 14 become characteristic vibrations that are detected by the sensor, while also being influenced by the relative position of each vibration unit to the position where the operation was performed and the characteristics of the shelf, etc. This makes it possible to identify which position in the height direction the operation was performed at.

[0035] Figure 5 shows another example of a vibration unit. Figure 5A is a perspective view of another example of a vibration unit. The vibration unit 35 has a roughly cubic shape. An elastic body such as a spring is provided inside the vibration unit 35, and the spring inside the vibration unit 35 vibrates in response to vibration applied from the outside of the vibration unit 35. A slide switch 351 is provided on one of the outer surfaces of the vibration unit 35. By sliding the slide switch 351, it is possible to selectively switch between a state in which the elastic body inside the vibration unit 35 can vibrate and a state in which it cannot vibrate.

[0036] Figure 5B illustrates the directions in which the vibration unit 35 can vibrate. The vibration unit 35 can vibrate in a total of six directions: vertically, horizontally, and both forward and backward, depending on the applied vibration. In addition, depending on the applied vibration, it can also vibrate diagonally.

[0037] Figure 5C is a cross-sectional view illustrating the internal structure of the vibration unit 35 along line AA in Figure 5A. A weight 35a is housed inside the vibration unit 35 and is attached to the base member via springs in the vertical, horizontal, front, and rear directions. For example, the weight 35a is attached to the base member 35c1 via spring 35b1 in the upward direction and to the base member 35c2 via spring 35b2 in the leftward direction. The weight 35a is approximately spherical in shape. The springs 35b1 and 35b2 are, for example, coil springs. The base members 35c1 and 35c2 are fixed by being bonded or adhesive to their inner circumferential surfaces. This allows the weight 35a to vibrate in any of the six directions, as well as diagonally. Such a vibration unit 35 can be used on its own, or it can be used as a first unit in combination with the second unit. Furthermore, since the weight 35a of the vibration unit 36 ​​is connected to a spring, the weight 35a vibrates in response to vibrations generated when the shelf T is operated on, thereby amplifying (enhancing) the vibrations generated by the operation on the shelf T. As a result, the vibrations generated by the operation can be detected by the sensor 14 as having more characteristic vibration properties, and the vibration generation position estimation system 4 (described later) can identify the position of the shelf T not only in the planar direction but also in the height direction.

[0038] Figure 5D is a cross-sectional view illustrating the internal structure of the vibration unit 36 ​​in a configuration different from that shown in Figure 5C. In Figure 5C, a coil spring is used in the vibration unit 35, while in Figure 5D, a grid spring is used in the vibration unit 36. A weight 36a is housed inside the vibration unit 36 ​​and is attached to the base member via grid springs in the vertical, horizontal, front, and rear directions. For example, the weight 36a is attached to the base member 36c1 via grid spring 36b1 in the upward direction and to the base member 36c2 via grid spring 36b2 in the leftward direction. The shape of the weight 36a is approximately cubic. The base members 36c1 and 36c2 are fixed by adhesion or bonding to their inner surfaces. This allows the weight 36a to vibrate in any of the six directions, as well as diagonally. Such a vibration unit 36 ​​can be used on its own, or it can be used as a first unit in combination with the second unit.

[0039] Figure 5E is a cross-sectional view illustrating the internal structure of the vibration unit 37 in a configuration different from that shown in Figure 5C. In Figure 5C, a coil spring is used in the vibration unit 35, but in Figure 5E, a damper is used in the vibration unit 37. The vibration unit 37 houses dampers, which are mounted via base members in the vertical, horizontal, front, and rear directions. For example, Figure 5E shows dampers 37a, 37b, 37c, and 37d. Damper 37a is attached to the inner surface of the right side via a base member 38a and dampens vibrations in the left-right direction. Damper 37b is attached to the inner surface of the upper side via a base member 38b and dampens vibrations in the up-down direction. The damper 37c is attached to the inner circumferential surface of the left side via the base member 38c and dampens vibrations in the left-right direction. The damper 37d is attached to the inner circumferential surface of the lower side via the base member 38d and dampens vibrations in the up-down direction. The base members 38a, 38b, 38c, and 38d are fixed by being bonded or adhered to their inner surfaces. This allows the weight 36a to vibrate in any of the six directions, as well as obliquely. Although the case where the vibration unit 37 is provided with multiple dampers has been described, the vibration unit 37 may also be provided with only one damper. Such a vibration unit 37 can be used on its own, or it can be used as a second unit in combination with the first unit. Furthermore, multiple vibration units 37 may be connected together.

[0040] Figure 5F is a cross-sectional view showing the case when multiple vibration units are connected together. In this figure, one outer surface of multiple vibration units 35A and one outer surface of vibration unit 35B are connected by adhesive or bonding. The natural vibration characteristics of vibration units 35A and 35B may be the same or different. By connecting vibration units 35A and 35B, the vibration characteristics of the combined vibration unit, which is the two connected vibration units, can be edited. If the natural vibration characteristics of vibration units 35A and 35B are to be different, the weight of the counterweights may be different, or the length and stiffness of the springs may be changed to obtain different spring constants. Each weight vibrates in response to vibrations applied from an external source, and resonance occurs depending on the type of vibration applied. For example, resonance occurs with respect to vibrations in the direction of connection. This allows vibrations from the connected vibration unit to be applied at the mounting position on the shelf, in addition to vibrations applied from an external source. This diagram illustrates the case where two vibration units are connected, but it is also possible to connect three or more vibration units.

[0041] Figure 5G is a cross-sectional view showing the configuration when multiple vibration units are connected together. In this figure, one outer surface of multiple vibration units 35 and one outer surface of vibration unit 37A are connected by adhesive or bonding. A weight is attached to the inner surface of vibration unit 35 via a coil spring, and a damper is attached to vibration unit 37A to reduce vibration in the left-right direction (in this case, the connection direction). By connecting vibration unit 35 and vibration unit 37A, the vibration characteristics of the combined vibration unit, which is the two connected vibration units, are edited. The weight vibrates due to vibrations applied from the outside, but the damper is also driven, so the damper is driven to dampen the vibration of the weight. As a result, vibrations based on the vibration characteristics corresponding to the combination of vibration unit 35 and vibration unit 37A are applied at the mounting position on the shelf, separately from vibrations applied from the outside. In this diagram, the case in which one vibration unit 35 and one vibration unit 37A are connected has been described, but it is also possible to connect multiple units of at least one of the vibration units 35 and 37A.

[0042] The vibration units 15, 25, 35, 36, 37, 35A, 35B, and 37A described above may be pre-installed in the fixture, or they may be installed later depending on the vibration characteristics of the fixture, the weight of the items placed on the fixture, their placement, etc. If installed later, vibration units that make it easier (or more advantageously) to detect operations on the fixture can be selected and installed. Furthermore, even if the number of shelves in the fixture is changed or the installation position of the shelves is changed, the position of the vibration units may be changed or they may be replaced with different vibration units in accordance with these changes. Furthermore, the vibration unit attached to the fixture can be installed at any desired position. Therefore, it can be installed in a position that does not get in the way when performing operations such as taking items out of or placing items on a shelf T. For example, if the vibration unit is attached to the underside of the shelf, it is not necessary to install the vibration unit on the top surface of the shelf, thus securing a placeable area on the shelf, providing a wider area for placing and taking items, and improving operability. Furthermore, if the vibration unit is installed at the back of the shelf, it can be positioned to avoid traffic flow when removing or placing items from the front of the shelf, thereby improving operability. Furthermore, if the vibration unit is installed in a position that is not easily visible from the front of shelf board T, it is possible to ensure that the aesthetic design is not compromised even when the vibration unit is attached to the shelf.

[0043] Figure 6 is a schematic block diagram illustrating the functions of the management system S. In this embodiment, an example is described in which the management system S is a smart shelf system. In the smart shelf system, the display shelf is an example of a fixture. The display shelf is a shelf having multiple shelves. In the smart shelf system, the products are an example of objects placed on the fixture and objects taken out of the fixture. Here, the display shelf may be shelf T as shown in Figure 1. Furthermore, the display shelf is equipped with a vibration unit. At least one of the vibration units 15, 25, 35, 36, 37, 35A, 35B, 37A, etc., described above can be used.

[0044] The management system S comprises a user terminal 2, a sensor device 3, a vibration generation location estimation system 4, a product management system 5, and a display device 6. The user terminal 2 and the vibration generation location estimation system 4 are connected in a communication manner. The sensor device 3 and the vibration generation location estimation system 4 are connected in a communication manner. The vibration generation location estimation system 4 and the product management system 5 are connected in a communication manner. The product management system 5 and the display device 6 are connected in a communication manner.

[0045] User terminal 2 is the device operated by the user. User terminal 2 can be, for example, a PC (personal computer), a smartphone, or a tablet device. The user uses applications and web browsers installed on user terminal 2 to input and confirm various types of information.

[0046] For example, the user operates the user terminal 2 to input training data that the vibration location estimation system 4 uses for machine learning. Through machine learning using this training data, a trained model is generated that is used to estimate the location where vibration occurs when a predetermined force is applied to the display shelf (hereinafter also referred to as the "vibration location"). The vibration location is also the location where the predetermined force is applied. The predetermined force is, for example, the force applied to the display shelf when a product is placed on it, or the force applied to the display shelf when a product is removed from it. Furthermore, the user can operate user terminal 2 to check the vibration location estimation results from the vibration location estimation system 4.

[0047] Sensor device 3 detects the force applied to the display shelf. This sensor device 3 can, for example, use the sensor 14 described above. Sensor device 3 transmits the detection result as sensor information to the vibration generation location estimation system 4.

[0048] The vibration generation location estimation system 4 includes a learning system 40 and an estimation system 50. The learning system 40 is a system that generates trained models. The learning system 40 can be implemented, for example, by a server device or a PC. The learning system 40 generates a trained model using training data received from the user terminal 2. The learning system 40 also inputs sensor information received from the estimation system 50 into the trained model and transmits position information indicating the vibration generation location to the estimation system 50.

[0049] The estimation system 50 is a system that estimates the location where vibration occurs on a display shelf. The estimation system 50 can be implemented, for example, by a server device or a PC. The estimation system 50 estimates the vibration generation location using a trained model generated by the learning system 40. Specifically, the estimation system 50 acquires sensor information from the sensor device 3 and transmits it to the learning system 40, and receives location information output from the trained model from the learning system 40 to estimate the vibration generation location. The estimation system 50 transmits the estimation results to the user terminal 2 and the product management system 5.

[0050] The product management system 5 is a system for managing products. For example, the product management system 5 manages the inventory of products displayed on shelves and displays product descriptions and related advertisements for products picked up by customers on the display device 6. The product management system 5 has product information that associates, for example, the location information of a product, inventory information, product descriptions, and related advertisements. Based on the location information indicated by the estimation results received from the estimation system 50, the product management system 5 updates the inventory information associated with that location information, and retrieves product descriptions and related advertisements from the product information and displays them on the display device 6.

[0051] Display device 6 is a display device that displays various types of information. Display device 6 can be, for example, a liquid crystal display, a plasma display, an organic electro-luminescence (OLED) display, or a signage display. Display device 6 is not limited to a display, and may also be a projector, for example. In the case of a projector, various types of information are displayed by projecting content from the projector onto a screen, wall, or the like. Display device 6 is installed in a store, for example, to display information for customers. Display device 6 displays product descriptions and related advertisements received from the product management system 5 as information for customers.

[0052] Figure 7 is a block diagram showing an example of the functional configuration of the learning system 40 according to this embodiment. As shown in Figure 7, the learning system 40 includes a communication unit 401, a storage unit 402, a learning data acquisition unit 403, a learning unit 404, an output processing unit 405, and a control unit 406.

[0053] The communication unit 110 has the function of sending and receiving various types of information. For example, the communication unit 110 receives training data from the user terminal 2. The communication unit 110 also receives sensor information from the estimation system 50. Furthermore, the communication unit 110 transmits position information of vibration generation locations output from the trained model to the estimation system 50. The communication by the communication unit 110 may be wireless communication, wired communication, or a combination of wireless and wired communication.

[0054] The memory unit 402 has the function of storing various types of information. The memory unit 402 includes a memory area for storing training data and a memory area for storing trained models. The training data consists of a set of data that includes positional information indicating the location where vibration occurs when a predetermined force is applied to any point on the display shelf, and sensor information corresponding to the variation in that vibration. The training data is prepared in advance by the user through measurements, etc. The user sets the location information of the training data according to the shape of the display shelf (e.g., number of shelves) and the location to be recognized by the application. The location information is given as (x, y, L), for example, by providing information L indicating which of multiple shelves it is and the 2D coordinates (x, y) on the surface of the shelf. Alternatively, the location information may be given as 3D coordinates (x, y, z) on the display shelf. In this case, the position in the height direction may be expressed as L, which identifies which shelf it is, or as z, which is the height relative to the bottom of the shelf. The user performs an operation such as placing or removing an object at the location indicated by the set location information, causing the sensor device 3 to detect the sensor information at that time. The user acquires the set location information and the sensor information detected by the sensor device 3 as a single training data. The user inputs the acquired training data into the user terminal 2 and transmits it to the learning system 40. This allows the relationship between location information, including height information, and detected vibrations to be learned as training data. The more data in the training data, the higher the accuracy of the output results of the trained model can be.

[0055] The pre-trained model memory area stores the pre-trained model.

[0056] The storage unit 402 is composed of a storage medium, such as an HDD (Hard Disk Drive), SSD (Solid State Drive), flash memory, EEPROM (Electrically Erasable Programmable Read Only Memory), RAM (Random Access read / write Memory), ROM (Read Only Memory), or any combination of these storage media.

[0057] The learning data acquisition unit 403 acquires learning data. For example, the learning data acquisition unit 403 acquires sensor information obtained from a sensor that detects vibrations in which at least one of the duration or intensity of the vibrations generated in response to an operation is changed, when a vibration unit is installed at any of the positions of a fixture in which multiple shelves are provided in the height direction, and the vibration unit vibrates in response to an operation in which an object is placed on or removed from any of the shelves of the fixture. The learning data acquisition unit 403 also acquires position information including the position in the height direction of the operated fixture. The learning data acquisition unit 403 may also acquire sensor information including the result of detecting vibrations in which both the duration and intensity of the vibrations generated in response to an operation have been changed. The learning data acquisition unit 403 acquires the learning data transmitted from the user terminal 2 via the communication unit 110 and writes it to the learning data storage area of ​​the storage unit 402 for storage.

[0058] The learning unit 404 generates a trained model that learns the relationship between sensor information and location information using the acquired training data. For example, the learning unit 404 generates a trained model by machine learning using the training data acquired by the training data acquisition unit 403. Specifically, the learning unit 404 uses the training data acquired by the training data acquisition unit 403 to learn the relationship between sensor information and location information, and generates a trained model that outputs location information when sensor information is input. The learning unit 404 writes the generated trained model to the trained model memory area of ​​the memory unit 402 and stores it.

[0059] The learning unit 404 uses training data to train a neural network model using deep learning. The neural network model is, for example, an LSTM (Long Short-Term Memory), which is a type of RNN (Recurrent Neural Network) suitable for time-series signal input. The learning unit 404 may perform preprocessing such as standardization or normalization on the training data before inputting it into the model.

[0060] The output processing unit 405 has a function to control the output of various types of information. For example, the output processing unit 405 causes the output results obtained from the trained model to be transmitted to the estimation system 50 via the communication unit 110.

[0061] The control unit 406 controls the overall operation of the learning system 40. The control unit 406 is implemented, for example, by causing the CPU (Central Processing Unit) provided as hardware in the learning system 40 to execute a program.

[0062] Figure 8 is a block diagram showing an example of the functional configuration of the estimation system 50 according to this embodiment. The estimation system 50 includes a communication unit 501, a storage unit 502, a sensor information acquisition unit 503, an estimation unit 504, a weight calculation unit 505, an output processing unit 506, and a control unit 507.

[0063] The communication unit 501 transmits and receives various types of information. For example, the communication unit 501 receives sensor information from the sensor device 3. The communication unit 501 also transmits the sensor information to the learning system 40. The communication unit 501 also receives location information of the vibration generation location from the learning system 40. The communication unit 501 also transmits the estimated vibration generation location to the user terminal 2 or the product management system 5. Furthermore, communication by the communication unit 501 may be wireless communication, wired communication, or a combination of wireless and wired communication.

[0064] The storage unit 502 is composed of a storage medium, such as an HDD, SSD, flash memory, EEPROM, RAM, ROM, or any combination thereof.

[0065] The sensor information acquisition unit 503 acquires sensor information. The sensor information acquisition unit 503 acquires sensor information obtained from a sensor that detects vibrations in which at least one of the duration or intensity of the vibrations generated in response to an operation is changed. The vibration unit is provided at one of the positions of a fixture in which multiple shelves are provided in the height direction, and the vibration unit vibrates in response to vibrations generated when an operation is performed in response to an operation in which an object is placed on or removed from one of the shelves of the fixture. The sensor information acquisition unit 503 may detect vibrations in which both the duration and intensity of the vibration generated in response to the operation have changed. In this case, the sensor information acquisition unit 503 can detect a continuous pattern of vibration intensity change within the duration by detecting vibrations in which both the duration and intensity of the vibration generated in response to the operation have changed. That is, the sensor information acquisition unit 503 may detect the change in intensity within the duration as a change amount pattern. For example, the sensor information acquisition unit 503 acquires sensor information received by the communication unit 501 from the sensor device 3. This sensor information may include not only the result of vibration detection but also the result of weight detection. When the sensor information acquisition unit 503 acquires the result of weight detection, the weight calculation unit 505, which will be described later, can calculate the difference in weight.

[0066] The estimation unit 504 estimates the vibration generation location. For example, the estimation unit 504 estimates the vibration generation location using a trained model. The estimation unit 504 inputs the acquired sensor information into a trained model that has learned the relationship between sensor information and the position on the operated furniture, and obtains position information including the position in the height direction on the operated furniture, thereby estimating the position on the operated furniture. Specifically, the estimation unit 504 transmits the sensor information acquired by the sensor information acquisition unit 503 from the communication unit 210 to the learning system 40, and inputs it into the trained model stored in the memory unit 402. The estimation unit 504 acquires position information output from the trained model from the learning system 40 via the communication unit 501. Then, the estimation unit 504 estimates the vibration generation location based on the position information indicated by the acquired output result. Specifically, the estimation unit 504 estimates that the position indicated by the position information indicated by the acquired output result is the vibration generation location.

[0067] The weight calculation unit 505 determines the difference in the weight of the fixture from the detection results of the weight sensors obtained before and after the operation.

[0068] The output processing unit 506 has a function to control the output of various types of information. For example, the output processing unit 506 causes the estimation result from the estimation unit 504 to be transmitted to the communication unit 501 to the user terminal 2 or the product management system 5.

[0069] The control unit 507 has the function of controlling the overall operation of the estimation system 50. The control unit 507 is implemented, for example, by causing the CPU, which is provided as hardware in the estimation system 50, to execute a program.

[0070] Figures 9 and 10 are images representing the simulation results of vibrations transmitted to a shelf when an object is placed on the shelf. Figure 9 shows the simulation results when no vibration unit is installed on the shelf, and Figure 10 shows the simulation results when a vibration unit is installed on the shelf. This section presents the results of a Finite Element Method (FEM) analysis of the changes in vibration characteristics when an operation is performed on the shelf.

[0071] Figure 9 shows the results of an FEM analysis performed when an object is placed at position A, approximately in the center of the top surface of the uppermost shelf, in a shelf without a vibration unit. The results show that the vibrations propagated to the bottom of the shelf propagate in a symmetrical manner with respect to the excitation point on the plane. In contrast, the shelf in Figure 10 is the same shape and structure as the shelf in Figure 9, but differs in that it is equipped with vibration units. Here, the vibration units, which are vortex members, are arranged in a pattern of 2 vertically and 12 horizontally for each of the four support columns of the shelf, and the results of an FEM analysis are shown for the case where an object is placed at the same position A as in Figure 9. As a result, the vibration propagated to the bottom of the shelf showed a different propagation pattern than that shown in Figure 9 (indicated by B).

[0072] Thus, the vibration detection results obtained by the sensor differ when a vibration unit is installed compared to when there is no vibration unit. Here, a single vibration unit may be installed on the shelf, or multiple vibration units may be combined to edit their vibration characteristics and then installed on the shelf. By editing the vibration characteristics of the vibration unit to make it easier to identify the operated position in the height direction, or by arbitrarily changing the mounting position on the shelf, it is possible to detect the operated position in the height direction while having a high degree of design flexibility. The ease of identifying the position in the height direction is such that, for example, when the shelves of the object being operated are different, the greater the degree of difference in the FEM analysis results, the easier it is to identify. Therefore, by changing various conditions in advance using FEM analysis and checking the analysis results, the ease of identifying the position in the height direction can be examined.

[0073] In this way, by attaching the vibration unit, it becomes easier to classify the position not only in the horizontal direction but also in the vertical direction using the trained model in the vibration generation position estimation system 4, thereby improving the accuracy of position identification in the vertical direction. In other words, the trained model is used to classify the position in each direction based on the detection results of the sensor device, but the vibration unit is used to make the classification pattern of the trained model clearer.

[0074] Figure 11 is a diagram illustrating the processing flow of the management system S, and Figure 12 is a flowchart showing an example of the operation of the vibration generation location estimation system 4 according to this embodiment. Learning Phase First, the operator places an object on one of the shelves of shelf T, or removes an object that is already on one of the shelves of shelf T. This applies a force to shelf T, and vibrations are transmitted to shelf T. The vibration unit vibrates in response to the vibrations generated in shelf T (Figure 11: Step S10). The sensor device detects the vibrations, which include the vibrations applied in response to the operation on shelf T and the vibrations generated by the vibration unit (Figure 11: Step S11).

[0075] Meanwhile, the operator inputs operation position data from the user device, indicating the number of shelves T that have been operated on. The control unit 406 of the learning system 40 acquires the detection result output from the sensor device and the operation position data input to the user device from the communication unit 401 and writes them to the storage unit 402 (Figure 11: Step S12). Here, multiple operations are performed on shelf T. Each time the operator performs an operation on shelf T, the operator inputs the height position where the operation occurred as operation position data from the user device. As a result, each time an operation is performed on shelf T, the combination of the detection result output from the sensor device and the operation position data is acquired by the learning system 40 and written as learning data to the storage unit 402 by the control unit 406.

[0076] The learning data acquisition unit 131 of the learning system 40 reads the learning data from the storage unit 402 when a certain amount of learning data has been written to the storage unit 402 (Figure 11: Step S13, Figure 12: Step S101). When training data is read, the learning unit 404 performs machine learning using the training data and generates a trained model (Figure 11: Step S14, Figure 12: Step S102).

[0077] The learning unit 404 writes the generated trained model to the trained model memory area of ​​the memory unit 402 for storage (Figure 11: Step S15, Figure 12: Step S103).

[0078] Execution Phase Once the trained model is generated, the estimation system 50 can perform location information estimation. Here, when an operation is performed to place an object on shelf T, or to remove an object placed on shelf T, the sensor device outputs detection results as sensor information corresponding to the operation performed on shelf T and the vibrations generated by the vibration unit. The sensor information acquisition unit 503 of the estimation system 50 acquires sensor information output from the sensor device (Figure 12: Step S104). Specifically, the sensor information acquisition unit 503 acquires sensor information detected by the sensor device 3 and received by the communication unit 210 of the estimation system 50. Next, the estimation unit 232 of the estimation system 50 transmits the sensor information to the learning system 40 (Figure 12: Step S105). Specifically, the estimation unit 504 transmits the sensor information acquired by the sensor information acquisition unit 503 from the communication unit 210 to the learning system 40, and inputs it into the trained model.

[0079] Next, the control unit 130 of the learning system 40 inputs sensor information into the trained model (Figure 11: step S16, Figure 12: step S106). Specifically, the control unit 130 inputs the sensor information received by the communication unit 110 from the estimation system 50 into the trained model stored in the storage unit 402. Next, the control unit 130 causes the communication unit 110 to transmit the output result from the trained model to the estimation system 50 (Figure 12: Step S107).

[0080] Next, the communication unit 210 of the estimation system 50 receives the output result transmitted from the learning system 40 (Figure 12: Step S108). Next, the estimation unit 232 estimates the vibration generation location based on the output result (Figure 11: step S17, Figure 12: step S109). Specifically, the estimation unit 232 estimates the location indicated by the location information of the output result as the vibration generation location. Next, the output processing unit 233 of the estimation system 50 outputs the estimation result (Figure 11: step S18, Figure 12: step S110). Specifically, the output processing unit 233 transmits the estimation result from the communication unit 210 to the user terminal 2 or the product management system 5.

[0081] Figure 13 shows an example of how the estimation results are displayed on the screen of the user terminal 2 or the display device 6. The display screen shows the shelves 11a, 11b, and 11c of shelf T, along with the arrangement of the three sensors 14 on the underside of shelf 11a. When estimation results are obtained, including the estimated coordinates in the horizontal direction and the estimated position in the vertical direction, a figure representing the object 18 is displayed on the display screen at the position corresponding to the estimation results. For example, if the estimated position in the vertical direction indicates that the object 18 is placed on the second shelf from the top, a figure representing the object 18 is displayed on the upper side of shelf 11b. In this case, the horizontal position where the object 18 is displayed is based on the horizontal position obtained as an estimation result.

[0082] Furthermore, when object 18 is removed from the shelf, the displayed shape of object 18 disappears based on the estimated position of its removal. This makes it possible to see on the screen that object 18 is no longer on the shelf. Furthermore, to determine whether object 18 was placed on shelf T or removed from shelf T, the weight calculation unit 505 calculates the difference between the weight detection result before the operation and the weight detection result after the operation, based on the weight detection result obtained from the sensor device. The estimation unit 504 can estimate that the item was removed if the weight has decreased before and after the operation, and estimate that the item was placed if the weight has increased before and after the operation.

[0083] By displaying these estimation results on the display device 6, it becomes possible to display, for example, what products are placed on the product display shelves and on which shelf they are placed, without having to photograph the shelves with a camera. This allows customers or store employees to check the product display status on the display screen of the display device 6 without having to directly check the product shelves or even if they are located away from the shelves. Furthermore, if the display device 6 is installed in an area where store employees perform their duties, such as the back room of a store, it can be used to check whether or not products need to be restocked on the shelves. Also, if the target of management is not the product shelves but the shelves in the warehouse, store employees can understand the inventory status by checking the display screen of the display device 6.

[0084] In the embodiments described above, the smart shelf system was described as being installed in a store, but it may also be installed in general households, public facilities, and other types of facilities. For example, a pump-type container capable of holding liquids such as shampoo or liquid soap can be placed on the shelf. In this case, when a user presses the pump of the pump-type container to dispense liquid detergent, the vibrations generated in response to this operation can be detected. This makes it possible to identify which shelf the operation was performed on. Furthermore, when liquid detergent is dispensed by pressing the pump, the weight of the container before and after the operation is detected, and the amount of liquid used can be determined by calculating the difference. In this case, in addition to displaying position information including the height position where the operation was performed on the display device, the remaining amount of liquid may also be displayed on the display device in addition to the calculated amount used. In this case, the display device may be installed near the shelf. This makes it easier to check the remaining amount when using the pump-type container. In this case, if sensor device 3 is a weight sensor, it is possible to obtain information that allows for the detection of position in the height direction, and also to detect the difference in weight (amount used) without adding any new sensors.

[0085] According to the embodiments described above, for example, as shown in Figure 1, a shelf having three shelves, an upper, middle, and lower shelf, where a combined vibration unit of vibration unit 35A and vibration unit 35B shown in Figure 5F is provided on the underside of the uppermost shelf, and a combined vibration unit of vibration unit 35 and vibration unit 37A shown in Figure 5G is provided on the underside of the middle shelf, when an object is manipulated on the upper shelf, vibration is transmitted to the vibration unit provided on the underside of the upper shelf, causing this vibration unit to vibrate. This strengthens and sustains the vibration of the entire shelf, starting from the upper shelf, and this vibration can be detected by a sensor device provided at the bottom of the shelf.

[0086] Furthermore, when an item is moved on the middle shelf, the vibration is transmitted to a vibration unit located on the underside of the middle shelf, causing this unit to vibrate. In this case, a damper is provided as the vibration unit 37A, which acts to suppress the vibration of the entire shelf. Thus, by varying the characteristics of the vibration units and the mounting position of the vibration units, the vibrations detected by the sensor device can be made to differ depending on the height of the shelf (level). This allows the learning system 40 to learn how to easily identify the height, making it easier to distinguish between different heights.

[0087] For example, if the horizontal axis represents the maximum amplitude and the horizontal axis represents the duration of the vibration, and the results are plotted based on the detection results detected by the sensor device, the results can be represented as shown in Figure 14. These detection results can then be clustered to identify which stage the operation corresponds to.

[0088] In the above-described embodiment, the surface on which objects are placed or removed from the shelf may be a smooth surface, but it may also have minute irregularities or be made rough. This can increase the vibrations generated when the operation is performed.

[0089] The learning system 40 and estimation system 50 in the above-described embodiment may be implemented using a computer. In that case, the program for implementing this function may be recorded on a computer-readable recording medium, and the program recorded on this recording medium may be loaded into the computer system and executed. Here, "computer system" includes hardware such as the OS and peripheral devices. Furthermore, "computer-readable recording medium" refers to portable media such as flexible disks, magneto-optical disks, ROMs, CD-ROMs, and storage devices such as hard disks built into the computer system. Moreover, "computer-readable recording medium" may also include those that dynamically hold programs for a short period of time, such as communication lines used when transmitting programs via networks such as the Internet or communication lines such as telephone lines, and those that hold programs for a certain period of time, such as volatile memory inside the computer system that acts as a server or client in such cases. Furthermore, the above-mentioned program may be for implementing a part of the above-mentioned function, or it may be a program that can implement the above-mentioned function in combination with a program already recorded in the computer system, or it may be implemented using a programmable logic device such as an FPGA (Field Programmable Gate Array).

[0090] Although embodiments of this invention have been described in detail above with reference to the drawings, the specific configuration is not limited to these embodiments and includes designs and the like that do not depart from the spirit of this invention. [Explanation of symbols]

[0091] 2...User terminal, 3...Sensor device, 4...Vibration generation location estimation system, 5...Product management system, 6...Display device, 11a,11b,11c...Shelf board, 12a,12b,12c,12d...Support column, 14,14a,14b,14c,14d...Sensor, 15,15a,15b,15c,15d...Vibration unit, 18,18a,18b...Object, 20,35c1,35c2...Base member, 25,25a,25b,35,35A,35B,36...Vibration unit, 25a1,251b1,250, 252b1...Vortex member, 25a3,25b3...Connecting members, 36c1,36c2,38a,38b,38c,38d,220...Base members, 37,37A...Vibration unit, 37a,37b,37c,37d...Damper, 251b...First unit, 251b2,25a2,252b2...Pin, 252b...Second unit, 255...Starting end, 256...Ending end, 257...Center point 351...Slide switch, 40...Learning system, 50...Estimation system, 403...Learning data acquisition unit, 401, 501...Communication unit, 402, 502...Storage unit, 403...Learning data acquisition unit, 404...Learning unit, 405, 506...Output processing unit, 406, 507...Control unit, 503...Sensor information acquisition unit, 504...Estimation unit, 505...Weight calculation unit, T...Shelf, S...Management system

Claims

1. A sensor information acquisition unit acquires sensor information from a sensor that detects vibrations in which at least one of the duration or intensity of the vibrations generated in response to an operation in which an object is placed on or removed from any of the shelves of the fixture is changed, and the vibration unit is vibrated in response to the vibrations generated in response to the operation, An estimation unit inputs the acquired sensor information into a trained model that has learned the relationship between the sensor information and the position of the operated fixture to obtain position information including the position in the height direction of the operated fixture. An estimation system having the following characteristics.

2. The number of vibration units may be one or more, and they may be attached to the fixture at any position. The estimation system according to claim 1.

3. The vibration unit is configured such that a first unit having at least one of the characteristics of extending the duration of the vibration or increasing the intensity of the vibration, and a second unit having at least one of the characteristics of shortening the duration of the vibration or decreasing the intensity of the vibration, can be combined to edit the characteristics of changing at least one of the duration or intensity of the vibration generated in response to the operation. The estimation system according to claim 2.

4. The vibration unit is At least one of several types of first units with different characteristics, At least one of several types of second units with different characteristics, These are combined The estimation system according to claim 3.

5. The aforementioned sensor is a weight sensor. The estimation system includes a weight calculation unit that determines the difference in the weight of the fixture from the detection results of the weight sensor obtained before and after the operation is performed. An estimation system according to any one of claims 1 to 3, having the following:

6. A vibration unit is provided at one of the positions of a fixture having multiple shelves in the height direction, and the vibration unit vibrates in response to vibrations generated when an object is placed on or removed from one of the shelves of the fixture, thereby acquiring sensor information obtained from a sensor that detects vibrations in which at least one of the duration or intensity of the vibration generated in response to the operation has changed, and position information including the position in the height direction of the operated fixture, A learning unit generates a trained model that learns the relationship between the sensor information and the position information using the acquired training data, A learning system that has the following features.

7. A vibration unit is provided at one of the positions of a fixture in which multiple shelves are provided in the height direction, and the vibration unit vibrates in response to vibrations that occur when an object is placed on or removed from one of the shelves of the fixture, thereby acquiring sensor information from a sensor that detects vibrations in which at least one of the duration or intensity of the vibrations that occur in response to the operation has changed, The acquired sensor information is input into a trained model that has learned the relationship between the sensor information and the position of the operated fixture, thereby obtaining position information including the position in the height direction of the operated fixture. Estimation method.

8. A vibration unit is provided at one of the positions of a fixture having multiple shelves in the height direction, and the vibration unit vibrates in response to vibrations generated when an object is placed on or removed from one of the shelves of the fixture, thereby acquiring sensor information obtained from a sensor that detects vibrations in which at least one of the duration or intensity of the vibration generated in response to the operation has changed, and position information including the position in the height direction of the operated fixture, Using the acquired training data, a trained model is generated that learns the relationship between the sensor information and the position information. Learning methods.

9. A vibration unit is provided at one of the positions of a fixture in which multiple shelves are provided in the height direction, and the vibration unit vibrates in response to vibrations that occur when an object is placed on or removed from one of the shelves of the fixture, thereby acquiring sensor information from a sensor that detects vibrations in which at least one of the duration or intensity of the vibrations that occur in response to the operation has changed, The acquired sensor information is input into a trained model that has learned the relationship between the sensor information and the position of the operated fixture, thereby obtaining position information including the position in the height direction of the operated fixture. A program that causes a computer to perform a task.

10. A vibration unit is provided at one of the positions of a fixture having multiple shelves in the height direction, and the vibration unit vibrates in response to vibrations generated when an object is placed on or removed from one of the shelves of the fixture, thereby acquiring sensor information obtained from a sensor that detects vibrations in which at least one of the duration or intensity of the vibration generated in response to the operation has changed, and position information including the position in the height direction of the operated fixture, Using the acquired training data, a trained model is generated that learns the relationship between the sensor information and the position information. A program that causes a computer to perform a task.

Images