Acoustic inventory of products

The system addresses the challenges of noise interference and installation complexity by converting product displacement into structure-borne sounds, enabling effective and flexible acoustic inventory management with improved sensitivity and ease of use.

WO2024242611A1PCT designated stage expired Publication Date: 2024-11-28PACKUSTICS AB
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
PCT/SE2024/050487
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-22
Filing Date
2024-05-20
Publication Date
2024-11-28

AI Technical Summary

Technical Problem

Existing acoustic inventory systems face challenges with background noise interference, sensitivity, directionality, and complex installation due to the need for precise placement of acoustic sensors near product movement sources, making them difficult to implement effectively in retail and warehouse settings.

Method used

A system that converts product displacement into characteristic structure-borne sounds using a converting element physically interacting with the product, allowing an acoustic sensor placed at a distance to detect these sounds through a structure, thereby reducing interference from background noise and simplifying installation.

Benefits of technology

This approach enables reliable and flexible product displacement detection with reduced noise interference and easier system installation, allowing for efficient inventory management by distinguishing between different products using unique sound signatures.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure SE2024050487_28112024_PF_FP_ABST
    Figure SE2024050487_28112024_PF_FP_ABST
Patent Text Reader

Abstract

The present disclosure relates to a system (100) for acoustic inventory of products. The system (100) comprises at least one converting element (10) for converting a displacement of a product (90) to a characteristic sound, an acoustic sensor (40) configured to be mechanically connected to a structure (50) at a distance from the at least one converting element (10), and a processor (42, 62) coupled to the acoustic sensor (40). The at least one converting element (10) is configured to physically interact with the product (90) upon displacement of the product (90) such that the physical interaction causes the converting element (10) to generate a characteristic sound, and to induce the characteristic sound in the structure (50). The acoustic sensor (40) is configured to register the characteristic sound generated by the converting element (10) as structure-borne sound propagating through the structure (50). The processor (42, 62) is configured to detect the displacement of the product (50) based on the characteristic sound registered by the acoustic sensor (40).
Need to check novelty before this filing date? Find Prior Art

Description

[0001] Acoustic inventory of products

[0002] TECHNICAL FIELD

[0003] The present disclosure relates to a system for acoustic inventory of products and a method for acoustic inventory of products. More specifically, the disclosure relates to a system for acoustic inventory of products and a method for acoustic inventory of products as defined in the introductory parts of the independent claims.

[0004] BACKGROUND ART

[0005] Monitoring sounds of product movements is a specific application of acoustic monitoring technology. It involves using microphones or acoustic sensors to detect and analyse sounds associated with the movement of products in a retail or warehouse setting.

[0006] The technology typically involves analysis of acoustic signatures of specific sounds associated with different types of product movement, such as the sound of a product being picked up, placed on a shelf, or moved on a conveyor belt. These sounds may be used to track the movement of products through the store or warehouse, allowing for real-time inventory tracking and management.

[0007] There are a number of different approaches to implementing sound-based inventory monitoring systems. Some systems use specialized microphones or acoustic sensors that are installed in specific locations throughout the store or warehouse, while others may rely on existing security cameras or other equipment to capture audio data.

[0008] The captured audio data may be processed using machine learning algorithms that can identify and categorize specific sounds, such as the sound of a particular product being moved or the sound of a shopping cart being rolled. These algorithms can be trained to recognize different products and product movements, allowing for accurate tracking of inventory levels and movement.

[0009] Overall, sound-based inventory monitoring is a promising technology for retailers and warehouse managers looking to improve inventory management, reduce shrinkage, and optimize store or warehouse layouts. US10586205 describes a system for monitoring product movement using acoustic sensors in a retail or warehouse setting. The system is configured to recognize the acoustic signatures of specific sounds associated with different types of product movement, such as the sound of a product being picked up from a shelf or the sound of a product being placed back on the shelf. The system may comprise a sound emitter device coupled with a support structure, such as a shelf. The sound emitter device may comprise a device configured to produce a sound in response to the movement of the support structure and / or an item placed on the support structure. The system may include a plurality of sound emitters where each emitter is configured to produce a sound that is distinguishable based on its sound characteristics, such as sound frequency. The sound emitter device may for example comprise a bendable foil or a clicker to produce the characteristic sound. The system further comprises a sound sensor or sound sensor array for detecting the sounds generated by the sound emitters and transmit the sound to a stock tracking system. The sound sensors may, e.g., be installed in the ceiling, pillars, beams, etc., of a shopping space.

[0010] There are several problems associated with the type of system disclosed in US10586205, including but not limited to background noise, sensitivity, directionality and interference. For example, it may be difficult to separate the sound from the sound emitters from background noise generated by shopping activity and nearby electronic devices. Furthermore, installing and configuring the system in a shopping space may be a complex and time-consuming task.

[0011] SUMMARY

[0012] It is an object of the present disclosure to mitigate, alleviate or eliminate one or more of the above-identified deficiencies and disadvantages in the prior art, and solve at least one of the above mentioned problems.

[0013] It is a particular object of the present disclosure to provide a system for acoustic inventory of products, which is simple, compact and easy to assemble and use.

[0014] According to a first aspect of the present disclosure, there is provided a system for acoustic inventory of products. The system comprises at least one converting element for converting a displacement of a product to a characteristic sound; an acoustic sensor configured to be mechanically connected to a structure at a distance from the at least one converting element; and a processor coupled to the acoustic sensor. The at least one converting element is configured to physically interact with the product upon displacement of the product, whereby the physical interaction causes the converting element to generate a characteristic sound and induce the characteristic sound in the structure. The acoustic sensor is configured to register the characteristic sound generated by the converting element as structure-borne sound propagating through the structure, and the processor is configured to detect the displacement of the product based on the characteristic sound registered by the acoustic sensor.

[0015] By means of the system as disclosed herein, a displacement or movement of a product that normally does not emit a sound or is difficult to register a sound from, can be converted to a characteristic sound that can be registered by means of an acoustic sensor. Thus, the system according to the present disclosure can convert an inaudible displacement of a product to audible sound by means of the converting element. The converting element is configured to physically interact with the product and thereby generate a sound, which is induced into the structure by the converting element. The converting element is suitably a simple mechanical construction which is mechanically coupled to the structure. In some embodiments, the converting element is attached to the structure. In other embodiments, the converting element may be integrally formed in the structure. In yet other embodiments, the converting element may be comprised in, or constituted by, a separate item that is placed on the structure and mechanically coupled to the structure by the weight exerted on the structure by the item.

[0016] The converting element will thus physically interact with the structure such that the characteristic sound, generated by the converting element in response to the physical interaction with the product, is induced into and propagates through the structure and can be registered by the acoustic sensor. By means of the structure, the acoustic sensor and the converting element will, at least temporarily, be mechanically connected. The structure may for example be a shelf in a store or a warehouse, or a purpose-made structure for enhanced mechanical coupling with the converting element and / or enhanced acoustic transmission of the characteristic sound propagating from the converting element to the acoustic sensor, via the structure. The structure-borne sound induced into the structure can propagate a long way and the acoustic sensor may therefore be arranged far away from the converting element and the location of the displacement of the product. The possibility of positioning the acoustic sensor far away from the product increases system flexibility and eliminates the need for acoustic sensors or other electronic equipment in the immediate surroundings of the product. Furthermore, the system allows multiple converting elements to be connected to the same structure and a single acoustic sensor can thereby be used to detect displacement of anyone of a plurality of different products on / along the same structure. By having an acoustic sensor that registers the characteristic sound of the converting element as structure-borne sound, surrounding sounds or noise will not interfere with the registered sound compared to when using a microphone for detecting airborne sound. The characteristic sound generated by the converting element will this way easily be registered and a reliable system for detecting displacement of products is achieved.

[0017] The characteristics of the sound generated by the converting element depend on the shape and configuration of the converting element. The characteristics of the sound allows the processor to determine that the registered sound originated from the converting element, and that a displacement of a product has occurred at the location of the converting element.

[0018] According to some embodiments, the system comprises a plurality of converting elements, wherein the characteristic sound generated by each converting element is associated with a product or product type, wherein the processor is configured to determine which product or product type was displaced based on the characteristic sound registered by the acoustic sensor. This way, the system can distinguish between displacements of different products or product types and may be configured to keep track of products that are picked up from or put back on a warehouse shelf or similar structure.

[0019] According to some embodiments, the acoustic sensor is a single acoustic sensor configured to register a characteristic sound created by any one of the plurality of converting elements as structure-borne sound propagating through the structure. By using a single acoustic sensor a compact and cost-efficient system can be achieved. According to some embodiments, the converting element comprises a sound-generating portion including at least one vibrational element configured to be brought into vibration by the displacement of the product, wherein the configuration of the at least one vibrational element causes the characteristic sound to comprise at least one characteristic frequency and / or a plurality of frequencies forming a characteristic sequence of frequencies. The at least one characteristic frequency and / or sequence of frequencies gives the converting element a unique identity, which identity may be associated to a product or product type, e.g., to keep track of product stock.

[0020] According to some embodiments, the at least one vibrational element is integrally formed in or attached to the structure to which the acoustic sensor is mechanically connected. In some embodiments, the system may be provided in the form of a pre-fabricated structure with a plurality of integrated converting elements. This way, the pre-fabricated structure can serve as product underlay for automatic inventory of products to be placed on the underlay. By providing this type of product underlay with integrated converting elements, system installation and implementation is facilitated. Furthermore, this allows the structure of the underlay to be optimized for acoustic transmission of structure-borne sound, thereby facilitating detection of product displacement. Yet further, it brings about the advantage of being able to optimize the size and / or shape of the product underlay to the environment in which it is to be used. For example, the size and / or shape of the product underlay can be tailored to the size and shape of a warehouse shelf onto which the product underlay is to be placed.

[0021] Preferably, the acoustic sensor is removably attached to the structure. This is advantageous in that the acoustic sensor can be removed from the structure, allowing easy cleaning of the structure and the converting elements. An exemplary field of use of the above mentioned product underlay is within pharmaceutics, where the product underlay could be used to detect movement and keep track of stock of medical packagings or other pharmaceutical products. By using a removable acoustic sensor and selecting the materials of the product underlay appropriately, the product underlay can be cleaned by means of autoclaving.

[0022] According to some embodiments, the converting element further comprises an actuating portion configured to be brought into movement by the displacement of the product, wherein the at least one vibrational element of the sound-generating portion is configured to be brought into vibration by the movement of the actuating portion. In this case, the movement of the actuating portion may itself generate a characteristic sound indicating that the product has been displaced. However, the vibrational element that is brought into vibration by the movement of the actuating portion will add at least one characteristic frequency to the characteristic sound and the converting element will this way be given a unique identity.

[0023] According to some embodiments, the actuating portion comprises a snapping mechanism configured to perform a snapping movement into a stable state of the snapping mechanism upon removal of the product from the snapping mechanism, wherein the at least one vibrational element is configured to be brought into vibration at least partly by the snapping movement of the snapping mechanism. The snapping movement is a sudden movement caused by a quick release of energy that is stored in the snapping mechanism in response to a force exerted on the snapping mechanism by the weight of the product. The snapping movement causes a sound pulse of short duration, which sound pulse, in some embodiments, may trigger the processor to perform a frequency analysis of subsequently registered sound originating from vibrations of the at least one vibrational element. The snapping mechanism may be a mono-stable snapping mechanism that is brought out of its stable state by the weight of the product, causing the snapping mechanism to snap back into its stable state upon removal of the product.

[0024] According to some embodiments, the snapping mechanism comprises a bulging portion configured to be brought into an instable state of the snapping mechanism by the weight of the product, and to perform the snapping movement into the stable state of the snapping mechanism upon removal of the product.

[0025] According to some embodiments, the at least one vibrational element is attached to or integrally formed in a product rack or packaging, wherein the at least one vibrational element is configured to be brought into vibration through physical interaction with at least one protruding part of the product upon removal or insertion of the product out of or into the product rack or packaging.

[0026] According to some embodiments, the at least one vibrational element is integrally formed in the product rack or packaging and defined by at least one slit that extends into the rack or packaging from an edge of an opening of the rack or packaging, such that the at least one vibrational element physically interacts with the at least one protruding part of the product when the product is removed from or inserted into the product rack or packaging, via said opening. This way, a converting element for converting a movement of a product into a characteristic sound that can be registered by the acoustic sensor can be formed by means of one or more slits in a product rack or packaging. In some embodiments, the at least one slit may define a flap or a tongue of a product packaging, which flap physically interacts with the protruding part of the product when the product is removed from or inserted into the packaging, causing the flap to vibrate with a characterizing frequency that can be registered by the acoustic sensor and used by the processor to detect that a specific product has been removed from or inserted into the packaging.

[0027] According to some embodiments, the at least one protruding part of the product comprises one or more braille characters. Braille characters are commonly used on products to enable identification of the product by visually impaired persons, especially in the pharmaceutical industry. By forming the flap or tongue of the product packaging such that it physically interacts with braille characters of the product when the product is removed from or inserted into the product packing, the system can be configured to detect displacement of a product without modifying the product.

[0028] According to some embodiments, the product comprises at least two protruding parts, and wherein the sound-generating portion comprises at least two vibrational elements with different frequency characteristics configured to physically interact with a respective protruding part of the product upon removal and / or insertion of the product out of and into the product rack or packaging, wherein the processor is configured to determine whether the product is inserted into or removed from the product rack or packaging based on the frequencies of the characteristic sound registered by the acoustic sensor. Using two or more vibrational elements with different frequency characteristics make the characterising sound comprise at least two different frequencies or tones. The at least two vibrational elements and / or the at least two protruding parts of the product may be arranged such that there is at least a small time delay between a physical interaction between a first vibrational element and a first protruding part and a physical interaction between a second vibrational element and a second protruding part, when the product is inserted into or removed from the product rack or packaging. This way, by analysing the order of frequencies registered by the acoustic sensor, the processor can determine whether the product has been inserted into or removed from the product rack or packaging.

[0029] According to some embodiments, the product rack or packaging is configured to store a plurality of products in the form of liquid containers, wherein the processor is configured to estimate an amount of liquid stored in the product rack or packaging based on the characteristic sound registered by the acoustic sensor. The amount of liquid stored in the rack or packaging may affect the characteristic sound created by the converting element upon product displacement, e.g., the frequency of the characteristic sound, whereby the processor may be configured to use differences in the characteristic sound registered by the acoustic sensor to estimate the amount of liquid stored in the rack or packaging. The processor may further be configured to generate a signal informing a user of low liquid stock when the estimated amount of liquid falls below a predetermined value. This may be advantageous in, e.g., hair studios, where hair products, such as hair colouring agents, may be stored in the proposed product rack or packaging in order for personnel to be automatically informed about a shortage of hair products.

[0030] According to a second aspect of the present disclosure there is provided a method for acoustic inventory of products. The method comprises providing at least one converting element for converting a displacement of a product to a characteristic sound, wherein the at least one converting element is configured to physically interact with the product upon displacement of the product such that the physical interaction causes the converting element to generate a characteristic sound and induce the characteristic sound into a structure; registering the characteristic sound generated by the converting element as structure-borne sound propagating through the structure by means of an acoustic sensor mechanically connected to the structure at a distance from the at least one converting element; and detecting the displacement of the product based on the characteristic sound registered by the acoustic sensor by means of a processor coupled to the acoustic sensor.

[0031] It is to be understood that all features and advantages with regard to the above-described system are also applicable to the method for acoustic inventory, which method is typically performed by means of the system. According to some embodiments, the method comprises providing a plurality of converting elements, each generating a characteristic sound associated with a respective product or product type, and determining, by means of the processor, which product or product type was displaced based on the characteristic sound registered by the acoustic sensor.

[0032] According to some embodiments, the converting element comprises a sound-generating portion including at least one vibrational element configured to be brought into vibration by the displacement of the product, thereby causing the characteristic sound to comprise at least one characteristic frequency and / or a plurality of frequencies forming a characteristic sequence of frequencies, wherein the method comprises determining, by means of the processor, the product or product type based on the at least one characteristic frequency and / or the plurality of frequencies forming the characteristic sequence of frequencies of the characteristic sound.

[0033] According to some embodiments, the method comprises providing a structure to which the acoustic sensor is mechanically connectable, wherein the at least one vibrational element of the sound-generating portion is attached to or integrally formed in the structure.

[0034] According to some embodiments, the at least one vibrational element is attached to or integrally formed in a product rack or packaging, such that the at least one vibrational element is brought into vibration through physical interaction with at least one protruding part of the product upon removal or insertion of the product out of or into the product rack or packaging.

[0035] According to some embodiments, the product comprises at least two protruding parts and the sound-generating portion comprises at least two vibrational elements with different frequency characteristics, wherein the at least two vibrational elements are configured to physically interact with a respective protruding part of the product upon removal and insertion of the product out of and into the product rack or packaging. The method comprises determining, by means of the processor, whether the product is inserted into or removed from the product rack or packaging based on the frequencies of the characteristic sound registered by the acoustic sensor.

[0036] According to some embodiments, the product rack or packaging is configured to store a plurality of products in the form of liquid containers, wherein the method comprises estimating, by means of the processor, an amount of liquid stored in the product or packaging based on the characteristic sound registered by the acoustic sensor.

[0037] According to some embodiments, the product rack or packaging is configured to store a plurality of products in the form of liquid containers, wherein the processor is configured to estimate an amount of liquid stored in the product rack or packing based on the characteristic sound registered by the acoustic sensor.

[0038] According to a another aspect of the present disclosure there is provided a computer program comprising computer-readable instructions which, when executed by a processor of a system for acoustic inventory of products as disclosed herein, causes the processor to perform the step of detecting the displacement of the product based on the characteristic sound generated by the converting element and registered by the acoustic sensor. The computer program may further comprise instructions for causing the processor of the system to perform any of, or any combination of, the method steps performed by the processor in the abovedescribed method.

[0039] The computer program or parts of the computer program may reside in a data storage medium of the acoustic sensor. In some embodiments, the computer program may be a distributed application comprising several computer program components configured to perform different steps of the above-described method. For instance, the computer program may comprise a first program component or application residing in the acoustic sensor, a second program component or application residing in a network server, and a third program component or application in form of a client application for presentation of data and interaction with a user, residing in a client device. In another example, the computer program may comprise a web application accessible via a web browser of the client device.

[0040] According to another aspect of the present disclosure there is provided a computer program product comprising at least one computer-readable medium, such as a non-transitory memory hardware device, storing the above mentioned computer program.

[0041] The present disclosure will become apparent from the detailed description given below. The detailed description and specific examples disclose preferred embodiments of the disclosure by way of illustration only. Those skilled in the art understand from guidance in the detailed description that changes and modifications may be made within the scope of the appended claims.

[0042] BRIEF DESCRIPTIONS OF THE DRAWINGS

[0043] The above objects, as well as additional objects, features and advantages of the present disclosure, will be more fully appreciated by reference to the following illustrative and nonlimiting detailed description of example embodiments of the present disclosure, when taken in conjunction with the accompanying drawings. In the accompanying drawings, like reference numerals are used to denote the same or similar objects or elements throughout the various figures. Consistent with this notation, the use of a specific reference numeral in one drawing figure should be understood to represent the same object or element when it appears in subsequent figures, unless otherwise specified.

[0044] Figure 1 illustrates a system for acoustic inventory of products, according to an exemplary embodiment of the present disclosure.

[0045] Figures 2A-2C illustrate an exemplary embodiment of a converting element of the system for acoustic inventory of products illustrated in Figure 1.

[0046] Figure 3 illustrates an acoustic sensor of the system for acoustic inventory of products, according to an exemplary embodiment of the disclosure.

[0047] Figure 4 illustrates a system for acoustic inventory of products, according to another exemplary embodiment of the present disclosure.

[0048] Figure 5 illustrates an exemplary embodiment of a converting element of the system for acoustic inventory of products illustrated in Figure 4

[0049] Figure 6 is a flow chart illustrating a method for acoustic inventory of products, according to an exemplary embodiment of the disclosure.

[0050] DETAILED DESCRIPTION

[0051] The present disclosure will now be described with reference to the accompanying drawings, in which preferred example embodiments of the disclosure are shown. The disclosure may, however, be embodied in other forms and should not be construed as limited to the herein disclosed embodiments. The disclosed embodiments are provided to fully convey the scope of the disclosure to the skilled person.

[0052] It is to be understood that the terminology used herein is for purpose of describing particular embodiments only, and is not intended to be limiting. It should be noted that, as used in the specification and the appended claims, the articles "a", "an", "the", and "said" are intended to mean that there are one or more of the elements unless the context explicitly dictates otherwise. Thus, for example, reference to "a unit" or "the unit" may include several devices, and the like. Furthermore, the terms "comprising", "including", "containing" and similar wordings are intended to be open-ended transitional terms that do preclude the possibility of additional elements or steps.

[0053] Figure 1 schematically illustrates a system 100 for acoustic inventory of products, according to an exemplary embodiment of the present disclosure. The system 100 comprises at least one converting element 10 for converting a displacement of a product 90 to a characteristic sound, an acoustic sensor 40 configured to be mechanically connected to a structure 50 at a distance from the at least one converting element 10, and at least one processor 42, 62 coupled to the acoustic sensor 40. The at least one converting element 10 is configured to physically interact with the product 90 upon displacement of the product 90, such that the physical interaction causes the converting element 10 to generate a characteristic sound, and to induce the characteristic sound into the structure 50. The acoustic sensor 40 is configured to register the characteristic sound generated by the converting element 10 as structure-borne sound propagating from the converting element 10 to the acoustic sensor 40, through the structure 50. The at least one processor 42, 62 is configured to detect the displacement of the product 90 based on the characteristic sound registered by the acoustic sensor 40.

[0054] In the illustrated example, the system 100 comprises a plurality of converting elements 10, wherein the characteristic sound generated by each converting element is associated with a specific product 90 or a specific product type. This way, the at least one processor 42, 62 may determine which product or product type was displaced based on the characteristic sound registered by the acoustic sensor 40.

[0055] Furthermore, in the illustrated example, a single acoustic sensor 40 is used to register characteristic sound created by any one of the plurality of converting elements 10 as structure-borne sound propagating through the structure 50. In other embodiments, the system may comprise a plurality of acoustic sensors located in different areas of the structure 50.

[0056] The converting elements 10 are integrally formed in the structure 50, which structure forms a plate-shaped product underlay for automatic inventory of products 90 placed on the converting elements 10 of the underlay. The underlay may be formed by a sheet material, such as sheet metal or a plastics sheet. In this case, the converting elements 10 may be formed by cutting or punching them out of the sheet material, e.g., through a punching process. The product underlay may be placed on any suitable surface, such as a shelf 5 in, e.g., a shop or a warehouse.

[0057] With reference to Figures 2A-2C, illustrating one of the converting elements 10 of Figure 1 in more detail, the converting element 10 comprises a sound-generating portion 20 including at least one vibrational element 22 configured to be brought into vibration by the displacement of the product 90. Each vibrational element 22 of a converting element 10 is typically an elongated element having a first free end and an opposite second end that is connected to the structure 50. The vibrations of the at least one vibrational element 22 propagates into the structure 50 where they manifest themselves as a characteristic structure-borne sound that can be registered by the acoustic sensor 40. The configuration of the at least one vibrational element 22 causes the characteristic sound to comprise at least one characteristic frequency and / or a plurality of frequencies forming a characteristic sequence of frequencies.

[0058] Each converting element 10 may further comprise at least one actuating portion 30 configured to be brought into movement by the displacement of the product 90, whereby the at least one vibrational element 22 of the sound-generating portion 20 is brought into vibration by the movement of the at least one actuating portion 30. In the illustrated example, the products 90 are positioned on top of the converting elements 10 of the product underlay such that the weight of each product 90 exerts a force on one or more actuating portions 30 of a respective converting element 10. The actuating portion 30 is configured such that the shape of the actuating portion is changed under influence of the force exerted on the actuating portion by the product 90. When the product 90 is removed from the converting element 10, the removal of the force causes the actuating portion 30 to regain its original shape through a sudden movement that causes vibration of the at least one vibration element 22 of the soundgenerating portion 20.

[0059] In some examples, the actuating portion 30 may comprise a snapping mechanism 32 configured to perform a snapping movement into a stable state of the snapping mechanism 32 upon removal of the product 90 from the snapping mechanism, wherein the at least one vibrational element 22 is configured to be brought into vibration at least partly by the snapping movement of the snapping mechanism 32. As best illustrated in Figures 2B-2C, the snapping mechanism 32 may comprise a bulging portion configured to be brought into an instable state of the snapping mechanism 32 by the weight of the product 90, and to perform the snapping movement into the stable state of the snapping mechanism 32 upon removal of the product 90. This type of snapping mechanism is sometimes referred to as a mono-stable snapping mechanism.

[0060] By giving each converting element 10 a unique configuration in terms of the frequency or sequence of frequencies generated by the at least one vibrational element of the converting element 10 upon displacement of the product 90, the at least one processor 42, 62 (see Figure 1) of the inventory system will be able to determine not only that an object has been displaced, but also which product has been displaced, based on the characteristic sound registered by the acoustic sensor 40.

[0061] Figure 3 schematically illustrates the acoustic sensor 40 of the acoustic inventory system shown in Figure 1. The acoustic sensor 40 comprises a housing 45, such as plastic housing, which is securely attached to the structure 50 to ensure tight mechanical coupling between the sensor housing 45 and the structure 50. The acoustic sensor 40 further comprises an acoustic sensor element 41 configured to register acoustic signals in form of structure-borne sound or vibrations propagating through the structure 50. The acoustic sensor element 41 may, e.g., be a piezoelectric acoustic sensor element or a contact microphone for registering structure-borne sound in the structure 50. The acoustic sensor 40 further comprises a processor 42 and a data storage medium 43. The data storage medium 43 stores a computer program component which, alone or in combination with computer program components residing in one or more other devices to which the acoustic sensor 40 is connectable, constitutes a computer program for acoustic inventory of products in accordance with the principles of the present disclosure.

[0062] In the illustrated example, the system 100 is a cloud-based system comprising a network server 60. The acoustic sensor 40 comprises a communication module 44 configured to communicate with a communication module 64 of the network server 60 using any known communication protocol. In some embodiments, the network server 60 is a web server connected to the Internet. The network server 60 further comprises a processor 62 and a data storage medium 63 storing a server-side component of the above mentioned computer program for acoustic inventory of products. The server-side component of the computer program may be an inventory management software that may comprise any functionality typically found in state-of-the-art inventory management software, including but not limited to product and stock information management. In some embodiments, the server-side component may comprise a web application that enables use of the inventory management software via a web browser of an electronic communication device 70, such as a laptop, a tablet computer or a smart phone. In other embodiments, some or all functionality of the inventory management software may reside in a client-side computer program component stored on the electronic communication device 70 itself.

[0063] Likewise, the logic required for processing the characteristic sound registered by the acoustic sensor 40 may reside in any of the acoustic sensor 40, the network server 60, and an electronic communication device 70 to which the acoustic sensor 40 is directly or indirectly connected.

[0064] In some embodiments, the acoustic sensor 40 is configured to perform a frequency analysis of the registered characteristic sound in order to determine which product was displaced and, optionally, whether the product was picked up from or put back on the product underlay. This information is then communicated to the network server 62 in order for a server-side inventory management software to take appropriate actions from an inventory management perspective.

[0065] In some embodiments, in order to avoid unnecessary processing of acoustic signals, the frequency analysis of the characteristic sound registered by the acoustic sensor 40 may be triggered by the sound of a snapping mechanism 32 returning to its stable state. The snapping movement of the snapping mechanism 32 causes a sound pulse of short duration to be generated, which sound pulse may be used to trigger the processor performing the frequency analysis to perform a frequency analysis of any sound registered by the acoustic sensor 40 immediately following registration of the sound pulse, which sound is likely to originate from vibrations of the at least one vibrational element 22 of the converting element 10.

[0066] Figure 4 schematically illustrates a system 100 for acoustic inventory of products, according to another exemplary embodiment of the present disclosure.

[0067] In this example, the product 90 is stored in a product rack or packaging 80, hereinafter referred to as the packaging. The packaging 80 is configured to physically interact with the product 90 upon displacement of the product, such that the physical interaction causes the packaging 80 to generate a characteristic sound, and to induce the characteristic sound into the structure 50 on which the acoustic sensor 40 is mounted, in order for the at least one processor 42, 62 to detect the displacement of the product 90 based on the characteristic sound registered by the acoustic sensor 40. Consequently, the product packaging 80 constitutes a converting element 10 configured to convert a displacement of the product 90 to a characteristic sound by physically interacting with the product upon displacement thereof, just like the converting elements 10 of the system illustrated in Figure 1.

[0068] In the embodiment illustrated in Figure 4, a plurality of packagings 80 are placed on the structure 50 onto which the acoustic sensor 40 is mounted. The structure 50 constitutes a product underlay that is somewhat different from the product underlay constituted by the structure 50 in Figure 1, since this product underlay does not contain any converting elements that are integrated into the structure. In other embodiments (not shown), the packagings 80 may be placed directly on the shelf 5, without any intervening product underlay, whereby the acoustic sensor 40 may be mounted directly onto the shelf, at a distance from the packagings 80, and be configured to register the characteristic sound generated by the physical interaction between a packaging and a product as structure-borne sound propagating through the shelf material. In such a scenario, the shelf constitutes the structure onto which the acoustic sensor is mounted to pick up the characteristic sound as structure-borne sound.

[0069] With simultaneous reference to Figure 5, illustrating a magnification of the area designated by reference character A in Figure 4, the product packaging 80 comprises at least one vibrational element 22 configured to be brought into vibration through physical interaction with at least one protruding part 91 of the product 90 upon removal or insertion of the product out of or into an opening 81 of the packaging 80. The at least one vibrational element 22 may be integrally formed in the packaging and be defined by at least one slit 25 that extends into the packaging 80 from an edge of the opening 81. In the illustrated example, the at least one protruding part 91 of the product comprises a sequence of braille characters which, for example, may express the name of the product in braille format.

[0070] When the product 91 is removed from or inserted into the opening 81 of the packaging 80, the at least one vibrational element 22 of the packaging 80 interacts with the braille characters embossed on the surface of the product 90, which causes the at least one vibrational element 21 to vibrate. The vibrations propagate throughout the packaging 80, which induces the vibrations into the structure 50 via the mechanical connection between the packaging 80 and the structure, caused by the packaging weight. The vibrations of the at least one vibrational element 22 thus propagate through the packaging 80 and into the structure 50, where they manifest themselves as a characteristic structure-borne sound that can be registered by the acoustic sensor 40. The configuration of the at least one vibrational element 22 causes the characteristic sound to comprise at least one characteristic frequency and / or a plurality of frequencies forming a characteristic sequence of frequencies. The at least one processor 42, 62 may then detect the displacement of the product based on the characteristic sound registered by the acoustic sensor 40.

[0071] In some embodiments (not shown), the at least one vibrational element 22 may comprise two or more vibrational elements with different frequency characteristics configured to physically interact with a respective protruding part 91 of the product 90 upon removal and insertion of the product 90 out of and into the product rack or packaging 80, wherein the at least one processor 42, 62 is configured to determine whether the product 90 is inserted into or removed from the product rack or packaging 80 based on the frequencies of the characteristic sound registered by the acoustic sensor 40. By analysing the order of the frequencies registered by the acoustic sensor 40, the processor can determine whether the product has been inserted into or removed from the product rack or packaging 80. The at least one vibrational element 22 is not necessarily integrally formed in the product rack or packaging 80. In some embodiments (not shown), the at least one vibrational element 22 is attached to the product rack or packing 80 in a manner that causes physical interaction between the at least one vibrational element 22 and the at least one protruding part 91 of the product 90 upon removal or insertion of the product out of or into the product rack or packaging 80.

[0072] In yet other embodiments (not shown), the acoustic sensor 40 may be attached directly to the product rack or packaging and configured to register the characteristic sound generated by the converting element as structure-borne sound propagating through the product rack or packaging. In such scenarios, the product rack or packaging serves as the structure conducting the characteristic sound from the converting element to the acoustic sensor.

[0073] Figure 9 is a flow chart illustrating a method for acoustic inventory of products, according to an exemplary embodiment of the disclosure. The method relates to a system 100 for acoustic inventory of products. The system may, for example, be devised and configured as disclosed in figure 1 or figure 4.

[0074] The method comprising providing slOl at least one converting element 10 for converting a displacement of a product 90 to a characteristic sound, wherein the at least one converting element 10 is configured to physically interact with the product 90 upon displacement of the product, the physical interaction causing the converting element 10 to generate a characteristic sound and induce the characteristic sound in the structure 50; registering sl02 the characteristic sound generated by the converting element 10 as structure-borne sound propagating through the structure 50 by means of an acoustic sensor 40 mechanically connected to the structure 50 at a distance from the at least one converting element 10; and detecting sl03 the displacement of the product 90 based on the characteristic sound registered by the acoustic sensor 40 by means of a processor 42, 62 coupled to the acoustic sensor 40.

[0075] The method may comprise providing a plurality of converting elements 10, each generating a characteristic sound associated with a respective product 90 or product type, and determining, by means of the at least one processor 42, 62, which product 90 or product type was displaced based on the characteristic sound registered by the acoustic sensor 40. The converting element 10 may comprise a sound-generating portion 20 including at least one vibrational element 22 configured to be brought into vibration by the displacement of the product 90, thereby causing the characteristic sound to comprise at least one characteristic frequency and / or a plurality of frequencies forming a characteristic sequence of frequencies, wherein the method comprises determining, by means of the at least one processor 42, 62, the product or product type based on the at least one characteristic frequency and / or the plurality of frequencies forming the characteristic sequence of frequencies of the characteristic sound.

[0076] The method may further comprise providing a structure 50 to which the acoustic sensor 40 is mechanically connectable, wherein the at least one vibrational element 22 of the soundgenerating portion is attached to or integrally formed in the structure 50.

[0077] The at least one vibrational element 22 may be attached to or integrally formed in a product rack or packaging 80, such that the at least one vibrational element 22 is brought into vibration through physical interaction with the at least one protruding part 91 of the product 90 upon removal or insertion of the product out of or into the product rack or packaging 80.

[0078] In some embodiments, the at least one vibrational element 22 may comprise at least two vibrational elements with different frequency characteristics attached to or integrally formed in the product rack or packaging 80, which vibrational elements are configured to physically interact with a respective protruding part 91 of the product 90 upon removal and insertion of the product out of and into the product rack or packaging 80. In this case, the method may comprise a step of determining, by means of the at least one processor 42, 62, whether the product 90 is inserted into or removed from the product rack or packaging 80 based on the frequencies of the characteristic sound registered by the acoustic sensor 40.

[0079] As clear from the foregoing description, the method is typically a computer-implemented method performed by one or more processors of the system 100 upon execution of a computer program. As also clear from the foregoing description, the computer program may be a distributed computer program comprising program components residing in both the acoustic sensor 40 and the network server 60. The method may hence be performed by both the processor 42 of the acoustic sensor 40 and the processor 62 of the network server 60. Furthermore, the person skilled in the art realizes that the present disclosure is not limited to the embodiments described above. The person skilled in the art realizes that modifications and variations are possible within the scope of the appended claims. For example, it should be realized that all or some of the functionality described herein as residing in the network node 60 may, in other embodiments, reside in the acoustic sensor 40. In yet other embodiments, all or some of the functionality described herein as residing in the network node 40 may reside in a client device in direct or indirect communication with the acoustic sensor 40, such as the electronic communication device 70. Consequently, it should be realized that the system 100 is not limited to any particular system configuration or system topology encompassed by the appended claims.

Claims

CLAIMS1. A system (100) for acoustic inventory of products, the system (100) comprising: at least one converting element (10) for converting a displacement of a product (90) to a characteristic sound; an acoustic sensor (40) configured to be mechanically connected to a structure (50) at a distance from the at least one converting element (10); and a processor (42, 62) coupled to the acoustic sensor (40), wherein the at least one converting element (10) is configured to physically interact with the product (90) upon displacement of the product (90), the physical interaction causing the converting element (10) to generate a characteristic sound and induce the characteristic sound in the structure (50), wherein the acoustic sensor (40) is configured to register the characteristic sound generated by the converting element (10) as structure- borne sound propagating through the structure (50), and wherein the processor (42, 62) is configured to detect the displacement of the product (50) based on the characteristic sound registered by the acoustic sensor (40).

2. The system (100) according to claim 1, wherein the system comprises a plurality of converting elements (10), wherein the characteristic sound generated by each converting element is associated with a product or product type, the processor (42, 62) being configured to determine which product or product type was displaced based on the characteristic sound registered by the acoustic sensor (40).

3. The system (100) according to claim 2, wherein the acoustic sensor (40) is a single acoustic sensor configured to register a characteristic sound created by any one of the plurality of converting elements (10) as structure-borne sound propagating through the structure (50).

4. The system (100) according to any of the preceding claims, wherein the converting element (10) comprises a sound-generating portion (20) including at least one vibrational element (22) configured to be brought into vibration by the displacement of the product (90), the configuration of the at least one vibrational element (22) causing thecharacteristic sound to comprise at least one characteristic frequency and / or a plurality of frequencies forming a characteristic sequence of frequencies.

5. The system (100) according to claim 4, wherein the at least one vibrational element (22) is integrally formed in or attached to the structure (50) to which the acoustic sensor (40) is mechanically connected.

6. The system (100) according to claim 4 or 5, wherein the converting element (10) further comprises an actuating portion (30) configured to be brought into movement by the displacement of the product (90), the at least one vibrational element (22) of the soundgenerating portion (20) being configured to be brought into vibration by the movement of the actuating portion (30).

7. The system (100) of claim 6, wherein the actuating portion (30) comprises a snapping mechanism (32) configured to perform a snapping movement into a stable state of the snapping mechanism (32) upon removal of the product (90) from the snapping mechanism, the at least one vibrational element (22) being configured to be brought into vibration at least partly by the snapping movement of the snapping mechanism (32).

8. The system (100) of claim 7, wherein the snapping mechanism (32) comprises a bulging portion configured to be brought into an instable state of the snapping mechanism (32) by the weight of the product (90), and to perform the snapping movement into the stable state of the snapping mechanism (32) upon removal of the product (90).

9. The system (100) according to claim 4 or 5, wherein the at least one vibrational element (22) is attached to or integrally formed in a product rack or packaging (80), the at least one vibrational element (22) being configured to be brought into vibration through physical interaction with at least one protruding part (91) of the product (90) upon removal or insertion of the product out of or into the product rack or packaging (80).

10. The system (100) according to claim 9, wherein the at least one vibrational element (22) is integrally formed in the product rack or packaging (80) and defined by at least one slit (25)that extends into the packaging from an edge of an opening (81) of the rack or packaging (80), such that the at least one vibrational element (22) physically interacts with the at least one protruding part (91) of the product (90) when the product is removed from or inserted into the product rack or packaging (80), via said opening (81).

11. The system (100) according to any one of the claims 9 or 10, wherein the at least one protruding part (91) of the product (90) comprises one or more braille characters.

12. The system (100) according to any one of the claims 9-11, wherein the product (90) comprises at least two protruding parts (91), and wherein the sound-generating portion (20) comprises at least two vibrational elements (22) with different frequency characteristics configured to physically interact with a respective protruding part (91) of the product (90) upon removal and insertion of the product (90) out of and into the product rack or packaging (80), the processor (40) being configured to determine whether the product (90) is inserted into or removed from the product rack or packaging (80) based on the frequencies of the characteristic sound registered by the acoustic sensor (40).

13. A method for acoustic inventory of products, the method comprising: providing (slOl) at least one converting element (10) for converting a displacement of a product (90) to a characteristic sound, wherein the at least one converting element (10) is configured to physically interact with the product (90) upon displacement of the product, the physical interaction causing the converting element (10) to generate a characteristic sound and induce the characteristic sound in a structure (50); registering (sl02) the characteristic sound generated by the converting element (10) as structure-borne sound propagating through the structure (50) by means of an acoustic sensor (40) mechanically connected to the structure (50) at a distance from the at least one converting element (10); and detecting (sl03) the displacement of the product (90) based on the characteristic sound registered by the acoustic sensor (40) by means of a processor (42, 62) coupled to the acoustic sensor (40).

14. The method according to claim 13, comprising:providing a plurality of converting elements (10), each generating a characteristic sound associated with a respective product (90) or product type, and determining, by means of the processor (40), which product (90) or product type was displaced based on the characteristic sound registered by the acoustic sensor (40).

15. The method according to any one of the claims 13-14, wherein the converting element (10) comprises a sound-generating portion (20) including at least one vibrational element (22) configured to be brought into vibration by the displacement of the product (90), thereby causing the characteristic sound to comprise at least one characteristic frequency and / or a plurality of frequencies forming a characteristic sequence of frequencies, the method comprising: determining, by means of the processor (40), the product or product type based on the at least one characteristic frequency and / or the plurality of frequencies forming the characteristic sequence of frequencies of the characteristic sound.

16. The method according to claim 15, comprising: providing a structure (50) to which the acoustic sensor (40) is mechanically connectable, wherein the at least one vibrational element (22) of the sound-generating portion is attached to or integrally formed in the structure (50).

17. The method according to claim 15 or 16, wherein the at least one vibrational element (22) is attached to or integrally formed in a product rack or packaging (80), such that the at least one vibrational element (22) is brought into vibration through physical interaction with at least one protruding part (91) of the product (90) upon removal or insertion of the product out of or into the product rack or packaging (80).

18. The method according to claim 17, wherein the product (90) comprises at least two protruding parts (91), and wherein the sound-generating portion (20) comprises at least two vibrational elements (22) with different frequency characteristics configured to physically interact with a respective protruding part (91) of the product (90) upon removal and insertion of the product out of and into the product rack or packaging (80), the method comprising:determining, by means of the processor (40), whether the product (90) is inserted into or removed from the product rack or packaging (80) based on the frequencies of the characteristic sound registered by the acoustic sensor (40).

Citation Information

Patent Citations

  • Apparatus and method for monitoring stock information in a shopping space

    US10586205B2

  • Systems and methods to trigger actions based on encoded sounds associated with containers

    US10878838B1

  • Factory Inventory System Utilizing the Natural Frequency of a Pallet Rack

    US20170091701A1

  • Apparatus and method for monitoring stock information in a shopping space

    US20170193436A1

  • System and Methods for Identifying Positions of Physical Objects Based on Sounds

    US20180188351A1