Device for generating a haptically perceptible area and configuration and control methods of such a device

Abstract

Device for generating a haptically perceptible area of ​​a virtual object within an interaction space (6) with a plurality of ultrasonic loudspeakers (2, 2') whose ultrasonic signals are tunable with respect to phase, frequency and amplitude, characterized in that the ultrasonic loudspeakers (2, 2') individually or in groups form several modules (3, 3') which can be positioned or arranged on the interaction space (6) in such a way that a) that at least one ultrasonic loudspeaker (2, 2') of one module (3, 3') is oriented at an angle to at least one ultrasonic loudspeaker (2, 2') of another module (3, 3') and / or b) that at least one ultrasonic loudspeaker (2, 2') of a module (3, 3') is spaced apart from a common plane of other ultrasonic loudspeakers (2, 2') and / or c) the modules (3, 3') have a selectable distance from each other, wherein the device has several ultrasonic microphones (5) which are arranged directly on the modules (3, 3').

Description

[0001] The present invention relates to a device for generating a haptically perceptible area of ​​a virtual object within an interaction space with a plurality of ultrasonic loudspeakers whose ultrasonic signals can be tuned for this purpose with regard to phase, frequency and amplitude.

[0002] Furthermore, the invention relates to a method for configuring and controlling such a device.

[0003] The operation of devices, as known from the state of the art, is usually achieved via switches, buttons, touch surfaces, or other physical control elements. By activating such control elements, the user receives immediate haptic feedback via the sense of touch of their hand or fingers, allowing the duration and pressure applied to the control element to be immediately perceptible.

[0004] In addition to such physically tangible and mass-based devices, virtual environments have also been created in recent years, particularly virtual reality (VR), augmented reality (AR), or holographic environments, in which any objects, devices, and / or user interfaces are virtually represented by suitable display systems. While such virtual objects are visually and spatially perceptible, they cannot be directly touched and thus perceived haptically or sensed.

[0005] The devices mentioned above were developed for the haptic perception of virtual objects. They use controlled ultrasonic loudspeakers with phase, frequency, and amplitude to create a haptically perceptible area at predefined points in an interaction space. The sound acts directly on the hand and can be felt and sensed there. One such device with a planar arrangement of ultrasonic loudspeakers is described, for example, in WO2016 / 038347 A1.

[0006] Another generic device and a method for generating a haptically perceptible area within an interaction space is described in WO 2018 / 200424 A1.

[0007] Currently known devices for generating a haptically perceptible area of ​​a virtual object have a large number of ultrasonic loudspeakers arranged flatly on a common plate. Above the plate extends the interaction space in which the tactile points or areas can be generated. The arrangement of all ultrasonic loudspeakers of a device on a common plate has proven disadvantageous in practice because the plate must have a relatively large number of ultrasonic loudspeakers to generate the desired effect, which is why it requires a large amount of space, which is not available in all devices at the required location, namely below the interaction space. This makes optimal positioning of the ultrasonic loudspeaker arrangement more difficult.Furthermore, in known devices, especially for generating holographic environments, the preferred positions of the image-generating devices and the preferred positions of the ultrasonic loudspeakers often conflict, requiring compromises in each case, which reduces the accuracy of the generated effects. Finally, with a planar arrangement of the ultrasonic loudspeakers, the interacting hand can only be stimulated from one direction, so that the perceptible effect on the palm of the hand decreases or disappears completely if the hand is twisted.

[0008] Overall, therefore, the known ultrasonic loudspeaker arrangements cannot be satisfactorily combined with the equally known imaging devices. Furthermore, the manual configuration of the ultrasonic loudspeakers or the ultrasonic signals has proven to be very complex, time-consuming, and error-prone.

[0009] Based on this, the object of the present invention is to provide a device and a method that enable flexible adaptation of the interaction space to the space in which virtual objects are created by an image generation device. Furthermore, the configuration of the ultrasound signals is to be simplified.

[0010] This object is firstly achieved by the device according to claim 1. According to the invention, the ultrasonic loudspeakers individually or in groups form several modules which can be positioned or arranged in the interaction space in such a way that a) at least one ultrasonic loudspeaker of a module is oriented at an angle to at least one ultrasonic loudspeaker of another module and / or b) at least one ultrasonic loudspeaker of a module is spaced from a common plane of other ultrasonic loudspeakers and / or c) the modules have a selectable distance from each other, wherein the device has a plurality of ultrasonic microphones arranged directly on the modules.

[0011] The positionability and thus the freely selectable position of the individual modules with ultrasonic loudspeakers allows the device to be optimally adapted to the available installation space, thus avoiding collisions between the preferred positions of the image-generating devices and the tactile sensory devices. The arbitrary, particularly angular, arrangement of individual loudspeakers and / or modules relative to one another ensures optimal three-dimensional alignment of the modules in space. These modules can be arranged at any position outside the interaction space and aligned in the desired orientation. This allows for a customized design of the device for generating haptic feedback while optimally utilizing the available installation space.Furthermore, the device with essentially freely positionable modules has a minimal space requirement and, in particular, the adjustability of the orientation of individual modules allows a high degree of individualization of the strength of the haptic effect to be achieved.

[0012] The ultrasonic microphones allow for easy, automatic configuration of the device. Furthermore, the ultrasonic microphones can be used as a three-dimensional acoustic camera, which can detect, identify, and continuously track objects within the interaction space. This eliminates the need for an otherwise required image capture device, such as a stereo camera. This will be discussed in more detail in the description of the method according to the invention.

[0013] Preferred embodiments of the present invention are described below and in the subclaims.

[0014] According to a first preferred embodiment, the ultrasonic loudspeakers of a module are arranged on a common module plane. The ultrasonic loudspeakers of a module are preferably oriented parallel to one another. While the specific configuration of such a module is not predetermined, particularly with regard to the number and arrangement of the ultrasonic loudspeakers, and is essentially arbitrary, a matrix-like arrangement of the ultrasonic loudspeakers has proven advantageous in practice. Square matrix arrangements of, for example, 4x4 ultrasonic loudspeakers are particularly preferred.

[0015] According to an advantageous embodiment of the invention, the device has at least three modules whose module planes are aligned at an angle to each other. This allows the modules to be optimally arranged around the interaction space and aligned with the interaction space, depending on the available space.

[0016] To configure the device depending on the individual position and orientation of the modules and the ultrasonic loudspeakers arranged on them, a control unit with software (firmware) and a programming interface (API) is preferably provided. The control unit calculates the phase, frequency, and amplitude of the individual ultrasonic signals depending on the individual module positions and module orientations in order to generate an area with haptic feedback at any point in the interaction space. If such an area is to be generated at multiple locations, suitable groups are formed from the existing modules, which are assigned to a specific point and generate the haptic feedback at that location.Because the haptic area is generated from three different directions with the present three-dimensional arrangement of the modules, the haptic area can be felt even with different hand orientations and any shadow areas within the interaction space can be effectively avoided.

[0017] The entire device, including the individual modules, can be controlled by the control unit, allowing the use of simple and therefore inexpensive modules. To ensure the correct timing of all modules, all cable lengths and signal propagation times must be known or equal.

[0018] Alternatively, each module is provided with a separate microcontroller for configuring and / or controlling the ultrasonic loudspeakers of a module and for communicating with the control unit. This results in optimal networking of the modules with the control unit, which serves as the main control unit, and communication is possible via a real-time capable bus system.

[0019] The problem underlying the present invention is additionally solved by the method according to claim 8. According to the invention, in order to configure the device, the modules successively emit defined test signals which are registered immediately and / or after reflection from one or more reflection bodies by ultrasonic microphones located directly on the modules, so that the relative positions and orientations of the modules and ultrasonic microphones result from the propagation times and / or the intensities of the registered test signals. Taking into account the thus determined and known positions and orientations of the modules and the ultrasonic microphones arranged thereon, the ultrasonic signals of the modules can be advantageously tuned with regard to their phase, frequency and amplitude such that haptically perceptible areas can be generated at any desired location within the interaction space.The ultrasonic microphones enable self-calibration of the device, simplifying configuration, which can be performed essentially automatically without manual input. The accuracy and resolution of the configuration increase with the number of modules and / or ultrasonic microphones.

[0020] In addition to the device configuration, the ultrasonic microphones can also be used as an acoustic camera, with which objects within the interaction space can be detected, identified, and tracked through continuous position determination. For this purpose, reflections from stationary or moving objects within the interaction space—such as a hand intervening in the interaction space—are preferably registered by the ultrasonic microphones, and the position, shape, and movement of the object are determined from the reflected signals. The ultrasonic frequencies that are emitted by the ultrasonic loudspeakers to generate the haptic effect can be used to detect and track objects within the interaction space. Alternatively, other frequencies can also be used.

[0021] Finally, according to a preferred embodiment of the method, objects can be identified using existing classification algorithms that compare the reflected signals with existing object-related data sets and, if there is a match, allow for unambiguous assignment of the object. This also enables specific and targeted stimulation of the object within the interaction space.

[0022] A concrete embodiment of the present invention is explained below with reference to the figure, which shows a schematic representation of a device for generating a haptically perceptible area of ​​a virtual object within an interaction space with a plurality of ultrasonic loudspeakers.

[0023] The device 1 shown has a plurality of cylindrical ultrasonic loudspeakers 2, 2', which in the present exemplary embodiment form groups of a total of 25 modules 3, 3', each with 16 ultrasonic loudspeakers 2, 2'. In total, the device shown has 400 ultrasonic loudspeakers 2, 2'. The ultrasonic loudspeakers 2, 2' of the individual modules 3, 3' are each arranged in a matrix of 4x4 ultrasonic loudspeakers 2, 2' on a common module plate 4 and aligned parallel to one another. The module plates 4 are flat and form module planes. The modules 3, 3' are ring-shaped and arranged at an angle to a horizontal, such that at least one ultrasonic loudspeaker 2 of a module 3 is oriented at an angle to at least one ultrasonic loudspeaker 2' of another module 3' and is spaced from a common plane of other ultrasonic loudspeakers.As a result, the modules 3, 3' and the ultrasonic loudspeakers 2, 2' can essentially be positioned freely and, in the embodiment shown, are aligned with the center of the interaction space 6 arranged above them.

[0024] For the configuration and control of the device 1, the modules 3, 3' at least partially have ultrasonic microphones 5, whereby in the present embodiment only three ultrasonic microphones 5 are shown as examples. To increase the accuracy of the configuration and control method, preferably all modules 3, 3' have at least one ultrasonic microphone 5. The ultrasonic microphones 5 receive test signals for configuration, which are emitted one after the other by the modules 3, 3' and which are optionally reflected by a reflection body (not shown). The positions and orientations of the individual modules 3, 3' can be clearly determined using the recorded signals, in particular their frequencies and amplitudes, so that the ultrasonic signals can be adjusted with regard to their phase, frequency and amplitude such that points with haptically perceptible areas can be generated at any position within the interaction space 6.In the figure, the interaction space 6 with a possible haptic feedback is shown essentially in a partially spherical shape, whereby the shape of the interaction space 6 also depends on the concrete and freely selectable arrangement of the modules 3, 3'.

[0025] In the interaction space 6 itself, haptically perceptible areas can be created at any position, which can be perceived by a person with a hand 7 entering the interaction space 6. The reflections of the ultrasonic signals from the hand 7—or another object—are registered by the ultrasonic microphones 5, so that the movement of the hand 7 in the interaction space 6 can be determined from the reflected signals.

[0026] A control unit 8 and a programming interface 9 are provided for controlling and configuring the device 1. In the illustrated embodiment, the control unit 8 is connected to the device 1 in such a way that the control signals are sent directly from the control unit 8 to the individual modules 3, 3'. Alternatively, the modules 3, 3' can also have separate microcontrollers (not shown) that are connected to the modules 3, 3' on the one hand and to the control unit 8 on the other hand for control purposes. List of reference symbols 1 device 2.2' ultrasonic speaker 3, 3' module 4 module plate 5 Ultrasonic microphone 6 Interaction space 7 Hand 8 Control unit 9 Programming interface

Claims

[1] Device for generating a haptically perceptible area of a virtual object within an interaction space (6) with a plurality of ultrasonic loudspeakers (2, 2'), the ultrasonic signals of which can be tuned for this purpose with regard to phase, frequency and amplitude, characterized by that the ultrasonic loudspeakers (2, 2') individually or in groups form several modules (3, 3') which can be positioned or arranged in the interaction space (6) in such a way that a) that at least one ultrasonic loudspeaker (2, 2') of a module (3, 3') is oriented at an angle to at least one ultrasonic loudspeaker (2, 2') of another module (3, 3') and / or b) that at least one ultrasonic loudspeaker (2, 2') of a module (3, 3') is spaced from a common plane of other ultrasonic loudspeakers (2, 2') and / or c) the modules (3, 3') have a selectable distance from one another, wherein the device has a plurality of ultrasonic microphones (5) which are arranged directly on the modules (3, 3'). [2] Device according to claim 1, characterized by that the ultrasonic loudspeakers (2, 2') of a module (3, 3') are arranged on a common module level. [3] Device according to one of claims 1 or 2, characterized by that the ultrasonic loudspeakers (2, 2') of a module (3, 3') have an orientation parallel to one another. [4] Device according to one of claims 1 to 3, characterized by that the ultrasonic loudspeakers (2, 2') of a module (3, 3') are arranged as a matrix, in particular as a square matrix. [5] Device according to one of claims 1 to 4, characterized by that the device has at least three modules (3, 3') whose module planes are aligned at an angle to one another. [6] Device according to one of claims 1 to 5, characterized by a control unit (8) with software contained thereon and a programming interface (9) for configuring the device (1) depending on the individual positions and orientations of the modules (3, 3'). [7] Device according to one of claims 1 to 6, characterized by that each module (3, 3') has a separate microcontroller for configuring and / or controlling the ultrasonic loudspeakers (2, 2') of a module (3, 3') and for communicating with the control unit (8). [8] Method for configuring and controlling a device (1) according to one of claims 1 to 7, characterized byin that, for the configuration of the device (1), the modules (3, 3') successively emit defined test signals which are registered immediately and / or after reflection on one or more reflection bodies by ultrasonic microphones (5) which are directly attached to the modules (3, 3'), so that the relative positions and orientations of the modules (3, 3') and ultrasonic microphones (5) result from the propagation times and / or the intensities of the registered test signals. [9] Method according to claim 8, characterized by that, with known positions and orientations of the modules (3, 3') and ultrasonic microphones (5), the ultrasonic signals of the modules (3, 3') are tuned with regard to their phase, frequency and amplitude in such a way that haptically perceptible areas can be generated at any desired location within the interaction space (6). [10] Method according to one of claims 8 or 9, characterized bythat reflections of stationary or moving objects within the interaction space (6) are registered by the ultrasonic microphones (5) and the position, shape and / or movement of the object is determined from the reflected signals.

Images