Audio haptic output device and home appliance including same

Abstract

An audio haptic output device and a home appliance, according to an embodiment of the present disclosure, include: a touch panel; a memory for storing a plurality of pieces of effect data corresponding to a plurality of touch inputs; a processor that generates an audio signal by calling, from the memory, effect data matched to a touch input through the touch panel; a speaker amplifier for amplifying and outputting a signal based on the audio signal; and an actuator that outputs at least one of audio and haptic on the basis of an output signal of the speaker amplifier, wherein the processor or the speaker amplifier converts and outputs a frequency characteristic of the audio signal if the effect data matched to the touch input through the touch panel includes audio data.

Description

Audio haptic output device and home appliance including same

[0001] The present disclosure relates to an audio-haptic output device and a home appliance including the same, and more particularly, to an audio-haptic output device capable of outputting audio and haptics by utilizing an actuator, and a home appliance including the same.

[0002] Electronic devices are equipped with various input / output methods. For example, electronic devices can output visual information through displays and auditory information through speakers. Furthermore, recently, haptic effects have been used to indicate a user's touch input or a specific status. Haptic technology uses force, vibration, and motion to output predetermined patterns, allowing the user to experience tactile sensations. Electronic devices may include a vibration device for providing audio and a vibration device for providing haptic effects to the user. Vibration devices are often implemented using actuators.

[0003] FIG. 21 and FIG. 22 are drawings for reference to an audio haptic output device utilizing a conventional actuator. FIG. 21 illustrates an input / output device that outputs haptics and voice comments / audio in response to input through a touch panel, and FIG. 22 illustrates the results of gravitational acceleration measurements by frequency of a general haptic actuator and an audio actuator.

[0004] In the order illustrated in Fig. 21, when a user touches a specific action button, the touch action button data is transmitted to a signal processor, and the signal processor retrieves the haptic / audio data of the corresponding button. The memory stores haptic data and audio data matched to each touch action button. The haptic / audio data retrieved by the signal processor are each decoded, and the decoded haptic signal and audio signal are transmitted to a haptic driver for amplification and an audio amplifier, respectively. The amplified haptic signal is transmitted to a vibration actuator, and the amplified audio signal is transmitted to a speaker. Since the haptic actuator and the audio actuator use different frequency ranges, it is difficult to use them for both purposes.

[0005] Existing technologies implement the driving system separately when implementing haptics and audio, and either receive input signals separately or generate a haptic signal through a separate algorithm from the audio signal, and then operate the haptics and audio. Prior art document 1 European Patent Publication EP 3 748 474 A1 combines separately received haptic and audio signals through a mixer and transmits them to a haptic actuator and a speaker audio actuator, and prior art document 2 U.S. Patent Publication US 2022 0197389 A1 generates a haptic signal from an audio signal by a processor and transmits it to a speaker and an actuator, respectively.

[0006] As previously discussed, audio typically uses speakers to produce output, while haptic actuators are used for haptic feedback. Because the operating frequency bands of actuators used for audio are different, audio actuators are not used for haptic feedback.

[0007] Referring to FIGS. 21 and 22, due to the resonance frequency characteristics of the actuator, haptics and audio are generally configured and used as separate systems. The actuator for haptics has a resonance frequency formed in a low frequency band (60 to 200 Hz) and the amount of vibration displacement at resonance is also very large, which is an advantageous characteristic for generating a sense of vibration. The actuator for audio has a resonance formed in a higher frequency band (300 to 600 Hz) and the level of reduction in vibration displacement after resonance is also formed more gradually than that for haptics.

[0008] For haptic actuators, since their purpose is vibration, their vibration displacement is weak in the audio frequency range. Furthermore, increasing the output of a haptic actuator to output audio can result in excessive vibration displacement around the resonant frequency, potentially leading to component collisions and actuator damage. Audio actuators, on the other hand, are designed for sound, and their output is very weak for low-frequency vibration signals, preventing them from generating sufficient vibration for haptic performance. Therefore, to ensure sufficient audio / haptic output, haptics and audio are configured as separate systems. However, implementing each function increases cost and system complexity.

[0009]

[0010] The purpose of the present disclosure is to provide an audio-haptic output device capable of outputting both audio and haptics with a single actuator and a home appliance including the same.

[0011] The purpose of the present disclosure is to provide an audio haptic output device with improved reliability and a home appliance including the same, which can prevent collision and damage between parts due to vibration of an actuator.

[0012] The purpose of the present disclosure is to provide an audio haptic output device and a home appliance including the same, which are easy to design due to reduced system complexity and reduced parts, thereby reducing cost.

[0013] The purpose of the present disclosure is to provide an audio haptic output device capable of outputting various effects and a home appliance including the same.

[0014] The purpose of the present disclosure is to provide an audio haptic output device capable of outputting high-quality audio and a home appliance including the same.

[0015] The purpose of the present disclosure is to provide a home appliance that can prevent moisture infiltration from the outside without the need to create a separate speaker hole and has a high degree of freedom in design.

[0016] The audio-haptic output device and home appliance according to embodiments of the present disclosure can output both audio and haptics with a single common actuator.

[0017] The audio haptic output device and home appliance according to embodiments of the present disclosure can prevent damage to the actuator and improve its lifespan by appropriately controlling the frequency of a signal input to the actuator according to the output.

[0018] The audio-haptic output device and home appliance according to embodiments of the present disclosure can output audio and haptics more effectively according to the situation by combining audio signals and haptic signals and storing them in memory.

[0019] An audio-haptic output device and a home appliance according to one embodiment of the present disclosure include a touch panel, a memory storing a plurality of effect data corresponding to a plurality of touch inputs, a processor generating an audio signal by calling effect data matching the touch input through the touch panel from the memory, a speaker amplifier amplifying and outputting a signal based on the audio signal, and an actuator outputting at least one of audio and haptics based on an output signal of the speaker amplifier, wherein the processor or the speaker amplifier converts and outputs a frequency characteristic of the audio signal when the effect data matching the touch input through the touch panel includes audio data, thereby stably outputting both audio and haptics with a single common actuator.

[0020] The above processor converts the called effect data into an analog audio signal or an audio signal according to an audio communication standard.

[0021] The processor or the speaker amplifier can prevent damage to the actuator and improve its lifespan by converting the frequency characteristics of the audio signal according to different frequency conversion logics when the actuator outputs both audio and haptics and when the actuator outputs only audio, and can output audio and haptics with high quality.

[0022] The above processor or the speaker amplifier, when the actuator outputs audio and haptics of voice guidance or button sounds, increases the gain only in the audio region frequency band preset for the voice guidance or button sounds, thereby converting the frequency characteristics of the audio signal.

[0023] The processor or the speaker amplifier, when the actuator outputs audio and haptics of voice guidance or button sounds, increases the gain of the audio frequency band preset for the voice guidance or button sounds to be greater than the gain of the haptic frequency band, thereby converting the frequency characteristics of the audio signal.

[0024] The processor or the speaker amplifier, when the actuator outputs audio and haptics of voice guidance or button sounds, applies a first gain at a first frequency among the preset audio range frequency bands for the voice guidance or button sounds, and applies a second gain greater than the first gain at a second frequency that is further from the resonant frequency of the actuator than the first frequency, thereby converting the frequency characteristics of the audio signal.

[0025] The processor or the speaker amplifier, when the actuator outputs only audio of voice guidance or button sounds, converts the frequency characteristics of the audio signal by amplifying only the audio region frequency band preset for the voice guidance or button sounds.

[0026] The processor or the speaker amplifier converts the frequency characteristics of the audio signal by attenuating a frequency band of a predetermined range from the resonant frequency of the actuator and amplifying at least a portion of the frequency band among other regions when the actuator outputs only music type audio.

[0027] The processor or the speaker amplifier does not convert the frequency characteristics of the audio signal when the actuator outputs only haptics, or converts the frequency characteristics of the audio signal by increasing a region outside a predetermined range of frequency bands from the resonant frequency of the actuator.

[0028] The processor or the speaker amplifier converts the frequency according to different frequency conversion logics when the actuator outputs only haptics and when the actuator outputs both haptics and audio, and the speaker amplifier amplifies the frequency-converted signal.

[0029] The plurality of effect data stored in the above memory include haptic data, audio data, and combined data in which haptic data and audio data are combined in advance.

[0030] The above haptic data, the audio data, and the combined data are encoded and stored as audio type data.

[0031] The above combined data is generated by sequentially combining the haptic data and the audio data in that order, or by mixing the haptic data and the audio data so that they overlap.

[0032] The processor or the speaker amplifier converts and outputs the frequency characteristics of the audio signal so that the actuator vibration is greater when a touch input is made to a second area located further from the first area with respect to the actuator than when a touch input is made to a first area of ​​the touch panel.

[0033] An audio-haptic output device and a home appliance according to one embodiment of the present disclosure include a touch panel, a memory storing a plurality of effect data corresponding to a plurality of touch inputs, and an actuator outputting at least one of audio and haptics based on effect data matching a touch input through the touch panel among the data stored in the memory, wherein the plurality of effect data stored in the memory include haptic data, audio data, and combined data in which haptic data and audio data are combined in advance, thereby enabling audio and haptics to be output more effectively according to a situation.

[0034] The above combined data may be generated by sequentially combining the haptic data and the audio data in that order, or by mixing the haptic data and the audio data so that they overlap.

[0035] An audio haptic output device and a home appliance according to one embodiment of the present disclosure further include a processor that calls effect data matching a touch input through the touch panel from the memory to generate an audio signal, and a speaker amplifier that amplifies and outputs a signal based on the audio signal, wherein the processor or the speaker amplifier can convert and output a frequency characteristic of the audio signal so that the actuator vibration upon a touch input to a second area located further from the first area with respect to the actuator is greater than the actuator vibration upon a touch input to a first area of ​​the touch panel.

[0036] An audio haptic output device and a home appliance according to one embodiment of the present disclosure further include a processor that calls effect data matching a touch input through the touch panel from the memory to generate an audio signal, a speaker amplifier that amplifies and outputs a signal based on the audio signal, and an actuator that outputs at least one of audio and haptics based on an output signal of the speaker amplifier, wherein the processor or the speaker amplifier can convert and output a frequency characteristic of the audio signal when the effect data matching the touch input through the touch panel includes audio data.

[0037] The above processor can convert the called effect data into an analog audio signal or an audio signal according to an audio communication standard.

[0038] The processor or the speaker amplifier may convert the frequency characteristics of the audio signal according to different frequency conversion logics when the actuator outputs both audio and haptics and when the actuator outputs only audio.

[0039] The above processor or the speaker amplifier can convert the frequency characteristics of the audio signal by increasing the gain only in the audio region frequency band preset for the voice guidance or button sound when the actuator outputs audio and haptics of the voice guidance or button sound.

[0040] The processor or the speaker amplifier can convert the frequency characteristics of the audio signal by increasing the gain of the audio frequency band preset for the voice guidance or button sound to be greater than the gain of the haptic frequency band when the actuator outputs audio and haptics of the voice guidance or button sound.

[0041] The processor or the speaker amplifier may, when the actuator outputs audio and haptics of voice guidance or button sounds, apply a first gain at a first frequency among the preset audio range frequency bands for the voice guidance or button sounds, and apply a second gain greater than the first gain at a second frequency that is further from the resonant frequency of the actuator than the first frequency, thereby converting the frequency characteristics of the audio signal.

[0042] The above processor or the speaker amplifier can convert the frequency characteristics of the audio signal by amplifying only the audio region frequency band preset for the voice guidance or button sound when the actuator outputs only the audio of the voice guidance or button sound.

[0043] The processor or the speaker amplifier can convert the frequency characteristics of the audio signal by attenuating a frequency band of a predetermined range from the resonant frequency of the actuator and amplifying at least a portion of the frequency band among other regions when the actuator outputs only music type audio.

[0044] The processor or the speaker amplifier may not convert the frequency characteristics of the audio signal when the actuator outputs only haptics, or may convert the frequency characteristics of the audio signal by increasing a region outside a predetermined range of frequency bands from the resonant frequency of the actuator.

[0045] The processor or the speaker amplifier may convert the frequency according to different frequency conversion logics when the actuator outputs only haptics and when the actuator outputs both haptics and audio, and the speaker amplifier may amplify the frequency-converted signal.

[0046] The processor or the speaker amplifier may convert and output the frequency characteristics of the audio signal so that the actuator vibration is greater when a touch input is made to a second area located further from the first area with respect to the actuator than when a touch input is made to a first area of ​​the touch panel.

[0047] An audio-haptic output device and a home appliance according to one embodiment of the present disclosure include a touch panel, a memory storing audio data and haptic data corresponding to a plurality of touch inputs, a processor calling at least one of audio data and haptic data matching a touch input through the touch panel from the memory and converting it into an audio signal, a speaker amplifier amplifying and outputting a signal based on the audio signal, and an actuator outputting at least one of audio and haptic according to an output signal of the speaker amplifier, wherein both audio and haptic can be output by a single common actuator.

[0048] According to at least one of the embodiments of the present disclosure, by measuring the quality of raw water and purified water with one sensor, it is possible to respond to water quality accidents caused by raw water and deterioration of purified water quality.

[0049] According to at least one of the embodiments of the present disclosure, sensing accuracy and efficiency can be improved by piping configuration and rinsing operation for sensor commonization.

[0050] According to at least one of the embodiments of the present disclosure, both audio and haptics can be output by a single actuator.

[0051] According to at least one of the embodiments of the present disclosure, collision and damage between parts due to vibration of the actuator can be prevented, thereby improving reliability and lifespan.

[0052] According to at least one of the embodiments of the present disclosure, system complexity is reduced, facilitating design and reducing costs by reducing parts.

[0053] According to at least one of the embodiments of the present disclosure, high quality audio and haptics can be output.

[0054] According to at least one of the embodiments of the present disclosure, there is no need to create a separate speaker hole in the home appliance, thereby preventing moisture from penetrating from the outside and increasing the degree of freedom in design.

[0055] Meanwhile, various other effects will be disclosed directly or implicitly in the detailed description according to the embodiments of the present disclosure to be described later.

[0056] FIG. 1 is a configuration diagram of a home appliance system including a home appliance according to one embodiment of the present disclosure.

[0057] FIG. 2 is a block diagram of the main configuration of an audio haptic output device according to an embodiment of the present disclosure.

[0058] FIG. 3 and FIG. 4 are drawings for reference in the description of data combination according to an embodiment of the present disclosure.

[0059] FIG. 5 and FIG. 6 are drawings for reference in the description of the frequency characteristics of an actuator according to an embodiment of the present disclosure.

[0060] Figure 7 is a diagram illustrating a haptic waveform.

[0061] Figure 8 is a diagram illustrating a button sound waveform.

[0062] Figure 9 is a diagram illustrating a voice guidance message waveform.

[0063] Figure 10 is a diagram illustrating haptic and button sound waveforms.

[0064] Figure 11 is a diagram illustrating a music sound source waveform.

[0065] Figures 12 and 13 are drawings illustrating collisions caused by actuator vibration.

[0066] FIG. 14 and FIG. 15 are drawings for reference in the description of haptic output of an audio haptic output device according to one embodiment of the present disclosure.

[0067] FIG. 16 and FIG. 17 are drawings for reference in a description of audio haptic output of an audio haptic output device according to one embodiment of the present disclosure.

[0068] FIG. 18 and FIG. 19 are drawings for reference in a description of audio output of an audio haptic output device according to an embodiment of the present disclosure.

[0069] FIG. 20 is a drawing for reference in a description of an internal block of a speaker amplifier of an audio haptic output device according to an embodiment of the present disclosure.

[0070] Figures 21 and 22 are drawings referring to an audio haptic output device utilizing a conventional actuator.

[0071] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the attached drawings. However, the present disclosure is not limited to these embodiments and can be modified in various forms.

[0072] In the drawings, parts that are not related to the description are omitted in order to clearly and concisely explain the present disclosure, and the same drawing reference numerals are used for the same or extremely similar parts throughout the specification.

[0073] Meanwhile, the suffixes "module" and "part" used in the following description are given solely for the convenience of writing this specification, and do not impart any particularly significant meaning or role to them. Therefore, the terms "module" and "part" may be used interchangeably.

[0074] Additionally, while terms such as "first" and "second" may be used in this specification to describe various elements, these elements are not limited by these terms. These terms are used only to distinguish one element from another.

[0075] FIG. 1 is a configuration diagram of a home appliance system including a home appliance according to one embodiment of the present disclosure.

[0076] Referring to FIG. 1, a home appliance system (1) according to one embodiment of the present disclosure may include one or more home appliances (10), a server (20), and a mobile terminal (30).

[0077] Home appliances (10) are electronic devices for users, and examples thereof include air conditioners (not shown), air purifiers (not shown), refrigerators (not shown), washing machines (not shown), cooking appliances (not shown), and vacuum cleaners (not shown).

[0078] The home appliance (10) has an internal communication unit (not shown) and can exchange data with electronic devices within the internal network or electronic devices accessible via an external network. To this end, the communication unit (not shown) can exchange data with a server (20), AP device, etc., either wired or wirelessly.

[0079] Meanwhile, the mobile terminal (30) can perform wireless communication with the home appliance (10), remotely control the wirelessly connected home appliance (10), or receive monitoring data from the home appliance (10).

[0080] Meanwhile, the server (20) can transmit and receive data with a home appliance (10) connected via a network, etc. For example, the server (20) can provide update information to the home appliance (10) at the request of a mobile terminal (30).

[0081] As another example, the server (20) can provide update information to the home appliance (10) when necessary, even if there is no request from the mobile terminal (30) or the home appliance (10).

[0082] Meanwhile, a home appliance (10) according to an embodiment of the present disclosure includes an audio-haptic output device (11). The home appliance (10) includes an operating unit (not shown) that receives user input, and the audio-haptic output device (11) may be configured as a part of the operating unit or may be connected to the operating unit to output audio effects, haptic effects, and audio / haptic effects corresponding to user input through the operating unit.

[0083] The above-described operating unit may include one or more buttons for receiving user input. The operating unit may be provided as a touch panel and may include one or more buttons for operating the home appliance (10).

[0084] For example, if the home appliance (10) is a washing machine or dryer, the control unit may include one or more control means for controlling the operation mode such as washing / drying.

[0085] For example, if the home appliance (10) is an air conditioner, the control unit may include buttons for selecting and controlling an operation mode such as cooling operation, automatic operation, or air purification operation.

[0086] For example, if the home appliance (10) is a water dispensing device, the control unit may include a capacity button for selecting the water dispensing capacity, a hot water button for selecting hot water and further selecting the temperature of the hot water to be dispensed, a purified water button for selecting purified water, a cold water button for selecting cold water, and other function buttons.

[0087] The above audio haptic output device (11) outputs a haptic effect in response to user input through the operating unit, so that the user can feel tactile feedback on his or her input.

[0088] The above audio haptic output device (11) can output audio such as button sounds, voice guidance messages, and music in response to user input through the operating unit. In addition, the audio haptic output device (11) can output sound effects for driving conditions and can output a predetermined warning sound when an error occurs.

[0089] In addition, the audio haptic output device (11) can provide composite feedback to the user by outputting audio and haptics together.

[0090] The configuration and operation of the above audio haptic output device (11) will be described in detail later with reference to FIGS. 2 to 20.

[0091] Meanwhile, the audio haptic output device (11) can be upgraded for audio effects, haptic effects, and composite effects combining audio and haptics. Therefore, by receiving data on new effects from the server (20) or the mobile terminal (30) and upgrading, a variety of effects can be expanded and provided to users without limitation. Accordingly, in addition to the effects provided only at the time of product launch, a variety of functions and effects can be provided using the audio haptic output device (11).

[0092] The home appliance (10) can run system applications, applications related to driving modes, or download applications internally.

[0093] Meanwhile, the update information may include firmware update information, software update information, etc. The firmware update information is data for updating the firmware in the home appliance (10), and may be, for example, update information regarding load control or safety control of the home appliance (10). Meanwhile, the software update information may include upgrade information related to a new operating mode. Based on the upgrade information related to the new operating mode, the operating mode can be upgraded. Meanwhile, the software update information may further include update information for a system application, an application related to the operating mode, or a download application. Based on this update information, the operating mode can be updated. The firmware update information may include data regarding the firmware, the operating system, or the framework. Based on this firmware update information, the firmware can be updated.

[0094] Upgrade information related to a new driving mode can be received from an external server (20) or mobile terminal (30), and an upgrade for the new driving mode can be performed based on the upgrade information related to the new driving mode.

[0095] Meanwhile, a home appliance (10) according to an embodiment of the present disclosure may perform updates related to changing settings for existing driving modes, in addition to upgrading to new driving modes. A home appliance (10) according to an embodiment of the present disclosure may perform updates related to an audio haptic output device (11).

[0096] Meanwhile, the home appliance system (1) may further include an access point (AP) device (not shown). The AP device may provide an internal network to adjacent electronic devices. In particular, it may provide a wireless network.

[0097] Meanwhile, the AP device can allocate a wireless channel using a predetermined communication method to electronic devices within the internal network and perform wireless data communication through the channel. Here, the predetermined communication method may be a WiFi communication method.

[0098] At this time, a mobile terminal (30) located within the internal network can perform monitoring, remote control, etc. of the home appliance (10) by connecting to the home appliance (10) through an AP device.

[0099] Meanwhile, the AP device can perform data communication with external electronic devices via an external network, in addition to the internal network. For example, the AP device can perform wireless data communication with a mobile terminal (30) located outside via the external network. In this case, the mobile terminal (30) located in the external network can monitor, remotely control, etc. the home appliance (10) by connecting to the home appliance via the external network and the AP device.

[0100] As another example, the AP device can perform wireless data communication with a server (20) located externally via an external network.

[0101] Meanwhile, the server (20) can store firmware information and driving information (course information, etc.) for the home appliance (10) and register product information for the home appliance (10). For example, the server (20) may be a server operated by a home appliance (10) manufacturer.

[0102] FIG. 2 is a block diagram of the main configuration of an audio haptic output device according to an embodiment of the present disclosure.

[0103] Referring to FIG. 2, an audio haptic output device (11) according to an embodiment of the present disclosure and a home appliance (10) including the same include a touch panel (110), an actuator (150) that outputs at least one of audio and haptic in response to a touch input (111) through the touch panel (110), and a memory (120) in which data for driving the actuator (150) is stored.

[0104] When there is a touch input on the touch panel (110), the actuator (150) can output an audio feedback effect, a haptic feedback effect, or a composite feedback effect combining audio and haptics, corresponding to the location where the touch input (111) is made.

[0105] In addition, the touch panel (110) includes a button, and the actuator (150) can output an audio feedback effect, a haptic feedback effect, or a composite feedback effect combining audio and haptics in response to the touched button.

[0106] The above actuator (150) can be placed on the rear side of the touch panel (110).

[0107] Fastening elements, such as screws, can make it difficult to generate haptic feedback by forcibly fixing the fastening portion. Furthermore, noise may be generated momentarily at the screw fastening portion, and vibration and interference with the actuator (150) may occur. Therefore, it is desirable to minimize the use of screws in the audio haptic output device (11).

[0108] For this purpose, the touch panel (110) may be attached to another mechanical structure using an adhesive material such as tape. Additionally, the actuator (150) may be attached to the rear surface of the touch panel (110) using tape.

[0109] Preferably, in order to eliminate the foreign noise caused by interference between the touch panel (110) and the internal injection molding material generated when the actuator (150) vibrates, a damping material may be added between the touch panel (110) and the internal injection molding material. For example, the internal injection molding material may be a mechanism for fixing and protecting internal components such as a printed circuit board (PCB).

[0110] For example, a polyurethane foam, Poron, may be additionally placed between the touch panel (110) and the inner injection material. When Poron is attached to the touch panel (110), the vibration of the touch panel (110) can attenuate energy transmitted to the housing coupled to the touch panel (110) through the damping of Poron.

[0111] Damping materials such as porons may be arranged at the center of the rear surface of the touch panel (110). In addition, the damping materials may be arranged between the buttons of the touch panel (110). The damping materials may be arranged in a plurality of lines passing between the buttons of the touch panel (110). For example, the damping materials may be arranged in a plurality of lines parallel in the horizontal direction. In addition, the damping materials may be arranged in a plurality of lines parallel in the vertical direction. In addition, the damping materials may be arranged in a grid shape.

[0112] The memory (120) stores a plurality of effect data corresponding to a plurality of touch inputs. Since the actuator (150) outputs at least one of audio and haptics, the memory (120) stores audio data and haptic data corresponding to a plurality of touch inputs. In addition, depending on the provided effect, the effect corresponding to a specific input is stored by combining the audio data and the haptic data. By combining the audio signal and the haptic signal and storing them in advance in the memory (120), the audio and haptics can be output more effectively according to the situation. That is, the plurality of effect data stored in the memory (120) includes haptic data, audio data, and combined data in which the haptic data and the audio data are combined in advance.

[0113] The actuator (150) outputs at least one of audio and haptic based on effect data matching the touch input (111) through the touch panel (110) among the data stored in the memory (120).

[0114] The actuator (150) can output vibrations having a frequency and amplitude corresponding to the pattern of the input electrical signal. The vibration of the actuator (150) is generated by a patterned electrical signal, and the vibration of the actuator (150) can be controlled by adjusting the electrical signal input to the actuator (150).

[0115] An audio haptic output device (11) according to one embodiment of the present disclosure and a home appliance (10) including the same further include a processor (130) that performs signal processing on data stored in the memory (120) into an electric signal input to the actuator (150) and a signal to be output by the actuator (150), and a speaker amplifier (140) that amplifies an output signal of the processor (130).

[0116] For example, the processor (130) can decode and output an encoded signal. The processor (130) may also be referred to as a decoder or signal processor.

[0117] Additionally, the processor (130) can convert the effect data called from the memory (120) into an analog audio signal or an audio signal according to an audio communication standard (e.g., I2S). The audio signal may be an analog audio signal or may be composed of data according to a known audio communication standard.

[0118] The haptic signal may include at least one of the amplitude, frequency, and duration of vibration of the actuator (150). Meanwhile, by converting the haptic signal into an analog audio signal or an audio signal according to an audio communication standard and storing it, data processing and conversion can be performed more effectively, and haptic effect data and audio effect data can be more effectively combined.

[0119] The plurality of effect data stored in the above memory (120) include haptic data, audio data, and combined data in which haptic data and audio data are combined in advance. In this case, the haptic data, the audio data, and the combined data may be encoded and stored as audio type data.

[0120] The above combined data may be generated by sequentially combining the haptic data and the audio data in that order. Alternatively, the combined data may be generated by mixing the haptic data and the audio data so that they overlap.

[0121] FIG. 3 and FIG. 4 are drawings for reference in the description of data combination according to an embodiment of the present disclosure.

[0122] Figure 3 illustrates an example of mixing and combining haptics and audio. Referring to Figure 3, when haptics of a predetermined length (e.g., 50 ms) or longer are simultaneously output with audio, audio and haptic data are mixed to overlap and stored in an audio data format.

[0123] Figure 4 illustrates an example of sequentially combining haptic and audio. Referring to Figure 4, for short haptic signals of less than a certain length (e.g., 50 ms), it is acceptable to transmit the haptic signal first and the audio later. This is because the short haptic action time allows for the haptic and audio to be perceived as being output simultaneously.

[0124] Audio data and haptic data are stored in the memory (120), and the processor (130) calls and decodes data corresponding to the touch input (111) and transmits it to the speaker amplifier (140). The signal amplified by the speaker amplifier (140) is input to the actuator (150), and the actuator (150) outputs audio or haptic depending on the input signal.

[0125] When audio data and haptic data are output simultaneously, the audio data and haptic data are combined in advance in the form of audio data and stored in the memory (120).

[0126] In this case, the processor (130) calls data corresponding to the touch input (111), decodes it, and transmits it to the speaker amplifier (140). The signal amplified by the speaker amplifier (140) is input to the actuator (150), and the actuator (150) outputs audio and haptic according to the input signal.

[0127] When audio and haptic data are individually converted and combined, sound and / or vibration can be distorted during the combination process. Processing the frequency characteristics of pre-synthesized data according to the type of output effect and frequency band offers the advantage of providing high-quality audio and haptic data without distortion.

[0128] According to an embodiment, the processor (130) calls effect data matching the touch input (111) through the touch panel (110) from the memory (120) to generate an audio signal, and the speaker amplifier (140) amplifies a signal based on the audio signal output by the processor (130) and outputs the signal to the actuator (150).

[0129] The above actuator (150) outputs at least one of audio (170) and haptic (160) based on the output signal of the speaker amplifier (140).

[0130] According to one embodiment of the present disclosure, by appropriately controlling the frequency of a signal input to an actuator (150) according to the output, damage to the actuator can be prevented and its lifespan can be improved. This frequency characteristic conversion (135) can be performed in the processor (130). Alternatively, the frequency characteristic conversion (135) can be performed in the speaker amplifier (140).

[0131] The above processor (130) or the speaker amplifier (140) converts and outputs the frequency characteristics of the audio signal when the effect data matching the touch input (111) through the touch panel (100) includes audio data.

[0132] The processor (130) or the speaker amplifier (140) converts the frequency characteristics of the audio signal according to different frequency conversion logics when the actuator (150) outputs both audio and haptics and when the actuator (150) outputs only audio.

[0133] For example, when the actuator (150) outputs audio and haptics of voice guidance or button sounds, the processor (130) or the speaker amplifier (140) increases the gain only in the audio region frequency band preset for the voice guidance or button sounds, thereby converting the frequency characteristics of the audio signal.

[0134] In addition, when the actuator (150) outputs audio of voice guidance or button sounds and haptics together, the processor (130) or the speaker amplifier (140) converts the frequency characteristics of the audio signal by increasing the gain of the audio frequency band preset for the voice guidance or button sounds to be greater than the gain of the haptic frequency band.

[0135] In addition, when the actuator (150) outputs audio and haptics of voice guidance or button sounds, the processor (130) or the speaker amplifier (140) applies a first gain at a first frequency among the preset audio range frequency bands for the voice guidance or button sounds, and applies a second gain greater than the first gain at a second frequency that is further from the resonance frequency of the actuator (150) than the first frequency, thereby converting the frequency characteristics of the audio signal.

[0136] In addition, when the actuator (150) outputs only audio of voice guidance or button sounds, the processor (130) or the speaker amplifier (140) converts the frequency characteristics of the audio signal by amplifying only the audio region frequency band preset for the voice guidance or button sounds.

[0137] In addition, when the actuator (150) outputs only music type audio, the processor (130) or the speaker amplifier (140) attenuates a frequency band within a predetermined range from the resonant frequency of the actuator (150) and amplifies at least a portion of the frequency band among other regions, thereby converting the frequency characteristics of the audio signal.

[0138] Meanwhile, the processor (130) or the speaker amplifier (140) does not convert the frequency characteristics of the audio signal when the actuator (150) outputs only haptics, or converts the frequency characteristics of the audio signal by increasing the area outside a predetermined range of frequency bands from the resonant frequency of the actuator (150).

[0139] According to one embodiment of the present disclosure, the processor (130) or the speaker amplifier (140) may convert the frequency according to different frequency conversion logics when the actuator (150) outputs only haptics and when the actuator (150) outputs both haptics and audio, and the speaker amplifier (140) may amplify the frequency-converted signal.

[0140] Meanwhile, the actuator (150) may be considerably smaller than the touch panel (110). In this case, when the actuator (150) vibrates for haptic output, the farther away the actuator (150) is, the weaker the tactile sensation will be felt. Accordingly, if the actuator (150) vibrates with the same intensity, it is difficult for the user to feel the haptic effect when touching an area / button that is far from the actuator (150) and when touching an area / button that is close to the actuator (150).

[0141] According to one embodiment of the present disclosure, the processor (130) or the speaker amplifier (140) may convert and output the frequency characteristics of the audio signal so that the actuator vibration upon a touch input to a second area located further away from the first area with respect to the actuator (150) is greater than the actuator vibration upon a touch input to a first area of ​​the touch panel (110). Accordingly, a constant tactile sensation can be felt by the user regardless of the touch area.

[0142] According to an embodiment, the processor (130) calls at least one of audio data and haptic data matching a touch input (111) through the touch panel (110) from the memory (120) and converts it into an audio signal, and the speaker amplifier (140) amplifies a signal based on the audio signal output by the processor (130) and outputs it to the actuator (150).

[0143] According to embodiments of the present disclosure, a single actuator (150) is utilized for both haptic (160) and audio (170) reproduction. Audio signals and haptic signals are stored in a memory (120), processed into audio signals through a processor (130) such as an MCU (Micom unit) or an AP (Application processor), and input to and output from a speaker amplifier (140).

[0144] In cases where a haptic signal and an audio signal must be output simultaneously, the haptic signal and the audio signal are combined in the form of audio data and stored in the memory (120), and then processed in the same process in that situation.

[0145] The haptic and audio actuator (150) must be designed to exhibit gentle damping at frequencies above the resonant frequency. To achieve gentle frequency damping, the vibration generating element k (stiffness) must be low. To achieve this, it is preferable to select and use an actuator (150) made of a material such as rubber.

[0146] FIG. 5 and FIG. 6 are drawings for reference in the description of the frequency characteristics of an actuator according to an embodiment of the present disclosure. FIG. 5 illustrates the frequency characteristics of various actuators (510, 520, 530, 540), and FIG. 6 illustrates the SPL (sound pressure level) for each frequency of the actuator (520) selected in FIG. 5.

[0147] For haptic actuators (510), a metal spring material with a high k (steffness) is typically used. To use the actuator for both haptic and audio applications, the resonance of the actuator must be formed within the haptic operating frequency range. Referring to FIGS. 5 and 6, the resonance frequency for implementing haptic performance in home appliances is in the range of 100 to 200 Hz.

[0148] A resonant frequency (530) below 100 Hz produces a heavy haptic sensation and large vibration displacement, increasing the likelihood of collisions between components in the product. Applying a resonant frequency (540) above 200 Hz for haptic applications in home appliances can generate unintended sounds along with the haptic vibration. Therefore, it is desirable to select an actuator (520) whose resonance is located within the haptic operating frequency and whose damping is gentle at subsequent frequencies.

[0149] To use an actuator for both haptic and sound applications, the actuator's resonance must be established within the haptic operating frequency range, and the damping must be gentle at frequencies above the resonant frequency. If the damping is too rapid, it will be difficult to use for sound applications. To achieve this, the actuator's vibration elements must have low stiffness. For this purpose, it is preferable to use a material with a low k (stiffness) (e.g., rubber) for the actuator's spring elements.

[0150] Meanwhile, to ensure sufficient audio output performance, vibration damping must be gentle at frequencies above the resonant frequency. Gentle vibration damping allows for frequency control algorithms to compensate for insufficient performance in the frequency band, enabling the implementation of full-range speaker frequency characteristics, as shown in Fig. 6. If vibration damping is too rapid, the audio amplifier's volume must be turned up significantly, and the frequency-specific volume level balance becomes unbalanced, making it difficult to apply for audio applications.

[0151] According to an embodiment of the present disclosure, a frequency control algorithm is applied to utilize the actuator (150) for both haptic and audio purposes. The frequency control algorithm compensates and attenuates the operating frequency band of the haptic or audio to implement performance suited to each function.

[0152] Comparing FIG. 2 and FIG. 21, the functions of the haptic driver and speaker amplifier are integrated into one speaker amplifier (140) and used, and the haptic actuator and speaker are also implemented into one actuator (150).

[0153] Referring to FIG. 2, when a user touches (111) a specific action button of a touch panel (110), touch action button data is transmitted from the memory (120) to the processor (130).

[0154] The memory (120) stores haptic data and audio data matched to each touch action button. The haptic data is stored in audio data format. In addition, when haptic data and audio data are simultaneously output in a specific button action, the haptic data and audio data are combined and stored as a single data (audio data format).

[0155] The processor (130) retrieves haptic / audio data corresponding to a button touched by the user from the memory (120). The processor (130) can decode the retrieved haptic / audio data into an audio signal (I2S or Analog Audio) format. For example, I2S (Integrated Interchip Sound) is a communication protocol for transmitting and receiving audio data, and there are audio file formats such as Wave and MPEG Layer-3 (mp3).

[0156] The decoded haptic / audio signal is frequency-converted through a frequency characteristic conversion algorithm (135). The frequency characteristic conversion algorithm (135) may be executed in the processor (130). Alternatively, if the speaker amplifier (140) has its own built-in DSP (digital signal processor), the frequency characteristic conversion algorithm (135) may be processed in the speaker amplifier (140).

[0157] The haptic / audio signal with converted frequency characteristics is amplified in a speaker amplifier (140), and the amplified haptic / audio signal is transmitted to an actuator (150) and output.

[0158] The actuator (150) can be attached to the touch panel (110) or the device and can use the touch panel or the device of the electronic device as a vibration plate to generate sound and vibration. The processor (130) or the speaker amplifier (140) can control the actuator (150) to vibrate at a predetermined frequency for a predetermined period of time. Depending on the vibration of the actuator (150), only sound, haptic sensation, or both sound and haptic sensation can be output.

[0159] According to one embodiment of the present disclosure, the operating frequency band of haptics or audio is compensated and attenuated through a frequency characteristic conversion algorithm (135) to implement performance suited to each function. The feedback effects output from the home appliance (10) come in various combinations, and each of the output feedback effects has different operating frequency band characteristics, requiring frequency control for each case.

[0160] FIG. 7 is a diagram illustrating a haptic waveform, where (a) illustrates a haptic waveform along the time axis (X-axis), and (b) illustrates a haptic waveform along the frequency axis (X-axis). Referring to FIG. 7, haptics uses a frequency below 200 Hz as a frequency band that generates a vibration effect. The haptic frequency is generated around the actuator resonance, and for example, the haptic frequency can be 180 Hz.

[0161] Fig. 8 is a diagram illustrating a button sound waveform, where (a) illustrates the button sound waveform along the time axis (X-axis), and (b) illustrates the button sound waveform along the frequency axis (X-axis). Since the button sound is a simple button sound for notification, it is mainly a single tone or a combination of multiple single tones, and a frequency band of 500 Hz or higher is generally used. Referring to Fig. 8, the button sound can be formed in the form of a single tone at 700 Hz and 3 KHz, which does not overlap with the haptic operation frequency.

[0162] Fig. 9 is a diagram illustrating a voice guidance comment waveform, where (a) illustrates a voice guidance comment waveform along the time axis (X-axis), and (b) illustrates a voice guidance comment waveform along the frequency axis (X-axis). The voice guidance comment can use a high-pitched voice with excellent clarity as the voice guidance comment. Referring to Fig. 9, the voice guidance comment waveform has energy concentrated in the frequency band of 200 Hz to 1 KHz. Fig. 9 illustrates a voice guidance comment frequency of 200 Hz or higher, and although the haptic frequency and the voice guidance comment frequency may overlap to some extent at the boundary, the boundary region can be viewed as a separate frequency region because the energy is small.

[0163] Fig. 10 is a diagram illustrating waveforms of haptics and button sounds, where (a) illustrates a waveform in which haptics and button sounds are combined along the time axis (X-axis), and (b) illustrates a waveform in which haptics and button sounds are combined along the frequency axis (X-axis). In order to output button sounds (or voice guidance comments) and haptics simultaneously, when button sounds (voice guidance comments) and haptics are combined, they can be transmitted separately because their respective operating frequency bands are different. Referring to Fig. 10, when haptics and button sounds are combined, the frequency bands are separated, and the operating frequencies of the two signals do not overlap.

[0164] Fig. 11 is a diagram illustrating a music sound source waveform, where (a) illustrates a music sound source waveform along the time axis (X-axis), and (b) illustrates a music sound source waveform along the frequency axis (X-axis). Since the frequency range of music is determined depending on the type of instrument used, the entire audio frequency can be considered as the operating range when outputting music, and the operating range of the actuator is 50 Hz to 10 KHz. Fig. 11 illustrates a music sound source frequency, and a wide frequency band is formed including the resonant frequency band of the actuator above 50 Hz.

[0165] Figures 12 and 13 are drawings illustrating collisions caused by actuator vibration. Figure 12 is a cross-sectional view of an actuator illustrating collisions at the actuator resonance frequency, and Figure 13 illustrates a case in which an actuator (1330) housed inside a housing (1320) vibrates a panel (1310), such as a touch panel or display panel.

[0166] Referring to Fig. 12, vibration can be generated using the electromagnetic force between a magnet and a coil (yoke). The actuator converts the electromagnetic force into a mechanical driving force, allowing the vibrator to vibrate up and down.

[0167] Meanwhile, the actuator of FIG. 12 is exemplary, and the present disclosure is not limited to the type of actuator. For example, a piezoelectric actuator, a linear resonant actuator, an electro-active polymer actuator, etc. may be used.

[0168] Referring to Fig. 12, if the input signal is too large at the actuator resonance frequency, the displacement of the vibrator may exceed the operating range, causing a collision by hitting the external housing of the actuator.

[0169] When music is played around the resonant frequency of the actuator, the actuator vibrator may move beyond its vibration displacement range, as shown in the example of Fig. 12. In this case, the actuator's outer housing and the vibrator or a component coupled to the vibrator may collide with the actuator's outer housing, resulting in noise.

[0170] Referring to FIG. 13, due to excessive vibration of the panel (1310) at the actuator resonance frequency, the actuator (1330) may collide with the housing (1320) coupled to the panel (1310), or the actuator (1330) may collide with another component coupled to the panel (1310).

[0171] As in the example of Fig. 13, if the amount of vibration that the actuator (1330) causes the panel (1310) to vibrate is too large, a collision due to the vibration may occur in the product. Therefore, vibration control at the actuator resonance frequency is required.

[0172] As described with reference to FIGS. 7 to 13, there are multiple combinations of output feedback effects, each of which has different operating frequency band characteristics. Accordingly, the frequency characteristic conversion algorithm (135) according to the present disclosure compensates and attenuates the operating frequency band of haptics or audio based on the combination of output feedback effects, thereby implementing performance suited to each function.

[0173] FIG. 14 and FIG. 15 are drawings for reference in the description of haptic output of an audio haptic output device according to an embodiment of the present disclosure, exemplifying a case where only haptics are output. FIG. 14 illustrates a driving area according to frequency when only haptics are output in a frequency and acceleration relationship, and FIG. 15 illustrates a frequency characteristic conversion algorithm when only haptics are played.

[0174] Referring to Figures 14 and 15, haptics requires a vibration intensity that is strong enough to be felt at the fingertips. Therefore, the haptic signal is generated in a band around the actuator resonance frequency. The range of the band around the resonance frequency is the region where the gravitational acceleration is attenuated to less than 30% of the maximum performance. In Figure 14, an actuator with a resonance point of 130 Hz is shown as an example, and in this case, a haptic signal is generated in the region of 110 Hz to 190 Hz. On the other hand, when the range is exceeded toward the high frequency band, the vibration attenuation is severe, so a very high gain setting is required, and contrary to the intention of generating only haptic vibration, sound may also be generated.

[0175] According to one embodiment of the present disclosure, a haptic signal can be output without frequency characteristic conversion.

[0176] Alternatively, the frequency control algorithm can be applied only to the band around the actuator resonance frequency (the attenuation region of less than 30% of the gravitational acceleration).

[0177] Additionally, the haptic signal optimized for the resonant frequency can be set to 0dB gain, and when using haptics in the surrounding frequency range, it can be set to a ±3dB gain range at that frequency.

[0178] Referring to Fig. 15, when generating an optimized haptic signal at the resonant frequency, there is no gain setting for each frequency.

[0179] When only haptic output is used, the frequency-specific control method (gain setting method) sets the haptic signal optimized for the resonant frequency to 0dB Gain (no gain setting for each frequency, Fig. 15), and the haptic signal in the band surrounding the resonant frequency can have its gain set to ±3dB. Setting the gain to ±3dB in the band surrounding the resonant frequency is the gain value required to optimize the vibration intensity to the level of the resonant frequency.

[0180] The frequency characteristic conversion algorithm (135) can increase or decrease the signal according to the gain set for each frequency band in the input signal.

[0181] FIG. 16 and FIG. 17 are drawings for reference in a description of audio haptic output of an audio haptic output device according to an embodiment of the present disclosure, and exemplify a case where haptics and voice guidance (or button sounds) are output. FIG. 16 illustrates a driving area according to frequency when haptics and voice guidance (or button sounds) are output in a frequency and acceleration relationship, and FIG. 17 illustrates a frequency characteristic conversion algorithm when playing haptics and voice guidance (or button sounds).

[0182] Referring to FIG. 16, the frequency range (1620) used for haptics and the frequency range (1610) used for voice guidance (or button sounds) are separated and can be distinguished.

[0183] The above processor (130) or the speaker amplifier (140) converts the frequency characteristics of the audio signal by increasing the gain only in the audio region frequency band (1610, 1700) preset for the voice guidance or button sound when the actuator (150) outputs audio and haptics of the voice guidance or button sound.

[0184] Alternatively, the processor (130) or the speaker amplifier (140) converts the frequency characteristics of the audio signal by increasing the gain of the audio frequency band (1610, 1700) preset for the voice guidance or button sound to be greater than the gain of the haptic frequency band (1620) when the actuator (150) outputs audio of voice guidance or button sound and haptics together.

[0185] Referring to Figures 16 and 17, the 200 Hz boundary between the haptic frequency band (1620) and the voice frequency band (1610) does not have a gain setting because it affects each frequency band. Above 300 Hz, the gain is set to improve the performance of voice guidance (button sound) output.

[0186] Referring to FIGS. 16 and 17, when haptic and voice guidance (or button sounds) are output together, the frequency characteristic conversion algorithm (135) increases the frequency-specific gain (e.g., 5 dB or more) in the voice guidance (button sound) range (e.g., 300 Hz or more) (1610).

[0187] In addition, with regard to haptics, the frequency characteristic conversion algorithm (135) described with reference to FIGS. 14 and 15 can be applied in the same manner. For example, a haptic signal optimized for a resonant frequency can be set to 0 dB gain, and when using haptics in a surrounding frequency range, the gain can be set to ±3 dB at that frequency.

[0188] The actuator's voice (button tone) frequency band (1610) exhibits greater performance attenuation as it moves away from the actuator resonant frequency. Therefore, to improve output performance, the gain can be increased by 5 dB or more above 300 Hz.

[0189] In addition, when the actuator (150) outputs audio and haptics of voice guidance or button sounds, the processor (130) or the speaker amplifier (140) applies a first gain (e.g., 5 dB) to a first frequency (e.g., fr1) among the preset audio area frequency bands for the voice guidance or button sounds, and applies a second gain (e.g., 7 dB) greater than the first gain to a second frequency (e.g., fr2) that is further from the resonance frequency of the actuator (150) than the first frequency, thereby converting the frequency characteristics of the audio signal.

[0190] For example, the gain can be set for each frequency (e.g., fr1 to fr6) in the gain increase region (1700). In this case, the gain can be set to be larger at frequencies fr2 to fr6, which are farther away from the resonant frequency of the actuator, than at frequency fr1, which is relatively close to the resonant frequency of the actuator.

[0191] Referring to FIGS. 16 and 17, the haptic frequency area (1620) sets the gain within ±3 dB, and the voice guidance (button sound) frequency area (1610) sets the gain to 5 dB or more in consideration of the attenuation level for each frequency.

[0192] Meanwhile, in the case where only voice guidance (or button sounds) are output, the gain for each frequency can be increased (by 5 dB or more) in the voice guidance (button sound) range (300 Hz or more) in the same manner. That is, in addition, when the actuator (150) outputs only the audio of voice guidance or button sounds, the processor (130) or the speaker amplifier (140) amplifies only the frequency band of the audio range preset for the voice guidance or button sounds, thereby converting the frequency characteristics of the audio signal.

[0193] The frequency of voice guidance (human voice) or button sounds is far from the actuator resonance frequency band, so the gain must be set to improve the performance of voice guidance (button sounds) output above 300 Hz.

[0194] Since the performance attenuation of the actuator's voice (button sound) frequency band increases as it moves away from the actuator resonance frequency, even when only voice guidance (or button sound) is output, the gain can be increased by 5 dB or more above 300 Hz to improve output performance.

[0195] FIG. 18 and FIG. 19 are diagrams for reference in a description of audio output of an audio haptic output device according to an embodiment of the present disclosure, exemplifying a case where only music audio is output. FIG. 18 illustrates a driving area according to frequency when only music is output, and FIG. 19 illustrates a frequency characteristic conversion algorithm when playing music.

[0196] Referring to FIGS. 18 and 19, the gain for each frequency is increased (by 5 dB or more) in the high frequency (200 Hz or more) range (1810) that is higher than the resonant frequency range (1820). In addition, a notch filter or a high-pass filter may be applied in the resonant frequency range (1820) to attenuate by 5 dB or more at the resonant frequency. In the case of music, the frequency band is formed widely from low frequency to high frequency. Therefore, a large signal level may be input at the haptic resonant frequency. In this case, a signal input that exceeds the output range of the actuator may be input, which may cause damage to the actuator or collision between parts. To solve this problem, the resonant frequency is attenuated and the gain is increased in the other audio frequency band (1810). The purpose of improving the audio frequency band (1810) is to compensate for the frequency characteristics of the actuator.

[0197] The processor (130) or the speaker amplifier (140) attenuates a frequency band (1820) within a predetermined range from the resonant frequency of the actuator (150) and amplifies at least a portion of the frequency band (1810) among other regions when the actuator (150) outputs only music type audio, thereby converting the frequency characteristics of the audio signal. + The resonant frequency region (1910) including the resonant frequency (fr7) may be attenuated by a notch filter or a high-pass filter. The gain may be set for each frequency (e.g., fr8 to fr13) of the gain increase region (1920). In this case, the gain may be set to be greater at frequencies fr9 to fr13 that are farther away than at frequency fr8 that is relatively close to the resonant frequency (fr7) of the actuator.

[0198] FIG. 20 is a drawing for reference in a description of an internal block of a speaker amplifier of an audio haptic output device according to an embodiment of the present disclosure, and exemplifies a speaker amplifier (2000) having a built-in DSP (digital signal processor).

[0199] It may be difficult to process the frequency conversion algorithm (135) at the hardware level of the microcomputer unit used in general home appliances. Therefore, it may be more desirable to process the frequency characteristic conversion algorithm (135) in the DSP (2020) built into the speaker amplifier (2000).

[0200] An audio signal in a format such as I2S, TDM, or analog is received through an audio interface (2010). The received audio signal is frequency controlled and processed in a DSP (2020) according to a frequency characteristic conversion algorithm (135) and transmitted to an amplifier for amplification (2040). The signal amplified in the amplifier (2040) is input to an actuator (150), and the actuator (150) vibrates according to the input signal and can output haptic, audio, or haptic / audio composite effects.

[0201] Depending on the embodiment, the speaker amplifier (2000) may further include a digital to analog converter (DAC) (2030) or an analog to digital converter (ADC).

[0202] According to an embodiment of the present disclosure, a speaker and vibration actuator can be replaced with a single actuator (150), and a haptic driver and audio amplifier can be replaced with a single audio speaker amplifier (140). This reduces system complexity, facilitating design, and reduces costs by reducing the number of components.

[0203] In addition, when using the actuator (150) as a speaker in a home appliance, there is no need to create a separate speaker hole, so moisture infiltration from the outside can be prevented, and an excellent design is possible.

[0204] According to one embodiment of the present disclosure, when audio and haptics are output simultaneously, audio data and haptic data are combined in advance in the form of audio data and stored in memory (120), and are called and used when necessary.

[0205] Audio and haptic data can be combined using a mixing method and / or a sequential output combining method. The mixing method overlaps audio and haptic data and synthesizes new data. The sequential combining method transmits the haptic data first, followed immediately by the audio data. Since haptic data can achieve haptic performance even with short data lengths of less than 1 msec, even if audio data is output immediately after the haptic transmission, it can be perceived as being output simultaneously.

[0206] According to one embodiment of the present disclosure, a frequency control algorithm (135) is applied to utilize the actuator (150) for both haptic and audio purposes.

[0207] When only haptics are output, the frequency conversion algorithm (135) may not be applied, or only the haptic operating frequency band may be fine-tuned. For example, the frequency conversion algorithm (135) may be applied only to the band surrounding the actuator resonance frequency (attenuation area of ​​less than 30% of gravitational acceleration). In addition, a haptic signal optimized for the resonance frequency may be set to 0 dB Gain, and when using haptics in the surrounding frequency range, the Gain may be set within a range of ±3 dB at the corresponding frequency.

[0208] In cases where only voice guidance (or button sounds) are output, the frequency conversion algorithm (135) can set the gain to increase only the audio region frequency. For example, the gain for each frequency can be increased in the voice guidance (button sounds) region of 300 Hz or higher.

[0209] Even when haptic and voice guidance (or button sounds) are output simultaneously, only the audio frequency can be increased. When haptic and voice guidance (or button sounds) are output simultaneously, the frequency bands of haptic and voice guidance (or button sounds) are separated, so the logic for outputting only haptic can be applied to haptic, and the logic for outputting only voice guidance (or button sounds) can be applied to voice guidance (or button sounds).

[0210] When only music audio is output, the resonant frequency is sharply attenuated, and the gain can be increased for audio frequencies outside the resonant region. For example, the gain for each frequency can be increased at frequencies above the resonant frequency region (over 200 Hz), and the resonant frequency can be attenuated by more than 5 dB.

[0211] Although the preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above, and various modifications may be made by a person skilled in the art to which the invention pertains without departing from the gist of the present invention as claimed in the claims. Furthermore, such modifications should not be understood individually from the technical idea or prospect of the present invention.

Claims

1. Touch panel; A memory storing multiple effect data corresponding to multiple touch inputs; A processor that generates an audio signal by calling effect data matching a touch input through the touch panel from the memory; A speaker amplifier that amplifies and outputs a signal based on the above audio signal; and An audio-haptic output device including an actuator capable of audio and haptic output based on an output signal of the speaker amplifier.

2. In paragraph 1, The above processor or the speaker amplifier is an audio haptic output device that converts and outputs the frequency characteristics of the audio signal when the effect data matching the touch input through the touch panel includes audio data.

3. In paragraph 2, The above processor or the above speaker amplifier, When the above actuator outputs both audio and haptics, and when the above actuator outputs only audio, An audio haptic output device that converts the frequency characteristics of the audio signal according to different frequency conversion logic.

4. In paragraph 3, The above processor or the above speaker amplifier, When the above actuator outputs both audio of voice guidance or button sound and haptics, By increasing the gain only for the audio area frequency band preset in the above voice guidance or button sound, An audio haptic output device that converts the frequency characteristics of the above audio signal.

5. In paragraph 4, The above processor or the above speaker amplifier, When the above actuator outputs both audio of voice guidance or button sound and haptics, The gain of the audio frequency band preset for the above voice guidance or button sound is increased to be greater than the gain of the haptic frequency band. An audio haptic output device that converts the frequency characteristics of the above audio signal.

6. In paragraph 3, The above processor or the above speaker amplifier, When the above actuator outputs audio of voice guidance or button sound, and haptics, The first gain is applied to the first frequency among the preset audio area frequency bands for the above voice guidance or button sound, By applying a second gain greater than the first gain at a second frequency farther from the resonant frequency of the actuator than the first frequency, An audio haptic output device that converts the frequency characteristics of the above audio signal.

7. In paragraph 3, The above processor or the above speaker amplifier, When the above actuator outputs only audio of voice guidance or button sounds, By amplifying the gain only for the audio area frequency band preset in the above voice guidance or button sound, An audio haptic output device that converts the frequency characteristics of the above audio signal.

8. In paragraph 3, The above processor or the above speaker amplifier, When the above actuator outputs only music type audio, At the resonant frequency of the above actuator, a frequency band of a predetermined range is attenuated, and at least some frequency bands of other regions are amplified. An audio haptic output device that converts the frequency characteristics of the above audio signal.

9. In paragraph 2, The above processor or the above speaker amplifier, When the above actuator only outputs haptics, By not converting the frequency characteristics of the above audio signal or increasing the frequency gain in a region outside a predetermined frequency band from the resonant frequency of the actuator, An audio haptic output device that converts the frequency characteristics of the above audio signal.

10. In paragraph 1, The above processor or the above speaker amplifier, When the actuator outputs only haptics and when the actuator outputs both haptics and audio, the frequency is converted according to different frequency conversion logic, The above speaker amplifier is an audio haptic output device that amplifies a frequency-converted signal.

11. In paragraph 1, Multiple effect data stored in the above memory are: An audio-haptic output device comprising haptic data, audio data, and combined data wherein haptic data and audio data are combined in advance.

12. In paragraph 1, An audio haptic output device in which the haptic data, the audio data, and the combined data are encoded and stored as audio type data.

13. In paragraph 12, The above combined data is, The above haptic data and the above audio data are sequentially combined and generated in that order. An audio haptic output device generated by mixing the above haptic data and the above audio data so that they overlap.

14. In paragraph 1, The above processor or the above speaker amplifier, The actuator vibration is greater than that when a touch input is made to the first area of ​​the above touch panel. When a touch input is made to a second area located further away from the first area based on the actuator, the vibration of the actuator is greater. An audio haptic output device that converts and outputs the frequency characteristics of the above audio signal.

15. Touch panel; A memory storing multiple effect data corresponding to multiple touch inputs; and An actuator capable of outputting at least one of audio and haptic based on effect data matching a touch input through the touch panel among data stored in the memory; Multiple effect data stored in the above memory are: An audio-haptic output device comprising haptic data, audio data, and combined data wherein haptic data and audio data are combined in advance.

16. In paragraph 15, The above combined data is, The above haptic data and the above audio data are sequentially combined and generated in that order. An audio haptic output device generated by mixing the above haptic data and the above audio data so that they overlap.

17. In paragraph 15, The above processor or the above speaker amplifier, The actuator vibration is greater when a touch input is made to a second area located further away from the first area with respect to the actuator than when a touch input is made to a first area of ​​the touch panel. An audio haptic output device that converts and outputs the frequency characteristics of the above audio signal.

18. In paragraph 15, A processor that generates an audio signal by calling effect data matching a touch input through the touch panel from the memory; and A speaker amplifier that amplifies and outputs a signal based on the above audio signal; Further comprising an actuator that outputs at least one of audio and haptic based on an output signal of the speaker amplifier; The above processor or the speaker amplifier is an audio haptic output device that converts and outputs the frequency characteristics of the audio signal when the effect data matching the touch input through the touch panel includes audio data.

19. In paragraph 18, The above processor, An audio haptic output device that converts the above-mentioned effect data into an analog audio signal or an audio signal according to an audio communication standard.

20. An audio haptic output device; comprising: The above audio haptic output device, touch panel; A memory storing audio data and haptic data corresponding to multiple touch inputs; A processor that calls at least one of audio data and haptic data matching a touch input through the touch panel from the memory and converts it into an audio signal; A speaker amplifier that amplifies and outputs a signal based on the above audio signal; and A water dispensing device comprising an actuator capable of outputting at least one of audio and haptic according to an output signal of the speaker amplifier.

Images