Haptic devices and methods of operation thereof

Abstract

A method of operating a haptic device according to one embodiment may include an operation of obtaining audio information generated from a video. The method may include an operation of identifying utterances included in the audio information based on the audio information. The method may include an operation of generating attribute information for each speaker who performed the utterances. The method may include an operation of controlling the haptic device based on the attribute information. The attribute information may include the gender, age group, and timbre of each speaker.

Description

Technology Field

[0001] The following disclosure relates to a haptic device and a method of operating the same. Background Technology

[0003] Vibration motor-based haptic technology is widely used as a means to convey surrounding information to people with hearing impairments. For example, research is being conducted on technologies that generate alerts about surrounding dangers or technologies that help them experience music.

[0004] Furthermore, wearable haptic devices are presenting the potential to efficiently convey high-level information based on various cognitive experiments.

[0005] The background technology described above is possessed or acquired by the inventor in the process of deriving the content of the disclosure of the present application, and cannot necessarily be considered as prior art disclosed to the general public prior to the filing of this application. The problem to be solved

[0007] One embodiment may provide a technique for identifying utterances included in voice information based on voice information obtained from a video.

[0008] One embodiment may provide a technique for generating attribute information for each speaker who has performed utterances.

[0009] One embodiment may provide a technology for controlling a haptic device based on characteristic information.

[0010] However, technical challenges are not limited to the technical challenges described above, and other technical challenges may exist. means of solving the problem

[0012] A method of operating a haptic device according to one embodiment may include an operation of obtaining audio information generated from a video. The method may include an operation of identifying utterances included in the audio information based on the audio information. The method may include an operation of generating attribute information for each speaker who performed the utterances. The method may include an operation of controlling the haptic device based on the attribute information. The attribute information may include the gender, age group, and timbre of each speaker.

[0013] The generating operation may include determining whether each of the utterers in the image has an utterance history. The generating operation may include generating first characteristic information for each of the first utterers among the utterers who do not have an utterance history. The generating operation may include obtaining stored second characteristic information for each of the second utterers among the utterers who have an utterance history.

[0014] The above speech history may be updated based on each of the first speakers.

[0015] The above-mentioned controlling operation may include an operation of generating a vibration pattern corresponding to the above-mentioned characteristic information.

[0016] The above-mentioned controlling operation may include controlling at least one motor included in the haptic device based on the vibration pattern.

[0017] The above identifying operation may include an operation of identifying the utterances included in the voice information through an artificial neural network.

[0018] A haptic device according to one embodiment may include a memory containing instructions and a processor electrically connected to the memory and for executing the instructions. When the instructions are executed by the at least one processor, the device may obtain audio information generated from a video, identify utterances included in the audio information based on the audio information, generate attribute information for each speaker who performed the utterances, and control the haptic device based on the attribute information. The attribute information may include the gender, age group, and timbre of each speaker.

[0019] When the above instructions are executed by the at least one processor, the device may determine whether each of the utterers in the image has an utterance history, generate first characteristic information for each of the first utterers among the utterers who do not have an utterance history, and obtain stored second characteristic information for each of the second utterers among the utterers who have an utterance history.

[0020] The above speech history may be updated based on each of the first speakers.

[0021] When the above instructions are executed by the at least one processor, the device may generate a vibration pattern corresponding to the characteristic information.

[0022] When the above instructions are executed by the at least one processor, the device may control at least one motor included in the haptic device based on the vibration pattern.

[0023] When the above instructions are executed by the at least one processor, the device may identify the utterances included in the voice information through an artificial neural network. Brief explanation of the drawing

[0025] FIG. 1 is a schematic diagram illustrating a haptic device according to one embodiment. FIG. 2 is a diagram illustrating the operation of a haptic device according to one embodiment. Figure 3 is a schematic block diagram of the haptic device illustrated in Figure 1. Figure 4 is a schematic block diagram of each module shown in Figure 3. Figure 5 is a flowchart illustrating a method for controlling a haptic device based on characteristic information obtained based on speech history. FIG. 6 is a flowchart illustrating a method for controlling a haptic device according to one embodiment. FIG. 7 is a schematic block diagram of a haptic device according to one embodiment. Specific details for implementing the invention

[0026] Specific structural or functional descriptions of the embodiments are disclosed for illustrative purposes only and may be modified and implemented in various forms. Accordingly, actual implementations are not limited to the specific embodiments disclosed, and the scope of this specification includes modifications, equivalents, or substitutions included in the technical concept described by the embodiments.

[0027] Terms such as "first" or "second" may be used to describe various components, but these terms should be interpreted solely for the purpose of distinguishing one component from another. For example, the first component may be named the second component, and similarly, the second component may be named the first component.

[0028] When it is stated that a component is "connected" to another component, it should be understood that it may be directly connected to or joined to that other component, or that there may be other components in between.

[0029] Singular expressions include plural expressions unless the context clearly indicates otherwise. In this document, phrases such as “A or B,” “at least one of A and B,” “at least one of A or B,” “A, B or C,” “at least one of A, B and C,” and “at least one of A, B, or C” may each include any one of the items listed together with the corresponding phrase, or all possible combinations thereof. In this specification, terms such as “comprising” or “having” are intended to designate the existence of the described feature, number, step, action, component, part, or combination thereof, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0030] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by those skilled in the art. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an ideal or overly formal sense unless explicitly defined in this specification.

[0031] As used herein, the term "module" may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be a component formed integrally, or a minimum unit of said component or a part thereof that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0032] As used in this document, the term "part" refers to a software or hardware component, such as an FPGA or ASIC, that performs certain roles. However, "part" is not limited to software or hardware. "Part" may be configured to reside in an addressable storage medium or configured to operate one or more processors. For example, "part" may include components such as software components, object-oriented software components, class components, and task components, as well as processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functions provided within the components and "parts" may be combined into a smaller number of components and "parts" or further separated into additional components and "parts." Furthermore, components and "parts" may be implemented to operate one or more CPUs within a device or secure multimedia card. Additionally, '~part' may include one or more processors.

[0033] Hereinafter, embodiments will be described in detail with reference to the attached drawings. In the description with reference to the attached drawings, identical components are given the same reference numeral regardless of the drawing number, and redundant descriptions thereof will be omitted.

[0035] FIG. 1 is a schematic diagram illustrating a haptic device according to one embodiment.

[0036] Referring to FIG. 1, according to one embodiment, a haptic feedback system (10) may include a haptic device (11). The haptic feedback system (10) may provide haptic feedback to a user (13) by analyzing audio information obtained from an image (19). For example, the haptic feedback system (10) may enable the haptic device (11) to provide haptic feedback by analyzing audio information obtained from an image (19).

[0037] The haptic device (11) can be designed to be wearable by the user (13). For example, the haptic device (11) can be implemented as a wristband, a wearable device, a headset, clothing, and / or other device that can be attached to a body part, but is not limited thereto. The haptic device (11) can provide the user (13) with various vibration patterns generated according to the present invention while the user (13) is wearing the haptic device (11), and the generated vibration patterns can be used for the purpose of responding to a speaker or notifying the user (13) of a certain event.

[0038] The haptic device (11) can acquire audio information generated from the image (19). The audio information may include utterances of speakers included in the image (19) and / or background music.

[0039] The haptic device (11) can identify utterances included in voice information based on voice information. For example, the haptic device (11) can extract only the utterances included in voice information from voice information.

[0040] The haptic device (11) can determine each speaker who has performed utterances. The haptic device (11) can generate attribute information for each speaker who has performed utterances. The haptic device (11) can store attribute information for each speaker who has performed utterances in a memory (not shown) included in the haptic device (11). The attribute information may include the gender, age group, and / or timbre of each speaker. The haptic device (11) can generate a vibration pattern based on the attribute information. For example, the haptic device (11) can generate a vibration pattern corresponding to the attribute information. The haptic device (11) can vibrate according to a vibration pattern and can transmit a vibration pattern to the user (13). The haptic device (11) can provide a vibration pattern to the user (13) when it is difficult to infer the speaker. For example, the haptic device (11) can provide a vibration pattern corresponding to the narration to the user (13) in a situation where only narration is output without a character in the video (19). As another example, the haptic device (11) can provide a vibration pattern corresponding to the voice beyond the handset of character in a call to the user (13). As yet another example, the haptic device (11) can provide a vibration pattern corresponding to the speaker in a situation where the mouth shape of the speaker in the video (19) can be confirmed.The haptic device (11) may provide a vibration pattern to the user (13) to assist subtitles displayed on the video (19). For example, the haptic device (11) may provide a vibration pattern to the user (13) to prevent a decrease in immersion in the video (19) due to rapid subtitle switching or long subtitles. The haptic device (11) may provide a vibration pattern to the user (13) to convey non-verbal information that can be obtained from the voice information of the video (19).

[0041] The haptic device (11) can control at least one motor based on a vibration pattern corresponding to the speaker who performed the speech at the same time as the speech occurs. The haptic device (11) can control at least one motor to resolve discrepancies between subtitles and video (19). The haptic device (11) can inform the user (13) who the speaker is among the characters appearing in the video (19) through the vibration pattern.

[0042] The haptic device (11) can generate vibration patterns using only voice information obtained from the image (19) without processing subtitles displayed on the image (19). Since the haptic device (11) processes voice information without processing subtitles displayed on the image (19), the user (13) can utilize it universally. The haptic device (11) can improve accessibility for the user (13) who is hearing impaired.

[0044] FIG. 2 is a diagram illustrating the operation of a haptic device according to one embodiment.

[0045] Referring to FIG. 2, according to one embodiment, the haptic device (11) can be designed to be wearable by a user (13) and is assumed to be implemented as a wristband, and the invention will be described.

[0046] The electronic device (20) can output an image (19). For example, the electronic device (20) may be implemented as at least one of a smartphone, a tablet personal computer, a mobile phone, a desktop personal computer, a laptop personal computer, a netbook computer, a workstation, a server, a PDA (personal digital assistant), a PMP (portable multimedia player), an MP3 player, a mobile medical device, a camera, or a wearable device. The haptic device (11) can obtain voice information from the image (19) output by the electronic device (20). For example, the haptic device (11) can directly obtain voice information output from a speaker built into or connected to the electronic device (20). As another example, the haptic device (11) can obtain voice information from the image (19) through a network connection with the electronic device (20). Network connections may include, for example, a local area network (LAN), a wide area network (WAN), the Internet, a telephone network, a wired or wireless data network, etc. Wireless data networks include, for example, 3G (generation), 4G, 5G, 3PP (3rd generation partnership project), Wi-Fi, Bluetooth communication, infrared communication, ultrasonic communication, visible light communication (VLC), Li-Fi, etc., but are not limited thereto. The operation of the haptic device (11) processing voice information will be explained in detail in FIG. 3.

[0048] Figure 3 is a schematic block diagram of the haptic device illustrated in Figure 1.

[0049] Referring to FIG. 3, according to one embodiment, a haptic device (11) may include a speech identification module (31), a speech analysis module (33), and / or a haptic signal generation module (35).

[0050] The utterance identification module (31) can acquire audio information generated from an image (e.g., image (19) of FIG. 1). The audio information may include utterances of speakers included in the image (19) and / or background music. The utterance identification module (31) can process the audio information into a form that can identify the utterances. Based on the audio information, the utterance identification module (31) can identify the utterances included in the audio information. For example, the utterance identification module (31) can extract only the utterances included in the audio information from the audio information. The utterance identification module (31) can transmit (or send) the utterances to the utterance analysis module (33).

[0051] The utterance analysis module (33) can acquire (or receive) utterances from the utterance identification module (31). The utterance analysis module (33) can determine each speaker who performed the utterances. For example, the utterance analysis module (33) can assign an ID value of the speaker who performed the utterances to each of the utterances. The utterance analysis module (33) can determine each speaker who performed the utterances based on the ID value of the speaker assigned to each of the utterances. The utterance analysis module (33) can determine whether the speaker who performed the utterances transmitted in succession has changed.

[0052] The utterance analysis module (33) can determine whether each of the speakers in the video (19) has an utterance history. The utterance history may be stored in a memory (not shown) included in the haptic device (11).

[0053] The utterance analysis module (33) can identify first speakers among the speakers who have performed utterances that have no utterance history. The utterance analysis module (33) can generate first characteristic information for each of the first speakers. For example, the utterance analysis module (33) can generate first characteristic information for each of the first speakers if some of the speakers who have performed utterances are identified as first speakers. The first characteristic information may include the gender, age group, and timbre of each of the first speakers. The utterance analysis module (33) can extract high-level features from the utterances performed by each of the first speakers. The utterance analysis module (33) can utilize ID values ​​to identify some of the speakers who have performed utterances as first speakers. The speech analysis module (33) can store first characteristic information of each of the first speech speakers and transmit (or send) it to the haptic signal generation module (35).

[0054] The speech analysis module (33) can acquire (or generate) the stored second characteristic information for each of the second speakers who have a speech history among the speakers who performed speeches. The second characteristic information may be that which was generated by the speech analysis module (33). The speech analysis module (33) may transmit (or send) the second characteristic information to the haptic signal generation module (35). The first speakers and the second speakers may be included among the speakers who performed speeches. The first characteristic information and the second characteristic information may be included in the characteristic information.

[0055] The haptic signal generation module (35) can acquire (or receive) characteristic information from the speech analysis module (33). For example, the haptic signal generation module (35) can acquire first characteristic information and / or second characteristic information. The haptic signal generation module (35) can control the haptic device based on the characteristic information. For example, the haptic signal generation module (35) can generate a vibration pattern corresponding to the characteristic information. The haptic signal generation module (35) can generate different vibration patterns based on the characteristic information generated in response to differences in speech speakers.

[0056] The haptic signal generation module (35) can provide a vibration pattern corresponding to each speaker to a user (e.g., user (13) of FIG. 1). Based on the vibration pattern, the haptic signal generation module (35) can generate a haptic signal for controlling at least one motor included in the haptic device (11). The vibration pattern corresponding to each speaker can be generated by a combination of multiple motors or by a difference in the vibration pattern of a single motor. For example, the haptic signal generation module (35) can generate a haptic signal that produces a vibration with a lower frequency and a longer duration when the speaker is male compared to when the speaker is female. As another example, the haptic signal generation module (35) can generate a haptic signal that produces a vibration with a higher frequency and a shorter duration when the speaker is female compared to when the speaker is male. As another example, the haptic signal generation module (35) can generate a haptic signal that produces a vibration providing a pulsating sensation when the speaker is female, compared to when the speaker is male.

[0057] The haptic signal generation module (35) can adjust the haptic signal based on the tone of the speaker included in the characteristic information. The haptic signal generation module (35) can adjust the frequency included in the haptic signal based on the tone of the speaker included in the characteristic information. The haptic signal generation module (35) can adjust the haptic signal based on the intensity (or volume) of the speaker's speech. The haptic signal generation module (35) can generate the size of the haptic signal in proportion to the intensity of the speaker's speech. The haptic signal generation module (35) can generate the size of the haptic signal in proportion to the intensity of the speaker's speech so that the user (13) can intuitively sense the intonation or intensity of the speech.

[0059] Figure 4 is a schematic block diagram of each module shown in Figure 3.

[0060] Referring to FIG. 4, according to one embodiment, the modules included in the haptic device (11) described in FIG. 3 (e.g., speech identification module (31), speech analysis module (33), haptic signal generation module (35)) may be implemented in detailed structures. The speech identification module (31) may include a voice information collection unit (311) and / or a speech identification unit (313). The voice information collection unit (311) may acquire (or collect) voice information from an image (e.g., the image (19) of FIG. 1). The voice information collection unit (311) may transmit (or send) the acquired voice information to the speech identification unit (313).

[0061] The speech identification unit (313) can acquire (or receive) voice information from the voice information collection unit (311). The operation of the speech identification unit (313) identifying speeches included in the voice information based on the voice information may be substantially the same as the operation of identifying speeches included in the voice information of the haptic device (11) described in FIGS. 1 to 3. The speech identification unit (313) may identify speeches included in the voice information through an artificial neural network. For example, the artificial neural network may be a convolutional neural network (CNN). The speech identification unit (313) can effectively identify speeches included in the voice information in response to the utilization of the haptic device (11).

[0062] The speech analysis module (33) can acquire (or receive) speeches included in voice information from the speech identification unit (313). The speech analysis module (33) can convert the speeches into data types such as Mel-Frequency Cepstral Coefficients (MFCC) to identify the speakers who performed the speeches and to acquire characteristic information. The speech analysis module (33) may include a speaker identification unit (331) and / or a characteristic information generation unit (333). The speaker identification unit (331) can identify the speakers who performed the speeches. The speaker identification unit (331) can identify the speakers who performed the speeches through an artificial neural network. For example, the artificial neural network may be a CNN. The speaker identification unit (331) can input the utterances into an artificial neural network to identify the speakers who performed the utterances.

[0063] The speaker identification unit (331) may assign an ID value to first speakers who have no speech history among the speakers who have performed speeches, and transmit a list of first speakers to the characteristic information generation unit (333). The speaker identification unit (331) may update the speech history based on each of the first speakers. For example, the speaker identification unit (331) may identify a speaker identified as a first speaker as a second speaker in the next speaker identification cycle. The speaker identification unit (331) may transmit second speakers who have a speech history among the speakers who have performed speeches to the haptic signal generation module (35). At this time, the speaker identification unit (331) may transmit the second characteristic information stored for each of the second speakers from memory (not shown) to the haptic signal generation module (35). The haptic device (11) can quickly generate haptic signals by not unnecessarily generating characteristic information of second speakers with a history of speaking.

[0064] The operation of the characteristic information generation unit (333) generating characteristic information for each of the speakers who have performed speeches may be substantially the same as the operation of generating characteristic information for the haptic device (11) described in FIGS. 1 to 3. The characteristic information generation unit (333) may generate first characteristic information for each of the first speakers who have no speech history among the speakers. The characteristic information generation unit (333) may transmit (or send) the generated first characteristic information to the haptic signal generation module (35).

[0065] The haptic signal generation module (35) may include a haptic signal generation unit (351) and / or a haptic feedback output unit (353). The operation of the haptic signal generation unit (351) generating a haptic signal may be substantially the same as the operation of the haptic device (11) in FIGS. 1 to 3 generating a haptic signal based on characteristic information. The haptic feedback output unit (353) may adjust the haptic signal generated based on characteristic information. The haptic feedback output unit (353) may adjust the haptic signal through an artificial neural network. For example, the haptic feedback output unit (353) may adjust the haptic signal by inputting characteristic information and the haptic signal generated based on characteristic information into an artificial neural network. The haptic feedback output unit (353) may use an artificial neural network to generate perceptual differences between haptic signals for the user (13). For example, the haptic feedback output unit (353) may generate perceptual differences between haptic signals based on characteristic information corresponding to each speaker, based on differences in tone, even if the speaker who performed the speech is male and of similar age. The haptic feedback output unit (353) may provide haptic signals having perceptual differences to the user (13). The haptic feedback output unit (353) may control at least one motor included in the haptic device (11) based on the haptic signals. The haptic feedback output unit (353) may transmit a haptic signal or a controlled haptic signal to at least one motor and control at least one motor.

[0067] Figure 5 is a flowchart illustrating a method for controlling a haptic device based on characteristic information obtained based on speech history.

[0068] Referring to FIG. 5, according to one embodiment, operations 510 to 580 may be operations performed by a haptic device (e.g., haptic device (11) of FIG. 1) described with reference to FIG. 1 to 7.

[0069] According to one embodiment, operations 510 to 570 may be understood to be performed in a processor (e.g., processor (730) of FIG. 7) of a haptic device (11) (e.g., electronic device (710) of FIG. 7) described with reference to FIG. 1.

[0070] In operation 510, the haptic device (11) can acquire audio information generated from an image (e.g., image (19) of FIG. 1). The audio information may include utterances of speakers included in the image (19) and / or background music.

[0071] In operation 520, the haptic device (11) can determine whether the voice information contains utterances. If the voice information does not contain utterances, the haptic device (11) can acquire the voice information again or wait until the next cycle of acquiring the voice information.

[0072] In operation 530, the haptic device (11) can identify utterances included in the voice information based on the voice information.

[0073] In operation 540, the haptic device (11) can determine whether each of the utterers who performed the utterances has an utterance history. The utterance history may be stored in a memory (not shown) included in the haptic device (11). The haptic device (11) can identify first utterers who have no utterance history among the utterers who performed the utterances. The haptic device (11) can identify second utterers who have an utterance history among the utterers who performed the utterances. The haptic device (11) can identify the utterers who performed the utterances through an artificial neural network. For example, the artificial neural network may be a CNN.

[0074] In operation 550, the haptic device (11) may acquire (or generate) stored second characteristic information for each of the second speakers. The second characteristic information may be generated at a prior time point prior to the haptic device (11) identifying the speakers who performed the utterances. The second characteristic information may include the gender, age group, and / or timbre of each of the second speakers.

[0075] In operation 560, the haptic device (11) can generate first characteristic information for each of the first speakers. The first characteristic information may include the gender, age group, and / or timbre of each of the first speakers.

[0076] In operation 570, the haptic device (11) can update the speech history based on each of the first speakers.

[0077] In operation 580, the haptic device (11) can generate a vibration pattern based on first characteristic information and second characteristic information. The haptic device (11) can control at least one motor included in the haptic device (11) based on the vibration pattern. The haptic device (11) can cause at least one motor to vibrate based on the vibration pattern.

[0078] Operations 510 through 580 may be performed sequentially, but are not limited thereto. For example, two or more operations may be performed in parallel.

[0080] FIG. 6 is a flowchart illustrating a method for controlling a haptic device according to one embodiment.

[0081] Referring to FIG. 6, according to one embodiment, operations 610 to 670 may be operations performed by a haptic device (e.g., haptic device (11) of FIG. 1) described with reference to FIG. 1 to 7.

[0082] According to one embodiment, operations 610 to 6170 can be understood as being performed in a processor (e.g., processor (730) of FIG. 7) of a haptic device (11) (e.g., electronic device (710) of FIG. 7) described with reference to FIG. 1.

[0083] In operation 610, the haptic device (11) can acquire audio information generated from an image (e.g., image (19) of FIG. 1). The haptic device (11) can acquire audio information of the image (19) using a microphone implemented in the haptic device (11) or a microphone implemented outside the haptic device (11) and electrically connected.

[0084] In operation 630, the haptic device (11) can identify utterances included in the voice information based on the voice information.

[0085] In operation 650, the haptic device (11) can generate attribute information for each of the utterers who performed the utterances. The haptic device (11) can acquire attribute information based on whether the utterers who performed the utterances have an utterance history.

[0086] In operation 670, the haptic device (11) may control or vibrate at least one motor included in the haptic device (11) based on generated characteristic information. The haptic device (11) may vibrate based on characteristic information.

[0088] FIG. 7 is a schematic block diagram of a haptic device according to one embodiment.

[0089] Referring to FIG. 7, according to one embodiment, an electronic device (710) (e.g., the haptic device (11) of FIG. 1) may include a memory (750) and a processor (730). The electronic device (710) may be substantially the same as the haptic device (11) described with reference to FIG. 1 through 6.

[0090] The memory (750) can store instructions (or programs) executable by the processor (730). For example, the instructions may include instructions for executing the operation of the processor (730) and / or the operation of each component of the processor (730).

[0091] The memory (750) may include one or more computer-readable storage media. The memory (750) may include non-volatile storage devices (e.g., magnetic hard disc, optical disc, floppy disc, flash memory, EPROM (electrically programmable memories), EEPROM (electrically erasable and programmable)).

[0092] The memory (750) may be a non-transitory medium. The term "non-transitory" may indicate that the storage medium is not implemented by a carrier wave or a propagated signal. However, the term "non-transitory" should not be interpreted as meaning that the memory (750) is immobile.

[0093] The processor (730) can process data stored in memory (750). The processor (730) can execute computer-readable code (e.g., software) stored in memory (750) and instructions triggered by the processor (730).

[0094] The processor (730) may be a data processing device implemented in hardware having a circuit having a physical structure for executing desired operations. For example, the desired operations may include code or instructions included in a program.

[0095] For example, a data processing device implemented in hardware may include a microprocessor, a central processing unit, a processor core, a multi-core processor, a multiprocessor, an Application-Specific Integrated Circuit (ASIC), and a Field Programmable Gate Array (FPGA).

[0096] The processor (730) can cause the electronic device (710) to perform one or more operations by executing code and / or instructions stored in memory (750). The operations performed by the electronic device (710) may be substantially the same as the operations performed by the haptic device (11) described with reference to FIGS. 1 through 7. Such redundant descriptions are omitted.

[0098] The embodiments described above may be implemented as hardware components, software components, and / or combinations of hardware and software components. For example, the devices, methods, and components described in the embodiments may be implemented using a general-purpose computer or a special-purpose computer, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing unit may execute an operating system (OS) and software applications executed on said operating system. Additionally, the processing unit may access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing unit may be described as being used as a single unit, but those skilled in the art will understand that the processing unit may include multiple processing elements and / or multiple types of processing elements. For example, the processing unit may include multiple processors or one processor and one controller. In addition, other processing configurations, such as parallel processors, are also possible.

[0099] Software may include computer programs, code, instructions, or a combination of one or more of these, and may configure a processing unit to operate as desired or command the processing unit independently or collectively. Software and / or data may be permanently or temporarily embodied in any type of machine, component, physical device, virtual equipment, computer storage medium or device, or transmitted signal wave in order to be interpreted by the processing unit or to provide instructions or data to the processing unit. Software may be distributed over networked computer systems and may be stored or executed in a distributed manner. Software and data may be stored on computer-readable recording media.

[0100] The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may store program instructions, data files, data structures, etc., either individually or in combination, and the program instructions recorded on the medium may be those specifically designed and configured for the embodiment or those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc.

[0101] The hardware device described above may be configured to operate as one or more software modules to perform the operation of the embodiment, and vice versa.

[0102] Although the embodiments have been described above with reference to the limited drawings, those skilled in the art can apply various technical modifications and variations based thereon. For example, suitable results may be achieved even if the described techniques are performed in a different order than described, and / or if the components of the described system, structure, device, circuit, etc. are combined or assembled in a form different from described, or replaced or substituted by other components or equivalents.

[0103] Therefore, other implementations, other embodiments, and equivalents to the claims also fall within the scope of the claims set forth below.

Claims

Claim 1 A method for operating a haptic device, comprising: obtaining audio information generated from a video; identifying utterances included in the audio information based on the audio information; generating attribute information for each speaker who performed the utterances; and controlling the haptic device based on the attribute information, wherein the attribute information includes the gender, age group, and timbre of each speaker. Claim 2 In claim 1, the generating operation comprises: determining whether each of the utterers in the image has an utterance history; generating first characteristic information for each of the first utterers among the utterers who do not have an utterance history; and acquiring stored second characteristic information for each of the second utterers among the utterers who have an utterance history. Claim 3 A method in which, in paragraph 2, the utterance history is updated based on each of the first utterers. Claim 4 A method according to claim 1, wherein the controlling operation includes an operation of generating a vibration pattern corresponding to the characteristic information. Claim 5 In claim 4, the controlling operation comprises controlling at least one motor included in the haptic device based on the vibration pattern. Claim 6 A method according to claim 1, wherein the identifying operation comprises identifying the utterances included in the voice information through an artificial neural network. Claim 7 A haptic device comprises: a memory containing instructions; and a processor electrically connected to the memory and for executing the instructions, wherein when the instructions are executed by the at least one processor, the device obtains audio information generated from a video, identifies utterances included in the audio information based on the audio information, generates attribute information of each speaker who performed the utterances, and controls the haptic device based on the attribute information, wherein the attribute information includes the gender, age group, and timbre of each speaker. Claim 8 A haptic device according to claim 7, wherein when the instructions are executed by the at least one processor, the device determines whether each of the utterers in the image has an utterance history, generates first characteristic information for each of the first utterers among the utterers who do not have an utterance history, and obtains stored second characteristic information for each of the second utterers among the utterers who have an utterance history. Claim 9 A haptic device according to claim 8, wherein the speech history is updated based on each of the first speakers. Claim 10 A haptic device according to claim 7, wherein when the instructions are executed by the at least one processor, the device generates a vibration pattern corresponding to the characteristic information. Claim 11 A haptic device according to claim 10, wherein, when the instructions are executed by the at least one processor, the device causes the device to control at least one motor included in the haptic device based on the vibration pattern. Claim 12 An electronic device according to claim 7, wherein, when the instructions are executed by the at least one processor, the device enables the device to identify the utterances included in the voice information through an artificial neural network. Claim 13 A computer program stored on a computer-readable recording medium in combination with hardware to execute the method of any one of claims 1 to 6.

Images