Audio output apparatus and method therefor

Abstract

An audio output apparatus is disclosed. The present apparatus comprises: a communication unit; a memory; an audio output unit; and a processor. When information regarding an external apparatus is received through the communication unit, the processor: determines a role of each apparatus on the basis of information stored in the memory and the received information; divides one piece of audio content into a plurality of audio elements; transmits an audio element corresponding to a role of the external apparatus to one external apparatus; and controls the audio output unit to output the audio element according to an output of the external apparatus. Accordingly, harmonious audio enjoyment becomes possible.

Description

Audio output device and method thereof

[0001] The present invention relates to an audio output device and a method thereof.

[0002] Thanks to advancements in electronic technology, various types of electronic products are becoming widespread and in use. Among these, electronic devices equipped with output devices, such as displays and speakers, that provide users with various content are steadily gaining popularity.

[0003] A prime example of this is the TV. While TVs display video frames, they can also output audio signals synchronized to those frames using their built-in speakers. Recent TVs often feature extremely thin bezels and side panels surrounding the display to enhance their aesthetics. In these cases, even if audio signals are output through the TV's built-in speakers, it can be difficult for users to hear at the desired level. To compensate for this, users often connect soundbars or external speakers to their TVs.

[0004] In this way, although many electronic products capable of audio output, i.e. audio output devices, are being used within the same environment, most users only use them to the extent of setting the audio output of some devices to output from a sound bar or external speakers instead.

[0005] Accordingly, the need for a method to effectively use multiple audio output devices has arisen in an environment where these devices are provided.

[0006] The present disclosure is in accordance with the above-described necessity, and according to an embodiment of the present disclosure, an audio output device includes a communication unit for performing communication with at least one external device, a memory storing information about the audio output device, an audio output unit, and a processor, wherein the processor, when information about the at least one external device is received through the communication unit, determines a role of each of the audio output device and the at least one external device based on the information stored in the memory and the received information, divides one audio content into a plurality of audio elements, transmits an audio element corresponding to the role of the at least one external device among the plurality of audio elements to the at least one external device through the communication unit, and controls the audio output unit to output an audio element corresponding to the role of the audio output device among the plurality of audio elements according to the output of the at least one external device, and when the state of one of the audio output unit and the at least one external device changes, at least one of the roles of each of the audio output device and the at least one external device and an audio element to be played can be adjusted.

[0007] Meanwhile, an output method according to an embodiment of the present disclosure includes the steps of receiving information of at least one external device, determining a role of each of the audio output device and the at least one external device based on the information of the audio output device and the received information, dividing one audio content into a plurality of audio elements, transmitting an audio element assigned to the at least one external device among the plurality of audio elements to the at least one external device, outputting an audio element assigned to the audio output device among the plurality of audio elements in accordance with an output of the at least one external device, and adjusting at least one of a role of each of the audio output device and the at least one external device and an audio element to be played when a state of one of the audio output device and the at least one external device is changed.

[0008] FIG. 1 is a drawing for explaining the operation of audio output devices according to at least one embodiment of the present disclosure;

[0009] FIG. 2 is a block diagram showing the configuration of an audio output device according to at least one embodiment of the present disclosure;

[0010] Figure 3 is a drawing for explaining a method for correcting audio output characteristics of an audio output device.

[0011] FIG. 4 is a block diagram showing the configuration of an audio output device according to various embodiments of the present disclosure;

[0012] Figure 5 is a drawing showing an example of a UI screen for setting the audio simultaneous playback function;

[0013] Figure 6 is a drawing showing an example of a notification message notifying that simultaneous audio playback is not possible.

[0014] Figures 7 and 8 are drawings for explaining a method of sensing a change in position of an external device.

[0015] Figure 9 is a drawing for explaining the operation of the audio output device according to the location of external devices.

[0016] FIG. 10 and FIG. 11 are drawings for explaining the operation of an audio output device according to various embodiments of the present disclosure;

[0017] FIG. 12 is a flowchart for explaining an output method according to at least one embodiment of the present disclosure;

[0018] Figure 13 is a timing diagram for explaining an example of a linking process of multiple audio output devices, and

[0019] FIG. 14 is a diagram for explaining the operation of a server device according to at least one embodiment of the present disclosure.

[0020] The terms used in the various embodiments of this disclosure have been selected from widely used, current terms, taking into account the functions of this disclosure. However, these terms may vary depending on the intentions of those skilled in the art, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, terms may be arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the description of the relevant disclosure. Therefore, the terms used in this disclosure should not be defined simply as names of terms, but rather based on the meanings of the terms and the overall content of this disclosure.

[0021] In this specification, expressions such as “has,” “can have,” “includes,” or “may include” indicate the presence of a feature (e.g., a number, function, operation, or component such as a part), and do not exclude the presence of additional features.

[0022] The expression "at least one of A and / or B" should be understood to mean either "A" or "B" or "A and B".

[0023] As used herein, the expressions “first,” “second,” “first,” or “second,” etc., may describe various components, regardless of order and / or importance, and are only used to distinguish one component from another, but do not limit the components.

[0024] When it is said that a component (e.g., a first component) is “(operatively or communicatively) coupled with / to” or “connected to” another component (e.g., a second component), it should be understood that the component may be directly coupled to the other component, or may be connected through another component (e.g., a third component).

[0025] Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, terms such as "comprise" or "consist of" are intended to indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but should be understood not to preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0026] In the present disclosure, a "module" or "part" performs at least one function or operation and may be implemented as hardware or software, or as a combination of hardware and software. Furthermore, multiple "modules" or multiple "parts" may be integrated into at least one module and implemented as at least one processor (not shown), excluding any "modules" or "parts" that need to be implemented as specific hardware.

[0027] In this specification, the term user may refer to a person using an electronic device or a device using an electronic device (e.g., an artificial intelligence electronic device).

[0028] An embodiment of the present disclosure will be described in more detail with reference to the attached drawings below.

[0029] FIG. 1 is a diagram illustrating the operation of audio output devices according to at least one embodiment of the present disclosure. According to FIG. 1, a plurality of audio output devices (100, 200-1, 200-2) can communicate with each other. In the present disclosure, an audio output device refers to a device capable of audio output. In the present disclosure, an audio output device may have a built-in speaker, but is not necessarily limited thereto, and a device that is connected to an external speaker and outputs audio using the external speaker may also be included.

[0030] In FIG. 1, the first audio output device (100) is illustrated as a TV, and the second and third audio output devices (200-1, 200-2) are illustrated as a sound bar and a wireless speaker, respectively. However, this is not limited thereto, and each audio output device (100, 200-1, 200-2) may be implemented as various types of electronic devices. Specifically, each audio output device (100, 200-1, 200-2) may be implemented as various types of products, such as a TV, a sound bar, a PC, a laptop PC, a mobile phone, a tablet PC, a refrigerator, a washing machine, an electric rice cooker, a cleaning robot, a speaker, a sound device, a projector, etc.

[0031] Each audio output device (100, 200-1, 200-2) can play and output a single audio content in conjunction with each other according to the user's selection. This function may be referred to by various terms such as an audio synchronous playback function, a symphony function (or Q-symphony function), a surround sound function, etc., but in the present disclosure, it is described as an audio synchronous playback function.

[0032] When the simultaneous audio playback function is selected, at least one of the audio output devices (100, 200-1, 200-2) may play a main role, and the remaining audio output devices may play sub roles. The main role may be a role that outputs at least one main element among a plurality of audio elements constituting one audio content. The sub role may be a role that outputs the remaining audio elements (i.e., sub elements) excluding the main element among a plurality of audio elements. In the present disclosure, the roles are divided into two roles, such as the main role and the sub role, but the number of roles may be divided in a wider variety. In the present disclosure, the main role and the sub role are described, but in addition, they may be expressed in various modified forms, such as the primary role and the secondary role, or the host role and other roles.

[0033] An audio element includes elements that constitute a single audio signal. Alternatively, it may be referred to as a sound source, a sound source object, etc., but in this disclosure, it is referred to as an audio element.

[0034] The plurality of audio output devices (100, 200-1, 200-2) can play either a main role or a sub role depending on the location or audio output characteristics of each device. The role division may be performed by one of the plurality of audio output devices (100, 200-1, 200-2), or the plurality of audio output devices (100, 200-1, 200-2) may each identify its own role. Alternatively, a server device (not shown) separately provided in addition to the plurality of audio output devices (100, 200-1, 200-2) may divide the roles by considering the location or audio output characteristics of each of the plurality of audio output devices (100, 200-1, 200-2).

[0035] An audio output device that plays the main role plays and outputs at least one audio element assigned to the main role among a plurality of audio elements. An audio output device that plays and outputs at least one audio element assigned to the sub role among a plurality of audio elements.

[0036] Audio output devices with primary and secondary roles synchronize with each other to play and output their own audio elements, allowing users to enjoy audio content with a three-dimensional feel. Specifically, since the primary and secondary roles are determined based on the location and audio output characteristics of each device, users can enjoy audio content played back at optimal quality for their respective locations. If the location or status of each device changes, the role assignment and element classification tasks can be re-performed. This allows for adaptive response to various events, maintaining user satisfaction.

[0037] The operation of dividing audio content into multiple audio elements and then classifying them into main elements and sub-elements may be performed by one of multiple audio output devices (100, 200-1, 200-2), or may be performed by each of the multiple audio output devices (100, 200-1, 200-2).

[0038] When implemented so that one of the plurality of audio output devices (100, 200-1, 200-2) performs the above-described role division and element classification operations, the audio output device that plays the main role may perform the above-described role division and element classification operations, but is not necessarily limited thereto, and even in the device that plays the sub-role among the plurality of audio output devices (100, 200-1, 200-2), if its computational capability or speed is superior to that of the other devices, it may perform at least one of the role division and element classification operations. Alternatively, if an external source is connected to one device, the device may perform the element classification operation to allocate audio elements for each role.

[0039] For convenience of explanation, the plurality of audio output devices (100, 200-1, 200-2) are referred to as first to third audio output devices, respectively.

[0040] For example, if the second audio output device (200-1) is positioned below or in front of the first audio output device (100) as shown in FIG. 1, and the third audio output device (200-2) is positioned on the right side of the first audio output device (100), then the first audio output device (100) or the second audio output device (200-1) located in the front direction based on the position of the user (10) can play the main role. If the positions of the first audio output device (100) and the second audio output device (200-1) are similar as shown in FIG. 1, the device with better audio output characteristics among the two devices can play the main role. As shown in Fig. 1, if the first audio output device (100) is a TV and the second audio output device (200-1) is a sound bar, since the audio output characteristics of the sound bar are generally better, the second audio output device (200-1) can play the main role, and the remaining audio output devices (100, 200-2) can play the sub roles. As a result, the second audio output device (200-1) can play and output audio elements corresponding to the main role (e.g., vocals, center signals, etc.), and the first audio output device (100) and the third audio output device (200-2) can play and output the remaining audio elements in the same manner. Alternatively, if the first audio output device (100) is positioned in the center based on the user's position and viewing direction as shown in FIG. 1, and the third audio output device (200-2) is positioned on the right, the first audio output device (100) may play the sub-element corresponding to the center among the sub-elements, and the third audio output device (200-2) may play the sub-element corresponding to the right. A method for classifying a main element corresponding to a main role and a sub-element corresponding to a sub-role among a plurality of audio elements will be described in detail again in the following section.

[0041] As described above, by having multiple different audio output devices harmoniously output a single audio content, user satisfaction can be greatly improved.

[0042] FIG. 2 is a block diagram showing the configuration of an audio output device according to an embodiment of the present disclosure. FIG. 2 is described based on the case where the first audio output device (100) illustrated in FIG. 1 performs both the role of determining the roles of each device and providing audio content by separating them by role. For convenience of explanation, the first audio output device (100) is hereinafter referred to as an audio output device, and all other audio output devices, including the second audio output device and the third audio output device (200-1, 200-2), are referred to as external devices. The remaining audio output devices other than the first audio output device (100) may have the same configuration as FIG. 2, but are not limited thereto and may include various additional configurations or modified configurations.

[0043] According to FIG. 2, the audio output device (100) includes a communication unit (110), a memory (120), an audio output unit (130), and a processor (140).

[0044] The communication unit (110) is configured to perform communication with various external devices. The communication unit (110) can transmit and receive various signals and data to and from external devices through communication methods such as Bluetooth, AP-based Wi-Fi (Wireless LAN network), Zigbee, wired / wireless LAN (Local Area Network), WAN (Wide Area Network), Ethernet, IEEE 1394, HDMI (High-Definition Multimedia Interface), USB (Universal Serial Bus), MHL (Mobile High-Definition Link), AES / EBU (Audio Engineering Society / European Broadcasting Union), Optical, Coaxial, etc. For example, the communication unit (110) can receive various information related to the device from other surrounding audio output devices, i.e., external devices.

[0045] The memory (120) is a configuration for storing various data, programs, commands, etc. The memory (120) may be implemented in the form of a memory embedded in the audio output device (100) or may be implemented in the form of a memory that is detachable from the audio output device (100) depending on the purpose of data storage. Specifically, the memory (120) may be implemented as one or more of DRAM (dynamic RAM), SRAM (static RAM), SDRAM (synchronous dynamic RAM), OTPROM (one time programmable ROM), PROM (programmable ROM), EPROM (erasable and programmable ROM), EEPROM (electrically erasable and programmable ROM), mask ROM, flash ROM, flash memory (e.g., NAND flash or NOR flash, etc.), hard drive, solid state drive (SSD), external hard drive, USB stick memory. In FIG. 2, the memory (120) is depicted as being a separate configuration from the processor (140), but depending on the embodiment, all or part of the memory may be integrated into the processor (140) and included as a single configuration.

[0046] The memory (120) can store various information and data received from an external device through the communication unit (110). In addition, the memory (120) can also store information and data to be transmitted to each external device. The memory (120) can store various information about the audio output device (100). For example, the memory (120) can store product information such as the product name, serial number, and manufacturing date of the audio output device (100), information about the manufacturing company or factory, location information, audio output characteristic information, detailed specification information such as processor performance or memory capacity, etc. The location information includes information that can specify the location of the audio output device (100) within a space (e.g., within a home) in which the audio output device (100) is installed. For example, if the audio output device (100) includes a Lidar sensor, the audio output device (100) can identify a location within the space based on a sensing value of the Lidar sensor. Alternatively, the user may directly input the location of the audio output device (100). The memory (120) may store the location information input by the user as is. In addition, the audio output characteristic information includes various information related to audio output. For example, the audio output characteristic information may include a volume curve, a sound pressure level (SPL), a maximum SPL, headroom, input sensitivity, maximum power, a crossover frequency, gain, speaker impedance, and other parameters.

[0047] A volume curve represents the amount of change in loudness according to a change in the volume level. Sound pressure level represents the loudness of sound in terms of sound pressure, and refers to the value expressed in decibels (dB) as a ratio to the reference sound pressure. Headroom refers to the difference between the peak level and the RMS (Root Mean Square) level of an audio signal. Generally, the greater the headroom, the more severe the loudness and fluctuations of the sound. Input sensitivity refers to the input voltage required for a specific audio device to produce the rated output. The lower the required input voltage value, the higher the input sensitivity can be said to be. The crossover frequency refers to the boundary of the divided sound range when the entire audible frequency band is divided into low-frequency range, high-frequency range, etc., and multiple speaker devices reproduce each sound range. Gain refers to the degree to which the input voltage signal is amplified, and is a value comparing the output level to the input level or reference level, and is usually expressed in decibels (dB). Speaker impedance is inversely proportional to the amplifier's output. If the speaker's impedance is low, the amplifier's output increases, and this can vary depending on the frequency band being played.

[0048] The processor (140) is a configuration for controlling the overall operation of the audio output device (100). The processor (140) may be implemented as a digital signal processor (DSP) for processing digital image signals, a microprocessor, a GPU (Graphics Processing Unit), an AI (Artificial Intelligence) processor, an NPU (Neural Processing Unit), or a TCON (Time Controller). However, the processor (140) is not limited thereto, and may include one or more of a central processing unit (CPU), an MCU (Micro Controller Unit), an MPU (micro processing unit), a controller, an application processor (AP), a communication processor (CP), or an ARM processor, or may be defined by the relevant terms. In addition, the processor (140) may be implemented as a SoC (System on Chip), an LSI (Large Scale Integration) having a built-in processing algorithm, or may be implemented in the form of an ASIC (Application Specific Integrated Circuit), or an FPGA (Field Programmable Gate Array). Alternatively, the processor (140) may be implemented as a processor capable of executing an artificial intelligence model stored in the memory (120). If the processor (140) includes an artificial intelligence-specific processor, the processor may be designed with a hardware structure specialized for processing a specific artificial intelligence model. For example, the processor (140) may include a hardware chip, such as an ASIC or FPGA, specialized for processing a specific artificial intelligence model.

[0049] The processor (140) can perform various operations based on various data, programs, commands, etc. stored in the memory (120).

[0050] When there are multiple external devices (200-1, 200-2) equipped with sound output functions in the vicinity as shown in Fig. 1, the processor (140) can transmit and receive various signals and data with these devices through the communication unit (110).

[0051] For example, when the processor (140) receives product information such as the product name or serial number of an external device, the date of manufacture, information about the manufacturer or factory, location information, audio output characteristic information, detailed specification information such as processor performance or memory capacity, etc., through the communication unit (110), the processor (140) may store such information in the memory (120). The location information of external devices may be received directly from the external devices, but is not necessarily limited thereto, and may be directly input by the user from an audio output device (100) other than the external device. When information on the location of an external device is input, the processor (140) may include the input location information in the information of the external device and store it together in the memory (120).

[0052] As described above, when information of one or more external devices is received through the communication unit (110), the processor (140) determines the role of each audio output device and external device based on the information stored in the memory (120) and the received information. If three audio output devices (100, 200-1, 200-2) are arranged in a space as shown in FIG. 1, the processor (140) can determine the role of each audio output device (100, 200-1, 200-2).

[0053] Role determination can be performed based on various criteria. For example, the processor (140) can determine the role based on the location of each device.

[0054] Taking Fig. 1 as an example, when communication with two external devices (200-1, 200-2) is connected, the processor (140) can first determine whether each external device (200-1, 200-2) is capable of performing a simultaneous audio playback function.

[0055] Specifically, the processor (140) checks information on each external device. If the audio output characteristics of the external device differ from the audio output characteristics of the audio output device (100) by a preset threshold or more, the processor (140) may determine that the simultaneous audio playback function is impossible. For example, if the external device outputs at a significantly lower sound pressure level than the sound pressure level of the audio output device (100), or if the difference between the frequency bands that can be played and output is large, the processor (140) may determine that the simultaneous audio playback with the external device is impossible. Alternatively, if the external device is too far away from the location of the audio output device (200), it is difficult to transmit audio of the same size to the user's location even if simultaneous audio playback is performed. Therefore, the processor (140) calculates the distance between the devices based on the location information of the audio output device (100) and the external devices (200-1, 200-2), and if the distance is greater than the preset threshold distance, the processor (140) may determine that the simultaneous audio playback function is impossible.

[0056] Alternatively, even if the distance between multiple audio output devices (100, 200-1, 200-2) is close, internal walls, doors, furniture, or other home appliances may block the connection between the devices. In this case, even if the simultaneous audio playback function is performed, the user may not be able to enjoy satisfactory audio content. The processor (140) determines whether there is an obstacle that may interfere with the transmission of audio signals between the devices based on the map of the space and the location information of each device. If the processor (140) determines that there is an obstacle, the simultaneous audio playback function may be impossible. The map information may be directly input by the user through a user interface (not shown) in the audio output device (100) or may be received from an external device and stored in the memory (120). For example, if there is a robot vacuum cleaner that cleans the inside of a space, the robot vacuum cleaner may generate a map that includes the size and shape of the space and the location, size, and shape of internal obstacles by using various sensors while performing indoor cleaning. The processor (140) can receive information about such a map from the robot vacuum cleaner and store it in the memory (120).

[0057] Alternatively, some audio output devices may not support or be unable to perform the simultaneous audio playback function due to hardware or software specifications. Furthermore, some devices may be unable to perform this function due to differences in manufacturers, operating systems, or other policy reasons. The processor (140) may determine whether a device does not support the simultaneous audio playback function based on information about each external device (200-1, 200-2).

[0058] If the processor (140) determines that the external devices (200-1, 200-2) are capable of simultaneous audio playback, the processor (140) can determine the role of each device based on information about the audio output device (100) and each of the external devices (200-1, 200-2).

[0059] For example, the processor (140) may identify the locations of each of the audio output device and the external devices (100, 200-1, 200-2), and then, based on the identified locations, determine the role of the audio output device and at least one external device as one of the main role and the sub role.

[0060] Taking FIG. 1 as an example, the user is looking at the first and second audio output devices (100, 200-1), and the third audio output device (200-2) is positioned to the right of the first and second audio output devices (100, 200-1).

[0061] The processor (140) of the first audio output device (100) can identify the user's location and then determine the device closest to the user among all audio output devices (100, 200-1, 200-2) as the main device. The processor (140) can identify the user's location based on sensing values ​​of various sensors, such as an image sensor, an infrared sensor, an ultrasonic sensor, and a microphone, provided in the first audio output device (100). For example, if an image sensor is provided, the processor (140) can identify an object corresponding to the user among a plurality of captured images captured by the image sensor, and analyze the size, location, shape, etc. of the object to identify how far the user is from the first audio output device (100), where the user is located in space, and which direction the user is looking at, etc. Alternatively, if an infrared sensor or an ultrasonic sensor is provided, the processor (140) can estimate the user's location and distance based on the size or reception direction of signals transmitted and reflected from each sensor. The processor (140) may use at least one of these sensors to identify the location of another audio output device (200-1, 200-2), but is not necessarily limited thereto, and may directly receive location information from another audio output device (200-1, 200-2) as described above, or may receive location information input from a user.

[0062] Alternatively, the processor (140) may determine at least one of the audio output devices (100, 200-1, 200-2) to play the main role based on location information directly input by the user through the user interface (e.g., touch screen, remote control, mouse, keyboard, etc.) of the first audio output device (100).

[0063] Among multiple audio output devices, there may be devices that are close to each other. As shown in FIG. 1, if the first audio output device (100) is a TV and the second audio output device (200-1) is implemented as a sound bar, the distances between the user (10) and the first and second audio output devices (100, 200-1) are almost the same. That is, the positions of the two devices can be considered to be within a similar range. The similar range means that the distance between the positions is within a preset distance range. For example, the processor (140) may determine that devices within a distance range of 50 cm are within a similar range. If the positions of each device are within a similar range, the processor (140) may compare the audio output characteristics among the information of each device and determine the role of the audio output device and at least one external device as either a main role or a sub role, respectively. For example, the processor (140) may determine the device with the highest Max SPL information as the main role and the remaining devices as sub roles. Here, the Max SPL information means the maximum value of the sound pressure level (SPL). Sound pressure level is a numerical index used to objectively measure the loudness of sound. Sound pressure level SPL is 20log 10 It can be calculated by an equation such as (P / Pref). In the equation, P is the root mean square (rms) of the instantaneous sound pressure, and Pref can be the reference sound pressure. The reference sound pressure Pref in air can be 20 μPa.

[0064] The processor (140) can also compare various audio output characteristics and determine a device with better audio output characteristics as the main role.

[0065] In the above embodiment, the role is first determined based on location, and then, if the location is similar, the role is determined again by comparing audio output characteristics. However, in other embodiments, the role may be determined solely based on audio output characteristics, not location. Alternatively, if multiple devices exist in similar locations, the processor (140) may determine the roles of all of these devices as the main role.

[0066] When the main role and sub role are determined, the processor (140) corrects the audio output characteristics of the audio output devices of the sub role according to the audio output characteristics of the audio output device of the main role. For example, when the audio output device (100) of FIG. 2 is determined as the main role, the processor (140) can transmit information about the audio output characteristics stored in the memory (120) to each of the external devices (200-1, 200-2) and control the external devices (200-1, 200-2) to correct their audio output characteristics according to this information.

[0067] On the other hand, if the audio output device (100) of FIG. 1 is determined to be a sub-role and the second audio output device (200-1) is determined to be a main role, the processor (140) corrects the audio output characteristics of the audio output device (100) itself according to the audio output characteristics of the second audio output device (200-1). The corrected audio output characteristics can be stored in the memory (120). The audio output unit (130) can output an audio signal according to the corrected characteristics. Information about the audio output characteristics of other devices can be directly used as information transmitted and received during the role division process, or information about other devices can be newly transmitted and received for audio output linkage after the role division is completed.

[0068] Figures 3 and 4 are diagrams illustrating various methods for correcting audio output characteristics. Figure 3 illustrates a case where a volume curve is corrected. Hereinafter, an example will be described in which the second audio output device (200-1) plays the main role and the first audio output device (100) and the third audio output device (200-2) play sub-roles.

[0069] The processor (140) of the audio output device (100) obtains a volume curve from among the audio output characteristic information received from an external device (200-1) and calculates the difference with its own volume curve. The volume curve can be measured as the amount of change in gain according to a change in volume level.

[0070] Figure 3 (a) shows that the volume curve (31) of the external device (200-1) as the main role and the volume curve (32) of the audio output device (100) as the sub role differ in some volume steps (e.g., about 17 to 45 steps). That is, even if the user sets the volume to the same level of 20, the volume level output from the audio output device (100) as the sub role becomes lower than the volume level (31) output from the audio output device (200-1) as the main role.

[0071] The processor (140) corrects its own volume curve (32) to match the volume curve (31) of the audio output device (200-1) serving as the main role. Here, correction means a task in which the processor (140) changes data for the volume curve (32) stored in the memory (120) into data for the volume curve (31) of the audio output device (200-1) serving as the main role. The data for the volume curve may include data for the audio amplification gain, speaker output voltage, etc. used for each audio volume. The processor (140) controls the audio output unit (130) to adjust the volume of the audio signal to match the changed volume curve data, thereby enabling it to be appropriately coordinated with the operation of the audio output device (200-1) serving as the main role.

[0072] Figure 3 (b) shows the corrected volume curve status. While products from the same company typically have data for the same volume curve, this isn't necessarily the case. Products from different companies can be used together. Therefore, aligning audio output characteristics before executing the aforementioned simultaneous audio playback function can significantly enhance user satisfaction.

[0073] The method of correcting audio output characteristics is not limited to this and can be performed in various ways. For example, volume characteristics can be adjusted through output power correction. In the above, the case where the characteristics of the remaining devices are corrected according to the characteristics of the device in the main role has been described, but it is not necessarily limited thereto. If some devices cannot operate according to the audio characteristics of the device in the main role, the device in the main role may adjust the audio output characteristics according to the audio output characteristics of the device in the main role. However, if the difference in audio output characteristics exceeds a threshold, the audio quality may actually deteriorate during simultaneous audio playback. Therefore, in such cases, the processor (140) may provide a guidance message indicating that simultaneous audio playback is impossible or guiding that the device be replaced with another device and tried again.

[0074] When the roles of each device (100, 200-1, 200-2) are determined, the processor (140) can perform a task of dividing the audio content into a plurality of audio elements so that one audio content can be played simultaneously on each device (100, 200-1, 200-2). The processor (140) can transmit an audio element corresponding to the role of at least one external device (200-1, 200-2) among the plurality of audio elements to at least one external device (200-1, 200-2) through the communication unit (110). The processor (140) controls the audio output unit (130) to output an audio element corresponding to the role of the audio output device (100) among the plurality of audio elements according to the output of at least one external device (200-1, 200-2).

[0075] The task of dividing audio content into multiple audio elements and dividing them into main elements and sub-elements can be performed according to various criteria.

[0076] For example, in the case of multi-channel sound such as 5.1 channel, 6.1 channel, 7.1 channel, etc., the processor (140) can classify audio elements by frequency channel. In the case of 5.1 channel content, the processor (140) can classify into multiple audio elements such as front left sound, center sound, front right sound, surround left sound, surround right sound, subwoofer sound, etc. As in Fig. 1, if the first and second audio output devices (100, 200-1) are positioned in the center, and the third audio output device (200-2) is positioned to the right with respect to the user, the processor (140) can classify the center sound, front left sound, and surround left sound as audio elements to be played by the first audio output device (100), and classify the center sound, front left sound, surround left sound, and subwoofer sound as audio elements to be played by the second audio output device (200-1), which plays the main role. The processor (140) can classify front light sound and surround light sound as audio elements to be played back by the third audio output device (200-2). However, this is merely an example, and the processor (140) can individually set the elements to be played back according to the role, location, and characteristics of each device.

[0077] Meanwhile, if the audio content includes only a left channel and a right channel, the processor (140) may reconstruct the audio content into a left element, a center element, and a right element. The processor (140) may classify the center element as a main element and the remaining elements as sub-elements. Alternatively, in a case such as FIG. 1, the center element and the left element may be classified as main elements, and the right element may be classified as a sub-element. In the example of FIG. 1, the first audio output device (100) performs a sub-role, but considering its position, it may also reproduce both the center element and the left element in the same manner as the main role.

[0078] Meanwhile, in the case of content such as dramas or movies, the audio signal may be composed of dialogue, music, sound effects, etc. The processor (140) can demultiplex multimedia content such as dramas or movies to extract video data, audio data, and other data. After decoding the audio data, the processor (140) can obtain audio elements corresponding to people's dialogue and at least one audio element corresponding to background music or other sound effects. The processor (140) can determine the audio element including people's dialogue as the main element, and determine the rest as sub-elements.

[0079] Alternatively, content such as a drama or movie may include a scene in which multiple speakers are having a conversation. In this case, the processor (140) may separate the voices of each speaker and classify them into multiple elements. To identify the speakers, the processor (140) may also consider video data in addition to audio data. Specifically, the processor (140) detects a face within a video frame and tracks changes in the lip area within the detected face. The processor (140) detects an audio signal output at the time when a change in the lip area is detected among audio signals output in synchronization with the corresponding video frame, and separates an audio signal in a frequency band corresponding to a human voice within the audio signal. When multiple characters are identified, the processor (140) identifies the face and lips of each character and estimates the voice corresponding to each person. The processor (140) matches the positions of the characters within the video frame with the positions of each audio output device (100, 200-1, 200-2) to set the voice to be output from each device. For example, if there are three characters and three audio output devices are arranged in a row, the processor (140) may be set to output the voice of the left character, the voice of the middle character, and the voice of the right character from the left, middle, and right audio output devices, respectively.

[0080] Alternatively, in the case of music content, the processor (140) may classify audio elements into audio elements including vocals and audio elements including sounds other than vocals (e.g., accompaniment, etc.). The processor (140) may determine the audio element including vocals as the main element, and determine the audio element including sounds other than vocals (e.g., accompaniment, etc.) as the sub element.

[0081] In addition, in the case of piano concerto content, elements containing piano sounds may be classified as main elements, and elements containing sounds of other instruments may be classified as sub-elements.

[0082] In addition, audio content can be divided into multiple audio elements in various ways depending on its type, and further classified into main elements and sub-elements.

[0083] Even after assigning elements for the main and sub roles, if the audio content changes, the processor (140) may reclassify the audio elements in the manner described above according to the type of the audio content and provide them to the audio output devices of the main and sub roles, respectively.

[0084] Meanwhile, if the status of at least one of the audio output devices (100, 200-1, 200-2) that are in charge of the main role and the sub role changes, the processor (140) can readjust the role of each audio output device (100, 200-1, 200-2) or the audio elements to be assigned to them. For example, if the audio output status of a third audio output device (200-2) assigned a sub role among the plurality of audio output devices (100, 200-1, 200-2) reaches a limit state, the processor (140) can control the audio element being played by the third audio output device (200-2) to be additionally played by the audio output unit (130) or played by the second audio output device (200-1). The limit state may include various states, such as reaching the maximum output limit, the battery falling below a certain amount, being turned off, or the distance increasing due to position movement. The processor (140) can receive a feedback signal from the third audio output device (200-2) and determine whether it is in a limit state based on the feedback signal. The processor (140) can check the state of the third audio output device (200-2) at regular intervals to determine whether it is in a limit state. Specifically, the processor (140) can transmit a query signal to the third audio output device (200-2) at regular intervals, and if the feedback signal is not received within a preset time, it can determine that it is in a limit state. Even if a feedback signal for notifying the limit state is received within a preset time, the processor (140) can determine that the third audio output device (200-2) is in a limit state. Even if there is no feedback signal, the processor (140) can determine that it is in a limit state if there is a part in the audio content being played that the third audio output device (200-2) cannot output.In addition, even when the positions of each device change, the processor (140) can readjust the roles of each device and the audio elements to be played. This will be described in detail later.

[0085] In the above embodiments, it has been described that the role determination, audio element classification, and distribution tasks of each audio output device (100, 200-1, 200-2) are all performed by the processor (140) of the first audio output device (100). However, as described above, this is merely an example, and the audio output devices (100, 200-1, 200-2) may each perform the above-described tasks. After exchanging information with each other, the audio output devices (100, 200-1, 200-2) can recognize their own roles by considering preset rules, their own location information, and the location of the user. In addition, the audio output devices (100, 200-1, 200-2) can extract audio elements corresponding to their own roles for the same audio content, and play and output the audio elements in synchronization with the playback and output timing of other devices. As described above, the entity performing each task may vary depending on the embodiment. For example, users can directly configure the role of each device and the audio elements that each device will play on the screen.

[0086] Fig. 4 illustrates an example of a detailed configuration of an audio output device. According to Fig. 4, the audio output device (100) includes a communication unit (110), a memory (120), an audio output unit (130), a processor (140), a display (150), and a sensor (160). Among the components of Fig. 4, duplicate descriptions of the same components as those described in Fig. 2 will be omitted.

[0087] The audio output unit (130) can play and output audio elements under the control of the processor (140). The audio output unit (130) is configured to play and output audio content or audio elements. Audio content may include not only sound source content streamed from an external device (e.g., a web server, etc.), but also audio files previously stored in the memory (120), audio data extracted from multimedia content, etc.

[0088] The audio output unit (130) may include at least one of various components, such as a decoder for decoding audio data, a digital-to-analog converter for converting a decoded signal into an analog audio signal, various filters, an amplifier circuit, and a speaker. Not only one speaker may be included, but at least one or more may be included depending on the type, size, specifications, etc. of the audio output device (100). When implemented as a TV, the speaker may be placed on the back or side of the display (150), or may be implemented as a front-top speaker placed in a forward direction from the upper side of the display (150).

[0089] The display (150) can display various screens under the control of the processor (140). The display (150) can be implemented as a display of various forms, such as an LCD (Liquid Crystal Display), an OLED (Organic Light Emitting Diodes) display, a PDP (Plasma Display Panel), etc. The display (150) can also include a driving circuit, a backlight unit, etc., which can be implemented as a form, such as an a-si TFT, an LTPS (low-temperature poly silicon) TFT, an OTFT (organic TFT), etc. In addition, the display (150) can be implemented as a flexible display, and can be a touch screen capable of touch input. For example, the processor (140) can control the display (150) to display a setting screen including at least one menu among various menus, such as a menu for selecting a simultaneous audio playback function, a menu for selecting a device to perform simultaneous audio playback, a menu for setting the role of each device, a menu for setting a classification method of audio elements, and a menu for selecting audio elements to be played on each device.

[0090] The sensor (160) may include at least one sensor for sensing the position of the audio output device (100), the position and distance of an external device, the position and distance of a user, the shape or size of the space in which the audio output device (100) is placed, the movement of an external device including a user, etc.

[0091] The processor (140) can determine whether the position of at least one external device has changed based on the sensing value of the sensor (160) relative to the position of the audio output device. If the processor (140) determines that the position has changed, it can readjust the roles of the audio output device (100) and at least one external device (200-1, 200-2), respectively.

[0092] Fig. 5 illustrates an example of the configuration of a settings screen. According to Fig. 5, a display (150) can display a settings screen (500).

[0093] The setting screen (500) may display a list (501) for distinguishing the main device and sub devices to participate in simultaneous playback, an input area (502) for setting each device on the list, a menu (503) for changing the device to be displayed in the input area (502), an input area (504) and a menu (505) for selecting the location of each device, etc. When the user selects the menu (503) for changing the device, the processor (140) may display all connected external devices (200-1, 200-2) as well as the audio output device (100) in the input area (502). In addition, when the menu (505) for location selection is selected, the processor (140) may display all settable locations in the form of a list on the input area (504). For example, at least one of various locations such as center, front right, front left, rear right, rear left, right, left, center above, and center below may be displayed in the input area (504).

[0094] In an environment like Fig. 1, the user can input data into each area of ​​the settings screen (500) as shown in Fig. 5. Accordingly, the sound bar is set as the main device, and its position is set to the center. Sub-device 1 is a TV, and its position is set to the center, the same as the sound bar. Sub-device 2 is a speaker, and its position is set to the right.

[0095] When using the setting screen (500) as shown in Fig. 5, the audio output device (100) can recognize the positions of the audio output device (100) and external devices (200-1, 200-2) even if the audio output device (100) does not have or use a sensor (160) or does not receive information from an external device.

[0096] Meanwhile, the processor (140) may provide various notification messages related to the execution of the audio simultaneous playback function through the display (150) or in the form of a voice message through the audio output unit (130).

[0097] Fig. 6 illustrates an example of a case where a message is displayed on the screen. According to Fig. 6, if there is an external device among the information of at least one external device whose audio output characteristics do not meet the preset allowable conditions, the processor (140) may determine that the performance of the audio simultaneous playback function for outputting audio content together with the external device is impossible. Accordingly, the processor (140) may display a message (600) notifying that the performance of the selected function is impossible. The message (600) may include various menus, such as a menu (601) for changing the setting status, a menu (602) for removing the message (600), a menu (not shown) for re-searching the surrounding external device, and a menu (not shown) for changing the audio content.

[0098] The allowable conditions may be information that is set individually or in combination for at least some of various audio output characteristics, such as a volume curve, a sound pressure level (SPL), a maximum SPL, headroom, input sensitivity, maximum power, crossover frequency, gain, speaker impedance, and other parameters, and pre-stored in the memory (120). For example, if the allowable condition for the maximum output difference is set to 10 dB, the processor (140) determines that an audio output device with a maximum output difference of 10 dB or more does not meet the allowable condition. If the audio output characteristics differ greatly, harmonious sound reproduction is impossible, and thus the audio simultaneous reproduction function is not supported.

[0099] The processor (140) can readjust at least one of the roles of each device (100, 200-1, 200-2) and the audio elements to be played, even if the position of at least one of the audio output device (100) or the external devices (200-1, 200-2) changes.

[0100] As in the configuration illustrated in FIG. 4, when the audio output device (100) comprises a sensor (160), the processor (140) can directly identify the location of each device or user based on the sensing value of the sensor (160).

[0101] FIG. 7 and FIG. 8 are drawings for explaining a case in which the position of an external device is sensed using multiple microphones.

[0102] According to FIG. 7, the sensor (160) includes a plurality of microphones (161-1, 161-2) distributed at different locations on the exterior of the audio output device (100). FIG. 7 illustrates two microphones (161-1, 161-2) distributed at both edges on the upper surface of the audio output device (100) implemented as a TV, but the locations and number of microphones may be varied.

[0103] The processor (140) transmits a control signal to an external device to output an arbitrary sound while activating the microphones (161-1, 161-2). In FIG. 7, an example is given of the case where the third audio output device (200-2) of FIG. 1 is an external device. The external device (200-2) outputs an arbitrary test sound according to the control signal. The processor (140) compares the times at which the test sound is received by each microphone (161-1, 161-2). As a result of the comparison, if the time at which the sound is received by the first microphone (161-1) is later than the time at which the sound is received by the second microphone (161-2), it can be determined that the first microphone (161-1) is further away than the second microphone (161-2). Accordingly, the processor (140) can estimate that the external device (200-2) is located to the right of where the second microphone (161-2) is located.

[0104] Alternatively, the processor (140) may calculate the distance (d1, d2) between each microphone (161-1, 161-2) and the external device (200-2) based on the time at which the test sound is received from each microphone (161-1, 161-2) and the time at which the test sound is output. In addition, since the received signal intensity of a sound at a further distance is weaker than the received signal intensity of a sound at a closer distance, the processor (140) may also consider the difference in the received signal intensity in addition to the time difference.

[0105] Fig. 8 illustrates a case where the external device (200-2) has moved to the left relative to the audio output device (100). The processor (140) may estimate the position of the external device (200-2) or calculate a new distance (d3, d4) between each microphone (161-1, 161-2) and the external device (200-2) in various ways as described above.

[0106] As shown in FIGS. 7 and 8, when the position of an external device (200-2) performing a sub role is changed from the right to the left, the processor (140) can adjust the audio elements to be played by the external device (200-2). Specifically, when a plurality of elements are classified into center sound, front left sound, front right sound, surround left sound, surround right sound, subwoofer sound, etc., the external device (200-2) can play and output the front right sound and the surround right sound before the position change, and can play and output the front left sound and the surround right sound after the position change. When the processor (140) detects that the external device (200-2) has moved to the left, it can control the audio output unit (130) to output the center sound, the front right sound, and the surround right sound.

[0107] Meanwhile, although FIG. 1 illustrates a case where a sound bar is included, there may also be cases where a sound bar is not present. FIG. 9 illustrates a case where three audio output devices other than a sound bar are connected to each other. For convenience of explanation, the TV is described as the first audio output device (100), and the remaining devices are described as the third and fourth audio output devices (200-3, 200-4), respectively. Although different reference numerals are used for the speaker (200-2) of FIG. 1, the third and fourth audio output devices (200-3, 200-4) may also be implemented as devices of the same type as the speaker (200-2) of FIG. 1.

[0108] According to FIG. 9, the first audio output device (100) is positioned in front of the user (10), and the third and fourth audio output devices (200-3, 200-4) are positioned on the left and right rear sides of the user (10), respectively.

[0109] In this case, the first audio output device (100) may play a main role based on location, and the third and fourth audio output devices (200-3, 200-4) may play a sub role.

[0110] If audio content can be classified into a total of five audio elements, such as center sound, front left sound, front right sound, surround left sound, and surround right sound, the first audio output device (100) plays and outputs the center sound, front left sound, and front right sound according to location. The third audio output device (200-3) plays the surround left sound, and the fourth audio output device (200-4) plays the surround right sound.

[0111] In this state, when the third and fourth audio output devices (200-3, 200-4) are respectively moved to the front side of the user and positioned on both sides of the first audio output device (100), the first audio output device (100) reproduces and outputs the center sound, front left sound, and front right sound as is. On the other hand, the third audio output device (200-3) reproduces and outputs the front left sound and the surround left sound, and the fourth audio output device (200-4) reproduces and outputs the front right sound and the surround right sound.

[0112] In this state, if the fourth audio output device (200-4) is turned off, has low battery, or is moved to a long distance, the first audio output device (100) and the third audio output device (200-3) can readjust at least one of the roles and audio elements to be played back. For example, the first audio output device (100) can play and output the front left sound, the center sound, the front right sound, and the surround right sound, and the third audio output device (200-3) on the left can play and output the front left sound and the surround left sound.

[0113] As described above, according to various embodiments of the present disclosure, role division and optimized sound reproduction are enabled based on the relative positions of each device. Furthermore, since the audio output characteristics of other devices are adjusted to match the audio output characteristics of the device playing the primary role, the same loudness of sound can be output at the same volume level, allowing users to enjoy audio content harmoniously.

[0114] While the above description focuses on audio output devices such as TVs, sound bars, and speakers, these devices can also be portable devices, such as smartphones or tablet PCs. Portable devices, often carried by users, can move more frequently than other audio output devices, and their small size can make them difficult to recognize.

[0115] Fig. 10 is a diagram illustrating a case in which a portable device is included. According to Fig. 10, a case in which a user (10) is positioned in front of the audio output devices (100, 200-1, 200-2) of Fig. 1 while holding a portable device (400) is shown. If the portable device (400) has an audio output function, the portable device (400) can also be classified as an audio output device.

[0116] The sensor (160) of the first audio output device (100) includes various sensors for detecting an external device or a user. For example, if the sensor (160) includes an image sensor (162), the processor (140) can analyze the captured data captured by the image sensor (162) to identify the location or distance of the external device or the user. If the sensor (160) includes a depth camera, the processor (140) can identify the location or distance of the external device or the user based on the depth value acquired from the depth camera. In the case of a portable device (400), since it can be considered that the user (10) is carrying the device, the location or distance of the user (10) can be identified, and the location or distance of the user (10) can be estimated as the location or distance of the portable device (400) at that point or at a point within a certain range therefrom.

[0117] The processor (140) can determine the roles of each of four audio output devices (100, 200-1, 200-2, 400) including the portable device (400), and then determine the audio elements to be played on each device.

[0118] In this case, the processor (140) can recognize that the device is a portable device based on product information among the information transmitted from the portable device (400). If the processor (140) recognizes that at least one external device is a portable device, the processor (140) can determine different roles depending on the distance (d) between the audio output device (100) and the portable device (400).

[0119] For example, if the distance (d) between the audio output device (100) and the portable device (400) is within a preset range (Dth), the processor (140) can control each device to distinguish between main and sub roles based on the location and audio output characteristics as described above and play audio elements suitable for each role and location. The processor (140) can classify audio elements including voices, such as dialogue or speech of a performer among audio contents, as main elements, and classify other sounds or sounds as sub elements.

[0120] On the other hand, if the distance (d) between the audio output device (100) and the portable device (400) exceeds the preset range (Dth), the processor (140) determines the portable device (400) as the main role, and determines the audio output device (100) and the remaining external devices (200-1, 200-2) as sub roles. If the distance (d) between the audio output device (100) and the portable device (400) exceeds the preset range (Dth), no matter how loudly the audio signal is output from the audio output device (100) and the remaining external devices (200-1, 200-2), the user may not be able to hear it well. In particular, in cases where the voice of the performer is important, such as in movies, dramas, or news, the user (10) may hear better the output from the portable device (400) located closest to the user (10) than from the audio output device (100) located far away and the remaining external devices (200-1, 200-2). Accordingly, the processor (140) may automatically determine the portable device (400) to play the main role when the distance (d) between the audio output device (100) and the portable device (400) exceeds a preset range (Dth).

[0121] In Fig. 10, it is described that the audio output device (100) directly senses the distance and readjusts the role, but it is not necessarily limited to this. For example, when a specific signal is received from the portable device (400), the processor (140) can set the portable device (400) to the main role regardless of the distance (d). The user (10) can simply set the portable device (400) to the main role by pressing a button on the body of the portable device (400) or selecting a menu on the UI screen.

[0122] When the portable device (400) is determined to be the main role, the processor (140) can provide an audio element including a voice among a plurality of audio elements to the portable device (400) through the communication unit (110).

[0123] When the same button or menu is selected again, the processor (140) may adjust the main role and sub role back to their original state.

[0124] Meanwhile, when there are multiple users or the space is very large, when audio elements are played separately for simultaneous audio playback, users in each location may not be able to fully enjoy the audio content. The audio output device (100) can identify the characteristics of the space in which the audio output device (100) is placed, the number of users within the space, etc., and perform different control operations based on the identification results.

[0125] FIG. 11 is a diagram for explaining the operation of an audio output device according to another embodiment of the present disclosure. According to FIG. 11, in addition to the audio output devices (100, 200-1, 200-2) of FIG. 1, a plurality of audio output devices (200-3 to 200-6) are additionally arranged, and a plurality of users (10-1 to 10-m) are present within the space (1100).

[0126] In the following, for the convenience of explanation, the description is based on the case where the first audio output device (100) among the audio output devices (100, 200-1 to 200-6) controls the remaining devices, but it is not necessarily limited to this, and as described above, each device can individually identify its own role and determine the audio element to be played.

[0127] In a system such as FIG. 11, the processor (140) can sense external objects using an image sensor (162) or other sensors among the sensors (160) included in the first audio output device (100). Here, the external objects include various objects such as internal walls or objects of a space (1100), moving users, and animals.

[0128] The processor (140) identifies the characteristics of the space (1100) where the first audio output device (100) is placed, such as the size and shape, and the number of users, based on the sensing value of the sensor (160). If the characteristics of the identified space or the number of users exceeds a preset threshold condition, the processor (140) controls the audio output unit (130) and external devices (200-1 to 200-6) to output the audio content in sync and identically without classifying the audio content into audio elements. The threshold condition may be a condition set by experimentally measuring a state where the effect of simultaneous audio playback is not felt. For example, when 8 or more users gather, their locations inevitably become dispersed. In this case, if an important audio element is output only from an audio output device at a specific location, a user at a distance will not be able to hear the audio element properly. Accordingly, if the threshold condition is set to 8 people and stored in the memory (120), when the processor (140) detects that there are 8 or more users, it can control all audio output devices (100, 200-1 to 200-m) to output audio content as is. In order to distinguish it from the aforementioned audio simultaneous playback function, this function can be named a party mode function or a group play function.

[0129] For example, when setting a threshold condition based on the size of a space rather than the number of users, a numerical value such as 20 m2 can be set as the threshold condition and stored in the memory (120). The threshold condition for the shape of the space can be set based on the ratio or location of pillars, walls, doors, furniture, home appliances, etc. placed within the space. For example, if a wall is placed between each audio output device, the processor (140) can determine that the threshold condition has been exceeded.

[0130] In the above, the case where the location, movement, distance, etc. of each device are identified based on the sensing value of the sensor or the user's input, etc. has been described, but the location, movement, distance, etc. of each device can also be identified in other ways.

[0131] For example, if there is a robot vacuum cleaner capable of creating a map while moving within a space, the processor (140) can receive the map created by the robot vacuum cleaner through the communication unit (110) and store it in the memory (120). The latest robot vacuum cleaners are equipped with various sensors such as a lidar sensor, an image sensor, an infrared sensor, and an ultrasonic sensor, and can generate semantic information that recognizes not only the size and shape of the entire space, but also each area or object within the space. For example, the robot vacuum cleaner can identify that a TV is located in the center of the living room, a sound bar is located below it, and a wireless speaker is located to the right of the sound bar, and generate identification information and map information for these. When the processor (140) receives various pieces of information created by the robot vacuum cleaner, it can identify the positional relationship and positional change status of all audio output devices, the user's position, spatial characteristics, etc. based on this information.

[0132] FIG. 12 is a flowchart illustrating an output method of an audio output device according to an embodiment of the present disclosure. According to FIG. 12 , the audio output device can receive information from an external device (S1210). In this case, the audio output device may transmit information of its own to the external device, but is not necessarily limited thereto.

[0133] The audio output device can determine the role of each device based on the received information and its own information (S1220). Once the role is determined, the audio output device can divide the audio content to be played into multiple audio elements (S1230). Since the method for determining the role and distinguishing the audio elements has been specifically described in the various embodiments described above, a detailed description thereof will be omitted.

[0134] An audio output device can transmit an audio element corresponding to the role of at least one external device among a plurality of audio elements to at least one external device (S1240). In this case, the location of the external device may be taken into consideration in addition to the role to transmit an appropriate audio element.

[0135] The audio output device outputs an audio element corresponding to the role of the audio output device among a plurality of audio elements, in accordance with the output of at least one external device (S1250). Accordingly, a simultaneous audio playback function can be performed, in which a single audio content is played by multiple devices.

[0136] Meanwhile, if the status of one of the audio output device and at least one external device changes (S1260), the audio output device can adjust the roles of the audio output device and at least one external device and at least one audio element to be played (S1270). Since the method for recognizing the status change and the method for adjusting the status change have been specifically described in the various embodiments described above, a redundant description will be omitted.

[0137] In the present method, a state change may include not only a state in which the device cannot output audio signals above a certain frequency range or above a certain loudness, but also a state in which the device can no longer output audio signals due to various reasons such as being moved too far away, a battery failure, or being turned off. When a device performing a sub-role enters a limit state, the audio output device can control the device performing a main role to additionally play audio elements that were previously output by the device in the limit state.

[0138] Additionally, when a portable device is included, the role or audio element to be played may be readjusted depending on the distance between the portable device and the audio output device, or the distance between the portable device and the device serving as the main role. Furthermore, the contents described in the various embodiments described above may also be equally incorporated into the output method according to various embodiments of the present disclosure.

[0139] FIG. 13 is a timing diagram illustrating the interaction of multiple audio output devices according to one embodiment of the present disclosure.

[0140] For convenience of explanation, only two audio output devices (100) and a second audio output device (200) are illustrated in FIG. 13, but as described above, the number of audio output devices may be varied.

[0141] According to Fig. 13, the first and second audio output devices (100, 200) communicate with each other and exchange various types of information (S1310). Specific details have been described in the other embodiments described above, so a redundant description will be omitted.

[0142] The first and second audio output devices (100, 200) set their respective main and sub roles based on the exchanged information (S1320). While the other embodiments described above assume that a single audio output device (100) sets the roles of all devices, as illustrated in FIG. 13, each device can also interact to set its own role.

[0143] Accordingly, when the first audio output device (100) is set to operate as a main role audio output device, i.e., a main device (S1330), and the second audio output device (200) is set to operate as a sub role audio output device, i.e., a sub device (S1340), the first and second audio output devices (100, 200) each perform operations corresponding to their respective roles.

[0144] First, the second audio output device (200) corrects its audio output characteristics to match the audio output characteristics of the first audio output device (100), which is the main device (S1341).

[0145] In this state, the first and second audio output devices (100, 200) each play and output audio elements corresponding to their respective roles (S1331, S1342).

[0146] The first and second audio output devices (100, 200) can transmit and receive a feedback signal to notify the current status when a specific event occurs during audio output or a certain period arrives (S1350).

[0147] Each of the first and second audio output devices (100, 200) corrects and outputs audio based on feedback from the other device. For example, if the output of the second audio output device (200) does not increase, the audio elements of the second audio output device (200) can be additionally played from the first audio output device (100).

[0148] Additionally, the various embodiments described above may be implemented alone or in combination at least in part.

[0149] The output method described in the various flowcharts above may be performed in an audio output device (100) having a configuration illustrated in the other embodiments described above, but is not necessarily limited thereto, and may be performed in an audio output device having a different configuration. Alternatively, it may be performed by a separately provided server device.

[0150] FIG. 14 is a diagram illustrating the operation of a server device according to at least one embodiment of the present disclosure. According to FIG. 14, a plurality of audio output devices (100, 200-1 to 200-4), a camera (1410), and a server device (1400) can communicate with each other.

[0151] The server device (1400) receives the shooting data captured by the camera (1410) and, based on the shooting data, can identify the location of each audio output device (100, 200-1 to 200-4), the distance between the devices, the location of the user (10), etc. In FIG. 14, each of the camera (1410) and the server device (1400) is illustrated, but these devices may be implemented in multiple units depending on the embodiment.

[0152] The server device (1400) can receive and store information of each audio output device (100, 200-1 to 200-4). If the server device (1400) determines that the user has selected the simultaneous audio playback function, the server device (1400) can identify the main device and the sub-device based on the information of the audio output devices (100, 200-1 to 200-4). The server device (1400) can control the sub-device to correct its own audio output characteristics according to the audio output characteristics of the main device.

[0153] When audio content to be played is selected by a user, the server device (1400) can assign audio elements to be played by each audio output device (100, 200-1 to 200-4) based on the role and location of the audio output device (100, 200-1 to 200-4).

[0154] In this state, if the server device (1400) identifies that the user's location has moved from point A to point B, it sets the audio output device (200-4) closest to point B as the main device, and can individually adjust the roles of the remaining audio output devices. In addition, the audio elements to be played can also be reallocated.

[0155] Since the specific role division, audio element allocation, and other operations have been specifically described in other embodiments described above, a duplicate description will be omitted.

[0156] As such, control tasks such as role division and audio element allocation can be performed by various operating entities.

[0157] In addition, although the above embodiments have been described as being divided into two main and sub roles, the sub roles may be further divided into multiple sub roles of different ranks depending on their performance or location. That is, they may be divided into multiple ranks, such as a first-ranking sub device, a second-ranking sub device, etc.

[0158] As described above, according to various embodiments of the present disclosure, multiple audio output devices can harmoniously reproduce and output a single audio content. Even if a status change occurs during output, the device can immediately respond to it, thereby maximizing user satisfaction.

[0159] The output methods according to the various embodiments described above may be stored and distributed on a non-transitory, readable recording medium in the form of program code for performing the method. Specifically, the program code for executing the method according to the various embodiments described above when executed by a device equipped with an audio output function may be stored on a recording medium and distributed or distributed online.

[0160] Here, the program code may sequentially perform the steps of: receiving information of at least one external device; determining a role of each of the audio output device and the at least one external device based on the information of the audio output device and the received information; dividing one audio content into a plurality of audio elements; transmitting an audio element corresponding to the role of at least one external device among the plurality of audio elements to the at least one external device; outputting an audio element corresponding to the role of the audio output device among the plurality of audio elements in accordance with the output of the at least one external device; and adjusting at least one of the roles of each of the audio output device and the at least one external device and the audio elements to be played when the state of one of the audio output device and the at least one external device is changed.

[0161] In addition, code may be used in which some steps are changed, deleted, or added to match the various embodiments described above.

[0162] A device equipped with a recording medium storing such program code can perform operations according to the various embodiments described above.

[0163] The recording medium can be various types of computer-readable media, such as ROM, RAM, memory chips, memory cards, external hard drives, hard drives, CDs, DVDs, magnetic disks, or magnetic tapes.

[0164] While the present invention has been described with reference to the attached drawings, the scope of the present invention is determined by the claims described below and should not be construed as being limited to the aforementioned embodiments and / or drawings. Furthermore, it should be clearly understood that improvements, modifications, and variations apparent to those skilled in the art, as defined in the claims, are also included within the scope of the present invention.

Claims

1. For audio output devices, A communication unit for performing communication with at least one external device; A memory storing information about the above audio output device; audio output section; and Processor; including; The above processor, When information of the at least one external device is received through the communication unit, the roles of each of the audio output device and the at least one external device are determined based on the information stored in the memory and the received information, A method of dividing a single audio content into a plurality of audio elements, transmitting an audio element corresponding to the role of at least one external device among the plurality of audio elements to the at least one external device through the communication unit, and controlling the audio output unit to output an audio element corresponding to the role of the audio output device among the plurality of audio elements according to the output of the at least one external device. An audio output device that adjusts at least one of the roles of the audio output device and the at least one external device and the audio elements to be played when the status of one of the audio output unit and the at least one external device changes.

2. In paragraph 1, The above processor, Identifying the locations of each of the audio output device and the at least one external device, and determining the roles of the audio output device and the at least one external device as one of a main role and a sub role based on the identified locations, An audio output device, wherein when the location of each device is within a preset similar range, audio output characteristics are compared among information of each device to determine the roles of the audio output device and the at least one external device as one of the main role and the sub role, respectively.

3. In paragraph 2, The above processor, When the audio output device is determined as the sub-role, the audio output characteristics stored in the memory are corrected according to the audio output characteristics of the external device determined as the main role among the at least one external device, and the audio output unit is controlled to play an audio element corresponding to the sub-role based on the corrected audio output characteristics. An audio output device, wherein when the audio output device is determined as the main role and the at least one external device is determined as the sub role, the audio output unit is controlled to play an audio element corresponding to the main role, and when there is an external device among the at least one external device playing an audio element corresponding to the sub role whose audio output status has reached a limit state, the audio output unit is controlled to additionally play an audio element that the external device was playing.

4. In paragraph 2, The above processor, An audio output device, wherein when the position of the at least one external device is changed based on the position of the audio output device, at least one of the roles of the audio output device and the at least one external device and the audio elements to be played are readjusted according to the changed positional relationship.

5. In paragraph 2, The above processor, An audio output device, wherein when the at least one external device includes a portable device, if the distance between the audio output device and the portable device exceeds a preset range, the portable device is determined to be the main role, and the audio output device and the remaining external devices are determined to be the sub roles.

6. In paragraph 5, The above processor, An audio output device that provides an audio element including a voice among the plurality of audio elements to the portable device through the communication unit when the portable device is determined to have the main role.

7. In paragraph 1, further comprising at least one sensor for sensing an external object; The above processor, An audio output device that identifies a characteristic of a space in which the audio output device is placed or a number of users based on a sensing value of at least one sensor, and controls each of the audio output unit and the at least one external device to output the audio content in sync and in the same manner when the characteristic of the space or the number of users each exceeds a preset threshold condition.

8. In paragraph 2, including display; The above processor, An audio output device that controls the display to output a message notifying that the function of simultaneously outputting the audio content with the external device is impossible if there is an external device among the information of the at least one external device whose audio output characteristics do not satisfy the preset allowable conditions.

9. In the output method of the audio output device, A step of receiving information from at least one external device; A step of determining a role of each of the audio output device and at least one external device based on information of the audio output device and the received information; A step of dividing one audio content into multiple audio elements; A step of transmitting an audio element assigned to at least one external device among the plurality of audio elements to the at least one external device; A step of outputting an audio element assigned to the audio output device among the plurality of audio elements in accordance with the output of at least one external device; An output method, comprising: a step of adjusting at least one of a role of the audio output device and the at least one external device and an audio element to be played when the state of one of the audio output device and the at least one external device changes; 10. In paragraph 9, The step of determining the role of each of the above audio output device and at least one external device is: A step of identifying the location of each of said audio output device and said at least one external device; A step of determining the roles of the audio output device and the at least one external device as one of a main role and a sub role, respectively, based on the identified location; An output method comprising: a step of comparing audio output characteristics among information of each device when the location of each device is within a preset similar range, and determining the roles of the audio output device and the at least one external device as one of the main role and the sub role, respectively; 11. In paragraph 10, The step of adjusting at least one of the roles of the above audio output device and the at least one external device and the audio element to be played, An output method comprising: a step of outputting an audio element corresponding to the sub-role in addition to an audio element corresponding to the main role when an audio output state of the at least one external device that plays an audio element corresponding to the sub-role reaches a limit state in a state where the audio output device is determined as the main role and the at least one external device is determined as the sub-role.

12. In paragraph 10, The step of adjusting at least one of the roles of the above audio output device and the at least one external device and the audio element to be played, An output method comprising: a step of adjusting at least one of the roles of the audio output device and the at least one external device and the audio elements to be played according to the changed positional relationship when the position of the at least one external device is changed based on the position of the audio output device.

13. In paragraph 10, The step of adjusting at least one of the roles of the above audio output device and the at least one external device and the audio element to be played, An output method further comprising: a step of determining the portable device as the main role and determining the audio output device and the remaining external devices as the sub roles when the distance between the audio output device and the portable device exceeds a preset range when the at least one external device includes a portable device; 14. In paragraph 10, A step of identifying the characteristics of the space in which the audio output device is placed or the number of users; and An output method further comprising: a step of providing the audio content to at least one external device and outputting the audio content in the same manner as the at least one external device in synchronization when the characteristics of the space or the number of users each exceed a preset threshold condition.

15. A non-transitory computer-readable storage medium storing computer instructions that, when executed by a processor of an audio output device, cause the audio output device to perform an operation, wherein the operation is: A step of receiving information from at least one external device; A step of determining a role of each of the audio output device and at least one external device based on information of the audio output device and the received information; A step of dividing one audio content into multiple audio elements; A step of transmitting an audio element assigned to at least one external device among the plurality of audio elements to the at least one external device; A step of outputting an audio element assigned to the audio output device among the plurality of audio elements in accordance with the output of at least one external device; A non-transitory computer-readable storage medium, comprising: a step of adjusting at least one of a role of the audio output device and the at least one external device and an audio element to be played when the state of one of the audio output device and the at least one external device changes;

Images