Electronic device and control method therefor

Abstract

This electronic device comprises: a speaker; a communication interface that performs communication with an external device for controlling the electronic device; a memory storing at least one instruction; and at least one processor connected to the memory to control the electronic device, wherein the at least one processor executes the at least one instruction to: control the speaker to reproduce content at a first volume; while the external device outputs guide audio for guiding a user's input, receive, from the external device through the communication interface, a first control command for decreasing an audio volume of the content; control, on the basis of the first control command, the speaker to decrease the audio volume of the content; receive, from the external device through the communication interface, a second control command for restoring the audio volume of the content to the first volume; and control, on the basis of the second control command, the speaker to restore the audio volume of the content.

Description

Electronic device and method of controlling the same

[0001] The present disclosure relates to an electronic device and a method for controlling the same, and more particularly, to a device for outputting guide audio that guides user input and a device for communicating with the device.

[0002] Mobile devices and TVs can be used in conjunction with each other to provide users with a variety of functions and conveniences. In particular, the talkback function of mobile devices is widely used as an assistive technology for the visually impaired. Talkback is a feature that outputs guide audio from mobile devices, providing information about the screen and other aspects.

[0003] When a user uses a mobile device and a TV simultaneously, the guide audio from the mobile device and the audio from the TV may overlap. For example, when a user controls the TV via a smartphone, the TV may output separate audio while the smartphone outputs the guide audio, resulting in overlapping audio. This can make it difficult for the user to clearly hear the guide audio, which can be inconvenient.

[0004] An electronic device according to one or more embodiments of the present disclosure includes a speaker, a communication interface for performing communication with an external device controlling the electronic device, a memory for storing at least one command, and at least one processor connected to the memory for controlling the electronic device, wherein the processor executes the at least one command to control the speaker to play content at a first volume, receive a first control command from the external device through the communication interface for reducing the audio volume of the content while the external device outputs guide audio for guiding a user's input, control the speaker to reduce the audio volume of the content based on the first control command, receive a second control command from the external device through the communication interface for restoring the audio volume of the content to the first volume, and control the speaker to output the audio volume of the content by restoring it to the first volume based on the second control command.

[0005] The at least one processor may execute the at least one command to receive information including a point in time at which the audio volume of the content decreases from the external device through the communication interface, and reduce the audio volume of the content at the point in time.

[0006] The at least one processor may execute the at least one command to control the speaker to output audio for an operation of the electronic device corresponding to the third control command when receiving a third control command from the external device through the communication interface.

[0007] The at least one processor may control the speaker to reduce the audio volume of the restored content when the control command is received from the external device through the communication interface while the external device outputs additional guide audio guiding additional user input by executing the at least one instruction.

[0008] The at least one processor may execute the at least one command, and upon receiving a user input including at least one text and a third control command controlling the operation of the electronic device through the communication interface, the processor may execute an application corresponding to the at least one text.

[0009] The at least one processor may execute the at least one command to detect the occurrence of an event, and transmit a reduction control command to the external device through the communication interface to reduce the volume of audio output by the external device to a second volume while outputting a voice notifying the occurrence of the event through the speaker, and when the output of the voice notifying the occurrence of the event through the speaker is completed, transmit a restoration control command to the external device through the communication interface to restore the reduced volume of audio output by the external device from the second volume.

[0010] The at least one processor may execute the at least one command to detect the occurrence of the event while playing the content, and reduce the audio volume of the content output through the speaker while a voice notifying the occurrence of the event is output through the speaker.

[0011] The at least one processor may transmit information including a time point at which a voice notifying the occurrence of the event is output to the external device through the communication interface, so as to reduce the volume of audio output by the external device at a time point at which a voice notifying the occurrence of the event is output, by executing the at least one command.

[0012] A method for controlling an electronic device according to one or more embodiments of the present disclosure includes: playing content at a first volume; receiving a first control command from an external device for reducing an audio volume of the content while the external device outputs guide audio for guiding a user's input; reducing the audio volume of the content based on the first control command; receiving a second control command from the external device for restoring the audio volume of the content to the first volume; and restoring the audio volume of the content to the first volume based on the second control command.

[0013] The above control method may further include a step of receiving information including a point in time at which the audio volume of the content decreases from the external device; and a step of decreasing the audio volume of the content at the point in time.

[0014] The above control method may further include a step of outputting audio for an operation of the electronic device corresponding to the third control command when receiving a third control command from the external device.

[0015] The above control method may further include a step of reducing the audio volume of the restored content when a control command is received from the external device while the external device outputs additional guide audio guiding additional user input.

[0016] The above control method may further include a step of executing an application corresponding to the at least one text when receiving a user input including at least one text and a third control command for controlling the operation of the electronic device.

[0017] The above control method may further include a step of detecting the occurrence of an event; a step of transmitting a reduction control command to the external device to reduce the volume of audio output by the external device to a second volume while outputting a voice notifying the occurrence of the event; and a step of transmitting a restoration control command to the external device to restore the reduced volume of audio output by the external device from the second volume when output of the voice notifying the occurrence of the event is completed.

[0018] The above control method may further include a step of reducing the audio volume of the content output through the speaker while a voice notifying the occurrence of the event is output.

[0019] The above control method may further include a step of outputting audio for an operation of the electronic device corresponding to the third control command when receiving a third control command for controlling an operation of the electronic device from the external device.

[0020] The control method may further include a step of reducing the volume of the restored content and outputting audio for an operation of the electronic device when receiving a control command from the external device to reduce the audio volume of the restored content while the external device outputs additional guide audio guiding additional user input.

[0021] The method may further include the step of executing an application corresponding to the at least one text upon receiving a user input for inputting at least one text and a third control command for controlling the operation of the electronic device.

[0022] The above control method may further include a step of detecting the occurrence of an event, a step of transmitting a control command to the external device for reducing the volume of audio output by the external device while outputting a voice notifying the occurrence of the event through the speaker, and a step of transmitting a control command to the external device for restoring the reduced volume of audio output by the external device when output of the voice notifying the occurrence of the event through the speaker is completed.

[0023] The above control method may further include a step of reducing the volume of audio corresponding to the content while a voice notifying the occurrence of the event is output through the speaker.

[0024] The above control method may further include a step of transmitting information about the time at which a voice notifying the occurrence of the event is output to the external device, in order to reduce the volume of audio output by the external device from the time at which a voice notifying the occurrence of the event is output.

[0025] A non-transitory computer-readable recording medium comprising a program for executing a control method of an electronic device according to one or more embodiments of the present disclosure, wherein the control method comprises the steps of: playing content; receiving a first control command from an external device for reducing an audio volume of the content while the external device outputs guide audio for guiding a user's input; reducing and outputting the audio volume of the content in response to the first control command; receiving a second control command from the external device for restoring the reduced audio volume of the content; and restoring and outputting the audio volume of the content in response to the second control command.

[0026] The step of receiving the first control command from the external device may include receiving information from the external device about a point in time when the volume of audio output by the external device decreases, and the step of reducing and outputting the audio volume of the content in response to the first control command may include reducing the audio volume of the content from a point in time when the volume of audio output by the external device decreases.

[0027] The above control method may further include a step of outputting audio for an operation of the electronic device corresponding to the third control command when receiving a third control command for controlling an operation of the electronic device from the external device.

[0028] The control method may further include a step of reducing the volume of the restored content and outputting audio for an operation of the electronic device when receiving a control command from the external device to reduce the audio volume of the restored content while the external device outputs additional guide audio guiding additional user input.

[0029] FIG. 1 is a diagram illustrating the operation of a first device and a second device according to one or more embodiments of the present disclosure.

[0030] FIG. 2 is a block diagram illustrating a configuration of a first device according to one or more embodiments of the present disclosure.

[0031] FIG. 3 is a block diagram illustrating a configuration of a second device according to one or more embodiments of the present disclosure.

[0032] FIG. 4 is a diagram illustrating a method for a second device to control a first device according to one or more embodiments of the present disclosure.

[0033] FIG. 5 is a drawing for explaining a UI displayed by a second device according to one or more embodiments of the present disclosure.

[0034] FIG. 6 is a diagram illustrating a mapping relationship between a user input and an operation of a second device according to one or more embodiments of the present disclosure.

[0035] FIG. 7 is a sequence diagram illustrating a method for a second device to control a first device according to one or more embodiments of the present disclosure.

[0036] FIG. 8 is a diagram illustrating a method for controlling the volume of audio over time by a first device and a second device according to one or more embodiments of the present disclosure.

[0037] FIG. 9 is a diagram illustrating a method for a first device to control a second device after obtaining a plurality of second user inputs according to one or more embodiments of the present disclosure.

[0038] FIG. 10 is a diagram illustrating a method for controlling the volume of audio over time by a first device and a second device according to one or more embodiments of the present disclosure.

[0039] FIG. 11 is a sequence diagram illustrating a method for a first device to control the volume of a second device according to one or more embodiments of the present disclosure.

[0040] FIG. 12 is a diagram illustrating a method for controlling the volume of audio by a first device and a second device according to one or more embodiments of the present disclosure.

[0041] FIG. 13 is a sequence diagram illustrating a method for a first device to control the volume of a second device according to one or more embodiments of the present disclosure.

[0042] FIG. 14 is a diagram illustrating a method for controlling the volume of audio by a first device and a second device according to one or more embodiments of the present disclosure.

[0043] FIG. 15 is a drawing for explaining a control method of a first device according to one or more embodiments of the present disclosure.

[0044] The present embodiments may be modified and have various embodiments. Therefore, specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the scope to specific embodiments, but should be understood to encompass various modifications, equivalents, and / or alternatives of the embodiments of the present disclosure. In connection with the description of the drawings, similar reference numerals may be used for similar components.

[0045] In describing the present disclosure, if it is determined that a specific description of a related known function or configuration may unnecessarily obscure the gist of the present disclosure, a detailed description thereof will be omitted.

[0046] Additionally, the following embodiments may be modified in various other forms, and the scope of the technical concepts of the present disclosure is not limited to the following embodiments. Rather, these embodiments are provided to further faithfully and completely convey the technical concepts of the present disclosure to those skilled in the art.

[0047] The terminology used in this disclosure is for the purpose of describing specific embodiments only and is not intended to limit the scope of the rights. Singular expressions include plural expressions unless the context clearly dictates otherwise.

[0048] In this disclosure, expressions such as “has,” “can have,” “includes,” or “may include” indicate the presence of a corresponding feature (e.g., a component such as a number, function, operation, or part), and do not exclude the presence of additional features.

[0049] In this disclosure, expressions such as “A or B,” “at least one of A and / or B,” or “one or more of A or / and B” can include all possible combinations of the listed items. For example, “A or B,” “at least one of A and B,” or “at least one of A or B” can all refer to (1) including at least one A, (2) including at least one B, or (3) including both at least one A and at least one B.

[0050] The expressions “first,” “second,” “first,” or “second,” etc., used in this disclosure can describe various components, regardless of order and / or importance, and are only used to distinguish one component from another, but do not limit the components.

[0051] When it is said that a component (e.g., a first component) is “(operatively or communicatively) coupled with / to” or “connected to” another component (e.g., a second component), it should be understood that said component may be directly coupled to said other component, or may be coupled via another component (e.g., a third component).

[0052] On the other hand, when it is said that a component (e.g., a first component) is "directly connected" or "directly connected" to another component (e.g., a second component), it can be understood that no other component (e.g., a third component) exists between said component and said other component.

[0053] The expression "configured to" used in the present disclosure may be used interchangeably with, for example, "suitable for," "having the capacity to," "designed to," "adapted to," "made to," or "capable of." The term "configured to" may not necessarily mean only "specifically designed to" in terms of hardware.

[0054] Instead, in some contexts, the phrase "a device configured to" may mean that the device, in conjunction with other devices or components, is "capable of" performing A, B, and C. For example, the phrase "a processor configured (or set) to perform A, B, and C" may refer to a dedicated processor (e.g., an embedded processor) for performing those operations, or a general-purpose processor (e.g., a CPU or application processor) that can perform those operations by executing one or more software programs stored in a memory device.

[0055] In the embodiments, a 'module' or 'part' performs at least one function or operation, and may be implemented as hardware or software, or as a combination of hardware and software. Furthermore, a plurality of 'modules' or 'parts' may be integrated into at least one module and implemented as at least one processor, except for a 'module' or 'part' that needs to be implemented as a specific hardware.

[0056] Meanwhile, the various elements and areas in the drawings are schematically drawn. Therefore, the technical concept of the present invention is not limited by the relative sizes or spacing depicted in the attached drawings.

[0057] Hereinafter, with reference to the attached drawings, embodiments according to the present disclosure will be described in detail so that a person having ordinary knowledge in the technical field to which the present disclosure pertains can easily implement the present disclosure.

[0058] FIG. 1 is a diagram illustrating the operation of a first device and a second device according to one or more embodiments of the present disclosure.

[0059] Referring to FIG. 1, a first device (100) can communicate with a second device (200).

[0060] In the present disclosure, the first device (100) may be referred to as an “electronic device” and the second device (200) may be referred to as an “external device.” Alternatively, the first device (100) may be referred to as an “external device” and the second device (200) may be referred to as an “electronic device.”

[0061] The second device (200) may be a device that controls the first device (100). The first device (100) may be implemented as a display device capable of playing content, such as a TV. The second device (200) may be implemented as a smartphone running a remote control application for remotely controlling the first device (100).

[0062] In the present disclosure, “remote control application” may be replaced with terms of the same / similar concept, such as “remote control function” or “remote control application.”

[0063] A remote control application running on a second device (200) can display a UI including a plurality of UI elements for controlling the first device (100).

[0064] The second device (200) can obtain user input through the UI. Here, the user input may be a touch input by touching, dragging, tapping, or swiping the display of the second device (200) with at least one finger.

[0065] In the present disclosure, “user input” may be replaced with terms of the same / similar concept, such as “user interaction”, “user touch”, “user touch input”, “user drag”, “user drag input”, “user swipe”, “user swipe input”, “user gesture”, “user gesture input”, “user touch gesture”, “user touch gesture input”, “user voice” or “user voice input”.

[0066] The second device (200) can perform an operation to support the control of a remote control application by a visually impaired person so that the visually impaired person can use the remote control application.

[0067] The second device (200) can output guide audio through the speaker to guide user input. Specifically, the second device (200) can output guide audio including information about the displayed UI. Alternatively, the second device (200) can output guide audio including information about a UI element activated or selected by the user. Alternatively, the second device (200) can output guide audio including information displayed on a screen for receiving user input. Accordingly, the user can manipulate the UI according to the user's intention through the guide audio without having to look at the screen being displayed by the second device (200).

[0068] For example, if a user selects a UI element to control a channel, the second device (200) may output guide audio that says “channel.”

[0069] In the present disclosure, “guide audio” may be replaced with terms of the same / similar concept, such as “guide audio,” “guide voice,” “voice feedback,” “feedback audio,” “talkback,” or “talkback audio.”

[0070] When the second device (200) outputs guide audio, the first device (100) may be outputting separate audio, such as audio of the content being played, at the first volume. In this case, a problem may arise where the guide audio output by the second device (200) cannot be heard smoothly due to the audio of the content being output by the first device (100).

[0071] Accordingly, while the second device (200) outputs the guide audio, the second device (200) can control the first device (100) to output the audio volume of the content at a reduced level.

[0072] And, when the output of the guide audio is completed, the second device (200) can control the first device (100) to restore the audio volume of the content to the first volume and output it.

[0073] The configuration and specific operation of the first device (100) and the second device (200) will be described with reference to the drawings below.

[0074] FIG. 2 is a block diagram illustrating a configuration of a first device according to one or more embodiments of the present disclosure.

[0075] Referring to FIG. 2, the first device (100) may include a first memory (110), a first communication interface (120), a first display (130), a first user interface (140), a first speaker (150), a first sensor (160), and a first processor (170). Some of the above components may be omitted, and the first device (100) may further include other components in addition to the above components.

[0076] The first memory (110) can store at least one instruction regarding the first device (100). The first memory (110) can store an operating system (O / S) for operating the first device (100). In addition, the first memory (110) can store various software programs or applications for operating the first device (100) according to various embodiments of the present disclosure. In addition, the first memory (110) can include a semiconductor memory such as a flash memory or a magnetic storage medium such as a hard disk.

[0077] According to one or more embodiments of the present disclosure, the first memory (110) may store information about content to be played by the first device (100). In this case, the information about the content may include a video signal and an audio signal for playing the content.

[0078] According to one or more embodiments of the present disclosure, the first memory (110) may store at least one command for controlling audio output by the first device (100) while the second device (200) outputs guide audio.

[0079] The first communication interface (120) includes circuitry and is configured to communicate with external devices and servers. The first communication interface (120) can communicate with external devices or servers based on wired or wireless communication methods. The first communication interface (120) may include a Bluetooth module (not shown), a Wi-Fi module (not shown), an IR (infrared) module, a LAN (Local Area Network) module, an Ethernet module, and the like. Here, each communication module may be implemented in the form of at least one hardware chip. In addition to the above-described communication method, the wireless communication module may include at least one communication chip that performs communication according to various wireless communication standards such as Zigbee, USB (Universal Serial Bus), MIPI CSI (Mobile Industry Processor Interface Camera Serial Interface), 3G (3rd Generation), 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution), LTE-A (LTE Advanced), 4G (4th Generation), 5G (5th Generation), etc. However, this is only one or more embodiments, and the first communication interface (120) may use at least one communication module among various communication modules.

[0080] According to one or more embodiments of the present disclosure, the first communication interface (120) can communicate with the second device (200) via the wireless communication standard described above. The first communication interface (120) can receive a control command for controlling the operation of the first device (100) from the second device (200). The first communication interface (120) can transmit a control command for controlling the operation of the second device (200) to the second device (200).

[0081] The first display (130) may be implemented as a display in various forms, such as a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display panel (PDP), etc. The first display (130) may also include a driving circuit, a backlight unit, etc., which may be implemented in forms such as an a-si TFT (amorphous silicon thin film transistor), an LTPS (low temperature poly silicon) TFT, an OTFT (organic TFT), etc. Meanwhile, the first display (130) may be implemented as a touch screen combined with a touch sensor, a flexible display, a three-dimensional display (3D display, three-dimensional dispaly), etc. In addition, the first display (130) according to one or more embodiments of the present disclosure may include a bezel that houses the display panel as well as a display panel that outputs an image. In particular, the bezel according to one or more embodiments of the present disclosure may include a touch sensor (not shown) for detecting user interaction.

[0082] According to one or more embodiments of the present disclosure, the first display (130) can display a video signal of the content.

[0083] The first user interface (140) may be implemented as a device such as a button, a touch pad, a mouse, and a keyboard, or as a touch screen capable of performing the aforementioned display and operation input functions. Here, the button may be a mechanical button, a touch pad, a wheel, or any other type of button formed on any area of ​​the front, side, or back of the main body of the first device (100).

[0084] The first speaker (150) is configured to output an audio signal. In particular, the first speaker (150) may include an audio output mixer, an audio signal processor, and an audio output module. The audio output mixer may synthesize a plurality of audio signals to be output into at least one audio signal. For example, the audio output mixer may synthesize an analog audio signal and another analog audio signal (e.g., an analog audio signal received from an external source) into at least one analog audio signal. The audio output module may include a speaker or an output terminal.

[0085] According to one or more embodiments of the present disclosure, the first speaker (150) may output audio of the content. In response to a control command received from the second device (200), the first speaker (150) may output audio of the content at a first volume or at a second volume lower than the first volume.

[0086] The first sensor (160) can detect the operating status or external environment of the first device (100) and generate an electrical signal or data value corresponding to the detected status. The first sensor (160) can include an image sensor, an acceleration sensor, a microphone sensor, or the like.

[0087] According to one or more embodiments of the present disclosure, a microphone sensor included in the first sensor (160) can detect a user's speech and perform an action corresponding to the user's speech.

[0088] The first processor (170) can control the overall operation and function of the first device (100). Specifically, the first processor (170) is connected to the configuration of the first device (100) including the first memory (110), and can control the overall operation of the first device (100) by executing at least one command stored in the first memory (110) as described above.

[0089] The first processor (170) may include one or more processors. Specifically, the one or more processors may include one or more of a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), an Accelerated Processing Unit (APU), a Many Integrated Core (MIC), a Digital Signal Processor (DSP), a Neural Processing Unit (NPU), a Main Processing Unit (MPU), a hardware accelerator, or a machine learning accelerator. The one or more processors may control one or any combination of other components of the electronic device, and may perform operations related to communication or data processing. The one or more processors may execute one or more programs or instructions stored in the first memory (110). For example, the one or more processors may perform a method according to one or more embodiments of the present disclosure by executing one or more instructions stored in the first memory (110).

[0090] The first processor (170) can control the configuration included in the first device (100) and control the operation of the first device (100) according to the present disclosure.

[0091] Specifically, the first processor (170) can control the first display (130) to display a screen of content and control the first speaker (150) to output audio of the content.

[0092] The first processor (170) can control the first speaker (150) to output the reduced audio volume of the content. The first processor (170) can control the first speaker (150) to restore the reduced audio volume of the content and output it.

[0093] The first processor (170) can control the first communication interface (120) to transmit a control command for controlling the operation of the second device (200) to the second device (200). In addition, the first processor (170) can receive a control command for controlling the operation of the first device (100) from the second device (200) through the first communication interface (120).

[0094] FIG. 3 is a block diagram illustrating a configuration of a second device according to one or more embodiments of the present disclosure.

[0095] Referring to FIG. 3, the second device (200) may include a second memory (210), a second communication interface (220), a second user interface (240), a second display (230), a second speaker (250), a second sensor (260), and a second processor (270).

[0096] The second memory (210) can store at least one instruction regarding the second device (200). The second memory (210) can store an operating system (O / S) for driving the second device (200). In addition, the second memory (210) can store various software programs or applications for the second device (200) to operate according to various embodiments of the present disclosure. In addition, the second memory (210) can include a semiconductor memory such as a flash memory or a magnetic storage medium such as a hard disk.

[0097] According to one or more embodiments of the present disclosure, the second memory (210) may store a remote control application capable of controlling the first device (100). Furthermore, the second memory (210) may store a keyboard application for obtaining text input from a user. Here, the keyboard application may be a Braille keyboard application supporting Braille functionality. For example, a user may input six-dot Braille using six fingers via the Braille keyboard application.

[0098] According to one or more embodiments of the present disclosure, the second memory (210) may store at least one command for controlling audio output by the first device (100) while the second device (200) outputs guide audio.

[0099] The second communication interface (220) includes circuitry and is configured to communicate with external devices and servers. The second communication interface (220) can communicate with external devices or servers based on wired or wireless communication methods. The first communication interface (120) may include a Bluetooth module (not shown), a Wi-Fi module (not shown), an IR (infrared) module, a LAN (Local Area Network) module, an Ethernet module, and the like. Here, each communication module may be implemented in the form of at least one hardware chip. In addition to the above-described communication method, the wireless communication module may include at least one communication chip that performs communication according to various wireless communication standards such as Zigbee, USB (Universal Serial Bus), MIPI CSI (Mobile Industry Processor Interface Camera Serial Interface), 3G (3rd Generation), 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution), LTE-A (LTE Advanced), 4G (4th Generation), 5G (5th Generation), etc. However, this is only one or more embodiments, and the second communication interface (220) may use at least one communication module among various communication modules.

[0100] According to one or more embodiments of the present disclosure, the second communication interface (220) may communicate with the first device (100) via the wireless communication standard described above. The second communication interface (220) may transmit a control command to the first device (100) for controlling the operation of the second device (200). Specifically, the second communication interface (220) may transmit a control command to the first device (100) for reducing the volume of audio output by the first device (100) or restoring the reduced volume of audio.

[0101] The second display (230) may be implemented as a display of various forms, such as a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display panel (PDP), etc. The second display (230) may also include a driving circuit, a backlight unit, etc., which may be implemented as a form, such as an a-si TFT (amorphous silicon thin film transistor), an LTPS (low temperature poly silicon) TFT, an OTFT (organic TFT), etc. Meanwhile, the second display (230) may be implemented as a touch screen combined with a touch sensor, a flexible display, a three-dimensional display (3D display, three-dimensional dispaly), etc. In addition, the second display (230) according to one or more embodiments of the present disclosure may include a bezel that houses the display panel as well as a display panel that outputs an image. In particular, the bezel according to one or more embodiments of the present disclosure may include a touch sensor (not shown) for detecting user interaction.

[0102] According to one or more embodiments of the present disclosure, the second display (230) can display a video signal constituting the content.

[0103] According to one or more embodiments of the present disclosure, the second display (230) may display a remote control application screen for controlling the first device (100). The second display (230) may display a UI screen for controlling the first device (100).

[0104] The second user interface (240) may be implemented as a device such as a button, a touch pad, a mouse, and a keyboard, or as a touch screen capable of performing the aforementioned display and operation input functions. Here, the button may be a mechanical button, a touch pad, a wheel, or any other type of button formed on any area of ​​the body exterior of the second device (200), such as the front, side, or back.

[0105] According to one or more embodiments of the present disclosure, a second user interface (240) implemented as a touch screen can obtain a user's touch input.

[0106] The second speaker (250) is configured to output an audio signal. In particular, the second speaker (250) may include an audio output mixer, an audio signal processor, and an audio output module. The audio output mixer may synthesize a plurality of audio signals to be output into at least one audio signal. For example, the audio output mixer may synthesize an analog audio signal and another analog audio signal (e.g., an analog audio signal received from an external source) into at least one analog audio signal. The audio output module may include a speaker or an output terminal.

[0107] According to one or more embodiments of the present disclosure, the second speaker (250) may output audio of the content. The second speaker (250) may output audio of the content at a first volume or at a second volume lower than the first volume.

[0108] According to one or more embodiments of the present disclosure, the second speaker (250) may output guide audio that guides user input.

[0109] The second sensor (260) can detect the operating status or external environment of the second device (200) and generate an electrical signal or data value corresponding to the detected status. The second sensor (260) can include an image sensor, an acceleration sensor, a microphone sensor, or the like.

[0110] According to one or more embodiments of the present disclosure, the acceleration sensor can detect the movement of the second device (200). According to one or more embodiments of the present disclosure, the microphone sensor can detect the user's speech. The microphone sensor can detect the user's speech.

[0111] The second processor (270) can control the overall operation and function of the second device (200). Specifically, the second processor (270) is connected to the configuration of the second device (200) including the second memory (210), and can control the overall operation of the second device (200) by executing at least one command stored in the second memory (210) as described above.

[0112] The second processor (270) may include one or more processors. Specifically, the one or more processors may include one or more of a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), an Accelerated Processing Unit (APU), a Many Integrated Core (MIC), a Digital Signal Processor (DSP), a Neural Processing Unit (NPU), a Main Processing Unit (MPU), a hardware accelerator, or a machine learning accelerator. The one or more processors may control one or any combination of other components of the electronic device, and may perform operations related to communication or data processing. The one or more processors may execute one or more programs or instructions stored in the second memory (210). For example, the one or more processors may perform a method according to one or more embodiments of the present disclosure by executing one or more instructions stored in the second memory (210).

[0113] The second processor (270) can control the configuration included in the second device (200) to control the operation of the first device (100) according to the present disclosure.

[0114] Specifically, the second processor (270) can control the second display (230) to display the screen of the content and control the second speaker (250) to output the audio of the content.

[0115] The second processor (270) can control the second speaker (250) to reduce the audio volume of the content or restore the reduced audio volume of the content to play it back.

[0116] The second processor (270) can control the second communication interface (220) to transmit a control command for controlling the operation of the first device (100) to the first device (100). In addition, the second processor (270) can receive a control command for controlling the operation of the second device (200) from the first device (100) through the second communication interface (220).

[0117] A method of controlling a first device (100) and a second device (200) according to one or more embodiments of the present disclosure will be described with reference to the drawings below.

[0118] FIG. 4 is a diagram illustrating a method for a second device to control a first device according to one or more embodiments of the present disclosure.

[0119] Referring to FIG. 4, the second device (200) may acquire a first user input to initiate a mode for outputting guide audio (S401). At this time, the second device (200) may acquire the first user input while the remote control application is running. Alternatively, if the first user input is acquired while the remote control application is not running, the second device (200) may execute the remote control application.

[0120] The guide audio output mode may be a mode in which the first device (100) outputs guide audio that guides user input.

[0121] In the present disclosure, “guide audio output mode” may be replaced with terms of the same / similar concept, such as “braille remote control mode,” “guide remote control mode,” “braille remote control input mode,” “visually impaired assistance mode,” “talkback mode,” or “accessibility mode.”

[0122] Specifically, the second device (200) can obtain the first user input through an acceleration sensor, a microphone sensor, or a second user interface (240).

[0123] The first user input may be a user input in which a preset movement of the second device (200) is detected. The second device (200) may detect the movement of the second device (200) through the acceleration sensor of the second sensor (260). In addition, the second device (200) may identify whether the detected movement of the second device (200) corresponds to the preset movement. For example, the preset movement may be a movement in which the second device (200) rotates so that the second display (230) faces the opposite direction of the user while the second display (230) is facing the user. In other words, the preset movement may be a movement detected by a gesture in which the user turns the second device (200) over and holds it.

[0124] Alternatively, the first user input may be a user input in which the user utters a specific sentence. The second device (200) may detect the user's utterance through the microphone sensor of the second sensor (260). If the detected user's utterance matches a preset utterance or has a similarity level greater than a preset value, the second device (200) may detect that the first user input has been acquired. For example, if a user utterance such as "Turn on the remote control" is detected, the second device (200) may detect that the first user input has been acquired.

[0125] Alternatively, the first user input may be a preset touch, drag, tap, or swipe input through the second display (230).

[0126] When the first user input is obtained, the second device (200) can start the guide audio output mode (S402).

[0127] When the guide audio output mode starts, the second device (200) can output audio through the second speaker (250) to indicate the start of the guide audio output mode. For example, the second device (200) can output a voice such as “Guide audio output mode has started” through the second speaker (250).

[0128] Meanwhile, the first user input may be acquired while a UI for controlling the first device (100) is displayed on the second display (230). Alternatively, if the first user input is acquired while the UI is not displayed on the second display (230) and the guide audio output mode is started, the second device (200) may display a UI for controlling the first device (100) through the second display (230). The UI displayed by the second device (200) will be described with reference to FIG. 5.

[0129] FIG. 5 is a drawing for explaining a UI displayed by a second device according to one or more embodiments of the present disclosure.

[0130] Referring to FIG. 5, a remote control application running on a second device (200) may display a UI (500) capable of controlling a first device (100). At this time, the UI screen may include a plurality of UI elements capable of controlling the first device (100). Each of the plurality of UI elements may be implemented in the form of an icon that can be selected by a user. When each of the plurality of UI elements is selected, the second device (200) may transmit a control command to the first device (100) to perform an operation corresponding to the selected UI element.

[0131] For example, the UI screen may include a UI element (501) that can turn the power of the first device (100) on and off, a UI element (502a, 502b, 502c, 502d) for inputting a direction key input (up, down, left, right) to the first device (100), a UI element (503) for returning to a previous operation, a UI element (504) for selecting an activated UI element, etc., a UI element (505) for displaying a home screen, a UI element (506) for displaying a menu screen, a UI element (507) for playing or pausing content, a UI element (508, 509) for increasing or decreasing the volume, a UI element (510) for muting the volume, and a UI element (511, 512) for increasing or decreasing a channel. Some of the above-described UI elements may be omitted, and the UI may include other UI elements in addition to the above-described UI elements.

[0132] Referring again to FIG. 4, the second device (200) can obtain a second user input that controls the UI displayed on the second display (230) (S403). At this time, the second user input may be a user input that touches, drags, or swipes the second display (230), but is not limited thereto.

[0133] Specifically, the second device (200) can obtain a second user input that activates one of a plurality of UI elements included in the UI.

[0134] Here, activation can mean a state in which the user is controlling, a state in which the user has selected, a state in which the user is touching, or a state in which additional input from the user can be received.

[0135] In this disclosure, “activation” may be replaced with terms of the same / similar concept, such as “focus state”, “selection state”, “selectable state”, “input-waiting state”, “ready state”, or “executable state”.

[0136] An activated UI element may be visually highlighted on the second display (230). The second device (200) may change at least one of the color, border, and size of the activated UI element.

[0137] Meanwhile, multiple UI elements included in the UI may have a predetermined order. The second user input may be a user input for activating a UI element in the previous order of the currently activated UI element or a user input for activating a UI element in the next order of the currently activated UI element.

[0138] Alternatively, the second user input may be a user input to activate a UI element located above, below, to the left or to the right of the currently activated UI element.

[0139] For example, when a user input is obtained in which the user touches a point on the second display (230), the second device (200) can activate the first UI element located at the point on the second display (230). At this time, when the user touches the point on the second display (230) and drags to the right, the second device (200) can activate the second UI element and deactivate the first UI element. Then, when the user touches the point on the second display (230) and drags to the left, the second device (200) can activate the first UI element again and deactivate the second UI element.

[0140] Alternatively, when a user input is obtained in which the user touches a first point of the second display (230), the second device (200) can activate the first UI element located at the first point. At this time, when a user input is obtained in which the user touches a point of the second display (230) and drags it to another point of the second display (230), the second device (200) can activate the second UI element located at the second point of the second display (230) and deactivate the first UI element located at the first point.

[0141] In response to the second user input, the second device (200) can output guide audio guiding the second user input through the second speaker (250) (S404).

[0142] Specifically, the guide audio may include information about a UI element activated by a second user input. Alternatively, the guide audio may include information about a UI screen being displayed through the second display (230). Alternatively, the guide audio may include information about a UI element being touched by the user.

[0143] For example, when a UI element for raising a channel of a first device (100) is activated, a second device (200) can output a guide audio that says “Raise channel” through a second speaker (250).

[0144] In addition, the second device (200) can obtain a third user input for controlling the first device (100) (S405). Specifically, the second device (200) can obtain a third user input for selecting an activated UI element.

[0145] The third user input may be, but is not limited to, a user input that touches, drags, taps, or swipes a point on the second display (230). For example, the third user input may be a touch input in which the user touches the second display (230) with two fingers and drags to the right.

[0146] Alternatively, the second device (200) may obtain a third user input to control the first device (100) to perform a preset action. At this time, the preset action may include at least one of returning to a previous action (or previous screen), displaying a menu screen, playing / pausing, displaying a home screen, changing channels up / down, increasing / decreasing volume, and muting. At this time, the preset actions may each be mapped to different touch inputs.

[0147] When a third user input is acquired, the second device (200) can transmit a control command corresponding to the third user input to the first device (100) (S406). Specifically, the second device (200) can transmit a control command corresponding to an activated UI element to the first device (100). For example, when a second user input is acquired while the “Channel Up” UI element is activated, the second device (200) can transmit a control command to the first device (100) for raising the channel of the first device (100).

[0148] In response to a received control command, the first device (100) can perform an operation corresponding to the received control command. For example, if a control command to raise the channel of the first device (100) is received, the first device (100) can raise the channel of the content and play it.

[0149] Meanwhile, the operation of the first device (100) corresponding to the received control command can be determined based on the state of the first device (100).

[0150] Specifically, the second device (200) can transmit at least one text to the first device (100) using a keyboard application installed in the second device (200).

[0151] Here, the keyboard application may be a Braille keyboard application accessible to the visually impaired. The Braille keyboard application may be an application that obtains user input, such as text or numbers, through a user interface configured in Braille.

[0152] For example, the second device (200) can input at least one text into the first device (100) through a keyboard application installed on the second device (200). Then, when at least one text is input into the first device (100), and a control command corresponding to the “Confirm” UI element is received, the first device (100) can perform an operation corresponding to the at least one text. At this time, the at least one text may be an abbreviation of a specific text.

[0153] For example, “MO” may be an abbreviation for “MOVIE.” In this case, if a control command is received while the “MO” text is input into the first device (100), the first device (100) may execute the “MOVIE” application corresponding to the “MO” text. In addition, if a control command is received while no text is input, the first device (100) may pause playback of the content or continue playback of the paused content.

[0154] Additionally, “ST” may be an abbreviation for “SLEEP TIMER.” In this case, when a control command is received while “ST” is input to the first device (100), the first device (100) may execute the “SLEEP TIMER” application corresponding to the “ST” text.

[0155] Additionally, the first device (100) can receive a control command from the second device (200) in a state where at least one text has not been input. At this time, the first device (100) can display a home screen or play / pause content being played, but is not limited thereto.

[0156] As described above, when a user input is acquired, the second device (200) can perform an operation corresponding to the user input. Meanwhile, the mapping relationship between the user input and the operation of the second device (200) on the remote control application may correspond to the mapping relationship between the user input and the operation of the second device (200) on the keyboard application installed on the second device (200).

[0157] For example, the mapping relationship between a user input and an operation of a second device (200) on a remote control application and the mapping relationship between a user input and an operation of a second device (200) on a keyboard application installed on the second device (200) may be as shown in FIG. 6.

[0158] FIG. 6 is a diagram illustrating a mapping relationship between a user input and an operation of a second device according to one or more embodiments of the present disclosure.

[0159] For example, referring to FIG. 6, when a user input of turning the device over is detected through an acceleration sensor, the second device (200) can execute a Braille keyboard input mode on a keyboard application and a guide audio output mode on a remote control application.

[0160] Additionally, when a user input of swiping with one finger on the second display (230) is obtained, the second device (200) can perform an action of moving a cursor on a keyboard application, an action of moving an activated UI element on a remote control application, or an action of moving an object selected on the first device (100).

[0161] Additionally, when a user input of swiping right with two fingers on the second display (230) is obtained, the second device (200) can perform an action of submitting the entered text on the keyboard application, an action of selecting an activated UI element on the remote control application, or an action of selecting an object selected on the first device (100).

[0162] That is, the second device (200) can load the mapping relationship of the keyboard application from the second memory (210) and apply it to the mapping relationship of the remote control application. As described above, since the mapping relationship of the keyboard application is also applied to the remote control application, the user can control the operation of the remote control application that is identical / similar to the operation of the keyboard application with the same user input as the keyboard application. Accordingly, a user who is already using the keyboard application can intuitively use the remote control application.

[0163] Meanwhile, when the second device (200) outputs guide audio, the first device (100) may also output audio of the content. In this case, a problem may arise where the guide audio output by the second device (200) is not easily audible to the user. Accordingly, the second device (200) may reduce the audio volume of the content output by the first device (100). This will be described with reference to the drawings below.

[0164] FIG. 7 is a sequence diagram illustrating a method for a second device (200) to control a first device (100) according to one or more embodiments of the present disclosure.

[0165] Referring to FIG. 7, the second device (200) can acquire a first user input (S701). At this time, the operation of the second device (200) may be the same as the operation in S401. As described above, if a preset movement of the second device (200) is detected, the second device (200) can detect that the first user input has been acquired.

[0166] When the first user input is acquired, the second device (200) may initiate a guide audio output mode (S702). At this time, the UI may include multiple UI elements capable of controlling the first device (100), as described with reference to FIG. 5.

[0167] Through the UI displayed on the second display (230), the second device (200) can obtain a second user input for controlling the UI (S703). At this time, the second user input may be an input for activating one of the UI elements included in the UI.

[0168] When a second user input is obtained, the second device (200) can activate one of a plurality of UI elements included in the UI.

[0169] For example, when a user input is obtained by dragging one finger to the right side of the second display (230) while the UI element corresponding to “Channel Up” is activated in the UI, the second device (200) can activate the UI element corresponding to “Channel Down.”

[0170] And, when a UI element capable of controlling a channel of the first device (100) is activated, the second device (200) can receive additional user input to control the channel of the first device (100). Alternatively, when a UI element capable of manipulating the audio volume of the first device (100) is activated, the second device (200) can receive additional user input to control the volume of the first device (100). For example, when a UI element corresponding to “channel down” is activated, the second device (200) can receive additional user input to transmit a control command for lowering the channel of the first device (100) to the first device (100). At this time, the user input for selecting the activated UI element may be an input of touching two fingers on the second display (230) and then swiping to the right.

[0171] Meanwhile, before the second device (200) outputs the guide audio, the first device (100) may be playing content through the first display (130) and the first speaker (150) (S704). At this time, the first device (100) may control the first speaker (150) to output the audio volume of the content at the first volume.

[0172] If the second device (200) outputs guide audio while the first device (100) outputs audio of the content, a problem may arise where the user cannot hear the guide audio smoothly. Accordingly, the second device (200) may perform an operation to control the audio volume of the content being played by the first device (100) to be reduced along with the operation of outputting the guide audio.

[0173] Specifically, the second device (200) can transmit a first control command to the first device (100) to reduce the audio volume of the content through the second communication interface (220) (S705). That is, the first device (100) can receive a first control command to reduce the audio volume of the content and output it from the second device (200) through the first communication interface (120). At this time, the first control command can include at least one of information on the degree to which the volume of the first speaker (150) is reduced and information on the timing of reducing the volume of the first speaker (150).

[0174] In addition, the second device (200) can output guide audio through the second speaker (250) (S706). At this time, the volume of the guide audio may be higher than the volume of the first speaker (150) that the first device (100) is outputting.

[0175] In response to the first control command, the first device (100) can control the first speaker (150) to output the audio volume of the content by reducing it (S707). At this time, if the first control command includes information for reducing the volume to a second level, the first device (100) can reduce the output volume of the first speaker (150) to the second level and play the content. In addition, if the first control command includes information regarding a specific point in time for reducing the audio volume of the content, the first device (100) can reduce the audio volume of the content from the specific point in time and play the content.

[0176] At this time, the period during which the first device (100) plays content with the reduced volume of the first speaker (150) and the period during which the second device (200) outputs guide audio may coincide or correspond.

[0177] When the output of the guide audio is completed, the second device (200) can transmit a second control command to the first device (100) via the second communication interface (220) to restore the reduced volume of the first speaker (150) (S708). That is, the first device (100) can receive the second control command to restore the reduced audio volume of the content from the second device (200) via the first communication interface (120). At this time, the second control command can include information about the time at which the first device (100) restores the volume of the first speaker (150).

[0178] In response to the second control command, the first device (100) can control the first speaker (150) to restore the audio volume of the content and output it (S709). The first device (100) can play the content by restoring the volume of the first speaker (150) from the point in time for restoring the volume of the first speaker (150) included in the second command.

[0179] Thereafter, the second device (200) can obtain a third user input for transmitting a third control command for controlling the operation of the first device (100) to the first device (100) (S710).

[0180] Here, the third user input may be a user input for selecting an active UI element on the UI of the remote control application. For example, the user may select a UI element by swiping two fingers to the right to "select" the active UI element.

[0181] When a third user input is obtained, the second device (200) can transmit a third control command to the first device (100) that controls the operation of the first device (100) (S711).

[0182] In response to the third control command, the first device (100) may perform an action corresponding to the third control command (S712). For example, if the "Channel Up" UI element is activated through a second user input and a third user input is obtained, the first device (100) may play the content of the switched channel.

[0183] In addition, the first device (100) can output an audio notification corresponding to the operation of the first device (100) (S713). Here, the audio notification may include information about the operation of the first device (100). For example, if the operation of the first device (100) is to raise the channel, the first device (100) can output an audio such as “Raise the channel.”

[0184] The change in the volume of audio output by the first device (100) and the second device (200) described with reference to Fig. 7 over time will be described in more detail with reference to Fig. 8.

[0185] FIG. 8 is a diagram illustrating a method for controlling the volume of audio over time by a first device and a second device according to one or more embodiments of the present disclosure.

[0186] Referring to FIG. 8, at time t1, the second device (200) can obtain a second user input.

[0187] At time t2, the second device (200) can transmit a first control command to the first device (100) to reduce the audio volume of the content.

[0188] From time t3, in response to the first control command, the first device (100) can reduce the audio volume of the content and play the content.

[0189] From time t3, the second device (200) can output guide audio from time t3.

[0190] When the output of the guide audio is completed at time t4, the second device (200) can transmit a second control command to the first device (100) to restore the audio volume of the content.

[0191] From time t4, in response to the second control command, the first device (100) can play the content at the audio volume of the restored content. Meanwhile, since the process of transmitting the control command may take some time, the first device (100) can play the content at the audio volume of the restored content from time t4 or a certain amount of time after time t4.

[0192] At time t5, the first device (100) can obtain a third user input to control the first device (100).

[0193] At time t6, the first device (100) can transmit a third control command corresponding to the third user input to the first device (100).

[0194] From the time point t7, in response to the third control command, the first device (100) may perform an action and output an action notification audio notifying that the first device (100) has performed the action.

[0195] From time t7 to time t8, the first device (100) can output motion notification audio through the first speaker (150).

[0196] From time t7 to time t8, the first device (100) can output the audio volume of the content by reducing it.

[0197] From time t8, the first device (100) can restore and output the audio volume of the content.

[0198] Meanwhile, according to one or more embodiments of the present disclosure, the second device (200) may acquire multiple second user inputs at regular intervals. At this time, the second device (200) may output guide audio whenever a second user input is acquired. In addition, whenever the guide audio is output, the second device (200) may control the first device (100) to reduce the audio output by the first device (100). This will be described with reference to FIG. 9.

[0199] FIG. 9 is a diagram illustrating a method for a first device to control a second device after obtaining a plurality of second user inputs according to one or more embodiments of the present disclosure.

[0200] Referring to FIG. 9, the second device (200) can obtain a third user input (S901). The operation of the second device (200) in step S901 may be the same as the operation in step 710 of FIG. 7. Accordingly, operations S701 to S709 prior to step S710 may be performed prior to step S901.

[0201] When a third user input is obtained, the second device (200) can transmit a third control command to the first device (100) to control the operation of the first device (100) (S902).

[0202] The first device (100) can output an operation notification audio at the first volume (S903).

[0203] While outputting the motion notification audio at the first volume, the first device (100) can output the motion notification audio at a second volume that is lower than the first volume.

[0204] Before the motion notification audio output is completed, the second device (200) can obtain a second user input (S904).

[0205] Accordingly, the second device (200) can transmit a first control command to the first device (100) to reduce the audio volume of the first device (100) (S905). Then, the second device (200) can output guide audio (S906).

[0206] In response to the first control command, the first device (100) can output the motion notification audio at a second volume that is reduced from the first volume while the guide audio is output (S907).

[0207] The change in the volume of audio output by the first device (100) and the second device (200) with respect to time, as described with reference to FIG. 9, will be described in more detail with reference to FIG. 10.

[0208] FIG. 10 is a diagram illustrating a method for controlling the volume of audio over time by a first device and a second device according to one or more embodiments of the present disclosure.

[0209] Referring to FIG. 10, at time t1, the second device (200) can obtain a second user input.

[0210] At time t2, the second device (200) can transmit a control command to control the operation of the first device (100) to the second device (200).

[0211] From time t3, the first device (100) can output the motion notification audio at the first volume and output the content at the second volume that is lower than the first volume.

[0212] Before the operation notification audio output of the first device (100) is completed, the second device (200) can obtain the first user input at time t4. Then, the second device (200) can transmit a first control command to the first device (100) to reduce the speaker output volume of the first device (100).

[0213] From time t5, the second device (200) can output guide audio. At this time, the second device (200) can output guide audio at the first volume.

[0214] From time t5 to time t6, the first device (100) can output the motion notification audio by reducing the volume from the first volume to the second volume.

[0215] At time t7, the output of the guide audio may be completed. Once the output of the guide audio is completed, the second device (200) may transmit a second control command to the first device (100) to restore the audio volume of the content.

[0216] Accordingly, the first device (100) can play content with the restored audio volume from the time point t7 or after the time point t7.

[0217] Meanwhile, according to one or more embodiments of the present disclosure, the first device (100) can be controlled by user input without going through the second device (200). Accordingly, the first device (100) can perform an action corresponding to the user input and output an action notification audio.

[0218] At this time, the second device (200) may be outputting audio. When the first device (100) and the second device (200) output audio together, a problem may occur in which the audio output by the first device (100) cannot be heard smoothly.

[0219] Accordingly, the first device (100) can reduce the volume of the second speaker (250) of the second device (200) while the motion notification audio is output. This will be described with reference to FIG. 11.

[0220] FIG. 11 is a sequence diagram illustrating a method for a first device to control the volume of a second device according to one or more embodiments of the present disclosure.

[0221] Referring to FIG. 11, the first device (100) can play the first content (S1101), and the second device (200) can play the second content (S1102).

[0222] The first device (100) can obtain user input for controlling the operation of the first device (100) through the first user interface (140) (S1103).

[0223] The method by which the first device (100) obtains user input may be at least partially the same as the method by which the second device (200) obtains user input.

[0224] The first device (100) can perform an operation corresponding to the acquired user input (S1104). For example, the first device (100) can obtain a user input for connecting the first device (100) and an external device via Bluetooth through the first user interface (140). Accordingly, the first device (100) can perform an operation for connecting to the external device via Bluetooth.

[0225] The first device (100) can transmit a control command to the second device (200) to reduce the speaker volume of the second device (200) (S1105).

[0226] In addition, the first device (100) can output an audio notification indicating the performed action and play the first content at a reduced audio volume (S1106). At this time, the first device (100) can output the first content at a second volume lower than the first volume.

[0227] Meanwhile, the order of the operations of step S1105 and step S1106 is merely one or more embodiments, and the operation of step S1106 may be performed first, or the operations of step S1105 and step S1106 may be performed simultaneously.

[0228] In response to a control command received from the first device (100), the second device (200) can play the second content with a reduced speaker volume (S1107).

[0229] When the motion notification audio output is completed, the first device (100) can transmit a control command to the second device (200) to restore the reduced speaker volume (S1108). Then, the first device (100) can restore the audio volume of the first content and play the first content (S1109).

[0230] In response to a control command received from the first device (100), the second device (200) can restore the reduced speaker volume and play the second content (S1110).

[0231] The change in the volume of audio output by the first device (100) and the second device (200) described with reference to Fig. 11 over time will be described in more detail with reference to Fig. 12.

[0232] FIG. 12 is a diagram illustrating a method for controlling the volume of audio by a first device and a second device according to one or more embodiments of the present disclosure.

[0233] Referring to FIG. 12, at time t1, the first device (100) can play the first content, and the second device (200) can play the second content.

[0234] At time t2, the first device (100) can obtain a user input for controlling the operation of the first device (100). Accordingly, the first device (100) can perform an operation corresponding to the user input.

[0235] At time t3, the first device (100) can transmit a control command to the second device (200) to reduce the volume of audio output by the second device (200).

[0236] From time t4, the first device (100) can output motion notification audio and play the first content with reduced audio volume.

[0237] When the motion notification audio output is completed at time t5, the first device (100) may restore and play the audio volume of the first content and transmit a control command to the second device (200) to restore the audio volume of the second device (200). Alternatively, the first device (100) may transmit a control command to the second device (200) to control the audio volume of the second device (200) to be restored t seconds before the motion notification audio output is completed.

[0238] The second device (200) can restore and play the audio volume of the second content from the time point t5 or a time point after the time point t5.

[0239] According to one or more embodiments of the present disclosure, when the occurrence of a preset event is detected, the first device (100) may output an audio notification signal indicating the occurrence of the event. This will be described in detail with reference to FIG. 13.

[0240] FIG. 13 is a sequence diagram illustrating a method for a first device to control the volume of a second device according to one or more embodiments of the present disclosure.

[0241] Referring to FIG. 13, the first device (100) can play the first content (S1301), and the second device (200) can play the second content (S1302).

[0242] The first device (100) can detect the occurrence of a preset event (S1303).

[0243] Here, the preset event may be an event in which the first device (100) receives notification information from an external device. Specifically, the first device (100) and at least one IoT device may constitute an IoT environment. At this time, the first device (100) may receive notification information from at least one IoT device. For example, the first device (100) may receive information from a refrigerator indicating that the refrigerator door is open.

[0244] When the occurrence of a preset event is detected, the first device (100) may output an event occurrence notification audio (S1304). For example, the first device (100) may output a notification audio that says, "The refrigerator door is open."

[0245] The first device (100) can transmit a control command to the second device (200) to reduce the audio volume of the second content (S1304).

[0246] In addition, the first device (100) can output an event occurrence notification audio and play the first content by reducing its audio volume (S1305). At this time, the first device (100) can output the first content at a second volume lower than the first volume.

[0247] Meanwhile, the order of the operations of step S1304 and step S1305 is merely one or more embodiments, and the operation of step S1305 may be performed first, or the operations of step S1304 and step S1305 may be performed simultaneously.

[0248] In response to a control command received from the first device (100), the second device (200) can reduce the audio volume of the second content and play it back (S1306).

[0249] When the audio output of the event occurrence notification is completed, the first device (100) can transmit a control command to the second device (200) to restore the audio volume of the second content and play it back (S1307). Then, the first device (100) can restore the audio volume of the first content and play the first content (S1308).

[0250] In response to a control command received from the first device (100), the second device (200) can restore the audio volume of the second content and play the second content (S1309).

[0251] The change in the volume of audio output by the first device (100) and the second device (200) described with reference to Fig. 13 over time will be described in more detail with reference to Fig. 14.

[0252] FIG. 14 is a diagram illustrating a method for controlling the volume of audio by a first device and a second device according to one or more embodiments of the present disclosure.

[0253] Referring to FIG. 14, at time t1, the first device (100) can play the first content, and the second device (200) can play the second content.

[0254] At time t2, the first device (100) can detect the occurrence of a preset event.

[0255] At time t3, the first device (100) can transmit a control command to the second device (200) to reduce the audio volume of the second content and play the second content.

[0256] From time t4, the first device (100) can output an event occurrence notification audio and play the first content by reducing the audio volume of the first content.

[0257] When the audio output of the event occurrence notification is completed at time t5, the first device (100) may transmit a control command to the second device (200) to restore the audio volume of the first content and play the first content, and to restore the audio volume of the second content and play the second content. Alternatively, the first device (100) may transmit a control command to the second device (200) to restore the audio volume of the second content t seconds after the event occurrence notification audio output is completed t seconds before the event occurrence notification audio output is completed, and to play the second content.

[0258] The second device (200) can play the second content by restoring the audio volume of the second content from the time point t5 or a time point after the time point t5.

[0259] FIG. 15 is a drawing for explaining a control method of a first device according to one or more embodiments of the present disclosure.

[0260] Referring to FIG. 15, the first device (100) can receive a first control command from the second device (200) to reduce the audio volume of the content while the external device outputs guide audio that guides the user's input (S1510).

[0261] The first device (100) can output the audio volume of the content by reducing it in response to the first control command (S1520).

[0262] Specifically, the first device (100) can receive information from the external device about the point in time when the volume of audio output by the external device decreases, and reduce the audio volume of the content from the point in time when the volume of audio output by the external device decreases.

[0263] The first device (100) can receive a second control command from an external device to restore the audio volume of the reduced content (S1530).

[0264] The first device (100) can restore and output the audio volume of the content in response to the second control command (S1540).

[0265] When the first device (100) receives a third control command for controlling the operation of the electronic device from the external device, it can output audio for the operation of the electronic device corresponding to the third control command.

[0266] The first device (100) may output audio for the operation of the electronic device by reducing the volume of the restored content when receiving a control command from the external device to reduce the audio volume of the restored content while the external device outputs additional guide audio guiding additional user input.

[0267] The first device (100) can execute an application corresponding to the at least one text when receiving a user input for inputting at least one text and a third control command for controlling the operation of the electronic device.

[0268] Meanwhile, the first device (100) can detect the occurrence of an event. Then, the first device (100) can transmit a control command to the external device to reduce the volume of audio output by the external device while outputting a voice notifying the occurrence of the event through the speaker. When the output of the voice notifying the occurrence of the event through the speaker is completed, the first device (100) can transmit a control command to the external device to restore the reduced volume of audio output by the external device.

[0269] The first device (100) can reduce the volume of audio corresponding to the content output through the speaker while a voice notifying the occurrence of the event is output through the speaker.

[0270] The first device (100) can transmit information about the time at which a voice notifying the occurrence of the event is output to the external device in order to reduce the volume of audio output by the external device from the time at which a voice notifying the occurrence of the event is output.

[0271] Although various embodiments have been described above, each embodiment is not necessarily implemented individually, and may be implemented together in a single product by being combined in whole or in part with at least one other embodiment.

[0272] Meanwhile, the terms "part" or "module" used in the present disclosure include units composed of hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A "part" or "module" may be an integrally composed component, a minimum unit performing one or more functions, or a portion thereof. For example, a module may be composed of an application-specific integrated circuit (ASIC).

[0273] Various embodiments of the present disclosure may be implemented as software including instructions stored in a machine-readable storage medium that can be read by a machine (e.g., a computer). The device, which is a device capable of calling instructions stored in the storage medium and operating according to the called instructions, may include an electronic device (100) according to the disclosed embodiments. When the instructions are executed by a processor, the processor may directly or under the control of the processor perform a function corresponding to the instructions using other components. The instructions may include code generated or executed by a compiler or interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, "non-transitory" means that the storage medium does not contain signals and is tangible, but does not distinguish between data being stored semi-permanently or temporarily in the storage medium.

[0274] According to one or more embodiments, the methods according to the various embodiments disclosed herein may be provided as a computer program product. The computer program product may be traded as a commodity between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or online through an application store (e.g., Play Store™). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0275] Each component (e.g., a module or a program) according to various embodiments may be composed of one or more entities, and some of the aforementioned sub-components may be omitted, or other sub-components may be further included in various embodiments. Alternatively or additionally, some components (e.g., a module or a program) may be integrated into a single entity, which may perform the same or similar functions as those performed by each of the respective components prior to integration. Operations performed by a module, program, or other component according to various embodiments may be executed sequentially, in parallel, iteratively, or heuristically, or at least some operations may be executed in a different order, omitted, or other operations may be added.

Claims

1. In electronic devices, speaker; A communication interface for communicating with an external device that controls the electronic device; a memory storing at least one instruction; and At least one processor connected to the memory and controlling the electronic device, The at least one processor executes the at least one instruction, Control the above speaker to play content at the first volume, Receive a first control command from the external device through the communication interface to reduce the audio volume of the content while the external device outputs guide audio that guides the user's input; Control the speaker to reduce the audio volume of the content based on the first control command; Receive a second control command from the external device through the communication interface to restore the audio volume of the content to the first volume; An electronic device that controls the speaker to restore the audio volume of the content to the first volume based on the second control command.

2. In paragraph 1, The at least one processor executes the at least one instruction, Receive information including the point in time at which the audio volume of the above content decreases from the external device through the communication interface, An electronic device that reduces the audio volume of the content at the above point in time.

3. In paragraph 1, The at least one processor executes the at least one instruction, An electronic device that controls the speaker to output audio for an operation of the electronic device corresponding to the third control command when receiving a third control command from the external device through the communication interface.

4. In paragraph 3, The at least one processor executes the at least one instruction, An electronic device that controls the speaker to reduce the audio volume of the restored content when receiving a control command from the external device through the communication interface while the external device outputs additional guide audio guiding additional user input.

5. In paragraph 1, The at least one processor executes the at least one instruction, An electronic device that, when receiving a user input including at least one text and a third control command controlling the operation of the electronic device through the communication interface, executes an application corresponding to the at least one text.

6. In paragraph 1, The at least one processor executes the at least one instruction, Detect the occurrence of an event, A reduction control command for reducing the volume of audio output by the external device to a second volume is transmitted to the external device through the communication interface while outputting a voice notifying the occurrence of the event through the speaker, An electronic device that, when the output of a voice notifying the occurrence of the event is completed through the speaker, transmits a restoration control command to the external device through the communication interface to restore the reduced volume of audio output by the external device from the second volume.

7. In paragraph 6, The at least one processor executes the at least one instruction, Detect the occurrence of the above event while playing the above content, An electronic device that reduces the audio volume of the content output through the speaker while a voice notifying the occurrence of the event is output through the speaker.

8. In paragraph 6, The at least one processor executes the at least one instruction, An electronic device that transmits information including the time at which a voice notifying the occurrence of the event is output to the external device through the communication interface, in order to reduce the volume of audio output by the external device at the time at which a voice notifying the occurrence of the event is output.

9. In a method for controlling an electronic device, Step of playing content at first volume; A step of receiving a first control command from an external device to reduce the audio volume of the content while the external device outputs guide audio that guides the user's input; A step of reducing the audio volume of the content based on the first control command; A step of receiving a second control command from the external device to restore the audio volume of the content to the first volume; and A control method comprising: a step of restoring the audio volume of the content to the first volume based on the second control command.

10. In paragraph 9, The above control method is, A step of receiving information from the external device including a point in time at which the audio volume of the content decreases; and A control method further comprising: a step of reducing the audio volume of the content at the above point in time.

11. In paragraph 9, The above control method is, A control method further comprising: a step of outputting audio for an operation of the electronic device corresponding to the third control command when a third control command is received from the external device; 12. In paragraph 11, The above control method is, A control method further comprising: a step of reducing the audio volume of the restored content when a control command is received from the external device while the external device outputs additional guide audio guiding additional user input.

13. In paragraph 9, The above control method is, A control method further comprising: upon receiving a user input including at least one text and a third control command for controlling the operation of the electronic device, executing an application corresponding to the at least one text.

14. In paragraph 9, The above control method is, A step for detecting the occurrence of an event; A step of transmitting a reduction control command to the external device to reduce the volume of audio output by the external device to a second volume while outputting a voice notifying the occurrence of the above event; and A control method further comprising: when the output of the voice notifying the occurrence of the above event is completed, transmitting a restoration control command to the external device to restore the reduced volume of the audio output by the external device from the second volume; 15. In paragraph 14, The above control method is, A control method further comprising: a step of reducing the audio volume of the content output through the speaker while a voice signaling the occurrence of the event is output.

Images