Method and electronic device for generating audio signal

Abstract

Provided are an electronic device and a method for performing operations thereof, the method comprising the steps of: obtaining size information of a video; obtaining environment information for playback of the video; determining the arrangement of one or more virtual speakers on the basis of the size information and the environment information; and generating an audio signal corresponding to the video on the basis of the arrangement of the one or more virtual speakers.

Description

Method for generating audio signals and electronic device

[0001] The present disclosure relates to a method for generating an audio signal and an electronic device. More specifically, it relates to a method and an electronic device for adjusting the arrangement of a channel layout in which an audio signal is output in correspondence with an output image.

[0002] With the advancement of display technology and the emergence of screens of various sizes, resolutions, and shapes, video and audio synchronization technologies are becoming increasingly important for enhancing the user experience. In particular, modern display devices have evolved beyond simply outputting video to establish themselves as a primary medium for immersive content consumption. The technology to design audio output appropriately based on the video's position, size, and environment is emerging as a core element of the user experience. For instance, when video is played across various screens, realism and immersion can only be provided if the audio is accurately output and positioned to correspond to each screen.

[0003] Audio channel layouts often followed a fixed channel configuration based on established rules. Consequently, there are limitations in adequately responding to diverse screen compositions and varied content formats. In particular, this increases the likelihood of disharmony between audio and video.

[0004] An electronic device according to one embodiment of the present disclosure may include a memory in which a program or one or more instructions are stored, and at least one processor. By having at least one processor execute the program or one or more instructions, either alone or in cooperation, the electronic device may obtain size information of an image. The electronic device may obtain environment information for playing the image. Based on the size information and the environment information, the electronic device may determine the placement of at least one virtual speaker. Based on the placement of at least one virtual speaker, the electronic device may generate an audio signal corresponding to the image.

[0005] A method for an electronic device to generate an audio signal according to one embodiment of the present disclosure may include the step of acquiring size information of an image. The method may include the step of acquiring environment information for playing the image. The method may include the step of determining the placement of at least one virtual speaker based on the size information and the environment information. The method may include the step of generating an audio signal corresponding to the image based on the placement of at least one virtual speaker.

[0006] A computer program according to one embodiment of the present disclosure may be stored on a medium to perform at least one of the embodiments of the disclosed method on a computer.

[0007] FIG. 1 is a conceptual diagram illustrating a system for generating an audio signal according to one embodiment of the present disclosure.

[0008] FIG. 2 is a flowchart illustrating a method for generating an audio signal according to one embodiment of the present disclosure.

[0009] FIG. 3a is a drawing illustrating a method in which an image is played back through a frame within a device according to one embodiment of the present disclosure.

[0010] FIG. 3b is a drawing illustrating a method in which an image is played back through a frame within a device according to one embodiment of the present disclosure.

[0011] FIG. 3c is a drawing illustrating a method in which an image is played back through a frame within a device according to one embodiment of the present disclosure.

[0012] FIG. 3d is a drawing illustrating a method in which an image is played back through a frame within a device according to one embodiment of the present disclosure.

[0013] FIG. 4a is a conceptual diagram illustrating the arrangement of a 5-channel layout according to one embodiment of the present disclosure.

[0014] FIG. 4b is a conceptual diagram illustrating the arrangement of a 7-channel layout according to one embodiment of the present disclosure.

[0015] FIG. 5a is a diagram illustrating the arrangement of a 5-channel layout when an image is output through a device according to one embodiment of the present disclosure.

[0016] FIG. 5b is a diagram illustrating the arrangement of a 5-channel layout when an image is output through a device according to one embodiment of the present disclosure.

[0017] FIG. 6a is a diagram illustrating the arrangement of a 5-channel layout when an image is output through a device according to one embodiment of the present disclosure.

[0018] FIG. 6b is a diagram illustrating the arrangement of a 5-channel layout when an image is output through a device according to one embodiment of the present disclosure.

[0019] FIG. 7a is a diagram illustrating the arrangement of a 7-channel layout when an image is output through a device according to one embodiment of the present disclosure.

[0020] FIG. 7b is a diagram illustrating the arrangement of a 7-channel layout when an image is output through a device according to one embodiment of the present disclosure.

[0021] FIG. 8 is a flowchart illustrating a method for controlling the arrangement of a channel layout according to one embodiment of the present disclosure.

[0022] FIG. 9 is a flowchart illustrating a process of rearranging the channel layout when user input to play an image is obtained according to one embodiment of the present disclosure.

[0023] FIG. 10 is a flowchart illustrating a process of rearranging the channel layout when a sensing value exceeding a threshold value is acquired according to one embodiment of the present disclosure.

[0024] FIG. 11 is a drawing illustrating the binaural rendering and output of an audio signal according to one embodiment of the present disclosure.

[0025] FIG. 12 is a drawing for explaining the configuration of an electronic device that generates an audio signal according to one embodiment of the present disclosure.

[0026] The terms used in this disclosure have been selected to be as widely used and general as possible, taking into account their functions within this disclosure; however, these terms may vary depending on the intent of those skilled in the art, case law, the emergence of new technologies, etc. Additionally, in specific cases, terms have been selected at the applicant's discretion, and in such cases, their meanings will be described in detail in the relevant description of the invention. Therefore, terms used in this disclosure should be defined not merely by their names, but based on their meanings and the overall content of this disclosure.

[0027] Singular expressions may include plural expressions unless the context clearly indicates otherwise. Terms used herein, including technical or scientific terms, may have the same meaning as generally understood by those skilled in the art as described in this specification. Additionally, terms including ordinal numbers, such as "first" or "second," used in this specification may be used to describe various components, but said components should not be limited by said terms. Such terms are used solely for the purpose of distinguishing one component from another.

[0028] When a part of a specification is described as "including" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components. Furthermore, terms such as "...part" or "module" as used in the specification refer to a unit that processes at least one function or operation, and this may be implemented in hardware or software, or as a combination of hardware and software.

[0029] Embodiments are described below with reference to the attached drawings so that those skilled in the art can easily implement them. However, the present disclosure may be embodied in various different forms and is not limited to the embodiments described herein. Furthermore, in order to clearly explain the present disclosure in the drawings, parts unrelated to the explanation have been omitted, and similar parts throughout the specification are denoted by similar reference numerals.

[0030] In this specification, 'channel (speaker) layout' may represent a combination of at least one channel and may specify the spatial arrangement of channels (speakers). Since the channel here is a channel where an audio signal is actually output, it may be referred to as a presentation channel.

[0031] For example, the channel layout may be an XYZ channel layout. Here, X may be the number of surround channels, Y may be the number of subwoofer channels, and Z may be the number of height channels. By the 'channel layout', the spatial location of each of the surround channels, subwoofer channels, and height channels can be specified.

[0032] For example, a ‘channel (speaker) layout’ may include, but is not limited to, a 1.0.0 channel (mono channel) layout, a 2.0.0 channel (stereo channel) layout, a 5.1.0 channel layout, a 5.1.2 channel layout, a 5.1.4 channel layout, a 7.1.0 layout, a 7.1.2 layout, and a 3.1.2 channel layout, and there may be various channel layouts.

[0033] The present disclosure describes the arrangement of surround channels. Accordingly, the channel layout may be an X-channel layout. Here, X may be the number of surround channels. The spatial location of the surround channels may be specified by the 'channel layout'.

[0034] For example, a 'channel (speaker) layout' may include, but is not limited to, a 5-channel layout and a 7-channel layout, and there may be various channel layouts.

[0035] Although the names of the channels specified by the 'channel (speaker) layout' may vary, they will be standardized for the sake of convenience of explanation.

[0036] Based on the spatial location of each channel, the channels of the 'channel (speaker) layout' can be named as follows.

[0037] For example, the first surround channel of a 1.0.0 channel layout may be named Mono Channel. The first surround channel of a 2.0.0 channel layout may be named L2 Channel. The second surround channel may be named R2 Channel.

[0038] Here, "L" indicates a channel located on the left side relative to the listener. "R" indicates a channel located on the right side relative to the listener. "2" indicates a surround channel when there are a total of 2 channels.

[0039] In a 5-channel layout, the first surround channel may be named L5, the second surround channel R5, the third surround channel C, the fourth surround channel Ls5, and the fifth surround channel Rs5. Here, "C" indicates a channel located at the center relative to the listener. "s" signifies a channel located at the side. "L5", "R5", "Ls5", and "Rs5" may also be referred to as "L", "R", "SL", and "SR", respectively. "L" may also be referred to as "FL (Front-Left)". "R" may also be referred to as "FR (Front-Right)". Additionally, "Ls5" may be referred to as "BL (Back-Left)". "Rs5" may also be referred to as "BR (Back-Right)".

[0040] The first surround channel of a 7-channel layout may be named L7 channel, the second surround channel R7 channel, the third surround channel C channel, the fourth surround channel Ls7 channel, the fifth surround channel Rs7 channel, the sixth surround channel Lb7 channel, and the seventh surround channel Rb7 channel. Here, "b" indicates that it is a channel located at the back. "Ls7", "Rs7", "Lb7", and "Rb7" may also be referred to as "SL", "SR", "BL", and "BR", respectively.

[0041] The first surround channel of the 3-channel layout can be named L3 channel, the second surround channel R3 channel, and the third surround channel C channel.

[0042] Here, some channels are named differently depending on the channel layout, but they may represent channels at the same location. For example, the R5 channel and the R7 channel may be channels at the same location. Similarly, the Rs5 channel may be a channel at the same location as the Rs7 channel, or a channel at the same location as the Rb7 channel.

[0043] Meanwhile, various channel names may be used, not limited to the names of the aforementioned channels.

[0044] The names of the channels in the channel layout for the aforementioned layout are summarized in Table 1 below.

[0045] Channel Layout Channel Names 1 Channel Layout Mono 2 Channel Layout L2 / R25 Channel Layout L5 / C / R5 / Ls5 / Rs57 Channel Layout L7 / C / R7 / Ls7 / Rs7 / Lb7 / Rb7

[0046] FIG. 1 is a conceptual diagram illustrating a system for generating an audio signal according to one embodiment of the present disclosure. Referring to FIG. 1, an electronic device (100) may include an azimuth adjustment module (110) and an audio adjustment module (120). A process within the electronic device (100) illustrated in FIG. 1 may be performed by a processor of the electronic device (100).

[0047] In one embodiment, the electronic device (100) can acquire an image. The electronic device (100) can acquire size information of the image by analyzing the image. For example, the electronic device (100) can acquire metadata including the image and data about the image. The metadata may include size information of the image. As another example, the electronic device (100) can acquire pixel data of the image. The electronic device (100) can acquire the number of pixels of the image. The electronic device (100) can acquire size information of the image based on the number of pixels.

[0048] In one embodiment, the size information of the image may include at least one of the width, height, and resolution of the image. For example, the size information of the image may include a ratio including the width and height of the image. As another example, the size information of the image may include the resolution of the image. As a specific example, the size information of the first image may have a resolution of 1920 x 1080 and may include the meaning that it consists of 1920 pixels in the width and 1080 pixels in the height.

[0049] In one embodiment, the electronic device (100) may include a display for displaying an image. The electronic device (100) may display an image through a frame that is part of the display. The image may be output through the entire display. Or, the image may be output through a part of the display. Accordingly, the electronic device (100) may display an image through a frame that is at least part of the display. The frame may include at least part of the display determined for outputting the image.

[0050] In one embodiment, the electronic device (100) may acquire environment information for playing an image. The environment information may include at least one of the size of a frame within a device that outputs an image and the arrangement of the device.

[0051] In one embodiment, the device for outputting the image may be an electronic device (100). The electronic device (100) may output the image and obtain environment information for outputting the image. In one embodiment, the device for outputting the image may be an external device. The electronic device (100) may obtain environment information for playing the image from an external device.

[0052] In one embodiment, the electronic device (100) can obtain the size of a frame for outputting an image. The electronic device (100) can output an image through a frame that is at least part of a display. The electronic device (100) can obtain the width and height of the frame.

[0053] In one embodiment, the electronic device (100) can obtain the orientation of the device. That is, the orientation in which a user views an image displayed through the electronic device (100) can be obtained. The orientation of the device may include a vertical orientation in which the device is positioned vertically and a horizontal orientation in which the device is positioned horizontally. For example, the device may include a first edge and a second edge that is longer than the first edge. The user may adjust the viewing angle of the device so that the first edge of the device faces the floor. Or, the user may adjust the viewing angle of the device so that the first edge of the device faces the wall. The electronic device can obtain information regarding the vertical orientation state in which the device is positioned so that the first edge faces the floor. The electronic device can obtain information regarding the horizontal orientation state in which the device is positioned so that the first edge faces the wall.

[0054] For reference, the distinction between images, devices, and frames, and the arrangement of devices will be explained in detail using the conceptual diagrams of FIGS. 3a to 3d.

[0055] In one embodiment, through steps S10 and S20, the electronic device (100) can obtain image size information and environment information for playing the image. The electronic device (100) can determine the placement of at least one virtual speaker using an azimuth adjustment module (110). The electronic device can determine the placement of at least one virtual speaker based on the size information and environment information.

[0056] In one embodiment, the electronic device (100) may obtain channel layout information. The channel layout information may be information related to audio playback using multiple channels. The electronic device (100) may obtain channel layout information regarding a channel layout to output an audio signal. For example, when outputting an audio signal according to a 5-channel layout (200), the electronic device (100) may obtain channel layout information regarding the 5-channel layout (200). As another example, when outputting an audio signal according to a 7-channel layout, the electronic device (100) may obtain channel layout information regarding the 7-channel layout.

[0057] The channel layout for outputting an audio signal containing channel layout information is not limited to 5-channel layouts and 7-channel layouts, and may include a stereo or surround system having symmetrical left and right channel pairs. The channel layout may include, for example, a 2-channel (stereo channel) layout, a 3-channel (L, R, C) layout, a 4-channel layout (FL, FR, BL, BR), as well as a 9-channel layout including four symmetrical channel pairs excluding the center surround channel (C).

[0058] In one embodiment, channel layout information may include information regarding the arrangement of channels (speakers) arranged according to the channel layout. For example, channel layout information regarding a 5-channel layout (200) may include information regarding the arrangement of the first surround channel (C channel) (210), the second surround channel (L5 channel) (221), the third surround channel (R5 channel) (222), the fourth surround channel (Ls5 channel) (231), and the fifth surround channel (Rs5 channel) (232) of the 5-channel layout (200).

[0059] In one embodiment, the electronic device (100) can determine the placement of at least one virtual speaker adjusted based on channel layout information. For example, the second surround channel (221) is a front angle (θ) from the straight line connecting the first surround channel (210) to the origin where the user of the 5-channel layout (200) is located. front It can be arranged at a position rotated by ). As another example, the fourth surround channel (231) is positioned at a surround angle (θ) from the straight line connecting the first surround channel (210) to the origin where the user of the 5-channel layout (200) is located. surround It can be arranged in a position rotated by )

[0060] In a 5-channel layout, the front angle (θ front ) can be 30°. Surround angle (θ surround ) may be 70°. In one embodiment of the present disclosure, the electronic device (100) has a front angle (θ front ) can be adjusted from 30°. For example, the electronic device (100) has a front angle (θ front By adjusting the angle to 35°, a virtual speaker corresponding to the second surround channel (221) can be placed. As another example, the electronic device (100) can have a surround angle (θ surround By adjusting the angle to 65°, a virtual speaker corresponding to the fourth surround channel (231) can be positioned. The electronic device (100) can determine the positioning of at least one virtual speaker according to a 5-channel layout having an adjusted arrangement.

[0061] However, although the arrangement of virtual speakers having an array adjusted based on a 5-channel layout in FIG. 1 has been described, it is obvious that a 7-channel layout may also be used. The types of channel layouts are merely examples and the present disclosure is not limited thereto.

[0062] In one embodiment, the electronic device (100) can determine the placement of at least one virtual speaker based on size information and environment information.

[0063] In one embodiment, the electronic device (100) can acquire size information. The electronic device (100) can acquire the width and height of the image. As the width of the image becomes relatively longer than the height, the electronic device (100) can change the arrangement of the channel layout so that audio can be provided over a wider width. For example, the electronic device (100) can [require] a front angle (θ front ) and surround angle (θ surround The arrangement of the second to fifth surround channels (221, 222, 231, 232) can be changed so that the ) becomes larger. The electronic device (100) can arrange a plurality of virtual speakers corresponding to the second to fifth surround channels (221, 222, 231, 232) of the changed arrangement.

[0064] In one embodiment, as the vertical length of the image becomes relatively longer than the horizontal length, the electronic device (100) may change the arrangement of the channel layout so that audio can be provided at a higher height. For example, the electronic device (100) has a front angle (θ front ) and surround angle (θ surround The arrangement of the second to fifth surround channels (221, 222, 231, 232) can be changed so that the ) becomes smaller. The electronic device (100) can arrange a plurality of virtual speakers corresponding to the second to fifth surround channels (221, 222, 231, 232) of the changed arrangement.

[0065] In one embodiment, the electronic device (100) can acquire environmental information. The electronic device (100) can acquire the size of a frame within a device to play an image. For example, the electronic device (100) can acquire the size of the device. The electronic device (100) can acquire the size of a frame within the device capable of outputting an image. The electronic device can acquire the width and height of a frame within the device to play an image. The electronic device can acquire the area of ​​a frame within the device to play an image.

[0066] As the area of ​​the frame increases, the electronic device (100) can change the arrangement of the channel layout so that audio can be provided over a wider width. For example, the electronic device (100) has a front angle (θ front ) and surround angle (θ surround The arrangement of the second to fifth surround channels (221, 222, 231, 232) can be changed so that the ) becomes larger. The electronic device (100) can arrange a plurality of virtual speakers corresponding to the second to fifth surround channels (221, 222, 231, 232) of the changed arrangement.

[0067] As the area of ​​the frame decreases, the electronic device (100) can change the arrangement of the channel layout so that audio can be provided in a narrower width. For example, the electronic device (100) has a front angle (θ front ) and surround angle (θ surround The arrangement of the second to fifth surround channels (221, 222, 231, 232) can be changed so that the ) becomes smaller. The electronic device (100) can arrange a plurality of virtual speakers corresponding to the second to fifth surround channels (221, 222, 231, 232) of the changed arrangement.

[0068] In one embodiment, the electronic device (100) may obtain a configuration of devices that output an image. The configuration of the devices may relate to the configuration of the devices when the image within the devices is viewed by a user.

[0069] In one embodiment, the arrangement of the device may include a vertical arrangement of the device and a horizontal arrangement of the device. A vertical arrangement of the device may refer to a state in which the device is arranged in a vertical direction. A vertical arrangement of the device may refer to a direction in which the screen is displayed vertically with a longer length, and the bottom edge may be shorter than the side edge. A horizontal arrangement of the device may refer to a state in which the device is arranged horizontally. A horizontal arrangement of the device may refer to a direction in which the screen is displayed horizontally with a longer length, and the bottom edge may be longer than the side edge.

[0070] In one embodiment, the electronic device (100) can change the arrangement of the channel layout depending on the arrangement of the device.

[0071] When the device is positioned horizontally, the electronic device (100) can change the arrangement of the channel layout so that audio can be provided over a wider width. For example, the electronic device (100) has a front angle (θ front ) and surround angle (θ surround The arrangement of the second to fifth surround channels (221, 222, 231, 232) can be changed so that the ) becomes larger. The electronic device (100) can arrange a plurality of virtual speakers corresponding to the second to fifth surround channels (221, 222, 231, 232) of the changed arrangement.

[0072] When the device is positioned vertically, the electronic device (100) can change the arrangement of the channel layout so that audio can be provided in a narrower width. For example, the electronic device (100) has a front angle (θ front ) and surround angle (θ surroundThe arrangement of the second to fifth surround channels (221, 222, 231, 232) can be changed so that the ) becomes smaller. The electronic device (100) can arrange a plurality of virtual speakers corresponding to the second to fifth surround channels (221, 222, 231, 232) of the changed arrangement.

[0073] In one embodiment, the electronic device (100) can adjust the audio signal based on the placement of virtual speakers determined using an azimuth adjustment module (110). The electronic device (100) can adjust the audio signal using an audio adjustment module (120). By adjusting the audio signal based on the adjusted placement of virtual speakers, the electronic device (100) can provide an effect that sounds as if they were generated from the adjusted placement of virtual speakers.

[0074] In one embodiment, the electronic device (100) can model sound distortion caused by the structure of a human head and ears using a Head-Related Transfer Function (HRTF). The electronic device (100) can modify the audio signal using the HRTF. The electronic device (100) can modify the audio signal so that sound is transmitted from an adjusted arrangement of virtual speakers using the HRTF. The HRTF can model the audio signal using physical changes from an audio signal generated in a specific direction until it reaches both ears. For example, the electronic device (100) can apply physical changes from sound generated from an adjusted arrangement of virtual speakers to both ears to the audio signal using the HRTF. For example, interactions with the head, ears, upper body, etc., can be applied to the audio signal. When a user hears the audio signal modified using the HRTF, they may feel as though they have heard sound generated from an adjusted arrangement of virtual speakers.

[0075] However, the method of using HRTF is merely an example, and the method of modifying an audio signal to output sound according to the adjusted placement of virtual speakers does not limit the technical concept of the present disclosure.

[0076] FIG. 2 is a flowchart illustrating a method for generating an audio signal according to one embodiment of the present disclosure.

[0077] Referring to FIG. 2, in step S210, the electronic device can obtain size information of the image.

[0078] In one embodiment, the electronic device may acquire an image and image size information. The image size information may include the width of the image, the height of the image, the ratio of the height to the width of the image, the number of pixels of the image, or the resolution of the image. For example, the image size information may have a resolution of 1280 x 720.

[0079] In step S220, the electronic device can acquire environment information for playing the image.

[0080] In one embodiment, the environment information may include information regarding a frame within a device for playing an image. The environment information may include information regarding the size of the frame. For example, the environment information may include at least one of the width of the frame, the height of the frame, and the area of ​​the frame.

[0081] In one embodiment, the environment information may include information regarding the placement of the device when playing a video. For example, when playing a video through the device, the user may place the device vertically or horizontally. That is, the user may place the device so that the screen of the device is displayed vertically, or place the device so that the screen of the device is displayed horizontally.

[0082] In step S230, the electronic device can determine the placement of at least one virtual speaker based on size information and environment information.

[0083] In one embodiment, the electronic device may obtain channel layout information. The electronic device may obtain information regarding the general arrangement of the channel layout for outputting an audio signal. For example, a 5-channel layout may be used to output an audio signal. The electronic device may obtain information regarding the general arrangement of the channels (speakers) of the 5-channel layout. The information regarding the general arrangement of the channels (speakers) of the 5-channel layout may include the arrangement of five surround channels.

[0084] As another example, a 7-channel layout may be used to output an audio signal. An electronic device may obtain information regarding the general arrangement of the channels (speakers) of the 7-channel layout. The information regarding the general arrangement of the channels (speakers) of the 7-channel layout may include the arrangement of 7 surround channels.

[0085] In one embodiment, the electronic device may determine the placement of at least one virtual speaker corresponding to at least one of the channels in the channel layout. The electronic device may determine to place the virtual speaker corresponding to a position adjusted from a reference position of at least one of the channels in the channel layout.

[0086] For example, the electronic device can acquire 5-channel layout information. The electronic device can change the arrangement of the second surround channel (L5 channel) of the 5-channel layout. The electronic device can place virtual speakers corresponding to the position of the changed second surround channel (L5 channel).

[0087] In one embodiment, the electronic device can determine the placement of at least one virtual speaker based on image size information.

[0088] As the horizontal length of the video becomes relatively longer than the vertical length, the electronic device may change the arrangement of the channel layout so that audio can be provided over a wider width. The electronic device may decide to place virtual speakers in correspondence with the changed arrangement of the channel layout.

[0089] As the vertical length of the video becomes relatively longer than the horizontal length, the electronic device may change the arrangement of the channel layout so that audio can be provided over a narrower width. The electronic device may decide to place virtual speakers in correspondence with the changed arrangement of the channel layout.

[0090] In one embodiment, the electronic device may determine the placement of at least one virtual speaker based on environment information for playing an image. The electronic device may obtain the size of a frame within the device for playing an image.

[0091] As the frame area increases, the electronic device can change the arrangement of the channel layout so that audio can be provided over a wider width. The electronic device can decide to place virtual speakers in correspondence with the changed arrangement of the channel layout.

[0092] As the frame area decreases, the electronic device can change the arrangement of the channel layout so that audio can be provided over a narrower width. The electronic device can decide to place virtual speakers in correspondence with the changed arrangement of the channel layout.

[0093] In one embodiment, the electronic device may determine the placement of at least one virtual speaker based on environment information for playing an image. The electronic device may obtain the placement of a device that outputs an image. The placement of the device may relate to the placement of the device when the image within the device is viewed by a user. The placement of the device may include a vertical placement of the device and a horizontal placement of the device. A vertical placement of the device may mean a placement in which the screen of the device is displayed vertically. A horizontal placement of the device may mean a placement in which the screen of the device is displayed horizontally.

[0094] When the device is positioned horizontally, the electronic device may change the arrangement of the channel layout so that audio can be provided over a wider width. The electronic device may decide to position virtual speakers in correspondence with the changed arrangement of the channel layout.

[0095] When the device is positioned vertically, the electronic device may change the arrangement of the channel layout so that audio can be provided over a narrower width. The electronic device may decide to position virtual speakers in correspondence with the channel layout of the changed arrangement.

[0096] In step S240, the electronic device can generate an audio signal corresponding to the image based on the arrangement of at least one virtual speaker.

[0097] In one embodiment, the electronic device can modify an audio signal so that sound is generated from a placement of at least one virtual speaker. Accordingly, a user listening to the modified audio signal may experience an illusion that sound is generated from a placement of at least one virtual speaker.

[0098] In one embodiment, the electronic device may apply a Head-Related Transfer Function (HRTF) to an audio signal to model sound distortion caused by the structure of the human head and ears. The electronic device may use the HRTF to modify the audio signal as if sound were generated from the arrangement of at least one virtual speaker. However, the method of modifying the audio signal is merely an example, and the technical concept of the present disclosure is not limited thereto.

[0099] FIG. 3a is a diagram illustrating a method in which an image is played back through a frame within a device according to an embodiment of the present disclosure. In one embodiment, the electronic device (300a) may be a device for outputting an image. That is, the electronic device (300a) may directly output the image (310). In one embodiment, the electronic device may also control the output of the image (310) through an external device (300a). Hereinafter, the device corresponding to reference numeral 300a is described based on the premise that it is the electronic device (300a).

[0100] Referring to FIG. 3a, an image (310) is shown. An electronic device (300a) can acquire the image (310). The electronic device (300a) includes a display and can output the image (310) through at least a part of the display. The electronic device (300a) can output the image (310) through a frame (320a).

[0101] In one embodiment, the electronic device (300a) can obtain an image (310) and size information of the image (310). In one embodiment, the size information of the image (310) may be expressed as resolution or the number of pixels, and the image (310) may be enlarged or reduced according to the size of the frame (320a) in which the image (310) is to be displayed.

[0102] In one embodiment, the electronic device (300a) can output an image (310) through a frame (320a). The electronic device (300a) of FIG. 3a is illustrated in a vertically elongated configuration. As the electronic device (300a) is vertically positioned, the frame (320a) capable of outputting the image (310) may be limited. The electronic device (300a) may determine the size of the frame (320a) such that the frame (320a), which is at least part of the display, has a size large enough to output the image (310). For example, the size of the frame (320a) may be determined such that the horizontal length of the electronic device (300a) matches the horizontal length of the frame (320a).

[0103] FIG. 3b is a drawing illustrating a method in which an image is played back through a frame within a device according to one embodiment of the present disclosure.

[0104] For the sake of convenience of explanation, any parts that overlap with those explained using FIG. 3a are simplified or omitted. Additionally, the electronic device (300b) of FIG. 3b refers to the same device as the electronic device (300a) of FIG. 3a, which is arranged vertically but is arranged horizontally.

[0105] Referring to FIG. 3b, in one embodiment, an electronic device (300b) can output an image (310) through a frame (320b). FIG. 3b illustrates the case where the electronic device (300b) is arranged horizontally. As the electronic device (300b) is arranged horizontally, the frame (320b) capable of outputting the image (310) may be limited. Compared to FIG. 3a, a wider area of ​​the display may be determined as the frame (320b). For example, the ratio of the vertical length to the horizontal length of the electronic device (300b) may match the resolution of the image (310). The electronic device (300b) may determine the size of the frame (320b) such that the frame (320b), which is at least part of the display, has a size large enough to output the image (310). Accordingly, the entire display of the electronic device (300b) can be determined as a frame (320b) that outputs an image (310).

[0106] FIG. 3c is a drawing illustrating a method in which an image is played back through a frame within a device according to one embodiment of the present disclosure.

[0107] For the sake of convenience of explanation, any parts that overlap with those explained using FIG. 3a are simplified or omitted. The electronic device (300a) of FIG. 3a and the electronic device (300c) of FIG. 3c represent different types of electronic devices. In one embodiment, the electronic device (300c) of FIG. 3c may represent a device having the function of being foldable or unfoldable.

[0108] Referring to FIG. 3c, in one embodiment, an electronic device (300c) can output an image (310) through a frame (320c). FIG. 3c illustrates the case where the electronic device (300c) is arranged horizontally. As the electronic device (300c) is arranged horizontally, the frame (320c) capable of outputting the image (310) may be limited. The electronic device (300c) may determine the size of the frame (320c) so that the frame (320c), which is at least part of the display, has a size large enough to output the image (310). For example, the size of the frame (320c) may be determined so that the vertical length of the electronic device (300c) matches the vertical length of the frame (320c).

[0109] FIG. 3d is a drawing illustrating a method in which an image is played back through a frame within a device according to one embodiment of the present disclosure.

[0110] For the sake of convenience of explanation, parts that overlap with those explained using FIGS. 3a and FIGS. 3c are simplified or omitted. Additionally, the electronic device (300d) of FIG. 3d is illustrated with the device unfolded and positioned vertically along the first axis (AX1). In comparison, the electronic device (300c) of FIG. 3c is illustrated with the device folded and positioned horizontally along the first axis (AX1).

[0111] Referring to FIG. 3d, in one embodiment, an electronic device (300d) can output an image (310) through a frame (320d). As the electronic device (300d) is positioned vertically, the frame (320d) capable of outputting the image (310) may be limited. The electronic device (300d) may determine the size of the frame (320d) so that the frame (320d), which is at least part of the display, has a size large enough to output the image (310). For example, the size of the frame (320d) may be determined so that the horizontal length of the electronic device (300d) matches the horizontal length of the frame (320d).

[0112] FIG. 4a is a conceptual diagram illustrating the arrangement of a 5-channel layout according to one embodiment of the present disclosure.

[0113] For the convenience of explanation, parts that overlap with those explained using Figure 1 are simplified or omitted.

[0114] Referring to FIG. 4a, the arrangement of a 5-channel layout (400) is illustrated. The 5-channel layout (400) may include a first surround channel (C channel) (410), a second surround channel (L5 channel) (421), a third surround channel (R5 channel) (422), a fourth surround channel (Ls5 channel) (431), and a fifth surround channel (Rs5 channel) (432). For example, a user located at the origin can listen to sound through the surround channels according to the arrangement of the 5-channel layout (400).

[0115] In one embodiment, the angle formed by the second surround channel (421) with the front direction toward the first surround channel (410) from the origin is the front angle (θ front It can be. Front angle (θ front ) may be an angle for specifying the position of the second surround channel (421). Front angle (θ front ) can likewise specify the location of the third surround channel (422), and the third surround channel (422) can be positioned symmetrically with respect to the second surround channel (421) with respect to the front direction.

[0116] In one embodiment, the angle that the fourth surround channel (431) makes with the front direction is the surround angle (θ surround It can be ). Surround angle (θ surround ) may be an angle for specifying the position of the fourth surround channel (431). Surround angle (θ surround ) can likewise specify the location of the 5th surround channel (432), and the 5th surround channel (432) can be positioned symmetrically with respect to the 4th surround channel (431) with respect to the front direction.

[0117] For example, in a 5-channel layout (400), the front angle (θ front ) can be 30°. Surround angle (θ surround ) may be 70°. In one embodiment of the present disclosure, the electronic device has a front angle (θ front ) and surround angle (θ surround By changing ) from 30° and 70° respectively, the placement of at least one virtual speaker corresponding to the changed surround channels can be determined. The electronic device can generate an audio signal based on the placement of at least one virtual speaker.

[0118] FIG. 4b is a conceptual diagram illustrating the arrangement of a 7-channel layout according to one embodiment of the present disclosure.

[0119] For the convenience of explanation, parts that overlap with those explained using Fig. 4a are simplified or omitted.

[0120] Referring to FIG. 4b, the arrangement of a 7-channel layout (450) is illustrated. The 7-channel layout (450) may include a first surround channel (C channel) (460), a second surround channel (L7 channel) (461), a third surround channel (R7 channel) (462), a fourth surround channel (Ls7 channel) (471), a fifth surround channel (Rs7 channel) (472), a sixth surround channel (Lb7) (481), and a seventh surround channel (Rb7) (482). Through the surround channels according to the arrangement of the 7-channel layout (450), for example, a user located at the origin can listen to the sound.

[0121] In one embodiment, the angle formed by the second surround channel (461) with the front direction toward the first surround channel (460) from the origin is the front angle (θ front It can be. Front angle (θ front ) may be an angle for specifying the position of the second surround channel (461). Front angle (θ front) can likewise specify the location of the third surround channel (462), and the third surround channel (462) can be positioned symmetrically with respect to the second surround channel (461) with respect to the front direction.

[0122] In one embodiment, the angle that the fourth surround channel (471) makes with the front direction is the side angle (θ side It can be. Side angle (θ side ) may be an angle for specifying the position of the fourth surround channel (471). Side angle (θ side ) can likewise specify the location of the 5th surround channel (472), and the 5th surround channel (472) can be positioned symmetrically with respect to the 4th surround channel (471) with respect to the front direction.

[0123] In one embodiment, the angle that the 6th surround channel (481) makes with the front direction is the angle (θ back It can be. Hundred angle (θ back ) may be an angle for specifying the position of the 6th surround channel (481). White angle (θ back ) can likewise specify the location of the 7th surround channel (482), and the 7th surround channel (482) can be positioned symmetrically with respect to the 6th surround channel (481) with respect to the front direction.

[0124] For example, in a 7-channel layout (450), the front angle (θ front ) can be 30°. Side angle (θ side ) can be 90°. 100° (θ back ) may be 45°. In one embodiment of the present disclosure, the electronic device has a front angle (θ front ) and white angle (θ back By changing the angles from 30° and 45°, respectively, the placement of at least one virtual speaker corresponding to the changed surround channels can be determined. In one embodiment, the electronic device has a side angle (θ side) can be maintained at a right angle. The electronic device can generate an audio signal based on the placement of at least one virtual speaker.

[0125] In one embodiment, the front angle (θ) of the 5-channel layout (400) front ) and surround angle (θ surround A configuration for determining the placement of at least one virtual speaker by adjusting ) is a front angle (θ) of a 7-channel layout (450). front ) and white angle (θ back It can correspond to a configuration that determines the placement of at least one virtual speaker by adjusting ).

[0126] FIG. 5a is a diagram illustrating the arrangement of a 5-channel layout when an image is output through a device according to one embodiment of the present disclosure.

[0127] Referring to FIG. 5a, the electronic device (500a) can obtain size information of an image. The electronic device (500a) can obtain environment information for playing the image. The size information of the image may include the resolution of the image and the number of pixels of the image. The environment information may include the size of a frame (520a) within the device for outputting the image and the arrangement of the device. For example, the environment information may include the size of a frame (520a) for outputting the image within the electronic device (500a) and the arrangement of the electronic device (500a).

[0128] The electronic device (500a) can determine the size of the frame (520a) such that the frame (520a), which is at least part of the display, has a size large enough to output an image. As illustrated in FIG. 5a, in one embodiment, the electronic device can determine the size of the frame (520a) such that the horizontal length of the frame (520a) matches the horizontal length of the electronic device (500a) so that the frame (520a) has a size large enough to output an image.

[0129] In one embodiment, the electronic device (500a) can determine the placement of at least one virtual speaker based on image size information and environment information for playing the image. For example, the electronic device (500a) can determine the placement of at least one virtual speaker corresponding to the position of a surround channel according to a 5-channel layout (50a). The electronic device (500a) can determine a front angle specifying the position of a second surround channel (51a) and a surround angle specifying the position of a fourth surround channel (52a) among the 5-channel layout (50a) based on size information and environment information. The electronic device (500a) can determine the placement of one virtual speaker corresponding to the position of a surround channel placed using at least one of the front angle or the surround angle.

[0130] In one embodiment, the electronic device (500a) can obtain size information. The electronic device (500a) can obtain the horizontal length and vertical length of the image. As the horizontal length of the image becomes relatively longer than the vertical length, the electronic device (500a) can change the arrangement of the channel layout so that audio can be provided over a wider width. For example, the electronic device (500a) can change the arrangement of the second surround channel (51a) and the fourth surround channel (52a) so that the front angle and the surround angle become larger. Of course, the third surround channel can be changed to a position symmetrical to the second surround channel (51a). The fifth surround channel can be changed to a position symmetrical to the fourth surround channel (52a). The electronic device (500a) can determine the placement of at least one virtual speaker corresponding to the position of the surround channel according to the five-channel layout (50a) with the changed arrangement.

[0131] In one embodiment, the electronic device (500a) can acquire environmental information. The electronic device (500a) can acquire the size of a frame (520a) for outputting an image. The electronic device (500a) can acquire the area of ​​the frame (520a), which is at least part of the display of the electronic device (500a). The larger the area of ​​the frame (520a), the more the electronic device (500a) can change the arrangement of the channel layout so that audio can be provided over a wider width. For example, the electronic device (500a) can change the arrangement of the second surround channel (51a) and the fourth surround channel (52a) so that the front angle and the surround angle increase. Of course, the third surround channel can be changed to a position symmetrical to the second surround channel (51a). The fifth surround channel can be changed to a position symmetrical to the fourth surround channel (52a). The electronic device (500a) can determine the placement of at least one virtual speaker in correspondence with the position of the surround channel according to the 5-channel layout (50a) that has been changed in placement.

[0132] In one embodiment, the electronic device (500a) can acquire environmental information. The electronic device (500a) can acquire the orientation of the device. The orientation of the device may include a vertical orientation in which the device is arranged lengthwise and a horizontal orientation in which the device is arranged lengthwise.

[0133] When the electronic device (500a) is positioned horizontally, the electronic device (500a) can change the arrangement of the channel layout so that audio can be provided over a wider width. For example, the electronic device (500a) can change the arrangement of the second surround channel (51a) and the fourth surround channel (52a) so that the front angle and the surround angle increase. Of course, the third surround channel can be changed to a position symmetrical to the second surround channel (51a). The fifth surround channel can be changed to a position symmetrical to the fourth surround channel (52a). The electronic device (500a) can determine the placement of at least one virtual speaker corresponding to the position of the surround channel according to the five-channel layout (50a) with the changed arrangement.

[0134] When the electronic device (500a) is positioned vertically, the electronic device (500a) can change the arrangement of the channel layout so that audio can be provided in a narrower width. For example, the electronic device (500a) can change the arrangement of the second surround channel (51a) and the fourth surround channel (52a) so that the front angle and the surround angle become smaller. Of course, the third surround channel can be changed to a position symmetrical to the second surround channel (51a). The fifth surround channel can be changed to a position symmetrical to the fourth surround channel (52a). The electronic device (500a) can determine the placement of at least one virtual speaker corresponding to the position of the surround channels according to the five-channel layout (50a) with the changed arrangement.

[0135] For example, as illustrated in FIG. 5a, when the electronic device (500a) is positioned vertically, the electronic device (500a) may change the arrangement of the channel layout so that audio can be provided in a narrower width. The electronic device (500a) may change the arrangement of the second surround channel (51a) and the fourth surround channel (52a) so that the front angle and the surround angle become smaller. The electronic device (500a) may determine the placement of at least one virtual speaker corresponding to the position of the surround channel according to the 5-channel layout (50a) with the changed arrangement.

[0136] In one embodiment, the front angle and surround angle may be determined to be larger values ​​as the horizontal length of the image becomes relatively longer than the vertical length, as the size of the frame (520a) becomes larger, and as the arrangement of the device becomes horizontal rather than vertical. For example, while mathematical formulas may be used to determine the front angle and surround angle, the degree of proportionality between the 'image size information and environment information' and the 'front angle and surround angle' does not limit the technical concept of the present disclosure.

[0137] For a specific example, the front angle can be calculated according to mathematical formulas 1 to 3.

[0138]

[0139] In mathematical equation 1, θ front can be the front angle. θ front θ may be an azimuth angle from the center axis of the second surround channel (L) or the third surround channel (R). front θ can be an azimuth rotated to the left or right relative to the listener. front It may be an azimuth rotated clockwise or counterclockwise relative to the direction in which the listener is facing forward.

[0140] In mathematical equation 1, θ 0, frontcan be the reference front angle. Based on international standards, for example, ITU-R BS.2051, θ 0, front It can be 30°.

[0141] In mathematical formula 1, W screen and W content can be a weight. W screen S that changes the front angle screen It can be a coefficient that determines the degree of influence of. W content R that changes the front angle content It can be a coefficient that determines the degree of influence of. W screen and W content is S screen and R content It can be set by the user by considering how much weight to give to each influence.

[0142] In mathematical formula 1, effect front can be an effective coefficient. effect front is S screen and R content Depending on the value of, it can be set by the user to consider how sensitively the front angle changes.

[0143] In mathematical formula 1, S screen can be a value representing the actual size of the image. For example, S screen It can be explained using mathematical formula 2.

[0144]

[0145] In mathematical equation 2, S, which is the actual size of the image screen can be a function that takes frame area and screen_orientation as variables.

[0146] In mathematical formula 2, Frame area may be the area of ​​a frame. Frame area may be the area of ​​a frame that is at least part of the device's display. For example, Frame area can be calculated as the product of the width and height of the frame.

[0147] In mathematical formula 2, screen_orientation may represent the placement of the device. For example, if the device is placed horizontally, screen_orientation may have a value of +1. For example, if the device is placed vertically, screen_orientation may have a value of -1.

[0148] In mathematical formula 2, the default area may be a frame area set as a standard. For example, the default area may be a value of 160cm * 90cm, but is not limited thereto. The default area may be introduced merely to explain the relative change (increase or decrease) of the frame area and does not limit the technical concept of the present disclosure.

[0149] In mathematical formula 2, s1 and s2 can be coefficients that are arbitrarily set. s1 and s2 can be set within a positive range.

[0150] In one embodiment, through Equation 2, the actual size of the image may have a larger value as the frame area increases or when the device is positioned horizontally. Accordingly, through Equation 1, the front angle may be larger as the actual size of the image increases.

[0151] In mathematical formula 1, R content can be a value representing the aspect ratio of the content (video). For example, R content It can be explained using mathematical formula 3.

[0152]

[0153] In mathematical formula 3, R, the screen ratio of the content (video).content It can be a function that takes content width and content height as variables.

[0154] In mathematical formula 3, content width can be the width of the image, i.e., the horizontal length. Content height can be the height of the image, i.e., the vertical length.

[0155] In mathematical formula 3, r1 and r2 can be coefficients that are arbitrarily set. r1 and r2 can be set within a positive range.

[0156] In one embodiment, through Equation 3, the aspect ratio of the image can have a larger value as the horizontal length of the image is greater than the vertical length of the image. Therefore, through Equation 1, the front angle can be larger as the aspect ratio of the image increases.

[0157] In addition, for example, the surround angle can be calculated according to Equations 2 through 4. Since Equations 2 and 3 have already been explained, the explanation will focus on Equation 4.

[0158]

[0159] In mathematical equation 4, θ surround can be the surround angle. θ surround θ may be an azimuth angle from the center axis of the fourth surround channel (SL) or the fifth surround channel (SR). surround It may be an azimuth rotated to the left or right relative to the listener.

[0160] In Equation 4, θ0, surround may be the reference surround angle of a 5-channel layout. Based on international standards, for example, ITU-R BS.2051, θ0, surround may be 70°.

[0161] In mathematical equation 4, W screen and W content can be a weight. W screen S that changes the front angle screenIt can be a coefficient that determines the degree of influence of. W content R that changes the front angle content It can be a coefficient that determines the degree of influence of. W screen and W content is S screen and R content It can be set by the user by considering how much weight to give to each influence.

[0162] In mathematical equation 4, effect surround can be an effective coefficient. Effect surround is S screen and R content Depending on the value, the user can set how sensitively the surround angle changes.

[0163] In mathematical equation 4, S screen can be a value representing the actual size of the image. For example, S screen can be explained using Equation 2. In Equation 4, R content can be a value representing the aspect ratio of the content (video). For example, R content It can be explained using mathematical formula 3.

[0164] In one embodiment, the electronic device (500a) can modify the audio signal as if sound were generated based on the arrangement of at least one virtual speaker determined. For example, the electronic device (500a) can model sound distortion caused by the structure of a human head and ear using HRTF. The electronic device (500a) can modify the audio signal as if sound were generated based on the arrangement of at least one virtual speaker determined using HRTF.

[0165] However, Equations 1 to 4 for calculating the front angle and surround angle are merely examples and do not limit the technical concept of the present disclosure. In addition, regarding a 5-channel layout, the arrangement of channels can be determined exemplarily using Equations 1 to 4, but the angles regarding the arrangement of channels can be calculated in the same way for a 7-channel layout.

[0166] FIG. 5b is a diagram illustrating the arrangement of a 5-channel layout when an image is output through a device according to one embodiment of the present disclosure.

[0167] For the convenience of explanation, parts that overlap with those explained using Fig. 5a are simplified or omitted.

[0168] For reference, the electronic device (500b) of FIG. 5b illustrates an arrangement in which the electronic device (500a) of FIG. 5a is rotated 90°.

[0169] Referring to FIGS. 5a and 5b, the electronic device (500b) can determine the size of the frame (520b) such that the frame (520b), which is at least part of the display, has a size large enough to output an image. As illustrated in FIG. 5b, in one embodiment, the electronic device (500b) can determine the size of the frame (520b) such that the frame (520b) matches the display of the electronic device (500b) so that the frame (520b) has a size large enough to output an image. In FIG. 5b, the size of the electronic device (500b) and the size of the frame (520b) match because the aspect ratio of the image matches the aspect ratio of the device, i.e., the electronic device (500b). Thus, the size of the frame (520b) can be determined so that the image fills the display of the electronic device (500b).

[0170] In one embodiment, the electronic device (500b) can determine the placement of at least one virtual speaker based on image size information and environment information for playing the image. For example, the electronic device (500b) can determine the placement of at least one virtual speaker corresponding to the position of the surround channel according to the 5-channel layout (50b) in which the placement has been changed.

[0171] In one embodiment, the area of ​​the frame (520b) illustrated in FIG. 5b may be larger than the area of ​​the frame (520a) illustrated in FIG. 5a. The electronic device (500b) may obtain the size of the frame (520b) for outputting an image. The electronic device (500b) may obtain the area of ​​the frame (520b) which is at least part of the display of the electronic device (500b). The larger the area of ​​the frame (520b), the more the electronic device (500b) may change the arrangement of the channel layout so that audio can be provided over a wider width. Accordingly, in the embodiment of FIG. 5b, the electronic device (500b) may change the arrangement of the second surround channel (51b) and the fourth surround channel (52b) so that the front angle and the surround angle are larger than in the embodiment of FIG. 5a. Conversely, in the embodiment of FIG. 5a, the electronic device (500a) can change the arrangement of the second surround channel (51a) and the fourth surround channel (52a) so that the front angle and the surround angle become smaller than in the embodiment of FIG. 5b.

[0172] Of course, the third surround channel may be repositioned to a position symmetrical to the second surround channel (51b). The fifth surround channel may be repositioned to a position symmetrical to the fourth surround channel (52b). The electronic device (500b) can determine the placement of at least one virtual speaker corresponding to the position of the surround channel according to the repositioned 5-channel layout (50b).

[0173] In one embodiment, there may be a standard for arranging the surround channels of a 5-channel layout. For example, a general standard for a 5-channel layout may be that the second surround channel (51b) and the third surround channel are arranged so that the front angle is 30°, and the fourth surround channel (52b) and the fifth surround channel are arranged so that the surround angle is 75°. The electronic device (500b) can adjust the arrangement of the surround channels so that the front angle and the surround angle increase as the area of ​​the frame (520b) increases. The electronic device (500b) can determine the arrangement of at least one virtual speaker corresponding to the position of the surround channels of the adjusted arrangement.

[0174] In one embodiment, the electronic device (500a) shown in FIG. 5a is positioned vertically, whereas the electronic device (500b) shown in FIG. 5b is positioned horizontally. When the electronic device (500b) is positioned horizontally as in FIG. 5b, the electronic device (500b) can change the arrangement of the channel layout so that audio can be provided over a wider width. For example, in the embodiment of FIG. 5b, the electronic device (500b) can change the arrangement of the second surround channel (51b) and the fourth surround channel (52b) so that the front angle and the surround angle are larger than in the embodiment of FIG. 5a. Conversely, if the electronic device (500a) is positioned vertically as in the embodiment of FIG. 5a, the electronic device (500a) can change the arrangement of the second surround channel (51a) and the fourth surround channel (52a) so that the front angle and the surround angle become smaller than in the embodiment of FIG. 5b.

[0175] Of course, the third surround channel may be repositioned to a position symmetrical to the second surround channel (51b). The fifth surround channel may be repositioned to a position symmetrical to the fourth surround channel (52b). The electronic device (500b) can determine the placement of at least one virtual speaker corresponding to the position of the surround channel according to the repositioned 5-channel layout (50b).

[0176] The numerical values ​​of the front angle and surround angle shown in FIGS. 5a and 5b are described as relative values ​​for illustrative purposes only and do not limit the technical concept of the present disclosure.

[0177] FIG. 6a is a diagram illustrating the arrangement of a 5-channel layout when an image is output through a device according to one embodiment of the present disclosure.

[0178] For the sake of convenience, parts that overlap with the explanation using Fig. 5a are simplified or omitted. Additionally, the explanation focuses on the parts that differ from the explanation using Fig. 5a.

[0179] Referring to FIG. 6a, the electronic device (600a) can determine the size of the frame (620a) so that the frame (620a), which is at least part of the display, has a size large enough to output an image. As illustrated in FIG. 6a, in one embodiment, so that the frame (620a) has a size large enough to output an image, the electronic device (600a) can determine the size of the frame (620a) such that the vertical length of the frame (620a) matches the vertical length of the electronic device (600a).

[0180] In one embodiment, the electronic device (600a) can determine the placement of at least one virtual speaker based on image size information and environment information for playing the image. For example, the electronic device (600a) can determine the placement of at least one virtual speaker corresponding to the position of the surround channel according to the 5-channel layout (50c) in which the placement has been changed.

[0181] In one embodiment, the electronic device (600a) can acquire environmental information. The electronic device (600a) can acquire the size of a frame (620a) for outputting an image. The electronic device (600a) can acquire the area of ​​the frame (620a), which is at least part of the display of the electronic device (600a). The larger the area of ​​the frame (620a), the more the electronic device (600a) can change the arrangement of the channel layout so that audio can be provided over a wider width. For example, the electronic device (600a) can change the arrangement of the second surround channel (51c) and the fourth surround channel (52c) so that the front angle and the surround angle increase. Of course, the third surround channel can be changed to a position symmetrical to the second surround channel (51c). The fifth surround channel can be changed to a position symmetrical to the fourth surround channel (52c). The electronic device (600a) can determine the placement of at least one virtual speaker in correspondence with the position of the surround channel according to the 5-channel layout (50c) that has been changed in placement.

[0182] In one embodiment, the electronic device (600a) can acquire environmental information. The electronic device (600a) can acquire the orientation of the device. The orientation of the device may include a vertical orientation in which the device is arranged lengthwise and a horizontal orientation in which the device is arranged lengthwise.

[0183] In one embodiment, the electronic device (600a) shown in FIG. 6a is positioned horizontally. As in FIG. 6a, when the electronic device (600a) is positioned horizontally, the electronic device (600a) can change the arrangement of the channel layout so that audio can be provided over a wider width.

[0184] FIG. 6b is a diagram illustrating the arrangement of a 5-channel layout when an image is output through a device according to one embodiment of the present disclosure.

[0185] For the convenience of explanation, parts that overlap with those explained using Fig. 5a are simplified or omitted.

[0186] For reference, the electronic devices (600a, 600b) of FIGS. 6a and 6b illustrate a device capable of folding and unfolding a display, and the electronic device (600b) of FIG. 6a is shown outputting an image when the display is folded. The electronic device (600b) of FIG. 6b is shown outputting an image when the display is unfolded. In one embodiment, the electronic devices (600a, 600b) of FIG. 6a and 6b can fold and unfold the display along a first axis (AX1).

[0187] Referring to FIG. 6a and FIG. 6b, the electronic device (600b) can determine the size of the frame (620b) such that the frame (620b), which is at least part of the display, has a size large enough to output an image. As shown in FIG. 6b, in one embodiment, so that the frame (620b) has a size large enough to output an image, the electronic device (600b) can determine the size of the frame (620b) such that the horizontal length of the frame (620b) matches the horizontal length of the electronic device (600b).

[0188] In one embodiment, the electronic device (600b) can determine the placement of at least one virtual speaker based on image size information and environment information for playing the image. For example, the electronic device (600b) can determine the placement of at least one virtual speaker corresponding to the position of the surround channel according to the 5-channel layout (50d) in which the placement has been changed.

[0189] In one embodiment, the area of ​​the frame (620b) illustrated in FIG. 6b may be larger than the area of ​​the frame (620a) illustrated in FIG. 6a. The electronic device (600b) may obtain the size of the frame (620b) for outputting an image. The electronic device (600b) may obtain the area of ​​the frame (620b) which is at least part of the display of the electronic device (600b). The larger the area of ​​the frame (620b), the more the electronic device (600b) may change the arrangement of the channel layout so that audio can be provided over a wider width. Accordingly, in the embodiment of FIG. 6b, the electronic device (600b) may change the arrangement of the second surround channel (51d) and the fourth surround channel (52d) so that the front angle and the surround angle are larger than in the embodiment of FIG. 6a. Conversely, in the embodiment of FIG. 6a, the electronic device (600a) can change the arrangement of the second surround channel (51c) and the fourth surround channel (52c) so that the front angle and the surround angle become smaller than in the embodiment of FIG. 6b.

[0190] Of course, the third surround channel may be repositioned to a position symmetrical to the second surround channel (51d). The fifth surround channel may be repositioned to a position symmetrical to the fourth surround channel (52d). The electronic device (600b) can determine the placement of at least one virtual speaker corresponding to the position of the surround channel according to the repositioned 5-channel layout (50d).

[0191] In one embodiment, the electronic device (600a) shown in FIG. 6a is arranged horizontally, whereas the electronic device (600b) shown in FIG. 6b is arranged vertically. When the electronic device (600b) is arranged vertically as in FIG. 6b, the arrangement of the channel layout of the electronic device (600b) can be changed so that audio can be provided in a narrower width.

[0192] However, compared to the electronic device (600a) illustrated in FIG. 6a, the electronic device (600b) illustrated in FIG. 6b has a dominant influence of a large frame area, so the arrangement of the second surround channel (51d) and the fourth surround channel (52d) can be changed so that the front angle and the surround angle become larger than in the embodiment of FIG. 6a. The difference between the influence due to the frame area and the influence due to the arrangement of the device can be explained by s2 in Equation 2. For example, if s2 is set large, the influence due to the arrangement of the device may increase. If s2 is set small, the influence due to the frame area may increase.

[0193] The numerical values ​​of the front angle and surround angle shown in FIGS. 6a and 6b are described as relative values ​​for illustrative purposes only and do not limit the technical concept of the present disclosure.

[0194] FIG. 7a is a diagram illustrating the arrangement of a 7-channel layout when an image is output through a device according to one embodiment of the present disclosure.

[0195] For the convenience of explanation, parts that overlap with those explained using Fig. 5a are simplified or omitted.

[0196] Referring to FIG. 7a, the electronic device (700a) can determine the size of the frame (720a) such that the frame (720a), which is at least part of the display, has a size large enough to output an image. As illustrated in FIG. 7a, in one embodiment, so that the frame (720a) has a size large enough to output an image, the electronic device can determine the size of the frame (720a) such that the horizontal length of the frame (720a) matches the horizontal length of the electronic device (700a).

[0197] In one embodiment, the electronic device (700a) can determine the placement of at least one virtual speaker based on image size information and environment information for playing the image. For example, the electronic device (700a) can determine the placement of at least one virtual speaker corresponding to the position of the surround channel according to the 7-channel layout (70a). The electronic device (700a) determines the front angle that specifies the position of the second surround channel (71a) among the 7-channel layout (70a) and the back angle (θ) that specifies the position of the sixth surround channel (72a). back ) can be determined based on size information and environmental information. The electronic device (700a) can determine the placement of one virtual speaker corresponding to the position of the surround channel placed using at least one of the front angle or back angle.

[0198] In one embodiment, the electronic device (700a) can obtain size information. The electronic device (700a) can obtain the horizontal length and vertical length of the image. As the horizontal length of the image becomes relatively longer than the vertical length, the electronic device (700a) can change the arrangement of the channel layout so that audio can be provided over a wider width. For example, the electronic device (700a) can change the arrangement of the second surround channel (71a) and the sixth surround channel (72a) so that the front angle and back angle become larger. Of course, the third surround channel can be changed to a position symmetrical to the second surround channel (71a). The seventh surround channel can be changed to a position symmetrical to the sixth surround channel (72a). The electronic device (700a) can determine the placement of at least one virtual speaker corresponding to the position of the surround channel according to the changed 7-channel layout (70a).

[0199] In one embodiment, the electronic device (700a) can acquire environmental information. The electronic device (700a) can acquire the size of a frame (720a) for outputting an image. The electronic device (700a) can acquire the area of ​​the frame (720a), which is at least part of the display of the electronic device (700a). The larger the area of ​​the frame (720a), the more the electronic device (700a) can change the arrangement of the channel layout so that audio can be provided over a wider width. For example, the electronic device (700a) can change the arrangement of the second surround channel (71a) and the sixth surround channel (72a) so that the front angle and the surround angle increase. Of course, the third surround channel can be changed to a position symmetrical to the second surround channel (71a). The seventh surround channel can be changed to a position symmetrical to the sixth surround channel (72a). The electronic device (700a) can determine the placement of at least one virtual speaker in correspondence with the position of the surround channel according to the 7-channel layout (70a) that has been changed in placement.

[0200] In one embodiment, the electronic device (700a) can acquire environmental information. The electronic device (700a) can acquire the orientation of the device. The orientation of the device may include a vertical orientation in which the device is arranged lengthwise and a horizontal orientation in which the device is arranged lengthwise.

[0201] When the electronic device (700a) is positioned horizontally as in FIG. 7a, the electronic device (700a) can change the arrangement of the channel layout so that audio can be provided over a wider width. For example, the electronic device (700a) can change the arrangement of the second surround channel (71a) and the sixth surround channel (72a) so that the front angle and back angle are increased. Of course, the third surround channel can be changed to a position symmetrical to the second surround channel (71a). The seventh surround channel can be changed to a position symmetrical to the sixth surround channel (72a). The electronic device (700a) can determine the placement of at least one virtual speaker corresponding to the position of the surround channel according to the changed 7-channel layout (70a).

[0202] When the electronic device (700a) is positioned vertically, the electronic device (700a) can change the arrangement of the channel layout so that audio can be provided in a narrower width. For example, the electronic device (700a) can change the arrangement of the second surround channel (71a) and the sixth surround channel (72a) so that the front angle and back angle become smaller. Of course, the third surround channel can be changed to a position symmetrical to the second surround channel (71a). The seventh surround channel can be changed to a position symmetrical to the sixth surround channel (72a). The electronic device (700a) can determine the placement of at least one virtual speaker corresponding to the position of the surround channels according to the changed 7-channel layout (70a).

[0203] In one embodiment, the front angle and back angle may be determined to be larger values ​​as the horizontal length of the image becomes relatively longer than the vertical length, as the size of the frame (720a) becomes larger, and as the arrangement of the device becomes horizontal rather than vertical. For example, while mathematical formulas may be used to determine the front angle and back angle, the degree of proportionality between the 'image size information and environment information' and the 'front angle and back angle' does not limit the technical concept of the present disclosure.

[0204] In one embodiment, Equation 4 for calculating the surround angle can likewise be used to calculate the hundred angle. In this case, θ in Equation 4 surround can be replaced with the white angle. θ0, surround can be replaced with the reference white angle in a 7-channel layout. θ0, surround can be 45° based on international standards. effect surround is S screen and R content It may be an effective coefficient set by the user, taking into account how sensitively the angle of view is changed depending on the value of .

[0205] In one embodiment, the electronic device (700a) can modify the audio signal as if sound were generated based on the arrangement of at least one virtual speaker determined. For example, the electronic device (700a) can model sound distortion caused by the structure of a human head and ear using HRTF. The electronic device (700a) can modify the audio signal as if sound were generated based on the arrangement of at least one virtual speaker determined using HRTF.

[0206] FIG. 7b is a diagram illustrating the arrangement of a 7-channel layout when an image is output through a device according to one embodiment of the present disclosure.

[0207] For the sake of convenience of explanation, parts that overlap with those explained using FIGS. 5a and FIGS. 7a are simplified or omitted.

[0208] For reference, the electronic device (700b) of FIG. 7b illustrates an arrangement in which the electronic device (700a) of FIG. 7a is rotated 90°.

[0209] Referring to FIGS. 7a and 7b, the electronic device (700b) can determine the size of the frame (720b) ​​such that the frame (720b), which is at least part of the display, has a size large enough to output an image. As illustrated in FIG. 7b, in one embodiment, the electronic device (700b) can determine the size of the frame (720b) ​​such that the frame (720b) ​​matches the display of the electronic device (700b) so that the frame (720b) ​​has a size large enough to output an image. In FIG. 7b, the size of the electronic device (700b) and the size of the frame (720b) ​​match because the aspect ratio of the image matches the aspect ratio of the device, i.e., the electronic device (700b). Thus, the size of the frame (720b) ​​can be determined so that the image fills the display of the electronic device (700b).

[0210] In one embodiment, the electronic device (700b) can determine the placement of at least one virtual speaker based on image size information and environment information for playing the image. For example, the electronic device (700b) can determine the placement of at least one virtual speaker corresponding to the position of the surround channel according to the 7-channel layout (70b) in which the placement has been changed.

[0211] In one embodiment, the area of ​​the frame (720b) ​​illustrated in FIG. 7b may be larger than the area of ​​the frame (720a) illustrated in FIG. 7a. The electronic device (700b) may obtain the size of the frame (720b) ​​for outputting an image. The electronic device (700b) may obtain the area of ​​the frame (720b) ​​which is at least part of the display of the electronic device (700b). The larger the area of ​​the frame (720b), the more the electronic device (700b) may change the arrangement of the channel layout so that audio can be provided over a wider width. Accordingly, in the embodiment of FIG. 7b, the electronic device (700b) may change the arrangement of the second surround channel (71b) and the sixth surround channel (72b) so that the front angle and back angle are larger than in the embodiment of FIG. 7a. Conversely, in the embodiment of FIG. 7a, the electronic device (700a) can change the arrangement of the second surround channel (71a) and the sixth surround channel (72a) so that the front angle and back angle become smaller than in the embodiment of FIG. 7b.

[0212] Of course, the third surround channel may be repositioned to a position symmetrical to the second surround channel (71b). The seventh surround channel may be repositioned to a position symmetrical to the sixth surround channel (72b). The electronic device (700b) can determine the placement of at least one virtual speaker corresponding to the position of the surround channel according to the repositioned 7-channel layout (70b).

[0213] In one embodiment, there may be a standard for arranging the surround channels of a 7-channel layout. For example, a general standard for a 7-channel layout may be that the second surround channel (71b) and the third surround channel are arranged so that the front angle is 35°, the fourth surround channel and the fifth surround channel are arranged so that the side angle is 90°, and the sixth surround channel (72b) and the seventh surround channel are arranged so that the back angle is 55°. The electronic device (700b) can adjust the arrangement of the surround channels so that the front angle and back angle increase as the area of ​​the frame (720b) ​​increases. The electronic device (700b) can determine the arrangement of at least one virtual speaker corresponding to the position of the surround channels of the adjusted arrangement.

[0214] In one embodiment, the electronic device (700a) shown in FIG. 7a is positioned vertically, whereas the electronic device (700b) shown in FIG. 7b is positioned horizontally. When the electronic device (700b) is positioned horizontally as in FIG. 7b, the electronic device (700b) can change the arrangement of the channel layout so that audio can be provided over a wider width. For example, in the embodiment of FIG. 7b, the electronic device (700b) can change the arrangement of the second surround channel (71b) and the sixth surround channel (72b) so that the front angle and back angle are larger than in the embodiment of FIG. 7a. Conversely, when the electronic device (700a) is positioned vertically as in the embodiment of FIG. 7a, the electronic device (700a) can change the arrangement of the second surround channel (71a) and the sixth surround channel (72a) so that the front angle and back angle are smaller than in the embodiment of FIG. 7b.

[0215] Of course, the third surround channel may be repositioned to a position symmetrical to the second surround channel (71b). The seventh surround channel may be repositioned to a position symmetrical to the sixth surround channel (72b). The electronic device (700b) can determine the placement of at least one virtual speaker corresponding to the position of the surround channel according to the repositioned 7-channel layout (70b).

[0216] The numerical values ​​of the front angle and back angle shown in FIGS. 7a and 7b are described as relative values ​​for illustrative purposes only and do not limit the technical concept of the present disclosure.

[0217] FIG. 8 is a flowchart illustrating a method for controlling the arrangement of a channel layout according to one embodiment of the present disclosure.

[0218] For the convenience of explanation, parts that overlap with those explained using Figure 2 are simplified or omitted.

[0219] Referring to FIG. 8, step S230 of FIG. 2 may include step S810 and step S820.

[0220] In step S810, the electronic device can obtain channel layout information related to audio playback using multiple channels.

[0221] In one embodiment, the electronic device may obtain information regarding a channel layout composed of a plurality of surround channels. The electronic device may obtain information regarding the arrangement of the plurality of surround channels according to the channel layout. The channel layout information may include at least one of a 5-channel layout and a 7-channel layout.

[0222] For example, an electronic device can obtain information regarding a 5-channel layout. The 5-channel layout may be a channel layout consisting of a first surround channel (C), a second surround channel (L), a third surround channel (R), a fourth surround channel (SL), and a fifth surround channel (SR). The electronic device can obtain information regarding the arrangement of the first surround channel (C), the second surround channel (L), the third surround channel (R), the fourth surround channel (SL), and the fifth surround channel (SR).

[0223] As another example, the electronic device can obtain information regarding a 7-channel layout. The 7-channel layout may be a channel layout consisting of a first surround channel (C), a second surround channel (L), a third surround channel (R), a fourth surround channel (SL), a fifth surround channel (SR), a sixth surround channel (BL), and a seventh surround channel (BR). The electronic device can obtain information regarding the arrangement of the first surround channel (C), the second surround channel (L), the third surround channel (R), the fourth surround channel (SL), the fifth surround channel (SR), the sixth surround channel (BL), and the seventh surround channel (BR).

[0224] In step S820, the electronic device can determine the placement of at least one virtual speaker corresponding to at least one channel by changing the placement of at least one channel among a plurality of channels arranged according to channel layout information.

[0225] For example, the electronic device may change the placement of the second surround channel (L) in a 5-channel layout. According to international standards, the second surround channel (L) in the 5-channel layout may be placed at a position rotated 30° to the left relative to the listener. The electronic device may change the placement of the second surround channel (L) in the 5-channel layout to a position rotated 35° to the left relative to the listener. The changed angles are merely examples, and the technical concept of the present disclosure is not limited thereto.

[0226] As another example, the electronic device may change the placement of the fourth surround channel (SL) in a five-channel layout. According to international standards, the fourth surround channel (SL) in the five-channel layout may be placed at a position rotated 110° to the left relative to the listener. The electronic device may change the placement of the fourth surround channel (SL) in the five-channel layout to a position rotated 105° to the left relative to the listener. The changed angles are merely examples, and the technical concept of the present disclosure is not limited thereto.

[0227] As another example, the electronic device may change the placement of the second surround channel (L) in the 7-channel layout. According to international standards, the second surround channel (L) in the 7-channel layout may be placed at a position rotated 30° to the left relative to the listener. The electronic device may change the placement of the second surround channel (L) in the 7-channel layout to a position rotated 35° to the left relative to the listener. The changed angles are merely examples, and the technical concept of the present disclosure is not limited thereto.

[0228] As another example, the electronic device may change the placement of the sixth surround channel (BL) in a 7-channel layout. According to international standards, the sixth surround channel (BL) in the 7-channel layout may be placed at a position rotated 135° to the left relative to the listener. The electronic device may change the placement of the sixth surround channel (BL) in the 7-channel layout to a position rotated 130° to the left relative to the listener. The changed angles are merely examples and the technical concept of the present disclosure is not limited thereto.

[0229] In one embodiment, the electronic device may place at least one virtual speaker at a changed location of at least one surround channel. In one embodiment, the electronic device may generate an audio signal corresponding to the image based on the placement of at least one virtual speaker. The electronic device may output audio corresponding to the image based on the placement of at least one virtual speaker. The electronic device may provide the user with an audio experience that appears as if sound is being generated from at least one virtual speaker.

[0230] FIG. 9 is a flowchart illustrating a process of rearranging the channel layout when user input to play an image is obtained according to one embodiment of the present disclosure.

[0231] For the convenience of explanation, parts that overlap with those explained using Figure 2 are simplified or omitted.

[0232] In step S910, the electronic device can acquire size information of the image. In step S920, the electronic device can acquire environment information for playing the image. In step S930, the electronic device can determine the placement of at least one virtual speaker based on the size information and the environment information. In step S940, the electronic device can generate an audio signal corresponding to the image based on the placement of at least one virtual speaker.

[0233] The description of steps S910 to S940 is omitted as it overlaps with the description of steps S210 to S240 of FIG. 2.

[0234] In step S950, the electronic device can determine whether user input for playing an image is obtained.

[0235] In one embodiment, the electronic device may acquire user input for selecting an image. The electronic device may acquire user input for playing an image. The electronic device may acquire the image selected according to the user input. When user input is acquired, the electronic device may re-acquire image size information and environment information for playing the image. That is, the electronic device may perform steps S910 to S940 to change the arrangement of at least one virtual speaker in correspondence with the image indicated by the user input. The electronic device may generate an audio signal corresponding to the image indicated by the user input. The electronic device may output audio corresponding to the image.

[0236] For example, steps S910 to S940 may be performed repeatedly when a new video is played or when the video to be played is changed according to user input. By repeatedly performing steps S910 to S940, the electronic device can generate an audio signal corresponding to the video selected in response to user input. When a new video is played or the video to be played is changed, the size information of the video may change. When user input is acquired, the electronic device can re-acquire the size information of the video. Based on the re-acquired size information of the video, the electronic device can change the placement of at least one virtual speaker.

[0237] FIG. 10 is a flowchart illustrating a process of rearranging the channel layout when a sensing value exceeding a threshold value is acquired according to one embodiment of the present disclosure.

[0238] For the convenience of explanation, parts that overlap with those explained using Figure 2 are simplified or omitted.

[0239] In step S1010, the electronic device can acquire size information of the image. In step S1020, the electronic device can acquire environment information for playing the image. In step S1030, the electronic device can determine the placement of at least one virtual speaker based on the size information and the environment information. In step S1040, the electronic device can generate an audio signal corresponding to the image based on the placement of at least one virtual speaker.

[0240] The description of steps S1010 to S1040 is omitted as it overlaps with the description using steps S210 to S240 of FIG. 2.

[0241] In step S1050, the electronic device can determine whether a sensing value exceeding a threshold is obtained.

[0242] In one embodiment, the electronic device may include a motion detection sensor. The motion detection sensor may include a sensor for detecting posture or a sensor for detecting the folding and unfolding of the device. The sensing value may be a sensing value for the tilt of the device or a sensing value for a change in the folding or unfolding of the device.

[0243] For example, the motion detection sensor may include a G-sensor that measures the movement and tilt of an object, an accelerometer that measures acceleration, a gyroscope that measures angular velocity, and a hinge sensor that detects the movement and angle of a hinge.

[0244] The electronic device may acquire a sensing value that exceeds a threshold value defining the boundary at which the size information of the image and the environment information for playing the image change. When the sensing value is acquired, the electronic device may re-acquire the size information of the image and the environment information for playing the image. That is, the electronic device may perform steps S1010 to S1040 to change the placement of at least one virtual speaker corresponding to the image when the size information and environment information change. The electronic device may generate an audio signal corresponding to the image based on the changed size information and environment information. The electronic device may output audio corresponding to the image.

[0245] For example, the electronic device can acquire a tilt sensing value that exceeds a threshold defining the boundary where a horizontally positioned device is positioned vertically. The electronic device can re-acquire the placement of the device. Based on the re-acquired placement of the device, the electronic device can change the placement of at least one virtual speaker.

[0246] As another example, the electronic device may acquire a sensing value from a hinge sensor that exceeds a threshold defining a boundary indicating that the device is folded. The electronic device may reacquire environmental information including the placement of the device and the size of the frame. Based on the reacquired environmental information, the electronic device may change the placement of at least one virtual speaker.

[0247] For example, steps S1010 to S1040 may be performed repeatedly when the size information or environmental information of the image changes, such as when the device is folded or its position is changed. By repeatedly performing steps S1010 to S1040, the electronic device can generate an audio signal corresponding to the image based on the changed size information and environmental information. When a sensing value exceeding a threshold is acquired, the electronic device can re-acquire the size information of the image and the environmental information for playing the image. Based on the re-acquired size information and environmental information, the electronic device can determine the placement of at least one virtual speaker.

[0248] FIG. 11 is a drawing illustrating the binaural rendering and output of an audio signal according to one embodiment of the present disclosure.

[0249] In one embodiment, the electronic device may obtain size information of an image. The electronic device may obtain environment information for playing the image. The environment information may include the size of a frame, which is at least part of the display of the first device (1110) that is the device for playing the image, and the arrangement of the first device (1110).

[0250] For example, the arrangement of the first device (1110) may be arranged horizontally as shown in FIG. 11, but this is merely an example and does not limit the technical concept of the present disclosure.

[0251] In one embodiment, the electronic device can determine the placement of at least one virtual speaker based on size information and environment information. The electronic device can generate an audio signal corresponding to an image based on the placement of at least one virtual speaker.

[0252] In one embodiment, the first device (1110) may be a device including a display, such as a mobile terminal, tablet, or laptop, for outputting an image.

[0253] In one embodiment, the electronic device can render an audio signal to a second device (1120), which is an audio device that outputs sound. For example, the electronic device can binaurally render the audio signal to the second device (1120). Binaural rendering is a method of rendering three-dimensional 3D audio, which may be a process of simulating sound heard by both of a person's ears to make it feel as if the sound is being heard in a real space. To this end, the second device (1120) may include two sound sources that output sound, and may be, for example, headphones or earphones.

[0254] In one embodiment, the second device (1120) can output an audio signal through two or more sound sources. In one embodiment, the electronic device can control the second device (1120) to output an audio signal through two or more sound sources.

[0255] In one embodiment, the second device (1120) can output an audio signal through two sound sources as illustrated in FIG. 11. In one embodiment, the electronic device can control the second device (1120) to output an audio signal through two sound sources. The two sound sources can provide sound independently to each ear of the user. By using two sound sources, sound is delivered independently to each ear, so crosstalk, where sound delivered to one ear affects the opposite ear, can be minimized.

[0256] In one embodiment, the electronic device may be a hub device that controls the operation of the first device (1110) and the second device (1120) or manages a network between the first device (1110) and the second device (1120). The electronic device may control the first device (1110) and the second device (1120) to output an image to the first device (1110) and simultaneously output an audio signal corresponding to the image to the second device (1120).

[0257] In one embodiment, the electronic device may be a first device (1110) or a second device (1120). For example, the electronic device may output an image as the first device (1110). The electronic device may generate an audio signal corresponding to the image. The electronic device may transmit the generated audio signal to the second device (1120). The generated audio signal may be output by the second device (1120). As another example, the electronic device may receive an audio signal corresponding to the image from the first device (1110). The electronic device may output an audio signal corresponding to the image as the second device (1120).

[0258] Hereinafter, with reference to FIG. 12, the configuration of an electronic device for performing the image processing operations described so far will be described. FIG. 12 is a drawing for explaining the configuration of an electronic device for generating an audio signal according to one embodiment of the present disclosure.

[0259] For the convenience of explanation, parts that overlap with those explained using FIGS. 1 to 11 are simplified or omitted.

[0260] Referring to FIG. 12, an electronic device (1200) according to one embodiment may include an input / output interface (1210), memory (1220), and a processor (1230). However, the components of the electronic device (1200) are not limited to the examples described above, and the electronic device (1200) may include more components than the components described above, or fewer components. In one embodiment, some or all of the input / output interface (1210), memory (1220), and processor (1230) may be implemented in the form of a single chip, and the processor (1230) may include one or more processors.

[0261] The input / output interface (1210) may include an input interface (e.g., touch screen, hard button, microphone, etc.) for receiving control commands or information from a user, and an output interface (e.g., display panel, speaker, etc.) for displaying the result of an operation or the status of an electronic device (1200) according to the user's control.

[0262] For example, when user input regarding image selection is obtained through the input / output interface (1210), the electronic device (1200) can obtain size information of the selected image and environment information for playing the image. The electronic device (1200) can determine the placement of at least one virtual speaker based on the size information and environment information. The electronic device (1200) can generate an audio signal corresponding to the image based on the placement of at least one virtual speaker. The electronic device (1200) can output the selected image and the generated audio signal through the input / output interface (1210). However, it goes without saying that the electronic device (1200) can control the output of the image through a separate device or output the audio signal through a separate device.

[0263] The processor (1230) of the electronic device (1200) can perform the operations described using FIGS. 1 to 11.

[0264] Memory (1220) is a configuration for storing various programs or data and may be composed of a storage medium or a combination of storage media such as ROM, RAM, hard disk, CD-ROM, and DVD. Memory (1220) may not exist separately but may be configured to be included in the processor (1230). Memory (1220) may be composed of volatile memory, non-volatile memory, or a combination of volatile and non-volatile memory. Programs or instructions for performing operations according to the embodiments described above with reference to FIGS. 1 through 11 may be stored in memory (1220). Memory (1220) may provide stored data to the processor (1230) upon the request of the processor (1230).

[0265] A processor (1230) is configured to control a series of processes to enable an electronic device (1200) to operate according to embodiments described above with reference to FIGS. 1 to 11, and may be composed of one or more processors. One or more processors included in the processor (1230) may be circuitry such as a System on Chip (SoC) or an Integrated Circuit (IC). In this case, one or more processors may be general-purpose processors such as a CPU, AP, or DSP (Digital Signal Processor), graphics-dedicated processors such as a GPU or VPU (Vision Processing Unit), or artificial intelligence-dedicated processors such as an NPU. For example, if one or more processors are artificial intelligence-dedicated processors, the artificial intelligence-dedicated processors may be designed with a hardware structure specialized for processing a specific artificial intelligence model.

[0266] The processor (1230) can write data to memory (1220) or read data stored in memory (1220), and in particular, can process data according to a predefined operation rule or artificial intelligence model by executing a program or at least one instruction stored in memory (1220). Accordingly, the processor (1230) can perform the operations described in the previously described embodiments, and the operations described as being performed by the electronic device (1200) in the previously described embodiments can be seen as being performed by the processor (1230) unless otherwise specified.

[0267] An electronic device according to one embodiment may include a memory in which a program or one or more instructions are stored, and at least one processor. By having at least one processor execute the program or one or more instructions, either alone or in cooperation, the electronic device may obtain size information of an image. The electronic device may obtain environment information for playing the image. Based on the size information and the environment information, the electronic device may determine the placement of at least one virtual speaker. Based on the placement of at least one virtual speaker, the electronic device may generate an audio signal corresponding to the image.

[0268] In one embodiment, the size information may include at least one of the width, height, and resolution of the image.

[0269] In one embodiment, the environment information may include at least one of the size of the frame within the device that outputs the image and the orientation of the device.

[0270] In one embodiment, by having at least one processor execute a program or one or more instructions, either alone or in cooperation, the electronic device can obtain channel layout information related to audio playback using multiple channels. By changing the arrangement of at least one channel among the multiple channels arranged according to the channel layout information, the electronic device can determine the arrangement of at least one virtual speaker corresponding to at least one channel.

[0271] In one embodiment, the channel layout information may include at least one of a 5-channel layout and a 7-channel layout.

[0272] In one embodiment, by having at least one processor execute a program or one or more instructions, either alone or in cooperation, the electronic device may obtain whether user input for playing an image is obtained. If user input is obtained, the electronic device may obtain size information. The electronic device may obtain environment information for playing the image. Based on the size information and the environment information, the electronic device may determine the placement of at least one virtual speaker.

[0273] In one embodiment, the electronic device may further include a sensor. By having at least one processor execute a program or one or more instructions, either alone or in cooperation, the electronic device may determine whether a sensing value exceeding a threshold value is obtained using the sensor. If the sensing value exceeds the threshold value, the electronic device may obtain size information. The electronic device may obtain environment information for playing an image. Based on the size information and environment information, the electronic device may determine the placement of at least one virtual speaker.

[0274] In one embodiment, the sensor may be a sensor that detects posture or a sensor that detects the folding and unfolding of the device.

[0275] In one embodiment, the electronic device can transmit an audio signal to an audio device by having at least one processor execute a program or one or more instructions, either alone or in cooperation. The electronic device can control the audio device so that the audio signal is output.

[0276] In one embodiment, the electronic device can output an audio signal through two or more sound sources by having at least one processor execute a program or one or more instructions, either alone or in cooperation.

[0277] A method for an electronic device to generate an audio signal according to one embodiment may include the step of acquiring size information of an image. The method may include the step of acquiring environment information for playing the image. The method may include the step of determining the placement of at least one virtual speaker based on the size information and the environment information. The method may include the step of generating an audio signal corresponding to the image based on the placement of at least one virtual speaker.

[0278] In one embodiment, the size information may include at least one of the width, height, and resolution of the image.

[0279] In one embodiment, the environment information may include at least one of the size of the frame within the device that outputs the image and the arrangement of the device.

[0280] In one embodiment, the step of determining the placement of at least one virtual speaker may include the step of obtaining channel layout information related to audio playback using a plurality of channels. The step of determining the placement of at least one virtual speaker may include the step of determining the placement of at least one virtual speaker corresponding to at least one channel by changing the placement of at least one channel among a plurality of channels arranged according to the channel layout information.

[0281] In one embodiment, the step of acquiring size information may include the step of acquiring whether user input for playing an image is acquired. The step of acquiring size information may include the step of acquiring size information when user input is acquired.

[0282] In one embodiment, the step of acquiring size information may include the step of acquiring whether a sensing value exceeding a threshold is acquired using a sensor. The step of acquiring size information may include the step of acquiring size information when the sensing value exceeds the threshold.

[0283] In one embodiment, the sensor may be a sensor that detects posture or a sensor that detects the folding and unfolding of the device.

[0284] In one embodiment, the step of generating an audio signal corresponding to an image may include the step of transmitting the audio signal to an audio device. The step of generating an audio signal corresponding to an image may include the step of controlling an audio device so that the audio signal is output.

[0285] In one embodiment, the step of generating an audio signal corresponding to an image may be to output the audio signal through two or more sound sources.

[0286] A computer-readable, non-transient recording medium may be provided on which a program for performing any one method according to one embodiment of the present disclosure is recorded.

[0287] A device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory storage medium' simply means that it is a tangible device and does not contain a signal (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily. For example, a 'non-transitory storage medium' may include a buffer in which data is stored temporarily.

[0288] According to one embodiment, the method according to the various embodiments disclosed herein may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or distributed online (e.g., download or upload) through an application store or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product (e.g., downloadable app) may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

Claims

1. Memory (1220) where a program or one or more instructions are stored; and It includes at least one processor (1230), By having at least one processor execute the program or one or more instructions alone or in cooperation, the electronic device (1200) Acquire image size information, and Obtain environment information to play the above video, and Based on the above size information and the above environment information, the placement of at least one virtual speaker is determined, and An electronic device that generates an audio signal corresponding to the image based on the arrangement of at least one virtual speaker.

2. In Paragraph 1, An electronic device comprising at least one of the width, height, and resolution of the image, wherein the size information above includes the width, height, and resolution.

3. In any one of paragraphs 1 to 2, An electronic device comprising at least one of the size of a frame within a device that outputs the image and the orientation of the device, wherein the above environment information is an environment information.

4. In any one of paragraphs 1 through 3, By having at least one processor execute the program or one or more instructions alone or in cooperation, the electronic device, Acquire channel layout information related to audio playback using multiple channels, and An electronic device that determines the arrangement of at least one virtual speaker corresponding to the at least one channel by changing the arrangement of at least one channel among the plurality of channels arranged according to the channel layout information.

5. In any one of paragraphs 1 through 4, By having at least one processor execute the program or one or more instructions alone or in cooperation, the electronic device, Obtaining whether user input for playing the above video is obtained, and When the above user input is obtained, the above size information is obtained, and Acquire the environment information for playing the above video, and An electronic device that determines the placement of at least one virtual speaker based on the size information and environment information.

6. In any one of paragraphs 1 through 4, Includes additional sensors, By having at least one processor execute the program or one or more instructions alone or in cooperation, the electronic device, Using a sensor, determine whether a sensing value exceeding a threshold is obtained, and If the above sensing value exceeds a threshold value, the above size information is obtained, and Acquire the environment information for playing the above video, and An electronic device that determines the placement of at least one virtual speaker based on the size information and environment information.

7. In any one of paragraphs 1 through 6, By having at least one processor execute the program or one or more instructions alone or in cooperation, the electronic device, The above audio signal is transmitted to an audio device, and An electronic device that controls the audio device so that the above audio signal is output.

8. In a method for an electronic device to generate an audio signal, Step of acquiring image size information; A step of obtaining environment information to play the above video; A step of determining the placement of at least one virtual speaker based on the size information and environment information; and A method comprising the step of generating an audio signal corresponding to the image based on the arrangement of at least one virtual speaker.

9. In Paragraph 8, The above size information includes at least one of the width, height, and resolution of the image.

10. In any one of paragraphs 8 through 9, An electronic device comprising at least one of the size of a frame within a device that outputs the image and the arrangement of the device, wherein the above environmental information includes the above environmental information.

11. In any one of paragraphs 8 through 10, The step of determining the placement of at least one virtual speaker is, A step of obtaining channel layout information related to audio playback using multiple channels; and A method comprising the step of determining the placement of at least one virtual speaker corresponding to the at least one channel by changing the placement of at least one channel among the plurality of channels arranged according to the channel layout information.

12. In any one of paragraphs 8 through 11, The step of obtaining the above size information is, A step of obtaining whether user input for playing the above video is obtained; and A method comprising the step of obtaining size information when the above user input is obtained.

13. In any one of paragraphs 8 through 11, The step of obtaining the above size information is, A step of obtaining whether a sensing value exceeding a threshold is obtained using a sensor; and A method comprising the step of obtaining size information when the above sensing value exceeds a threshold value.

14. In any one of paragraphs 8 through 13, The step of generating the audio signal corresponding to the above image is, A step of transmitting the above audio signal to an audio device; A method comprising the step of controlling the audio device so as to output the above audio signal.

15. A computer-readable recording medium having a program recorded thereon for performing the method of any one of paragraphs 8 through 14 on a computer.

Images