Electronic device and method for processing speech signal

Abstract

The present document relates to an electronic device and method for processing a voice signal. According to various embodiments, the electronic device comprises a housing configured to fold at least one part, at least one microphone, at least one speaker, an audio module electrically connected to the at least one microphone and the at least one speaker, and at least one processor electrically connected to the audio module. The at least one processor may be configured to identify a voice signal received through the microphone, acquire angle information related to the folding of at least one part of the housing, identify a state in which at least one part of the housing is being folded based on the angle information, adjust control information of the audio module for processing the voice signal based on the angle information, and control the audio module to process the voice signal based on the adjusted control information while at least one part of the housing is being folded. Other embodiments are also possible.

Description

Technology Field

[0001] Various embodiments of this document relate to electronic devices and methods for voice signal processing. Background Technology

[0002] Recently, electronic devices are evolving into various forms for user convenience, providing diverse services or functions, and are being implemented as devices suitable for multimedia services while maintaining portability.

[0003] In order to fully utilize the various services or functions provided by the electronic device, the size of the electronic device's display may be increased. Accordingly, a foldable flexible display may be placed across the entire area of ​​a housing structure separated to be foldable, and the housing structure may be implemented in a form that includes one or more speakers and microphones.

[0004] When processing voice signals using electronic devices capable of shape deformation such as folding or unfolding (e.g., mobile phones, IoT speakers, TVs, etc.), voice signal processing can be performed to remove echo and noise from the voice signal. The problem to be solved

[0005] In electronic devices capable of shape deformation (e.g., foldable electronic devices), if part of the housing is folded as the flexible display is folded, the positions of the microphone and speaker change, and the relative distance between the microphone and speaker changes, which may cause voice signal processing to malfunction and degrade. As a result, noise or echo caused by ambient sound may occur along with the speaker's voice signal, leading to severe sound quality degradation or the speaker's voice may not be transmitted clearly.

[0006] According to various embodiments of the present document, an electronic device and a method for processing a voice signal can be provided to maintain the performance of voice signal processing consistently according to variations in the form of the electronic device. means of solving the problem

[0007] According to various embodiments of the present document, an electronic device comprises a housing configured to fold at least one part, at least one microphone, at least one speaker, an audio module electrically connected to the at least one microphone and the at least one speaker, and at least one processor electrically connected to the audio module, wherein the at least one processor may be configured to identify a voice signal received through the microphone, acquire angle information related to the folding of at least one part of the housing, identify a state in which at least one part of the housing is being folded based on the angle information, adjust control information of the audio module for processing the voice signal based on the angle information, and control the audio module to process the voice signal based on the adjusted control information while at least one part of the housing is being folded.

[0008] According to various embodiments, a method of operation in an electronic device comprising at least one microphone and at least one speaker may include: receiving a voice signal through the at least one microphone; obtaining angle information related to the folding of at least one part of the housing of the electronic device; identifying the state of the at least part of the housing being folded based on the angle information; adjusting control information of an audio module of the electronic device for processing the voice signal based on the angle information; and processing the voice signal based on the adjusted control information while at least one part of the housing is being folded by the audio module.

[0009] According to various embodiments, in a non-transient storage medium storing a program, the program may include an executable instruction such that, when executed by a processor, the processor performs the operation of receiving a voice signal through at least one microphone, the operation of obtaining angle information related to the folding of at least one part of the housing of the electronic device, the operation of identifying the state of at least one part of the housing being folded based on the angle information, the operation of adjusting control information of an audio module of the electronic device for processing the voice signal based on the angle information, and the operation of processing the voice signal based on the adjusted control information while at least one part of the housing is being folded by the audio module. Effects of the invention

[0010] According to one embodiment, by providing an electronic device and method for processing a voice signal, control information for enhancing the performance of voice signal processing is adjusted according to a change in the shape of the electronic device to remove echoes and noise that are introduced together with the speaker's voice signal, thereby ensuring that constant performance is maintained without degradation during voice signal processing, and thus providing a more enhanced voice signal without degrading the sound quality. Brief explanation of the drawing

[0011] FIG. 1 is a block diagram of an electronic device in a network environment according to various embodiments. FIGS. 2A and FIGS. 2B are drawings showing examples of configurations of electronic devices according to various embodiments. FIG. 3 is a drawing showing examples of configurations of electronic devices according to various embodiments. FIG. 4 is a drawing showing examples of the folded state of an electronic device according to various embodiments. FIG. 5 is a diagram illustrating examples of the configuration of an audio module of an electronic device according to various embodiments. FIG. 6 is a diagram illustrating examples of the configuration of an audio module of an electronic device according to various embodiments. FIG. 7 is a drawing showing an example of a method of operation in an electronic device according to various embodiments. FIG. 8 is a drawing showing an example of a method of operation in an electronic device according to various embodiments. FIG. 9 is a drawing showing an example of a method of operation in an electronic device according to various embodiments. In relation to the description of the drawings, the same or similar reference numerals may be used for identical or similar components. Specific details for implementing the invention

[0012] Hereinafter, electronic devices according to various embodiments will be examined with reference to the attached drawings. In the various embodiments, the term "user" may refer to a person using the electronic device or a device using the electronic device (e.g., an artificial intelligence electronic device).

[0013] FIG. 1 is a block diagram of an electronic device (101) in a network environment (100) according to various embodiments. Referring to FIG. 1, in the network environment (100), the electronic device (101) may communicate with an electronic device (102) through a first network (198) (e.g., a short-range wireless communication network) or with an electronic device (104) or a server (108) through a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) through a server (108). According to one embodiment, the electronic device (101) may include a processor (120), memory (130), input module (150), sound output module (155), display module (160), audio module (170), sensor module (176), interface (177), connection terminal (178), haptic module (179), camera module (180), power management module (188), battery (189), communication module (190), subscriber identification module (196), or antenna module (197). In some embodiments, at least one of these components (e.g., connection terminal (178)) may be omitted from the electronic device (101), or one or more other components may be added. In some embodiments, some of these components (e.g., sensor module (176), camera module (180), or antenna module (197)) may be integrated into a single component (e.g., display module (160)).

[0014] The processor (120) can control at least one other component (e.g., hardware or software component) of the electronic device (101) connected to the processor (120) by executing software (e.g., program (140)), for example, and can perform various data processing or operations. According to one embodiment, as at least part of the data processing or operations, the processor (120) can store commands or data received from other components (e.g., sensor module (176) or communication module (190)) in volatile memory (132), process the commands or data stored in volatile memory (132), and store the resulting data in non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., central processing unit or application processor) or an auxiliary processor (123) that can operate independently or together with it (e.g., graphics processing unit, neural processing unit (NPU), image signal processor, sensor hub processor, or communication processor). For example, if the electronic device (101) includes a main processor (121) and an auxiliary processor (123), the auxiliary processor (123) may be configured to use lower power than the main processor (121) or to be specialized for a designated function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as part thereof.

[0015] The auxiliary processor (123) may control at least some of the functions or states associated with at least one component of the electronic device (101) (e.g., display module (160), sensor module (176), or communication module (190)) on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. According to one embodiment, the auxiliary processor (123) (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera module (180) or communication module (190)). According to one embodiment, the auxiliary processor (123) (e.g., neural network processing unit) may include a hardware structure specialized for processing an artificial intelligence model. The artificial intelligence model may be generated through machine learning. Such learning may be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or through a separate server (e.g., server (108)). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model may include a plurality of artificial neural network layers.An artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to the hardware structure, the artificial intelligence model may include a software structure, either additionally or substantially.

[0016] The memory (130) can store various data used by at least one component of the electronic device (101) (e.g., processor (120) or sensor module (176)). The data may include, for example, software (e.g., program (140)) and input data or output data for related commands. The memory (130) may include volatile memory (132) or non-volatile memory (134).

[0017] The program (140) may be stored as software in memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0018] The input module (150) can receive commands or data to be used for a component of the electronic device (101) (e.g., processor (120)) from outside the electronic device (101) (e.g., user). The input module (150) may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0019] The sound output module (155) can output a sound signal to the outside of the electronic device (101). The sound output module (155) may include, for example, a speaker or a receiver. The speaker may be used for general purposes, such as multimedia playback or recording playback. The receiver may be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part thereof.

[0020] The display module (160) can visually provide information to an external (e.g., user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling said device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of the force generated by said touch.

[0021] The audio module (170) can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150) or output sound through the sound output module (155) or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphones) connected directly or wirelessly to the electronic device (101).

[0022] The sensor module (176) can detect the operating state of the electronic device (101) (e.g., power or temperature) or the external environmental state (e.g., user state) and generate an electrical signal or data value corresponding to the detected state. According to one embodiment, the sensor module (176) may include, for example, a gesture sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an accelerometer sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biosensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0023] The interface (177) may support one or more specified protocols that can be used for the electronic device (101) to be connected directly or wirelessly to an external electronic device (e.g., electronic device (102)). According to one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0024] The connection terminal (178) may include a connector through which the electronic device (101) can be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0025] The haptic module (179) can convert an electrical signal into a mechanical stimulus (e.g., vibration or movement) or an electrical stimulus that the user can perceive through tactile or kinesthetic senses. According to one embodiment, the haptic module (179) may include, for example, a motor, a piezoelectric element, or an electric stimulation device.

[0026] The camera module (180) can capture still images and video. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0027] The power management module (188) can manage the power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented, for example, as at least part of a power management integrated circuit (PMIC).

[0028] The battery (189) can supply power to at least one component of the electronic device (101). According to one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0029] The communication module (190) can support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between an electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may include one or more communication processors that operate independently of the processor (120) (e.g., application processor) and support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., cellular communication module, short-range wireless communication module, or GNSS (global navigation satellite system) communication module) or a wired communication module (194) (e.g., LAN (local area network) communication module, or power line communication module). The corresponding communication module among these communication modules can communicate with an external electronic device (104) through a first network (198) (e.g., a short-range communication network such as Bluetooth, WiFi (wireless fidelity) direct, or IrDA (infrared data association)) or a second network (199) (e.g., a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can identify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) using subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module (196).

[0030] The wireless communication module (192) can support 5G networks and next-generation communication technologies following 4G networks, for example, new radio access technology. NR access technology can support high-speed transmission of high-capacity data (enhanced mobile broadband (eMBB)), minimization of terminal power and connection of multiple terminals (massive machine type communications (mMTC)), or high reliability and low latency (ultra-reliable and low-latency communications (URLLC)). The wireless communication module (192) can support a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate, for example. The wireless communication module (192) can support various technologies for securing performance in the high-frequency band, such as beamforming, massive MIMO (multiple-input and multiple-output), full-dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large-scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), external electronic device (e.g., electronic device (104)), or network system (e.g., second network (199)). According to one embodiment, the wireless communication module (192) can support a Peak data rate (e.g., 20 Gbps or more) for realizing eMBB, loss coverage (e.g., 164 dB or less) for realizing mMTC, or U-plane latency (e.g., downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less) for realizing URLLC.

[0031] An antenna module (197) can transmit a signal or power to or from an external source (e.g., an external electronic device). According to one embodiment, the antenna module (197) may include an antenna comprising a radiator made of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as a first network (198) or a second network (199), may be selected from the plurality of antennas, for example, by a communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the selected at least one antenna. According to some embodiments, in addition to the radiator, other components (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module (197).

[0032] According to various embodiments, the antenna module (197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent to a first surface (e.g., bottom surface) of the printed circuit board and capable of supporting a specified high frequency band (e.g., mmWave band), and a plurality of antennas (e.g., array antennas) disposed on or adjacent to a second surface (e.g., top surface or side surface) of the printed circuit board and capable of transmitting or receiving a signal of the specified high frequency band.

[0033] At least some of the above components can be connected to each other via a communication method between peripheral devices (e.g., bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)) and exchange signals (e.g., commands or data) with each other.

[0034] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) through a server (108) connected to a second network (199). Each of the external electronic devices (102, or 104) may be of the same or different type as the electronic device (101). According to one embodiment, all or part of the operations performed on the electronic device (101) may be performed on one or more of the external electronic devices (102, 104, or 108). For example, if the electronic device (101) needs to perform a function or service automatically or in response to a request from a user or another device, the electronic device (101) may request one or more external electronic devices to perform at least part of the function or service instead of performing the function or service itself or additionally. One or more external electronic devices that receive the above request may execute at least part of the requested function or service, or additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may provide the result as is or additionally processed as at least part of the response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device (101) may provide ultra-low latency services using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server using machine learning and / or neural networks. According to one embodiment, the external electronic device (104) or the server (108) may be included within a second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0035] FIGS. 2a and 2b are drawings showing examples of configurations of electronic devices according to various embodiments, FIG. 3 is a drawing showing examples of configurations of electronic devices according to various embodiments, and FIG. 4 is a drawing showing examples of folded states of electronic devices according to various embodiments.

[0036] Referring to FIGS. 1, FIGS. 2a, FIGS. 2b and FIGS. 3, an electronic device (101) according to various embodiments (e.g., the electronic device (101) of FIG. 1) may be configured such that at least a portion of the housing (210) is deformed and folded. The housing (210) may be divided into a first housing area (211) and a second housing area (213), and at least one of the first housing area (211) or the second housing area (213) may be configured to be folded in a direction in which the front of the first housing area (211) (e.g., the side where the display of the electronic device (101) is exposed) and the front of the second housing area (213) (e.g., the side where the display of the electronic device (101) is exposed) face each other or in an opposite direction with respect to a hinge structure (215) (or reference line).

[0037] According to various embodiments, an electronic device (101) (e.g., the electronic device (101) of FIG. 1) may be configured to include an audio module (301) (e.g., the audio module (176) of FIG. 1) and at least one processor (120). The electronic device (101) may be configured to include a memory (e.g., the memory (130) of FIG. 1) that is electrically connected to at least one processor (120) and stores instructions for operations performed by at least one processor (120) when the at least one processor (120) is executed, at least one sensor (e.g., the sensor module (176) of FIG. 1) and a communication module (e.g., the communication module (190) of FIG. 1). According to one embodiment, the electronic device (101) may be configured to include at least one separately configured microphone (220) (e.g., input module (150) of FIG. 1) and at least one speaker (230) (e.g., acoustic output module (155) of FIG. 1), as shown in FIG. 2a. According to another embodiment, the electronic device (101) may be configured to include an acoustic component (not shown) in which the microphone structure and the speaker structure are implemented as an integrated structure. According to another embodiment, the electronic device (101) may be configured to include acoustic components disposed respectively inside a portion of a first housing area (211) and a second housing area (213).

[0038] Referring to FIGS. 2a and 2b, at least one microphone (220) according to one embodiment may be placed inside a portion of the first housing area (211) and / or the second housing area (213) of the housing (210). For example, the microphone (220) may be placed in a position facing the speaker (230) as the first housing area (211) or the second housing area (213) of the housing (210) folds. The microphone (220) may receive a voice signal and transmit the received voice signal to an audio module (301). The voice signal received through the microphone (220) may include the voice signal of a speaker located at a short distance (e.g., near speaker), the voice signal of a speaker located at a long distance (e.g., far speaker), and echoes and noise from the signal output from the speaker (230). According to one embodiment, the speaker (230) may be placed in a part of the first housing area (211) or the second housing area (213) of the housing (210) and may include a configuration formed to generate sound based on an electrical signal and / or a configuration formed to convert sound into an electrical signal. At least one microphone (220) may be a beamforming microphone and may enhance the voice signal of a near-end speaker and remove noise by adding and canceling voice signals having opposite phases, and may process incoming voice signals to have directionality. For example, as shown in FIG. 2a, at least one microphone (220) may adjust the beamforming operation factor to have directionality in a direction facing the mouth of a near-end speaker. For example, the electronic device (101) may include a plurality of beamforming microphones (220) as shown in FIG. 2b, and since the plurality of beamforming microphones (220) may be implemented to have different directions of directionality, interference may occur depending on the distance between the microphones (220).In this case, when a part of the housing is folded, the distance between the microphones (220) changes, so the beamforming bandwidth according to the change in distance (or change in angle) can be adjusted to eliminate echo or noise introduced due to interference.

[0039] According to various embodiments, at least one speaker (230) may be placed inside a portion of the first housing area (211) or the second housing area (213). At least one speaker (230) may output sound in a direction toward the first housing area (211), and the output sound may travel through an acoustic waveguide (not shown) provided in the internal space of the housing (210) and radiate to the external space from one side of the second housing area (213). For example, the speaker (230) may be provided as a speaker for a receiver that outputs a sound in voice call mode. In another example, in multimedia mode, the speaker (230) may function as a loudspeaker and may be combined with other speaker modules not shown to provide stereo sound.

[0040] According to another embodiment, the acoustic component is configured such that the speaker structure and the receiver structure (e.g., microphone structure) are integrated and may be placed inside a portion of the first housing area (211) and the second housing area (213). For example, the electronic device (101) may provide different acoustic operation modes depending on the unfolded state and the folded state. For example, when the electronic device (101) is in the unfolded state, the acoustic component may operate in speaker mode, and when the electronic device (101) is in the folded state, the acoustic component may operate in speaker mode or receiver mode (e.g., microphone mode) depending on the user's selection. For example, the speaker mode and / or receiver mode may each provide different output voltages. According to another embodiment, when the electronic device (101) is folded for a call, the user may be able to transmit and receive in any direction by means of integrated acoustic components located within the first housing area (211) and the second housing area (213). The electronic device (101) may determine the state (e.g., position) of the electronic device (101) by means of a sensor module (176) (e.g., accelerometer and / or geomagnetic sensor) placed inside, thereby providing short-range transmission and reception suitable for the user. For example, if the user has their mouth located in the second housing area (213) and their ear located in the first housing area (211), the sensor module (176) determines the state of the electronic device, and the acoustic component located in the first housing area (211) may operate as a speaker, and the acoustic component located in the second housing area (213) may operate as a microphone.As another example, if the user has their ear located in the second housing area (213) and their mouth located in the first housing area (211), the sensor module (176) determines the state of the electronic device (101), and the acoustic component located in the first housing area (211) can operate as a microphone, and the acoustic component located in the second housing area (213) can operate as a speaker.

[0041] Referring to FIGS. 1, FIGS. 2a, FIGS. 2b, and FIGS. 3, an audio module (301) according to various embodiments may be electrically connected to at least one processor (120), a microphone (220), and a speaker (230) (or acoustic component), and may be configured to process voice signals input to the microphone (220) (or acoustic component) and to process signals output to the speaker (230) (or acoustic component) under the control of at least one processor (120). According to one embodiment, the audio module (301) may be configured to include a control module (310) and a voice signal processing module (320). The control module (310) is electrically connected to the processor (120) and the voice signal processing module (320), may operate according to control commands of the processor (120), and may be configured as a software module or a hardware module. According to another embodiment, the control module (310) may be configured to be included in the processor (120) as a software module. The control module (310) may receive angle information related to the folding of at least a portion of the housing (210) (e.g., a first housing area (211) and / or a second housing area (213)) detected by at least one sensor (e.g., the sensor module (176) of FIG. 1) from the processor (120). The control module (310) may transmit a control signal to the voice signal processing module (320) for processing the received voice signal based on the received angle information. The angle information may be calculated by the processor (120) based on movement information of at least a portion of the housing (210) (e.g., a first housing area (211) and / or a second housing area (213)) detected by at least one sensor (e.g., the sensor module (176) of FIG. 1) under the control of the processor (120).

[0042] According to various embodiments, the voice signal processing module (320) of the audio module (301) can perform an operation to process a received voice signal under the control of at least one processor (120). The voice signal processing module (320) can perform a voice signal processing operation to remove echoes included in the received voice signal based on angle information under the control of the processor (120) and the adjusted control information. The voice signal processing module (320) can perform voice signal processing to remove noise from the received voice signal. The control information may be information specified in correspondence with angle information acquired while a part of the housing (210) is being folded. For example, the control information may be adjusted at a certain ratio from the previous control information whenever the folding angle changes to an angle within a specified range (e.g., 15 degrees) while a part of the housing (210) is being folded by the voice signal processing module (320) under the control of the processor (120). As another example, the control information may be pre-specified information corresponding to the angle information and may be stored in memory (130) in the form of a table.

[0043] Referring to FIGS. 1, FIGS. 2a, FIGS. 2b, FIGS. 3 and FIGS. 4, at least one processor (120) of an electronic device (101) according to various embodiments can identify a voice signal received through a microphone (220), obtain angle information related to the folding of at least one part of a housing (210) (e.g., a first housing area (211) or / and a second housing area (213)), and identify a state in which at least a part of the housing (210) is folded based on the obtained angle information. The state in which at least a part of the housing (210) is folded may be, for example, a state in which the first housing area (211) or / and the second housing area (213) is rotating (or moving) in a direction facing or opposite to the hinge structure (215).

[0044] According to various embodiments, at least one processor (120) can detect movement (or rotation) of, for example, a first housing area (211) as part of the housing (210) as shown in FIG. 4 through at least one sensor (e.g., sensor module (176) of FIG. 1) and obtain angle information according to folding calculated based on the detected movement information. The received angle information may represent an angle between the first housing area (211) and the second housing area (213). For example, at least one processor (120) can identify that the electronic device (101) is in an unfolded state if the angle information is 180 degrees, as in FIG. 4 (a). For example, at least one processor (120) can identify that the electronic device (101) is in a folded state from an unfolded state when it is identified that the angle information changes to 150 degrees as the first housing region (211) of the housing (210) rotates more than a specified threshold angle (e.g., 30 degrees) in a direction facing the second housing region (213), as in FIG. 4 (b). At least one processor (120) can adjust the previous control information based on the control information specified corresponding to the angle information being 150 degrees, or adjust the previous control information at a certain ratio based on the angle change within a specified range. For example, at least one processor (120) can identify that the electronic device (101) is in a folded state when the angle information is identified to 90 degrees as the first housing region (211) of the housing (210) continues to rotate in a direction facing the second housing region (213), as in FIG. 4 (c). At least one processor (120) can adjust previous control information based on control information specified with angle information corresponding to 90 degrees, or adjust previous control information at a certain rate based on angle change within a specified range.For example, at least one processor (120) can identify that the first housing area (211) of the housing (210) is in a completely folded state when the angle information is identified as 0 degrees as the first housing area (211) of the housing (210) continues to rotate in a direction facing the second housing area (213) as in (d) of FIG. 4, and after a certain period of time has elapsed, identify that the audio module (301) for removing echo from the voice signal is in a stable state and stop adjusting the control information. At least one processor (120) can remove echo from the received voice signal by maintaining the adjusted control information when the angle information is 0 degrees. In another example, as shown in FIG. 4 (c), if there is no change in the angle information acquired for a certain period of time while the angle information is 90 degrees, at least one processor (120) identifies that the folding operation of the first housing area (211) of the housing (210) is completed and stops adjusting the control information, and when the angle information is 90 degrees, maintains the adjusted control information to remove echo from the received voice signal. In another example, if movement of the second housing area (213) of the housing (210) is detected or if the first housing area (211) and the second housing area (213) move simultaneously, as described in FIG. 4, at least one processor (120) can acquire angle information between the first housing area (211) and the second housing area (213) and adjust the control information based on the acquired angle information.

[0045] According to various embodiments, at least one processor (120) can identify a state in which a first housing region (211) is folded as at least a part of a housing as illustrated in FIG. 4, and control a voice signal processing module (320) to adjust control information for processing a received voice signal based on angle information obtained while the first housing region (211) is folded.

[0046] Referring again to FIG. 1, FIG. 2a, FIG. 2b, and FIG. 3, at least one processor (120) according to various embodiments may control an audio module (301) based on controlled information to remove linear echoes from a voice signal while at least a portion of the housing (210) is being folded, and to remove non-linear echoes from a received voice signal due to changes in the relative distance between the microphone (220) and the speaker (230). The control information may include at least one of operation parameter information (or factors) for voice signal processing of the voice processing module (320), weighting information related to the beamforming performance of the microphone (220), or input sensitivity information of the microphone (220). Here, the operational parameter information (or factor) for voice signal processing may include at least one of an echo cancellation operational parameter (or factor) (e.g., intensity of a residual echo canceller (RES) (e.g., noise cancellation intensity), linear / non-linear echo cancellation intensity, automatic gain control (AGC), coefficients of an adaptive filter (e.g., first filter) and convergence rate or an echo path change detector) and / or a noise cancellation operational parameter (or factor) (e.g., beamforming bandwidth). The convergence rate of the adaptive filter may be a factor for controlling the adaptation rate by following the angle change due to folding while a part of the housing (210) is being folded. The intensity of the residual echo canceller (RES) (e.g., second filter) may be a factor for removing echoes (e.g., non-linear echo components) caused by the hardware characteristics of the microphone (220) and speaker (230) while at least a part of the housing (210) is being folded. For example, the control information may include preset parameter sets by mapping angle information specified in at least one of motion parameter information, weight information, or input sensitivity information of a microphone for voice signal processing, and the preset parameter sets may be stored in a table form in memory (130).

[0047] According to various embodiments, at least one processor (120) can identify a folding speed corresponding to an angle change based on angle information acquired while at least a portion of the housing (210) is being folded, and can identify that at least a portion of the housing (e.g., a first housing area (211) and / or a second housing area (213)) is in a folding state when the identified folding speed exceeds a specified threshold. At least one processor (120) can identify that the folding operation of at least a portion of the housing (210) is stopped (e.g., a folding stop state) when the identified folding speed in the folding state (or unfolding state) is below a specified threshold and a specified time has elapsed. Here, the folding speed may represent the degree of change of the angle between the first housing area (211) and the second housing area (213) per second.

[0048] According to various embodiments, at least one processor (120) may terminate a control operation for voice processing when the operation of folding the first housing region (211) of the housing (210) is completed and the voice call or voice input (or recording) is completed. For example, at least one processor (120) may execute an application related to a voice call or voice input (or recording) to perform a control operation for voice processing.

[0049] According to various embodiments, at least one processor (120) can identify the relative distance between the microphone (220) and the speaker (230) based on angle information obtained while at least a part of the housing (210) (e.g., the first housing area (211) of FIG. 4) is folded when a voice signal is received, and can control the audio module (301) to increase the directional information (e.g., RES strength) included in the control information when the relative distance changes to a small value over time and becomes closer.

[0050] According to various embodiments, at least one processor (120) can adjust parameter information for voice signal processing of an audio module (301) included in control information (e.g., at least one of an echo cancellation operation parameter or a noise cancellation operation parameter) in response to angle information continuously acquired while at least a part of the housing (210) is being folded.

[0051] According to various embodiments, at least one processor (120) identifies a distance value between a microphone (220) and a speaker (230) based on angle information while at least a portion of the housing (210) is folded, and when the identified distance value decreases below a specified distance threshold, the sensitivity of the microphone (220) can be adjusted to a specified low value. When the identified distance value exceeds a specified distance threshold, the sensitivity of the microphone (220) can be adjusted to a specified high value.

[0052] According to the various embodiments described above, at least one processor (120) can perform the same operations even while at least a portion of the housing (210) is unfolded. In addition to the foldable form of the electronic device (101), the operations according to the various embodiments described above can be applied in the same way even when a portion of the electronic device (101) is folded or unfolded, such as in a rollable or sliderable form. In the description of FIGS. 2a, 2b and FIGS. 4, as an example, a microphone (220) is placed in the first housing area (211) and a speaker (230) is placed in the second housing area (213), but this is not limited thereto, and microphones (e.g., stereo microphones) may be mounted in the first housing area (211) and the second housing area (213), respectively, and speakers (e.g., stereo speakers) may be mounted in the first housing area (211) and the second housing area (213), respectively.

[0053] FIG. 5 is a drawing illustrating an example of the configuration of an audio module (301) of an electronic device according to various embodiments.

[0054] Referring to FIGS. 3 and 5, according to various embodiments, the voice signal processing module (320) of the audio module (301) may be configured to include an echo removal module (510) for removing echoes from a received voice signal. The voice signal processing module (320) can remove echoes from a voice signal of a far-speech or a voice signal input by a speaker (230) by means of the echo removal module (510). The voice signal processing module (320) can remove noise from the voice signal by means of a noise removal module. For example, the echo removal module (510) can control echoes in the received voice signal based on an echo reference corresponding to a signal from a far-speech output to the speaker (230). The echo removal module (510) can receive the echo reference through the control module (310). For example, the echo control module (510) may include a first filter (e.g., adaptive filter) (511) and a subtractor (513) for removing linear components of echo based on an echo reference in a voice signal. The echo control module (510) may further include a second filter (e.g., RES: residual echo suppressor) (515) for removing non-linear residual echo and non-linear components. The first filter (511) is a filter that predicts an acoustic path linearly, and generates a replica of the echo component based on an echo reference to pass the linear echo component to the subtractor (513), and obtains an echo-removed voice signal by subtracting the linear echo component passed from the first filter (511) from the received voice signal at the subtractor (513).The second filter (515) can remove non-linear residual echo and non-linear components caused by hardware features of the microphone (220) and speaker (230) from the voice signal from which linear echo has been removed by the first filter (511).

[0055] FIG. 6 is a diagram illustrating an example of the configuration of an audio module (301) of an electronic device according to various embodiments.

[0056] Referring to FIGS. 3 and FIGS. 6, according to various embodiments, the voice signal processing module (320) of the audio module (301) may be configured to include a noise removal module (610) for removing noise from a received voice signal. For example, the noise removal module (610) may remove noise from a voice signal output after the echo is removed by an echo removal module (e.g., the echo removal module (510) of FIG. 5). In another example, the noise removal module (610) may remove noise from a voice signal received from a microphone (220) and transmit the noise-removed signal to an echo removal module (e.g., the echo removal module (510) of FIG. 5) to remove the echo from the noise-removed voice signal.

[0057] According to various embodiments, the noise removal module (610) may include a sound source localizer (not shown) that tracks the location or direction of a speaking near-end speaker, a beamformer (611) that improves the signal-to-noise ratio by removing (or attenuating) noise entering from another direction while preserving the voice signal of the near-end speaker input from the tracked location or direction (target direction), and a noise suppressor (613) as a post filter that removes noise input together from the tracked direction (target direction) of the near-end speaker. The noise removal module (610) may further include a noise suppressor (615) that receives a voice signal from a microphone (220) via a different path from the beamformer (611) and removes noise input together from the tracked direction (target direction) of the near-end speaker.

[0058] According to various embodiments, the noise removal module (610) may output a first signal, which is a noise-removed voice signal, through a first path (601) of a beamformer (611) and a noise remover (613), and may output a second signal, which is a noise-removed voice signal, through a second path (603) of a noise remover (615). The noise removal module (610) may receive weighting information set based on angle information received from the control module (310). The noise removal module (610) may output an enhanced voice signal by mixing the first signal, the second signal, and the weighting information to prevent the signal output from changing abruptly as a part of the housing (210) is folded.

[0059] According to various embodiments, the noise removal module (610) may apply a maximum weight value (1.0) to the second signal output from the second path (603) in the initial stationary state, apply a minimum weight value (0) to the first signal output from the first path (601), and mix the first signal and the second signal to which the respective weights are applied. Here, if the weight is applied as 0, it may mean that beamforming by the beamformer (611) in the first path (601) is applied 100%. According to various embodiments, if at least one processor (120) identifies that a part of the housing (210) is being folded, it may increase the weight applied to the first signal and decrease the weight applied to the second signal based on angle information. For example, at least one processor (120) may adjust the weight applied to the first signal or the second signal at a specified rate whenever the change in angle information obtained while a part of the housing (210) is being folded changes to a specified range. In another example, at least one processor (120) can adjust the weight to be applied to the first signal and / or the second signal with specified weight information corresponding to the angle information obtained while a part of the housing (210) is being folded. According to various embodiments, when the state of the part of the housing (210) being folded is completed, at least one processor (120) can increase the weight of the second signal output to the second path (603), and after a certain period of time, when it is confirmed that the beamformer (611) has stabilized, the weight of the second signal can be adjusted to a maximum value (1.0). According to various embodiments, the noise removal module (610) receives weight information including the adjusted weight of the first signal and / or the second signal based on the angle information from at least one processor (120), and can output a more enhanced voice signal by applying the weight to the first signal and / or the second signal based on the received weight information to remove noise according to the angle change while folding.

[0060] Referring to FIGS. 1, 2 and 3, an electronic device (101) according to various embodiments may implement a software module (e.g., program (140) of FIG. 1) for processing a voice signal to obtain an enhanced voice signal by removing echo and / or noise from a voice signal based on angle information obtained while at least a portion of the housing (210) is folded or unfolded. A memory (130) of the electronic device (101) may store instructions (e.g., instructions) to implement the software module. At least one processor (120) may execute the instructions stored in the memory (130) to implement the software module and may control hardware associated with the function of the software module (e.g., sensor module (176), input module (150), acoustic output module (155), audio module (170), or communication module (190) of FIG. 1). According to one embodiment, the software module may be configured to include an application (e.g., module, manager, or program) related to a call or voice recording. The application may include an application received from an external electronic device (e.g., server (108) or electronic device (102, 104)). According to one embodiment, the application may include a preloaded application or a third-party application downloadable from a server. The components of the software module and the names of the components according to the illustrated embodiment may vary depending on the type of operating system. According to one embodiment, at least a portion of the software module may be implemented as software, firmware, hardware, or a combination of at least two of these. At least a portion of the software module may be implemented (e.g., executed) by a processor (e.g., AP), for example.At least a portion of the software module may include, for example, a module, program, routine, set of instructions, or process for performing at least one function.

[0061] As such, in various embodiments, the main components of the electronic device have been described through the electronic device (101) of FIGS. 1, 2, and 3. However, in various embodiments, the components illustrated in FIGS. 1, 2, and 3 are not all essential components, and the electronic device (101) may be implemented with more components than those illustrated, or with fewer components. Additionally, the positions of the main components of the electronic device (101) described above through FIGS. 1, 2, and 3 may be changed according to various embodiments.

[0062] According to various embodiments, an electronic device (e.g., an electronic device (101) of FIG. 1, FIG. 2 and FIG. 3) comprises a housing configured to fold at least one part, at least one microphone, at least one speaker, an audio module electrically connected to the at least one microphone and the at least one speaker, and at least one processor electrically connected to the audio module, wherein the at least one processor may be configured to identify a voice signal received through the microphone, obtain angle information related to the folding of at least one part of the housing, identify a state in which at least one part of the housing is being folded based on the angle information, adjust control information of the audio module for processing the voice signal based on the angle information, and control the audio module to process the voice signal based on the adjusted control information while at least one part of the housing is being folded.

[0063] According to various embodiments, the at least one processor may be configured to identify a folding speed corresponding to a change in folding angle based on the angle information, identify that at least one part of the housing is in a folding state when the identified folding speed exceeds a specified threshold, and identify that at least one part of the housing is in a stopped state when the identified folding speed is below the specified threshold for a specified time elapsed in the folding state.

[0064] According to various embodiments, the electronic device further includes at least one sensor that detects a folding movement of at least one part of the housing around a hinge structure to acquire the angle information, and the audio module may include a voice signal processing module that performs signal processing to remove echo or noise from the voice signal, and a control module that receives the angle information from the at least one processor and controls the voice signal processing module based on the received angle information.

[0065] According to various embodiments, the control information includes at least one of operation parameter information for voice signal processing of the audio module, the weighting information, or the input sensitivity information of the microphone, and the operation parameter information for voice signal processing may include an echo removal operation parameter and a noise removal operation parameter.

[0066] According to various embodiments, the at least one processor may be configured to adjust the echo removal operation parameter included in the operation parameter information based on the angle information while at least a part of the housing is being folded, and to control the audio module to remove echo from the voice signal based on the echo removal operation parameter adjusted while at least a part of the housing is being folded.

[0067] According to various embodiments, the at least one processor may be configured to remove linear echo components from the voice signal based on an echo removal operation parameter adjusted according to an angle change of the angle information during the folding of at least one part of the housing, and to remove residual echo and non-linear echo components from the voice signal based on an echo removal operation parameter adjusted according to a change in the relative distance between the at least one microphone and the at least one speaker during the folding of at least one part of the housing.

[0068] According to various embodiments, the at least one processor is configured to adjust the noise removal operation parameter included in the operation parameter information based on the change in relative distance between the at least one microphone and the at least one speaker based on the angle information while at least a part of the housing is being folded, and to control the audio module to remove noise from the voice signal based on the noise removal operation parameter adjusted while at least a part of the housing is being folded, and the noise removal operation parameter may include a beamforming bandwidth adjusted according to the angle information.

[0069] According to various embodiments, the audio module includes a noise removal module, and the noise control module may be configured to control the audio module to obtain a noise-removed voice signal by receiving weight information set based on angle information, applying different weight information to each of the first signal and the second signal, and mixing the first signal and the second signal.

[0070] According to various embodiments, the at least one processor may be configured to identify the relative distance between the at least one microphone and the at least one speaker based on the angle information during the folding of at least a portion of the housing, and to control the audio module to adjust the sensitivity of the microphone to a specified low value when the identified relative distance decreases to a specified distance threshold or lower, and to control the audio module to adjust the sensitivity of the microphone to a specified high value when the identified relative distance exceeds the specified distance threshold.

[0071] According to various embodiments, the electronic device may further include a communication module that transmits the voice signal processed based on the regulated control information to an external electronic device.

[0072] FIG. 7 is a drawing showing an example of a method of operation in an electronic device according to various embodiments.

[0073] Referring to FIG. 7, an electronic device according to various embodiments (e.g., the electronic device (101) of FIG. 1, FIG. 2 and FIG. 3) can execute an application related to voice calls or voice recording and perform an operation to process a voice signal received through the executed application.

[0074] In operation 701, an electronic device according to various embodiments may receive a voice signal through a microphone (e.g., input module (150) of FIG. 1, microphone (220) of FIG. 2 and FIG. 3). The voice signal received through the microphone may include the voice signal of a speaker located at a short distance (e.g., near speaker) along with the voice signal of a speaker located at a long distance (e.g., far speaker) and echoes and noise from the signal output from the speaker (230). The received voice signal may be transmitted to an audio module (e.g., audio module (301) of FIG. 3) and processed to obtain an enhanced voice signal by removing the echoes and / or noise contained in the voice signal.

[0075] In operation 703, the electronic device may obtain angle information related to the folding of at least a portion of the housing (e.g., the housing (210) of FIG. 2). The angle information may be calculated by a processor (e.g., the processor (120) of FIG. 1 and FIG. 3) based on movement information of at least a portion of the housing (e.g., the first housing area (211) and / or the second housing area (213) of FIG. 2) detected by at least one sensor (e.g., the sensor module (176) of FIG. 1).

[0076] In operation 705, the electronic device can identify a state in which at least a portion of the housing is being folded based on acquired angle information. For example, if the acquired angle information is below a specified threshold or is in an unfolded state (e.g., the angle between the first housing area (211) and the second housing area (213) in FIGS. 2 and 4 is 180 degrees), the electronic device can identify that at least a portion of the housing is not being folded. For example, if the acquired angle information is an angle greater than or equal to a threshold for a specified angle, or if the folding speed identified based on the angle information exceeds a threshold for a specified speed, the electronic device can identify that at least a portion of the housing (e.g., the first housing area (211) and / or the second housing area (213)) is in a folding state. For example, if the folding speed identified in the folding state is below a specified threshold and a specified time has elapsed, the electronic device can identify that at least a portion of the housing is in a state where the folding operation is completed (or interrupted) (e.g., a stopped folding state). Here, the folding speed may represent the degree to which the angle between the first housing area (211) and the second housing area (213) changes per second.

[0077] In operation 707, the electronic device may adjust control information for processing a received voice signal based on acquired angle information. For example, the control information may be adjusted at a certain ratio from the previous control information whenever the folding angle changes to an angle within a specified range (e.g., 15 degrees) while a part of the housing is being folded by the voice signal processing module (e.g., voice signal processing module (320) of FIG. 3). As another example, the control information may be information pre-specified in correspondence with the angle information and may be stored in advance in a memory (e.g., memory (130) of FIG. 1) in the form of a table. The control information may include at least one of operation parameter information (or factors) for voice signal processing by the voice processing module (320), weighting information related to the beamforming performance of the microphone (220), or input sensitivity information of the microphone (220). Here, the operational parameter information (or factor) for voice signal processing may include at least one of an echo removal operational parameter (or factor) (e.g., intensity of a residual echo remover (RES), linear / non-linear echo removal intensity (e.g., noise removal intensity), automatic gain control (AGC), coefficients of an adaptive filter (e.g., first filter), convergence rate, or echo path change detector) and / or a noise removal operational parameter (or factor) (e.g., beamforming bandwidth). The convergence rate of the adaptive filter may be a factor for controlling the adaptation rate by following the angle change due to folding while a part of the housing (210) is being folded. The intensity of the residual echo remover (RES) (e.g., second filter) may be a factor for removing echoes (e.g., non-linear echo components) caused by the hardware characteristics of the microphone (220) and speaker (230) while at least a part of the housing (210) is being folded.For example, the control information may include preset parameter sets by mapping angle information specified in at least one of motion parameter information, weight information, or input sensitivity information of a microphone for voice signal processing, and the preset parameter sets may be stored in a table form in memory (130).

[0078] In operation 709, the electronic device may perform an operation to obtain an enhanced voice signal by removing echoes and / or noise from the voice signal while at least one part of the housing is being folded based on controlled control information. For example, the electronic device may remove linear components of echoes from the voice signal based on an echo reference by a voice signal processing module, and remove non-linear residual echoes and non-linear components due to hardware characteristics of the microphone (220) and speaker (230). For example, the electronic device may improve the signal-to-noise ratio by removing noise from other directions while preserving the voice signal from the location or direction of the near-end speaker by a voice signal processing module.

[0079] According to various embodiments, when performing the 709 operation, for example, the electronic device may obtain an enhanced voice signal by performing an operation to remove echo from a voice signal and then performing an operation to remove noise from the voice signal from which echo has been removed. In another example, the electronic device may obtain an enhanced voice signal by performing an operation to remove noise from a voice signal and then performing an operation to remove echo from a voice signal. In yet another example, the electronic device may perform either an operation to remove echo or an operation to remove noise from a voice signal.

[0080] According to various embodiments, the electronic device may terminate a control operation for voice processing when the operation of folding a part of the housing is completed and a voice call or voice input (or recording) is completed.

[0081] FIG. 8 is a drawing showing an example of a method of operation in an electronic device according to various embodiments.

[0082] Referring to FIG. 8, an electronic device according to various embodiments (e.g., the electronic device (101) of FIG. 1, FIG. 2 and FIG. 3) may receive a voice signal through a microphone (e.g., the input module (150) of FIG. 1, the microphone (220) of FIG. 2 and FIG. 3) in operation 801. The voice signal received through the microphone may include the voice signal of a speaker located at a short distance (e.g., a near speaker), the voice signal of a speaker located at a long distance (e.g., a far speaker), and echoes and noise from the signal output from the speaker (230).

[0083] In operation 803, the electronic device may obtain angle information related to the folding of at least a portion of the housing (e.g., the housing (210) of FIG. 2). The angle information may be calculated by a processor (e.g., the processor (120) of FIG. 1 and FIG. 3) based on movement information of at least a portion of the housing (e.g., the first housing area (211) and / or the second housing area (213) of FIG. 2) detected by at least one sensor (e.g., the sensor module (176) of FIG. 1).

[0084] In operation 805, the electronic device can determine whether a part of the housing is being folded based on acquired angle information, and whether the angular velocity (hereinafter referred to as folding speed), which indicates the degree of change of the angle of the part being folded per second while the part of the housing is being folded, exceeds a specified threshold value.

[0085] If, as a result of checking in operation 805, the folding speed is less than or equal to a specified threshold, in operation 807, the electronic device identifies that a part of the housing is in a stopped state where it is not folded, and can perform an operation to remove echo using operation parameters set in a voice signal processing module (e.g., voice signal processing module (320) of FIG. 3). After performing operation 807, in operation 815, the electronic device checks whether the execution of an application related to voice calls or voice recording is terminated, for example, or whether the voice processing operation is completed by a specified event; if it is completed, the electronic device terminates the voice processing operation, and if not, the electronic device can continue to perform operation 801.

[0086] If, as confirmed in operation 805, the folding speed exceeds a specified threshold, in operation 809, the electronic device may adjust an operating factor (e.g., a parameter for the update (or convergence) speed of the first filter) to enhance the performance of the first filter (e.g., the first filter (511) of FIG. 5) (adaptive filter) included in the echo removal module included in the control information based on the angle information. The electronic device may adjust a residual echo removal operating factor (e.g., RES strength) included in the control information based on the angle information. For example, if the degree of angle change over time in the angle information decreases while a part of the housing is folding, causing the relative distance between the part being folded (e.g., the first housing area (211)) and the part facing it (e.g., the second housing area (213)) to become closer, the electronic device may increase the operating factor of the first filter (e.g., a parameter for the update (or convergence) speed of the first filter) to follow the instantaneous rapid change in the folding motion (e.g., a change in the acoustic echo path). Accordingly, the first filter updates (or The convergence speed can be adapted quickly. For example, when the degree of angle change over time of the angle information decreases while a part of the housing is being folded, the electronic device can increase the residual echo removal operation factor (e.g., RES strength) when the relative distance between the part being folded (e.g., the first housing area (211) and the part facing it (e.g., the second housing area (213)) becomes closer.

[0087] In the 811 operation, the electronic device can remove the linear echo component from the voice signal through the first filter based on the operating factor of the adjusted first filter.

[0088] In operation 813, the electronic device can remove residual echoes and non-linear echo components caused by hardware characteristics of the microphone and speaker from a voice signal in which linear echo components have been removed through the first filter via a residual echo remover (e.g., the second filter (515) of FIG. 5) of the voice signal processing module. For example, as the residual echo removal operation factor (e.g., RES strength) increases, the residual echo remover of the voice signal processing module can operate with a stronger RES strength when the relative distance between the folding part and the facing part becomes closer than when the relative distance becomes farther.

[0089] In operation 815, the electronic device can determine whether a part of the housing is in a stationary state where it is not folded to the facing part. For example, as shown in (d) of FIG. 4, if the angle information is identified as 0 degrees, the electronic device can identify that the first housing region (211) of the housing (210) is in a completely folded state, and after a certain period of time, identify that it is in a stationary state. The electronic device can terminate the voice signal processing operation if it confirms the stationary state in operation 815 and identifies that the voice signal processing module is in a stable state. For example, if the echo cancellation module of the voice signal processing module is in a stable state, the electronic device can adjust the convergence speed of the echo cancellation module to a small value or return it to its original state to allow it to adapt slowly. Otherwise, the electronic device can perform operation 801 to continue performing the voice signal processing operation.

[0090] FIG. 9 is a drawing showing an example of a method of operation in an electronic device according to various embodiments.

[0091] Referring to FIG. 9, an electronic device according to various embodiments (e.g., the electronic device (101) of FIG. 1, FIG. 2 and FIG. 3) may receive a voice signal through a microphone (e.g., the input module (150) of FIG. 1, the microphone (220) of FIG. 2 and FIG. 3) in operation 901. The voice signal received through the microphone may include the voice signal of a speaker located at a short distance (e.g., a near speaker), the voice signal of a speaker located at a long distance (e.g., a far speaker), and echoes and noise from the signal output from the speaker (230).

[0092] In operation 903, the electronic device may obtain angle information related to the folding of at least a portion of the housing (e.g., the housing (210) of FIG. 2). The angle information may be calculated by a processor (e.g., the processor (120) of FIG. 1 and FIG. 3) based on movement information of at least a portion of the housing (e.g., the first housing area (211) and / or the second housing area (213) of FIG. 2) detected by at least one sensor (e.g., the sensor module (176) of FIG. 1).

[0093] The electronic device can remove noise from the received voice signal by a noise removal module (e.g., the noise removal module (610) of FIG. 6) included in the voice signal processing module (320).

[0094] In operation 905, the electronic device removes noise from a voice signal by passing through a first path (e.g., the first path (601) of FIG. 6) of a beamformer (e.g., the beamformer (611) of FIG. 6) and a noise remover (e.g., the noise remover (613) of FIG. 6), as shown in FIG. 6, and outputs a first signal with noise removed using beamforming. At the same time or sequentially, in operation 907, separate from operation 905, the electronic device removes noise from a voice signal by passing through a second path (e.g., the second path (603) of FIG. 6) of a noise remover (e.g., the noise remover (615) of FIG. 6), as shown in FIG. 6, and outputs a second signal with noise removed without using beamforming.

[0095] In operation 909, the electronic device can obtain weight information to be applied to each of the first signal and the second signal based on the acquired angle information.

[0096] In the 911 operation, the electronic device may mix the first signal, the second signal, and weight information to prevent the signal output from changing abruptly as a part is folded, and output the mixed signal, which is a more enhanced voice signal. According to various embodiments, the electronic device may apply a maximum weight value (1.0) to the second signal output from the second path (603) in the initial stationary state and apply a minimum weight value (0) to the first signal output from the first path (601) by a noise removal module included in the voice processing module, and mix the first signal and the second signal to which the respective weights are applied. Here, if the weight is applied as 0, it may mean that beamforming by the beamformer (611) in the first path (601) is applied 100%.

[0097] According to various embodiments, when the electronic device identifies that a portion of the housing is being folded, it can increase the weight applied to the first signal and decrease the weight applied to the second signal based on angle information. For example, the electronic device can adjust the weight applied to the first signal or the second signal at a specified rate whenever the change in angle information acquired while the portion of the housing is being folded changes to a specified range. In another example, the electronic device can adjust the weight applied to the first signal and / or the second signal with specified weight information corresponding to the angle information acquired while the portion of the housing is being folded. According to various embodiments, when the state of the portion of the housing being folded is completed, the electronic device can increase the weight of the second signal output through the second path, and after a certain period of time, when it is confirmed that the beamformer has stabilized, it can adjust the weight of the second signal to a maximum value (1.0). According to various embodiments, the noise removal module receives weight information from a processor including adjusted weights of a first signal and / or a second signal based on angle information, and applies weights to the first signal and / or the second signal based on the received weight information to remove noise according to the angle change during folding, thereby outputting a more enhanced voice signal.

[0098] According to various embodiments, the electronic device may terminate a control operation for voice processing when the operation of folding a part of the housing is completed and a voice call or voice input (or recording) is completed.

[0099] The echo removal operation by the echo removal module of the voice processing module described in FIG. 8 and the noise removal operation by the noise removal module of the voice processing module described in FIG. 9 may be performed in combination or separately. For example, depending on the configuration of the voice processing module of the electronic device, the electronic device may perform the operation method described in FIG. 8 and then additionally perform the operation described in FIG. 9. As another example, depending on the configuration of the voice processing module of the electronic device, the electronic device may perform the operation described in FIG. 9 and then additionally perform the operation described in FIG. 8.

[0100] When performing the operation described in at least one of the above-described FIGS. 7, 8, and 9, an additional operation to adjust the input sensitivity of the microphone according to angle information may be performed.

[0101] According to various embodiments, the electronic device identifies a distance value between a microphone and a speaker based on angle information while at least a portion of the housing is folded, and when the identified distance value decreases below a specified distance threshold, the input sensitivity of the microphone can be adjusted to a specified low value. When the identified distance value exceeds the specified distance threshold, the electronic device can adjust the input sensitivity of the microphone to a specified high value.

[0102] According to various embodiments, a method of operation in an electronic device (e.g., electronic device (101) of FIGS. 1, 2, and 3) comprising at least one microphone (e.g., microphone (220) of FIGS. 1, 2, and 3) and at least one speaker (e.g., speaker (230) of FIGS. 1, 2, and 3) may include: receiving a voice signal through the at least one microphone; obtaining angle information related to the folding of at least one part of the housing of the electronic device; identifying the state of the at least part of the housing being folded based on the angle information; adjusting control information of an audio module of the electronic device for processing the voice signal based on the angle information; and processing the voice signal based on the adjusted control information while at least one part of the housing is being folded by the audio module.

[0103] According to various embodiments, the operation of identifying a state in which at least a portion of the housing is being folded may include the operation of identifying a folding speed corresponding to a change in folding angle based on the angle information, and the operation of identifying that at least a portion of the housing is being folded when the identified folding speed exceeds a specified threshold.

[0104] According to various embodiments, the method may further include an operation of identifying a stopped state in which at least one part of the housing is stopped from folding when the identified folding speed is below a specified threshold for a specified time while at least one part of the housing is being folded.

[0105] According to various embodiments, the method comprises, wherein the control information includes at least one of operation parameter information for voice signal processing of the audio module, the weighting information, or the input sensitivity information of the microphone, and the operation parameter information for voice signal processing may include an echo removal operation parameter and a noise removal operation parameter.

[0106] According to various embodiments, the operation of processing the voice signal based on the adjusted control information may include the operation of removing linear echo components from the voice signal based on the echo removal operation parameter adjusted according to the angle change of the angle information while at least one part of the housing is folded by the audio module, and the operation of removing residual echo and non-linear echo components from the voice signal based on the echo removal operation parameter adjusted according to the relative distance change between at least one microphone and at least one speaker while at least one part of the housing is folded by the audio module.

[0107] According to various embodiments, the operation of processing the voice signal based on the adjusted control information includes the operation of removing noise from the voice signal based on the noise removal operation parameter adjusted according to the change in relative distance between the at least one microphone and the at least one speaker while at least a part of the housing is folded by the audio module, and the noise removal operation parameter may include a beamforming bandwidth adjusted according to the angle information.

[0108] According to various embodiments, the operation of processing the voice signal based on the adjusted control information may include: an operation of outputting a first signal in which noise is removed according to the beamforming bandwidth through a first path of the noise removal module from the voice signal while at least one part of the housing is folded by the audio module; an operation of outputting a second signal in which noise is removed through a second path of the noise removal module from the voice signal while at least one part of the housing is folded by the audio module; an operation of receiving weight information set based on the angle information by the audio module; an operation of applying different weight information to each of the first signal and the second signal by the audio module; and an operation of obtaining a voice signal with noise removed by mixing the first signal and the second signal by the audio module.

[0109] According to various embodiments, the operation of processing the voice signal based on the adjusted control information comprises: adjusting the sensitivity of the microphone to a specified low value when the relative distance between the at least one microphone and the at least one speaker, identified based on the angle information while at least a part of the housing is being folded by the audio module, decreases to a specified distance threshold or lower;

[0110] When the relative distance identified by the audio module exceeds a specified distance threshold, the operation may include adjusting the sensitivity of the microphone to a specified high value.

[0111] According to various embodiments, a non-transient storage medium storing a program may include an executable instruction such that, when executed by a processor, the program performs the following operations: receiving a voice signal through at least one microphone; obtaining angle information related to the folding of at least one part of the housing of the electronic device; identifying the state of at least one part of the housing being folded based on the angle information; adjusting control information of an audio module of the electronic device for processing the voice signal based on the angle information; and processing the voice signal based on the adjusted control information while at least one part of the housing is being folded by the audio module.

[0112] Furthermore, the embodiments disclosed in this document are presented for the purpose of explaining and understanding the disclosed technical content and are not intended to limit the scope of the technology described in this document. Accordingly, the scope of this document should be interpreted to include all modifications or various other embodiments based on the technical concept of this document.

[0113] The electronic device according to the various embodiments disclosed in this document may be of various forms. The electronic device may include, for example, a portable communication device (e.g., a smartphone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a consumer electronics device. The electronic device according to the embodiments of this document is not limited to the devices described above.

[0114] The various embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of said items unless the relevant context clearly indicates otherwise. In this document, phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B or C," "at least one of A, B and C," and "at least one of A, B, or C" may each include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used simply to distinguish said components from other said components and do not limit said components in any other aspect (e.g., importance or order). Where any (e.g., 1st) component is referred to as “coupled” or “connected” to another (e.g., 2nd) component, with or without the terms “functionally” or “communicationly,” it means that said any component may be connected to said other component directly (e.g., via a wire), wirelessly, or through a third component.

[0115] The term “module” as used in the various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. A module may be a component formed integrally, or a minimum unit of said component or a part thereof that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0116] Various embodiments of the present document may be implemented as software (e.g., program (140)) comprising one or more instructions stored in a storage medium (e.g., internal memory (136) or external memory (138)) readable by a machine (e.g., electronic device (101)). For example, a processor (e.g., processor (120)) of the machine (e.g., electronic device (101)) may call at least one of the one or more instructions stored in the storage medium and execute it. This enables the machine to be operated to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The storage medium readable by the machine may be provided in the form of a non-transitory storage medium. Here, 'non-temporary' simply means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily.

[0117] According to one embodiment, the method according to the various embodiments disclosed herein may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or an application store (e.g., Play Store). TM It can be distributed online (e.g., downloaded or uploaded) through ) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0118] According to various embodiments, each component (e.g., module or program) of the components described above may include a singular or multiple entities, and some of the multiple entities may be separated and placed in other components. According to various embodiments, one or more of the components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Generally or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the multiple components in the same or similar manner as those performed by the corresponding component among the multiple components prior to integration. According to various embodiments, operations performed by the module, program, or other components may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added. Explanation of the symbols

[0120] 100: Network Environment 101: Electronic Device 120: Processor 130: Memory 160: Display module 190: Communication module 210: Housing 220: Microphone 230: Speaker

Claims

Claim 1 In an electronic device, a housing configured such that at least one part is foldable; at least one microphone; at least one speaker; and an audio module electrically connected to the at least one microphone and the at least one speaker; An electronic device comprising at least one processor electrically connected to the audio module, wherein the at least one processor is configured to identify a voice signal received through the microphone, acquire angle information related to the folding of at least one part of the housing, identify a state in which at least one part of the housing is being folded based on the angle information, adjust control information of the audio module for processing the voice signal based on the angle information, and control the audio module to process the voice signal based on the adjusted control information while at least one part of the housing is being folded, and wherein the at least one processor is configured to remove linear echo components from the voice signal based on an echo removal operation parameter adjusted according to an angle change of the angle information while at least one part of the housing is being folded, and to remove residual echo and non-linear echo components from the voice signal based on an echo removal operation parameter adjusted according to a change in the relative distance between at least one microphone and at least one speaker while at least one part of the housing is being folded. Claim 2 In claim 1, the at least one processor is configured to identify a folding speed corresponding to a change in folding angle based on the angle information, identify that at least one part of the housing is in a folding state when the identified folding speed exceeds a specified threshold, and identify that at least one part of the housing is in a stopped state when the identified folding speed is below the specified threshold for a specified time elapsed in the folding state, and the electronic device further includes at least one sensor that detects the folding movement of at least one part of the housing centered on a hinge structure to acquire the angle information, and the audio module includes a voice signal processing module that performs signal processing to remove echo or noise from the voice signal; and a control module that receives the angle information from the at least one processor and controls the voice signal processing module based on the received angle information, wherein the control information includes at least one of operation parameter information for voice signal processing of the audio module, weighting information, or input sensitivity information of the microphone, and the operation parameter information for voice signal processing includes an echo removal operation parameter and a noise removal operation parameter. Claim 3 delete Claim 4 delete Claim 5 An electronic device according to paragraph 2, wherein the at least one processor is configured to control the audio module to remove echo from the voice signal based on the at least one echo removal operation parameter included in the operation parameter information based on the angle information while at least a part of the housing is being folded, and based on the echo removal operation parameter controlled while at least a part of the housing is being folded. Claim 6 delete Claim 7 An electronic device according to paragraph 2, wherein the at least one processor is configured to control the noise removal operation parameter included in the operation parameter information according to the change in relative distance between the at least one microphone and the at least one speaker based on the angle information while at least a part of the housing is being folded, and to control the audio module to remove noise from the voice signal based on the noise removal operation parameter controlled while at least a part of the housing is being folded, and the noise removal operation parameter includes a beamforming bandwidth adjusted according to the angle information. Claim 8 An electronic device according to claim 1, wherein the audio module includes a noise removal module, and the noise removal module outputs a first signal in which noise is removed according to a beamforming bandwidth from the voice signal through a first path of the noise removal module while at least one part of the housing is folded, outputs a second signal in which noise is removed from the voice signal through a second path of the noise removal module while at least one part of the housing is folded, receives weighting information set based on the angle information, applies different weighting information to each of the first signal and the second signal, and controls the audio module to obtain a noise-removed voice signal by mixing the first signal and the second signal. Claim 9 The electronic device according to claim 1, wherein the at least one processor is configured to identify the relative distance between the at least one microphone and the at least one speaker based on the angle information during the folding of at least a portion of the housing, and to control the audio module to adjust the sensitivity of the microphone to a specified low value when the identified relative distance decreases to a specified distance threshold or less, and to control the audio module to adjust the sensitivity of the microphone to a specified high value when the identified relative distance exceeds the specified distance threshold, and the electronic device further comprises a communication module that transmits the voice signal processed based on the adjusted control information to an external electronic device. Claim 10 delete Claim 11 A method of operation in an electronic device comprising at least one microphone and at least one speaker, comprising: receiving a voice signal through the at least one microphone; obtaining angle information related to the folding of at least one part of the housing of the electronic device; identifying the state of the at least part of the housing being folded based on the angle information; adjusting control information of an audio module of the electronic device for processing the voice signal based on the angle information; and processing the voice signal based on the adjusted control information while the at least part of the housing is being folded by the audio module, wherein the operation of processing the voice signal comprises: removing a linear echo component from the voice signal based on an echo removal operation parameter adjusted according to an angle change of the angle information while the at least part of the housing is being folded; and removing residual echo and non-linear echo components from the voice signal based on an echo removal operation parameter adjusted according to a change in the relative distance between the at least one microphone and the at least one speaker while the at least part of the housing is being folded. Claim 12 In claim 11, the operation of identifying a state in which at least a portion of the housing is being folded comprises: identifying a folding speed corresponding to a change in folding angle based on the angle information; and identifying that at least a portion of the housing is being folded when the identified folding speed exceeds a specified threshold, wherein the method further comprises identifying a stopped state in which at least a portion of the housing is being folded when a specified time elapses at the identified folding speed below a specified threshold while at least a portion of the housing is being folded. Claim 13 delete Claim 14 delete Claim 15 delete Claim 16 delete Claim 17 delete Claim 18 delete Claim 19 delete Claim 20 A non-transient storage medium storing a program, wherein the program comprises, when executed by a processor, an executable instruction to execute the following operations: receiving a voice signal through at least one microphone; obtaining angle information related to the folding of at least one part of the housing of an electronic device; identifying the folding state of at least one part of the housing based on the angle information; adjusting control information of an audio module of the electronic device for processing the voice signal based on the angle information; and processing the voice signal based on the adjusted control information while at least one part of the housing is being folded by the audio module, wherein the executable instruction includes removing a linear echo component from the voice signal based on an echo removal operation parameter adjusted according to an angle change of the angle information while at least one part of the housing is being folded, and removing residual echo and non-linear echo components from the voice signal based on an echo removal operation parameter adjusted according to a change in the relative distance between at least one microphone and at least one speaker while at least one part of the housing is being folded.

Images