Sound collection method and related apparatus

Abstract

The present application relates to the technical field of electronics, and discloses a sound collection method and a related apparatus. The method is applied to a foldable electronic device. The electronic device comprises a display screen, a first microphone, a second microphone, and a third microphone, wherein the first microphone is arranged at the top end of the electronic device, the second microphone is arranged at the bottom end of the electronic device, and the third microphone is arranged at a side edge of the electronic device. When the electronic device is in a folded state, upon detecting that a microphone function of the electronic device is enabled, an acoustic wave signal can be collected by means of the third microphone. Since the third microphone is arranged on the side edge of the electronic device, when a user holds the electronic device that is in the folded state, the third microphone is unlikely to be blocked, thereby reducing the risk of microphone blockage and improving the quality of sound collection.

Description

A sound acquisition method and related device

[0001] This application claims priority to Chinese Patent Application No. 202411661762.6, filed on November 19, 2024, entitled "A Sound Acquisition Method and Related Apparatus", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of electronic technology, and in particular to a sound acquisition method and related apparatus. Background Technology

[0003] With the continuous development of electronic technology, more and more electronic devices are equipped with microphones to facilitate user sound collection. Microphones on electronic devices with foldable displays are typically located at the top and bottom of the device, resulting in a poor user experience. Summary of the Invention

[0004] This application provides a sound acquisition method and related apparatus, which sets up a microphone on the side of the electronic device, reducing the risk of the microphone being blocked and improving the quality of sound acquisition.

[0005] In a first aspect, this application provides a sound acquisition method applied to a foldable electronic device. The electronic device includes a display screen, a first microphone, a second microphone, and a third microphone, wherein the first microphone is disposed at the top of the electronic device, the second microphone is disposed at the bottom of the electronic device, and the third microphone is disposed at the side of the electronic device. The method includes: detecting that the electronic device has enabled the microphone function when the electronic device is in a folded state; and acquiring sound wave signals through the third microphone.

[0006] In this way, when the electronic device is folded, the user may easily block the microphones at the top and / or bottom when holding the device. The electronic device can adjust to use a third microphone on the side to collect sound signals when it detects that it is folded. Because the third microphone is located on the side of the electronic device, it is difficult for it to be blocked when the user is holding the folded device. Therefore, the risk of microphone blockage is reduced, and the quality of sound acquisition is improved.

[0007] In one possible implementation, before acquiring the acoustic signal via the third microphone, the method further includes detecting that the first microphone and / or the second microphone is blocked.

[0008] In this way, if the first and / or second microphones are detected to be blocked, the electronic device can use the third microphone on the side to collect sound, avoiding poor sound pickup due to blockage and improving the quality of sound collection.

[0009] In one possible implementation, the electronic device further includes a gravity sensor and / or a gyroscope sensor; detecting that the first microphone and / or the second microphone is blocked specifically includes: detecting the way the electronic device is held using the gravity sensor and / or the gyroscope sensor; and determining that the first microphone and / or the second microphone is blocked based on the way the electronic device is held.

[0010] The electronic device can detect the way it is held using a gravity sensor and / or a gyroscope sensor, and determine whether the first and second microphones are blocked based on this holding method. If the device is detected to be held in a preset manner, and the first and / or second microphones are determined to be blocked, the third microphone on the side can be used to collect sound.

[0011] In one possible implementation, the electronic device includes a low-power camera; detecting that the first microphone and / or the second microphone is blocked specifically includes: detecting the way the electronic device is held via the low-power camera; and determining that the first microphone and / or the second microphone is blocked based on the way the electronic device is held.

[0012] In this way, images can be captured using a low-power camera, and the way the electronic device is held can be determined based on the positional relationships of objects in the captured images. This holding method can then be used to determine whether the first and second microphones are blocked. If it is determined that the first and / or second microphones are blocked, a third microphone can be used to capture sound.

[0013] In one possible implementation, detecting that the first microphone and / or the second microphone is blocked specifically includes: acquiring a first test signal through the first microphone; determining that the first microphone is blocked based on the first test signal; and / or, acquiring a second test signal through the second microphone; determining that the second microphone is blocked based on the second test signal.

[0014] In this way, test signals can be directly acquired through the microphone under test (e.g., the first microphone and the second microphone), and it can be determined whether the microphone under test is blocked based on the relationship between the strength of the test signal and a preset strength threshold. If the strength of the test signal is less than or equal to the preset strength threshold, it is determined that the microphone acquiring the test signal is blocked; if the strength of the test signal is greater than the preset strength threshold, it is determined that the microphone acquiring the test signal is not blocked.

[0015] In one possible implementation, detecting that the electronic device has enabled the microphone function includes: detecting that the electronic device is making or receiving a phone call; or detecting that the electronic device has started recording.

[0016] In this way, in call or recording scenarios, the system can determine whether to use a third microphone to collect sound based on the device configuration of the electronic device, thus avoiding blocking the microphone.

[0017] In one possible implementation, the electronic device further includes a fourth microphone, which is disposed on the side of the electronic device, and the third and fourth microphones are disposed on different sides of the electronic device; when the electronic device is in a folded state, the activation of the microphone function of the electronic device is detected, specifically including: receiving a first operation to enable the directional sound acquisition function; the method further includes: in response to the first operation, determining the location of a second sound source; constructing a second beam based on the location of the second sound source; and acquiring sound wave signals through the third microphone, specifically including: acquiring sound wave signals through the third and fourth microphones based on the second beam.

[0018] Thus, in a directional sound acquisition scenario, if the electronic device is in a folded state, it can construct a second beam based on the direction of the second sound source, with the second beam pointing towards that direction. At this time, the electronic device uses multiple microphones to acquire sound wave signals based on the second beam, thereby enhancing the sound at the direction of the second sound source and suppressing sound from other directions.

[0019] In one possible implementation, in response to the first operation, determining the location of the second sound source specifically includes: determining the location of the second sound source based on the first operation; or, acquiring location sound wave signals through a third microphone and a fourth microphone; and determining the location of the second sound source based on the location sound wave signals.

[0020] In this way, the electronic device can determine the location of the second sound source based on the user's first operation, or it can collect the location sound wave signal on its own and determine the location of the second sound source based on the location sound wave signal.

[0021] In one possible implementation, when the electronic device is not in a folded state, the method further includes: receiving a second operation to enable directional sound acquisition; in response to the second operation, determining the location of a third sound source; constructing a third beam based on the location of the third sound source; and acquiring sound wave signals through a third microphone, specifically including: acquiring sound wave signals through a third microphone, a first microphone, and a second microphone based on the third beam.

[0022] Thus, in a directional sound acquisition scenario, if the electronic device is not in a folded state, it can construct a third beam based on the direction of the third sound source, with the third beam pointing towards that direction. At this point, the electronic device uses multiple microphones to acquire sound signals based on the third beam, thereby enhancing the sound from the direction of the third sound source and suppressing sound from other directions.

[0023] In one possible implementation, when the device form of the electronic device is not folded, the method further includes: receiving a third operation to record video; in response to the third operation, determining a first sound source region; constructing a fourth beam based on the first sound source region; and acquiring sound wave signals through a third microphone, specifically including: acquiring sound wave signals based on the fourth beam through the third microphone, the first microphone, and the second microphone.

[0024] Thus, in a video recording scenario, if the electronic device is not in a folded state, it can construct a fourth beam based on the first sound source region, with the fourth beam pointing towards the first sound source region. At this time, the electronic device uses multiple microphones to collect sound wave signals based on the fourth beam, thereby amplifying the sound from the first sound source region and suppressing sound from other regions.

[0025] In one possible implementation, when the electronic device is in a folded state, the method further includes: receiving a third operation to record video; in response to a fourth operation, determining a second sound source region; constructing a fifth beam based on the second sound source region; and acquiring sound wave signals through a third microphone, specifically including: acquiring sound wave signals based on the fifth beam through the third microphone, the first microphone, and the second microphone.

[0026] Thus, in a video recording scenario, if the electronic device is in a folded state, it can construct a fifth beam based on the second sound source region, with the fifth beam pointing towards that region. At this time, the electronic device uses multiple microphones to collect sound wave signals based on the fifth beam, thereby amplifying the sound from the second sound source region and suppressing sound from other regions.

[0027] Secondly, this application provides a sound acquisition method applied to a foldable electronic device. The electronic device includes a display screen, a first microphone, a second microphone, and a third microphone, wherein the first microphone is disposed at the top of the electronic device, the second microphone is disposed at the bottom of the electronic device, and the third microphone is disposed at the side of the electronic device. The method includes: when the electronic device is in a folded state, detecting that the call mode of the electronic device has switched to a hands-free mode; and acquiring sound wave signals through the third microphone.

[0028] Thus, when the electronic device is folded and in hands-free calling mode, because the speaker is generally located at the top or bottom, the distance between the top microphone and the top speaker, and between the bottom microphone and the bottom speaker microphone, is too close. The sound wave signal emitted by the speaker can interfere with the sound pickup of the top and bottom microphones. When the electronic device detects that it is in hands-free calling mode, it can use the side microphone to collect sound wave signals, reducing echo interference and improving the sound pickup quality. In one possible implementation, the method further includes: detecting the way the electronic device is held; after detecting that the calling mode of the electronic device has switched to hands-free mode, outputting a first prompt based on the way the electronic device is held, the first prompt being used to prompt the user to rotate the electronic device so that the third microphone is closer to the user's mouth.

[0029] In this way, during hands-free calling, the electronic device can also prompt the user to bring the side microphone close to the user's mouth so as to capture the user's voice.

[0030] In one possible implementation, the electronic device further includes a fourth microphone, which is disposed on the side of the electronic device, and the third microphone and the fourth microphone are disposed on different sides of the electronic device; after detecting that the electronic device has switched its call mode to hands-free mode, the method further includes: obtaining the location of a first sound source; constructing a first beam based on the location of the first sound source; and acquiring sound wave signals through the third microphone, specifically including: acquiring sound wave signals through the third microphone and the fourth microphone based on the first beam.

[0031] In this way, in hands-free calling mode, the electronic device can also determine the location of the first sound source and construct a first beam based on the location of the first sound source, so as to enhance the sound emitted by the user, suppress the sound from other directions, and improve the quality of the call.

[0032] In one possible implementation, obtaining the location of the first sound source specifically includes: detecting the way the electronic device is held; and determining the location of the first sound source based on the way the electronic device is held.

[0033] In this way, the electronic device can determine the location of the first sound source based on how the electronic device is held.

[0034] In one possible implementation, obtaining the location of the first sound source specifically includes: acquiring location sound wave signals through a third microphone and a fourth microphone; and determining the location of the first sound source based on the location sound wave signals.

[0035] In this way, electronic devices can determine the location of the first sound source based on the positioning sound wave signal.

[0036] Thirdly, this application provides a sound acquisition method applied to a foldable electronic device. The electronic device includes a display screen, a first microphone, a second microphone, a third microphone, a fourth microphone, and a fifth microphone. The first microphone is located at the top of the electronic device, the second microphone is located at the bottom of the electronic device, the third and fourth microphones are located on the two sides of the electronic device, and the fifth microphone is located on the back of the electronic device. The method includes: receiving an operation to record spatial audio when the electronic device is in a folded state; acquiring a first sound wave signal for a first time period through the second microphone; acquiring a second sound wave signal for the first time period through the third microphone; acquiring a third sound wave signal for the first time period through the fourth microphone; acquiring a fourth sound wave signal for the first time period through the fifth microphone; and generating a first audio file based on the first, second, third, and fourth sound wave signals.

[0037] Thus, in a spatial audio recording scenario, if the electronic device is in a folded state, it can determine whether to use the second, third, fourth, and fifth microphones to record spatial audio. At this time, the electronic device can use these microphones to acquire sound wave signals and then generate a spatial audio file based on the acquired sound wave signals, thereby providing users with richer recording effects.

[0038] In one possible implementation, generating a first audio file based on a first sound wave signal, a second sound wave signal, a third sound wave signal, and a fourth sound wave signal specifically includes: determining an X-dimensional component based on the first and second sound wave signals; determining a Y-dimensional component based on the second and third sound wave signals; determining a Z-dimensional component based on the second and fourth sound wave signals; and generating the first audio file based on the X-dimensional component, the Y-dimensional component, and the Z-dimensional component.

[0039] In this way, in the folded state, the components of sound in each dimension can be determined based on the collected sound wave signal, thereby generating a spatial audio file.

[0040] In one possible implementation, the method further includes: receiving an operation to record spatial audio when the electronic device is in an unfolded state; acquiring a fifth sound wave signal for a second time period via a second microphone; acquiring a sixth sound wave signal for a second time period via a third microphone; acquiring a seventh sound wave signal for a second time period via a fourth microphone; acquiring an eighth sound wave signal for a second time period via a fifth microphone; and generating a second audio file based on the fifth, sixth, seventh, and eighth sound wave signals.

[0041] Thus, in a spatial audio recording scenario, if the electronic device is in its deployed state, it can determine whether to use the second, third, fourth, and fifth microphones to record spatial audio. At this time, the electronic device can use these microphones to acquire sound wave signals and then generate a spatial audio file based on the acquired sound wave signals, thereby providing users with richer recording effects.

[0042] In one possible implementation, a second audio file is generated based on the fifth, sixth, seventh, and eighth sound wave signals, specifically including: determining the X-dimensional component based on the fifth and sixth sound wave signals; determining the Y-dimensional component based on the sixth and seventh sound wave signals; determining the Z-dimensional component based on the sixth and eighth sound wave signals; and generating the second audio file based on the X-dimensional, Y-dimensional, and Z-dimensional components.

[0043] In this way, in the unfolded state, the components of sound in each dimension can be determined based on the collected sound wave signal, thereby generating a spatial audio file.

[0044] In one possible implementation, the method further includes: receiving an operation to record spatial audio when the electronic device is in a semi-folded state; acquiring a ninth sound wave signal of a second time period through a first microphone; acquiring a tenth sound wave signal of a second time period through a second microphone; acquiring an eleventh sound wave signal of a second time period through a third microphone; acquiring a twelfth sound wave signal of a second time period through a fourth microphone; and generating a third audio file based on the ninth, tenth, eleventh, and twelfth sound wave signals.

[0045] Thus, in a spatial audio recording scenario, if the electronic device is in a semi-folded state, it can determine whether to use the first, second, third, and fourth microphones to record spatial audio. At this time, the electronic device can use the first, second, third, and fourth microphones to collect sound wave signals, and then generate a spatial audio file based on the collected sound wave signals, thereby providing users with richer recording effects.

[0046] In one possible implementation, a third audio file is generated based on the ninth, tenth, eleventh, and twelfth sound wave signals, specifically including: determining the X-dimensional component based on the eleventh and twelfth sound wave signals; determining the Y-dimensional component based on the ninth and tenth sound wave signals; determining the Z-dimensional component based on the tenth and eleventh sound wave signals; and generating the third audio file based on the X-dimensional, Y-dimensional, and Z-dimensional components.

[0047] In this way, in the semi-folded state, the components of sound in each dimension can be determined based on the collected sound wave signal, thereby generating a spatial audio file.

[0048] Fourthly, this application provides an electronic device, which includes one or more processors, one or more memories, a foldable display screen, a first microphone, a second microphone, and a third microphone; wherein the one or more memories are coupled to one or more processors, and the one or more memories are used to store computer instructions, which, when the one or more processors execute the computer instructions, implement the sound acquisition method in any possible implementation of any of the above aspects.

[0049] Fifthly, this application provides a chip system comprising: a processing circuit and an interface circuit, wherein the interface circuit is used to receive code instructions and transmit them to the processing circuit, and the processing circuit is used to execute the code instructions to perform the sound acquisition method in any possible implementation of any of the above aspects.

[0050] Sixthly, this application provides a readable storage medium storing computer instructions that, when executed by a processor, implement the sound acquisition method in any of the possible implementations of any of the above aspects.

[0051] In a seventh aspect, this application provides a computer program product comprising computer instructions that, when executed by a processor, implement the sound acquisition method in any of the possible implementations of any of the above aspects.

[0052] The beneficial effects of aspects four through seven can be referenced from the beneficial effects of aspects one through three mentioned above. Attached Figure Description

[0053] Figures 1A-1D are schematic diagrams of the device configuration of an electronic device provided in an embodiment of this application;

[0054] Figure 2 is a schematic diagram of a scenario in which a user holds a folded electronic device to make a voice call, according to an embodiment of this application.

[0055] Figure 3A is a schematic diagram of the principle of an echo double-talk effect provided in an embodiment of this application;

[0056] Figure 3B is a schematic diagram of the positional relationship between the microphone and the speaker when the electronic device A is in a folded state, according to an embodiment of this application.

[0057] Figure 3C is a schematic diagram of a spatial audio technology provided in an embodiment of this application;

[0058] Figure 3D is a schematic diagram of the microphone position when an electronic device records an audio file according to an embodiment of this application;

[0059] Figures 4A-4D are schematic diagrams of the device configuration of an electronic device provided in an embodiment of this application;

[0060] Figure 5 is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application;

[0061] Figure 6A is a schematic diagram of a scenario in which a user holds a folded electronic device to make a voice call, according to an embodiment of this application;

[0062] Figure 6B is a schematic diagram of the positional relationship between the microphone and the speaker when the electronic device is in a folded state, according to an embodiment of this application.

[0063] Figure 6C is a schematic flowchart of a sound acquisition method provided in an embodiment of this application;

[0064] Figures 7A-7C are schematic diagrams illustrating how a user holds an electronic device in an unfolded state, according to an embodiment of this application.

[0065] Figures 7D-7E are schematic diagrams illustrating a user's grip on an electronic device in a semi-folded state, according to an embodiment of this application.

[0066] Figures 7F-7I are schematic diagrams illustrating how a user holds an electronic device in a folded state, according to an embodiment of this application.

[0067] Figure 7J is a flowchart illustrating another sound acquisition method provided in an embodiment of this application;

[0068] Figure 8A is a schematic diagram of a scenario for recording spatial audio when an electronic device is in an unfolded state, according to an embodiment of this application.

[0069] Figure 8B is a schematic diagram of a scenario for recording spatial audio when an electronic device is in a semi-folded state, according to an embodiment of this application.

[0070] Figure 8C is a schematic diagram of a scenario for recording spatial audio when an electronic device is in a folded state, according to an embodiment of this application.

[0071] Figure 8D is a schematic flowchart of a sound acquisition method in a spatial audio recording scenario provided by an embodiment of this application;

[0072] Figure 9 is a flowchart illustrating another sound acquisition method in a spatial audio recording scenario provided in an embodiment of this application;

[0073] Figure 10A is a schematic diagram of an acoustic signal processing model provided in the application embodiment;

[0074] Figure 10B is a schematic diagram of a spatial audio algorithm model provided in an embodiment of this application;

[0075] Figure 11A is a schematic diagram of the direction of the sound wave beam when the electronic device is in an interview scenario, according to an embodiment of this application.

[0076] Figure 11B is a schematic diagram of the direction of the sound wave beam when the electronic device is in a call scenario, according to an embodiment of this application.

[0077] Figure 11C is a schematic diagram of the direction of the sound wave beam when the electronic device is in an interview scenario, according to another embodiment of this application.

[0078] Figure 12 is a schematic flowchart of another method for electronic devices to acquire sound based on simulated acoustic beams, provided in an embodiment of this application.

[0079] Figure 13 is a schematic diagram of a sound source location identification model provided in an embodiment of this application;

[0080] Figure 14 is a schematic diagram of an acoustic beamforming model provided in an embodiment of this application;

[0081] Figure 15A is a schematic diagram of the direction of the sound wave beam when an electronic device is recording video, according to an embodiment of this application.

[0082] Figure 15B is a schematic diagram of the direction of the sound wave beam when an electronic device is recording video, according to an embodiment of this application.

[0083] Figure 16 is a schematic flowchart of a method for an electronic device to collect sound during video recording, according to an embodiment of this application.

[0084] Figure 17 is a schematic diagram of the functional modules of an electronic device provided in an embodiment of this application;

[0085] Figure 18 is a schematic diagram of the physical structure of an electronic device provided in an embodiment of this application;

[0086] Figure 19 is a schematic flowchart of a sound acquisition method provided in an embodiment of this application;

[0087] Figure 20 is a flowchart illustrating another sound acquisition method provided in an embodiment of this application;

[0088] Figure 21 is a flowchart illustrating another sound acquisition method provided in an embodiment of this application. Detailed Implementation

[0089] The technical solutions in the embodiments of this application will be clearly and thoroughly described below with reference to the accompanying drawings. In the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B; the word "and / or" in the text is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, in the description of the embodiments of this application, "multiple" refers to two or more than two.

[0090] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0091] The term "user interface (UI)" used in the following embodiments of this application refers to the medium interface through which an application or operating system interacts and exchanges information with the user. It realizes the conversion between the internal form of information and the form that the user can accept. The user interface is source code written in a specific computer language such as Java or Extensible Markup Language (XML). The interface source code is parsed and rendered on the electronic device, ultimately presenting content that the user can recognize. A common form of user interface is the graphical user interface (GUI), which refers to a user interface related to computer operation displayed graphically. It can be visible interface elements such as text, icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, and widgets displayed on the screen of an electronic device.

[0092] The following describes the device configuration of an electronic device A provided in an embodiment of this application.

[0093] Figures 1A-1D show schematic diagrams of the device configuration of an electronic device A provided in an embodiment of this application.

[0094] As shown in Figure 1A, electronic device A may include a foldable display screen 11, a top microphone (mic) 12, a bottom microphone 13, a top speaker (speaker) 14, and a bottom speaker 15, and may also include a rear camera (not shown in Figure 1A). Electronic device A can be approximated as an imperfect hexahedron, with a certain curvature at the connection points between adjacent faces. Of the six faces of electronic device A, the face containing the foldable display screen 11 can be called the front, and the face opposite the front is called the back (i.e., the back of electronic device A). The rear camera can be located on the back of electronic device A. Optionally, a front camera can also be located on the front of electronic device A. There may be four sides between the front and back of electronic device A. Two opposite sides may each have a speaker; these two sides can be called the top and bottom of electronic device A. The speaker at the top is called the top speaker 14, and the speaker at the bottom is called the bottom speaker 15. Furthermore, the top microphone 12 and the bottom microphone 13 are also located at the top and bottom of electronic device A, respectively. Of the four sides, the two opposite sides other than the top and bottom can be referred to as the left and right sides of electronic device A, respectively. The left and right sides of electronic device A can also be collectively referred to as the sides of electronic device A. Optionally, one or more buttons, such as volume buttons and power buttons, can be provided on the right side of electronic device A.

[0095] The foldable display screen 11 may include a display area 11A and a display area 11B. A hinge 11C exists between display areas 11A and 11B, allowing them to fold towards each other along the hinge 11C, such that the angle between display areas 11A and 11B varies between 0° and 180°. In the embodiment shown in Figure 1A, the angle between display areas 11A and 11B is approximately 180°, at which point the foldable display screen 11 is in a fully unfolded state. When the angle between display areas 11A and 11B falls within a preset unfolded angle range (e.g., greater than or equal to 150° and less than or equal to 180°), the electronic device A can be considered to be in an unfolded state.

[0096] As shown in Figure 1B, in the foldable display screen 11 of electronic device A, the angle between display area 11A and display area 11B is approximately 90°. At this time, the foldable display screen 11 is in a semi-folded state. When the angle between display area 11A and display area 11B is within a preset semi-folded angle range (e.g., greater than or equal to 30° and less than or equal to 150°), electronic device A can be considered to be in a semi-folded state.

[0097] As shown in Figure 1C, in the foldable display screen 11 of electronic device A, the angle between display area 11A and display area 11B is approximately 0°. At this time, the foldable display screen 11 is in a fully folded state. When the angle between display area 11A and display area 11B is within a preset folding angle range (e.g., greater than or equal to 0° and less than or equal to 30°), electronic device A can be considered to be in a folded state.

[0098] The embodiment shown in Figure 1C illustrates the A-side of the electronic device A in a folded state, i.e., the back side of the display area 11A. A rear-facing camera may also be mounted on the back side of the display area 11A.

[0099] As shown in Figure 1D, in the foldable display screen 11 of electronic device A, the angle between display area 11A and display area 11B is 0°. At this time, the foldable display screen 11 is in a fully folded state, and the device form of electronic device A in this state can be called the folded state. The embodiment shown in Figure 1D illustrates the B side of electronic device A in the folded state, that is, the back side of display area 11B.

[0100] Figure 2 shows a schematic diagram of a scenario where a user holds a folded electronic device A to make a voice call, according to an embodiment of this application.

[0101] When electronic device A is folded and held by a user with one hand, as shown in Figure 2, there is a high probability that the top microphone 12 and bottom microphone 13 of electronic device A will be blocked by the user's finger. In this situation, if electronic device A is in a call, it needs to collect the user's sound wave signal through the top microphone 12 and bottom microphone 13 and send the collected sound wave signal to the other electronic device. When the top microphone 12 and bottom microphone 13 are blocked by the user's finger, electronic device A cannot properly collect the sound wave signal emitted by the user through the top microphone 12 and bottom microphone 13, which affects the user's normal use of the call function.

[0102] The echo double-talk effect is described below.

[0103] Figure 3A shows a schematic diagram of the principle of an echo double-talk effect provided in an embodiment of this application.

[0104] As shown in Figure 3A, user P1 can conduct a voice call with user P2 using electronic device B via network transmission through electronic device A. Electronic device A used by user P1 may include a microphone M1 and a speaker S1, while electronic device B used by user P2 may include a microphone M2 and a speaker S2.

[0105] During a voice call between user P1 and user P2, microphone M1 can capture user P1's acoustic signal As1 and transmit it to electronic device B via the network. Electronic device B can then play user P1's acoustic signal As1 to user P2 via speaker S2. Similarly, microphone M2 can capture user P2's acoustic signal As2 and transmit it to electronic device A via the network. Electronic device A can then play user P2's acoustic signal As2 to user P1 via speaker S1. In this way, voice communication between multiple electronic devices can be achieved.

[0106] It should be noted that during a voice call between user P1 and user P2, if user P1 is emitting an acoustic signal As1 while speaker S1 is simultaneously playing user P2's acoustic signal As2 transmitted over the network, microphone M1 will not only capture user P1's acoustic signal As1 but also the echo signal Es2 caused by the acoustic signal As2 played by speaker S1. The presence of echo signal Es2 will interfere with microphone M1's acquisition of acoustic signal As1, affecting the normal conversation between user P1 and user P2.

[0107] This interference between simultaneous voice activity at both ends during voice communication and the resulting echo can be called the echo double-talk effect. To reduce or eliminate the impact of echo signals on normal communication, built-in echo cancellation algorithms can be used to suppress the echo signals.

[0108] Generally, with the built-in echo cancellation algorithm remaining unchanged, the closer the distance between speaker S1 and microphone M1, the more difficult it becomes for the built-in echo cancellation algorithm in electronic device A to distinguish between the sound wave signal As1 emitted by user P1 and the echo signal Es2, resulting in a worse suppression effect on the echo signal. Therefore, when the distance between speaker S1 and microphone M1 is less than a certain distance threshold, the echo cancellation algorithm cannot effectively distinguish between the sound wave signal As1 emitted by user P1 and the echo signal Es2, affecting the user's normal communication.

[0109] Figure 3B shows a schematic diagram of the positional relationship between the microphone and the speaker when the electronic device A is in a folded state, according to an embodiment of this application.

[0110] As shown in Figure 3B, in electronic device A, the distance between the top microphone 12 and the top speaker 14 is fixed, and the distance between them is relatively small; the distance between the bottom microphone 13 and the bottom speaker 15 is also fixed, and the distance between them is relatively small. When electronic device A is in a folded state, the top and bottom edges of electronic device A are folded together. At this time, the distance between the top microphone 12 and the bottom speaker 15 is small, and the distance between the bottom microphone 13 and the top speaker 14 is also small. That is, the distance between any microphone and any speaker in electronic device A is less than a preset distance threshold. In this case, the echo cancellation algorithm built into electronic device A has difficulty distinguishing the echo signal Es2 from the sound wave signal As1, and the echo signal Es2 will interfere with the user's normal call.

[0111] The principles of spatial audio technology are explained below.

[0112] Figure 3C shows a schematic diagram of the principle of a spatial audio technology provided in an embodiment of this application.

[0113] As shown in Figure 3C, a three-dimensional spatial coordinate system XYZ can be established in three-dimensional space. This system can include three mutually perpendicular coordinate axes: the X-axis, the Y-axis, and the Z-axis. The origin of the XYZ system can be the origin O. The acoustic signal As0 in space can be decomposed into components along these three coordinate axes based on the XYZ system. In some embodiments, the component along the Z-axis can represent the vertical component of the acoustic signal As0 in space, also called the Z-axis component; the component along the X-axis can represent the left-right component of the acoustic signal As0 in space, also called the X-axis component; and the component along the Y-axis can represent the front-back component of the acoustic signal As0 in space, also called the Y-axis component. Furthermore, in this three-dimensional space, a spherical coordinate system can be created with the origin O as the origin. The sphere of this coordinate system can be called the W-plane, and the components of the acoustic signal As0 on the W-plane can represent the omnidirectional components of the acoustic signal As0, also called the W-axis components.

[0114] Spatial audio technology is an audio processing technique designed to provide listeners with an immersive three-dimensional sound experience. By simulating how sound propagates in real space, it allows listeners to perceive the source, distance, and direction of sound, thereby creating a more realistic sound environment.

[0115] When creating a spatial audio file, it is necessary to obtain the spatial information of the sound wave signal As0 to simulate the source, distance, and direction of the sound. In some embodiments, the spatial information of the sound wave signal As0 can be determined by the components of the sound wave signal in various directions. Generally, it is necessary to obtain the X-axis, Y-axis, Z-axis, and W-axis components of the sound wave signal As0. Then, a spatial audio file can be generated based on the X-axis, Y-axis, Z-axis, and W-axis components of the sound wave signal As0 using a spatial audio algorithm model. In other embodiments, the W-axis component of the sound wave signal As0 can also be determined based on the X-axis, Y-axis, and Z-axis components. Therefore, at least the X-axis, Y-axis, and Z-axis components of the sound wave signal As0 must be obtained to generate a spatial audio file.

[0116] Figure 3D shows a schematic diagram of the microphone position when an electronic device A records an audio file, according to an embodiment of this application.

[0117] As shown in Figure 3D, electronic device A is in an unfolded state. When electronic device A records an audio file, if the direction parallel to the top edge of electronic device A is taken as the X-axis, and the direction parallel to the side edge of electronic device A is taken as the Y-axis, then in the embodiment shown in Figure 3D, electronic device A can collect the component of the sound wave signal in the Y-axis direction through the top microphone 12 and the bottom microphone 13, but cannot obtain the components in other directions. Therefore, electronic device A cannot record spatial audio.

[0118] The following describes the device configuration of an electronic device 100 provided in an embodiment of this application.

[0119] As shown in Figure 4A, the electronic device 100 may include a foldable display screen 41, a top microphone (mic) 42, a bottom microphone 43, a top speaker (speaker) 44, a bottom speaker 45, and a side microphone 46. Optionally, it may also include a rear microphone 47. Further optionally, it may also include a front-facing camera and a rear-facing camera (not shown in Figure 4A).

[0120] The electronic device 100 can be approximated as an imperfect hexahedron, with a certain curvature at the connection between adjacent faces. Of the six faces of the electronic device 100, the face containing the foldable display screen 41 can be called the front, and the face opposite the front is called the back (i.e., the back of the electronic device 100). Optionally, a rear camera can be located on the back of the electronic device 100, and a front camera can also be located on the front. There can be four sides between the front and back of the electronic device 100. Two opposite sides can each have a speaker; these two sides can be called the top and bottom of the electronic device 100. The speaker at the top is called the top speaker 44, and the speaker at the bottom is called the bottom speaker 45. Additionally, a top microphone 42 and a bottom microphone 43 are also located at the top and bottom of the electronic device 100, respectively. Of these four sides, the other two opposite sides besides the top and bottom can be called the left and right sides of the electronic device 100, and the left and right sides can also be collectively referred to as the sides of the electronic device 100. The side microphone 46 may be a microphone located on the side of the electronic device 100. In some embodiments, the side microphone 46 may include a right microphone 46a and / or a left microphone 46b, wherein the right microphone 46a is located on the right side of the electronic device 100, and the left microphone 46b is located on the left side of the electronic device 100. Optionally, one or more buttons, such as volume buttons, power buttons, etc., may be provided on the right side of the electronic device 100.

[0121] The foldable display screen 41 may include a display area 41A and a display area 41B. A hinge 41C exists between display areas 41A and 41B, allowing them to fold towards each other along the hinge 41C, such that the angle between display areas 41A and 41B varies between 0° and 180°. In the embodiment shown in FIG. 4A, the angle between display areas 41A and 41B is approximately 180°, at which point the foldable display screen 41 is in a fully unfolded state. When the angle between display areas 41A and 41B falls within a preset unfolded angle range (e.g., greater than or equal to 450° and less than or equal to 180°), the electronic device 100 is considered to be in an unfolded state.

[0122] As shown in Figure 4B, in the foldable display screen 41 of the electronic device 100, the angle between display area 41A and display area 41B is approximately 90°. At this time, the foldable display screen 41 is in a semi-folded state. When the angle between display area 41A and display area 41B falls within a preset semi-folded angle range (e.g., greater than or equal to 30° and less than or equal to 450°), the electronic device 100 can be considered to be in a semi-folded state.

[0123] As shown in Figure 4C, in the foldable display screen 41 of the electronic device 100, the angle between display area 41A and display area 41B is approximately 0°. At this time, the foldable display screen 41 is in a fully folded state. When the angle between display area 41A and display area 41B is within a preset folding angle range (e.g., greater than or equal to 0° and less than or equal to 30°), the electronic device 100 can be considered to be in a folded state.

[0124] The embodiment shown in Figure 4C illustrates the A-side of the electronic device 100 in a folded state, i.e., the back side of the display area 41A. A rear-facing camera may also be provided on the back side of the display area 41A. Optionally, a rear microphone 47 may also be provided on the back side of the display area 41A.

[0125] As shown in Figure 4D, in the foldable display screen 41 of the electronic device 100, the angle between display area 41A and display area 41B is 0°. At this time, the foldable display screen 41 is in a fully folded state, and the device form of the electronic device 100 in this state can be called the folded state. The embodiment shown in Figure 4D shows the B side of the electronic device 100 in the folded state, that is, the back side of display area 41B.

[0126] It is understood that the embodiments shown in Figures 4A-4D are only examples. In the embodiments of this application, the electronic device 100 may also include more, fewer, or different devices than those in the above embodiments, such as including more microphones, etc. This application does not limit it.

[0127] It should be noted that in the embodiments shown in Figures 4A-4D above, the pivot 41C can be parallel to the intersection line of the foldable display screen 41 and the top of the electronic device 100. In this case, the foldable display screen 41 of the electronic device 100 can be folded vertically. In other embodiments, the pivot 41C can be parallel to the intersection line of the foldable display screen 41 and the side (e.g., left or right) of the electronic device 100. In this case, the foldable display screen 41 of the electronic device 100 can be folded horizontally. This application does not limit this.

[0128] The hardware structure of the electronic device 100 provided in the embodiments of this application is described below.

[0129] Figure 5 shows a schematic diagram of the hardware structure of an electronic device 100 provided in an embodiment of this application.

[0130] Electronic device 100 may be a mobile phone, tablet computer, desktop computer, laptop computer, handheld computer, notebook computer, ultra-mobile personal computer (UMPC), netbook, as well as cellular phone, personal digital assistant (PDA), augmented reality (AR) device, virtual reality (VR) device, artificial intelligence (AI) device, wearable device, in-vehicle device, smart home device and / or smart city device. The embodiments of this application do not impose any special restrictions on the specific type of electronic device.

[0131] Electronic device 100 may include a processor 110, internal memory 121, charging management module 140, power management module 141, battery 142, wireless communication module 160, audio module 170, sensor module 180, etc. Optionally, electronic device 100 may also include one or more of the following: external memory interface 120, universal serial bus (USB) interface 130, mobile communication module 150, antenna 1, antenna 2, button 190, motor 191, indicator 192, camera 193, display screen 194, etc. Sensor module 180 may include a gyroscope sensor 180B, magnetic sensor 180D, accelerometer sensor 180E, touch sensor 180K, etc. Optionally, sensor module 180 may also include one or more of the following: pressure sensor, barometric pressure sensor, distance sensor, proximity sensor, fingerprint sensor, temperature sensor, ambient light sensor, bone conduction sensor, etc.

[0132] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0133] Processor 110 may include one or more processing units, such as application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU). These different processing units may be independent devices or integrated into one or more processors.

[0134] The controller can generate operation control signals based on the instruction opcode and timing signals to complete the control of instruction fetching and execution.

[0135] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0136] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0137] USB port 130 is a USB standard compliant interface, specifically a Mini USB port, Micro USB port, USB Type-C port, etc. USB port 130 can be used to connect a charger to charge electronic device 100, and can also be used for data transfer between electronic device 100 and peripheral devices. It can also be used to connect headphones for audio playback. This interface can also be used to connect other electronic devices, such as AR devices.

[0138] It is understood that the interface connection relationships between the modules illustrated in the embodiments of the present invention are merely illustrative and do not constitute a structural limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0139] The charging management module 140 receives charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 receives charging input from the wired charger via the USB interface 130. In some wireless charging embodiments, the charging management module 140 receives wireless charging input via the wireless charging coil of the electronic device 100. While charging the battery 142, the charging management module 140 can also supply power to the electronic device via the power management module 141.

[0140] The power management module 141 connects the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, providing power to the processor 110, internal memory 121, display screen 194, camera 193, and wireless communication module 160, etc. The power management module 141 can also monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage current, impedance). In some other embodiments, the power management module 141 may also be located within the processor 110. In other embodiments, the power management module 141 and the charging management module 140 may be located in the same device.

[0141] The wireless communication function of electronic device 100 can be realized through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor, etc.

[0142] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with tuning switches.

[0143] The mobile communication module 150 can provide solutions for wireless communication, including 2G / 3G / 4G / 5G, applied to the electronic device 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1. In some embodiments, at least some functional modules of the mobile communication module 150 may be housed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 may be housed in the same device.

[0144] The modem processor may include a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs sound signals through an audio device (not limited to speaker 170A, receiver 170B, etc.) or displays images or videos through the display screen 194. In some embodiments, the modem processor may be a separate device. In other embodiments, the modem processor may be independent of the processor 110 and may be housed in the same device as the mobile communication module 150 or other functional modules.

[0145] The wireless communication module 160 can provide solutions for wireless communication applications on the electronic device 100, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, demodulates and filters the electromagnetic wave signals, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, frequency modulate and amplify them, and then convert them into electromagnetic waves for radiation via antenna 2.

[0146] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, enabling electronic device 100 to communicate with networks and other devices via wireless communication technology. The wireless communication technology may include Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time-Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technologies, etc. The GNSS may include the Global Positioning System (GPS), the Global Navigation Satellite System (GLONASS), the BeiDou Navigation Satellite System (BDS), the Quasi-Zenith Satellite System (QZSS), and / or satellite-based augmentation systems (SBAS).

[0147] In some embodiments, the electronic device 100 implements display functions via a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU performs mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0148] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), or it can be manufactured using organic light-emitting diodes (OLEDs), active-matrix organic light-emitting diodes (AMOLEDs), flexible light-emitting diodes (FLEDs), minimized LEDs, microLEDs, micro-OLEDs, quantum dot light-emitting diodes (QLEDs), etc. In some embodiments, electronic device 100 may include one or N displays 194, where N is a positive integer greater than 1.

[0149] Internal memory 121 may include one or more random access memory (RAM) and one or more non-volatile memory (NVM). The RAM can be directly read and written by the processor 110 and can be used to store executable programs (e.g., machine instructions) of the operating system or other running programs, as well as user and application data. The NVM can also store executable programs and user and application data, and can be pre-loaded into the RAM for direct read and write operations by the processor 110.

[0150] The external memory interface 120 can be used to connect to external non-volatile memory, thereby expanding the storage capacity of the electronic device 100. The external non-volatile memory communicates with the processor 110 through the external memory interface 120 to perform data storage functions. For example, music, video, and other files can be stored in the external non-volatile memory.

[0151] Electronic device 100 can implement audio functions through audio module 170 and application processor, such as making calls, playing music, and recording audio. Audio module 170 may include speaker 170A and microphone 170C. Optionally, audio module 170 may also include one or more of the following: receiver 170B and headphone jack 170D.

[0152] The audio module 170 is used to convert digital audio information into analog audio signals for output, and also to convert analog audio input into digital audio signals. The audio module 170 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 170 may be located in the processor 110, or some functional modules of the audio module 170 may be located in the processor 110.

[0153] The speaker 170A, also known as a "loudspeaker," is used to convert audio electrical signals into sound signals. The electronic device 100 can listen to music or make hands-free calls through the speaker 170A.

[0154] The receiver 170B, also known as the "earpiece," is used to convert audio electrical signals into sound signals. When the electronic device 100 answers a telephone call or voice message, the receiver 170B can be brought close to the ear to listen to the voice.

[0155] Microphone 170C, also known as a "microphone" or "voice transducer," is used to convert sound signals into electrical signals. When making a phone call or sending a voice message, the user can speak by bringing their mouth close to microphone 170C, inputting the sound signal into microphone 170C. Electronic device 100 may have at least one microphone 170C. In some embodiments, electronic device 100 may have two microphones 170C, which, in addition to collecting sound signals, can also perform noise reduction. In other embodiments, electronic device 100 may also have three, four, or more microphones 170C, which can collect sound signals, reduce noise, identify the sound source, and perform directional recording, etc.

[0156] The 170D headphone jack is used to connect wired headphones. The 170D headphone jack can be a USB 130 interface or a 3.5mm Open Mobile Terminal Platform (OMTP) standard interface, a CTIA (Cellular Telecommunications Industry Association of the USA) standard interface.

[0157] In this embodiment of the application, the electronic device 100 can detect the device state of the electronic device 100 through any one or more of the gyroscope sensor 180B, magnetic sensor 180D and accelerometer sensor 180E. The device state of the electronic device 100 may include an unfolded state, a semi-folded state and a folded state.

[0158] The gyroscope sensor 180B can be used to determine the motion attitude of the electronic device 100. In some embodiments, the gyroscope sensor 180B can determine the angular velocity of the electronic device 100 about three axes (i.e., the x, y, and z axes). The gyroscope sensor 180B can be used for image stabilization. For example, when the shutter is pressed, the gyroscope sensor 180B detects the angle of the shake of the electronic device 100, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to counteract the shake of the electronic device 100 by moving in the opposite direction, thus achieving image stabilization. The gyroscope sensor 180B can also be used in navigation and motion-sensing game scenarios.

[0159] The magnetic sensor 180D includes a Hall sensor. The electronic device 100 can use the magnetic sensor 180D to detect the opening and closing of the flip cover. In some embodiments, when the electronic device 100 is a flip phone, the electronic device 100 can detect the opening and closing of the flip cover using the magnetic sensor 180D. Then, based on the detected opening and closing state of the cover or the flip cover, features such as automatic flip unlocking can be set.

[0160] The 180E accelerometer can detect the magnitude of acceleration of electronic device 100 in various directions (typically three axes). When electronic device 100 is stationary, it can detect the magnitude and direction of gravity. It can also be used to identify the posture of electronic devices and applied to applications such as screen orientation switching and pedometers.

[0161] Touch sensor 180K, also known as a "touch device," can be located on display screen 194. The touch sensor 180K and display screen 194 together form a touchscreen, also known as a "touchscreen." Touch sensor 180K detects touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 194. In other embodiments, touch sensor 180K may also be located on the surface of electronic device 100, in a different position than display screen 194.

[0162] Buttons 190 include a power button, volume buttons, etc. Buttons 190 can be mechanical buttons or touch-sensitive buttons. Electronic device 100 can receive button input and generate key signal inputs related to user settings and function control of electronic device 100.

[0163] Motor 191 can generate vibration alerts. Motor 191 can be used for incoming call vibration alerts or for touch vibration feedback. For example, different vibration feedback effects can correspond to touch operations performed on different applications (such as taking photos, playing audio, etc.). Motor 191 can also correspond to different vibration feedback effects for touch operations performed on different areas of the display screen 194. Different application scenarios (such as time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also be customized.

[0164] Indicator 192 can be an indicator light, used to indicate charging status, power changes, or to indicate messages, missed calls, notifications, etc.

[0165] This application provides a sound acquisition method applied to a foldable electronic device 100. The electronic device 100 may include a top microphone 42, a bottom microphone 43, and a side microphone 46, and optionally, a rear microphone 47. When the electronic device 100 receives a user's microphone activation operation, the electronic device 100 can determine whether the current device form is folded. If the electronic device 100 is folded, the electronic device 100 can determine to use the side microphone 46 to acquire sound; if the electronic device 100 is folded, the electronic device 100 can determine to use the top microphone 42 and / or the bottom microphone to acquire sound.

[0166] This avoids the microphone being blocked and affecting sound acquisition when the electronic device is folded due to the user holding the device.

[0167] Figure 6A shows a schematic diagram of a scenario where a user holds a folded electronic device 100 to make a voice call, according to an embodiment of this application.

[0168] As shown in Figure 6A, when the electronic device 100 is in a folded state and the user holds it in one hand, the top microphone 42 and bottom microphone 43 of the electronic device 100 are highly likely to be blocked by the user's fingers. Meanwhile, the side microphones 46 (e.g., right microphone 46a and / or left microphone 46b) will not be blocked by the user's fingers. In this situation, if the electronic device 100 is in a scenario requiring sound acquisition, such as a voice call, the electronic device 100 can acquire sound wave signals through the side microphones 46.

[0169] In this way, even if the user holds the folded electronic device 100 to make a voice call, the user's voice can be collected through the side microphone 46 to facilitate the voice call function.

[0170] Figure 6B shows a schematic diagram of the positional relationship between the microphone and the speaker when the electronic device 100 is in a folded state, according to an embodiment of this application.

[0171] As shown in Figure 6B, in the electronic device 100, the distance between the top microphone 42 and the top speaker 44 is fixed, and the distance between them is relatively small; the distance between the bottom microphone 43 and the bottom speaker 45 is also fixed, and the distance between them is relatively small. When the electronic device 100 is in a folded state, the top and bottom edges of the electronic device 100 are folded together. At this time, the distance between the top microphone 42 and the bottom speaker 45 is small, and the distance between the bottom microphone 43 and the top speaker 44 is also small. Simultaneously, the distance between the side microphones 46 (e.g., right microphone 46a and / or left microphone 46b) and any speaker in the electronic device 100 is relatively large, exceeding a preset distance threshold. In this case, if the electronic device 100 activates hands-free calling mode, it can collect the user's voice through the side microphones 46, allowing the echo cancellation algorithm to easily distinguish between the sound wave signal emitted by the user and the echo signal generated by the sound played by the speaker, thereby better suppressing the interference of the echo signal.

[0172] In this way, even if the user holds the folded electronic device 100 to make a voice call, the user's voice can be collected through the side microphone, thereby reducing the interference of echo signals.

[0173] Figure 6C shows a schematic flowchart of a sound acquisition method provided in an embodiment of this application.

[0174] As shown in Figure 6C, a specific process for a sound acquisition method may include the following steps:

[0175] S601. Electronic device 100 receives an operation from a user to enable the microphone function 1.

[0176] Operation 1 can be used to trigger electronic device 100 to enable the microphone function.

[0177] In some embodiments, operation 1 can be used to initiate a call. For example, operation 1 can be used to trigger electronic device 100 to make or receive a call, or to trigger electronic device 100 to make or receive a voice / video call through other application software.

[0178] In other embodiments, operation 1 can be used to trigger electronic device 100 to record audio, or to trigger electronic device 100 to record video, etc.

[0179] It should be noted that operation 1 can be an operation on the controls on the display screen, an operation on the buttons of the electronic device 100, or a gesture operation.

[0180] S602. Electronic device 100 determines whether the device is in a folded state.

[0181] In response to operation 1, electronic device 100 can determine whether the device is in a folded state at this time.

[0182] The electronic device 100 can obtain its device configuration through one or more devices such as an accelerometer, a gyroscope, or a magnetic sensor.

[0183] If the device mode of the electronic device 100 is folded, the electronic device 100 can perform the following step S603.

[0184] If the device mode of electronic device 100 is not folded, then electronic device 100 can perform the following step S604.

[0185] S603. Electronic device 100 acquires sound wave signals via side microphone 46.

[0186] When the electronic device 100 is in a folded state, as shown in Figures 6A-6B above, there is a possibility that the top microphone 42 and the bottom microphone 43 may be blocked, and the distance between the top microphone 42 and the bottom microphone 43 and the speaker is small. Therefore, using a side microphone 46 (e.g., right microphone 46a and / or left microphone 46b) to collect sound wave signals can avoid the microphone being blocked and can also reduce echo interference of the electronic device 100 during a call, providing a better user experience.

[0187] In some embodiments, when the electronic device 100 is in a folded state, the electronic device 100 may also use the side microphone 46 as the main microphone and one or more other microphones (e.g., the top microphone 42, the bottom microphone 43, the rear microphone 47, etc.) as auxiliary microphones. The electronic device 100 can simultaneously acquire sound wave signals through the main microphone and auxiliary microphones, and determine the final sound wave signal by combining the sound wave signal acquired by the main microphone with the sound wave signal acquired by the auxiliary microphone.

[0188] S604. Electronic device 100 acquires sound wave signals through top microphone 42 and / or bottom microphone 43.

[0189] When the electronic device 100 is not in a folded state, the top microphone 42 and the bottom microphone 43 are less likely to be blocked, and the distance between the top microphone 42 and the bottom speaker 45 is relatively large, as is the distance between the bottom microphone 43 and the top speaker 44. Therefore, the sound acquisition function can be realized by using the top microphone 42 and / or the bottom microphone 43 to collect sound wave signals.

[0190] In some embodiments, when the electronic device 100 is not in a folded state, it may use one of its microphones (e.g., top microphone 42 or bottom microphone 43) as the main microphone and one or more other microphones (e.g., side microphone 46, rear microphone 47, top microphone 42, bottom microphone 43, etc.) as auxiliary microphones. The electronic device 100 can simultaneously acquire sound wave signals through the main microphone and auxiliary microphones, and determine the final sound wave signal by combining the sound wave signal acquired by the main microphone with the sound wave signal acquired by the auxiliary microphones.

[0191] In one possible implementation, if the electronic device 100 is in a call state, after acquiring sound wave signals through a microphone (e.g., side microphone 46, top microphone 42, or bottom microphone 43), the electronic device 100 can obtain the spectrum of the sound wave signals based on time-frequency transformation, and obtain the spectral characteristics of the sound wave signals based on the spectrum. Then, a deep learning algorithm can be used to identify the spectral characteristics of the sound wave signals, thus distinguishing human voices from other sounds. The deep learning algorithm can be trained by the electronic device 100 (or other electronic devices) based on preset speech spectral characteristics of multiple human voices. This allows for the extraction of human voices while discarding other sounds.

[0192] It is understood that the embodiment shown in Figure 6C is only an example. In the embodiments of this application, the electronic device 100 may also use more, fewer or different steps than the above embodiments to realize the sound acquisition function. For example, different microphones may be used to acquire sound wave signals in the unfolded state and the semi-folded state. This application does not limit this.

[0193] In this way, when the electronic device 100 collects sound wave signals, it can determine the microphone used to collect the sound wave signals based on the device configuration of the electronic device 100, avoiding the impact of blocking the hole on sound collection, and also reducing echo interference during the call, giving users a better call experience.

[0194] In other embodiments, when the electronic device 100 enables the microphone function, it can determine the microphone to be used not only based on the device configuration of the electronic device 100, but also based on the microphone's aperture occlusion status. Specifically, when the electronic device 100 detects that the microphone function is enabled, the electronic device 100 can measure the aperture occlusion status of one or more microphones and determine, based on the aperture occlusion status, to use an unoccluded microphone.

[0195] In one possible implementation, the electronic device 100 can determine whether each microphone is blocked by acquiring test signals. For example, if the microphone under test includes a top microphone 42, test signal 1 can be acquired through the top microphone 42, and the blockage of the top microphone 42 can be determined based on the relationship between the signal strength (e.g., decibel level) of test signal 1 and a preset strength threshold. If the signal strength of test signal 1 is less than or equal to the preset strength threshold, the top microphone 42 is determined to be blocked; if the signal strength of test signal 1 is greater than the preset strength threshold, the top microphone 42 is determined not to be blocked. It is understood that the embodiments here are merely illustrative. While it is possible to determine whether a microphone under test is blocked by acquiring test signals, the electronic device 100 can also use a similar method to determine whether other microphones are blocked in this application embodiment. It should be noted that when the microphone under test includes multiple microphones, the electronic device 100 can test whether multiple microphones are blocked simultaneously or test whether each microphone under test is blocked sequentially; this application does not limit this.

[0196] In another possible implementation, the electronic device 100 can determine whether a microphone is blocked based on how the user holds the device. When the electronic device 100 is not held by the user, there is no blocked microphone. When the electronic device 100 is held by the user, there may be a blocked microphone.

[0197] Figures 7A-7C illustrate schematic diagrams of how a user holds an electronic device 100 in an unfolded state, according to an embodiment of this application.

[0198] As shown in Figure 7A, the electronic device 100 is in its unfolded state, held by a user with one hand, and the foldable display screen 41 of the electronic device 100 is facing the user, with the top of the electronic device 100 pointing towards the user's palm. In this configuration, the user's way of holding the electronic device 100 can be described as a one-handed top-grip holding method in the unfolded state. At this time, as shown in Figure 7A, the top microphone 42 may be blocked, preventing it from properly capturing sound.

[0199] As shown in Figure 7B, the electronic device 100 is in its unfolded state, held by a user with one hand, and the foldable display screen 41 of the electronic device 100 is facing the user, with the bottom of the electronic device 100 facing the user's palm. In this configuration, the user's grip on the electronic device 100 can be described as a one-handed bottom-grip grip in the unfolded state. At this time, as shown in Figure 7B, the bottom microphone 43 may be blocked, preventing it from properly capturing sound.

[0200] As shown in Figure 7C, the electronic device 100 is in its unfolded state. The user holds the electronic device 100 with one hand, and the foldable display screen 41 of the electronic device 100 faces the user. The back of the electronic device 100 faces the user's palm, and the left and right sides of the electronic device 100 are close to the user's fingers. In this configuration, the user's grip on the electronic device 100 can be described as a one-handed side-grip grip in the unfolded state. At this time, as shown in Figure 7C, the left microphone 46b, right microphone 46a, and rear microphone 47 may be blocked, preventing them from properly collecting sound.

[0201] Figures 7D-7E illustrate schematic diagrams of how a user holds an electronic device 100 in a semi-folded state, according to an embodiment of this application.

[0202] As shown in Figure 7D, the electronic device 100 is in a semi-folded state, with the user holding it in one hand. The display area 41A of the foldable display screen 41 of the electronic device 100 faces the user, and the hinge 41C is perpendicular to the direction of gravity. In this case, the way the user holds the electronic device 100 can be called a single-handed horizontal grip in the semi-folded state. At this time, as shown in Figure 7D, none of the microphones are blocked.

[0203] As shown in Figure 7E, the electronic device 100 is in a semi-folded state, with the user holding it in one hand. The display area 41A of the foldable display screen 41 of the electronic device 100 faces the user, and the hinge 41C is parallel to the direction of gravity. In this case, the way the user holds the electronic device 100 can be called a single-handed vertical grip in the semi-folded state. At this time, as shown in Figure 7E, the left microphone 46b, the right microphone 46a, and the top microphone 42 may be blocked, causing the left microphone 46b, the right microphone 46a, and the top microphone 42 to be unable to collect sound properly.

[0204] Figures 7F-7I illustrate a schematic diagram of how a user holds an electronic device 100 in a folded state, according to an embodiment of this application.

[0205] As shown in Figure 7F, the electronic device 100 is in a folded state. When a user holds the electronic device 100 with one hand, the B-side (the side without the rear camera) of the folded electronic device 100 faces the user's palm, and the hinge 41C is parallel to the direction of gravity. In this configuration, the user's way of holding the electronic device 100 can be described as a longitudinal grip on the A-side in the folded state. At this time, as shown in Figure 7F, both the top microphone 42 and the bottom microphone 43 may be blocked, preventing them from properly collecting sound.

[0206] As shown in Figure 7G, the electronic device 100 is in a folded state. When a user holds the electronic device 100 with one hand, the B-side (the side without the rear camera) of the folded electronic device 100 faces the user's palm, and the pivot 41C is perpendicular to the direction of gravity. In this configuration, the user's way of holding the electronic device 100 can be described as a horizontal grip on the A-side in the folded state. At this time, as shown in Figure 7G, both the right microphone 46a and the left microphone 46b may be blocked, preventing them from properly collecting sound.

[0207] As shown in Figure 7H, the electronic device 100 is in a folded state. When a user holds the electronic device 100 with one hand, the A-side (the side with the rear camera) of the folded electronic device 100 faces the user's palm, and the hinge 41C is parallel to the direction of gravity. In this configuration, the user's way of holding the electronic device 100 can be referred to as the B-side longitudinal holding method in the folded state. At this time, as shown in Figure 7H, the top microphone 42, bottom microphone 43, and rear microphone 47 may be blocked, preventing them from properly collecting sound.

[0208] As shown in Figure 7I, the electronic device 100 is in a folded state. When a user holds the electronic device 100 with one hand, the A-side (the side with the rear camera) of the folded electronic device 100 faces the user's palm, and the pivot 41C is perpendicular to the direction of gravity. In this case, the user's way of holding the electronic device 100 can be called the B-side horizontal holding method in the folded state. At this time, as shown in Figure 7G, the rear microphone 47, right microphone 46a, and left microphone 46b may be blocked, causing them to be unable to collect sound properly.

[0209] It is understood that the embodiments shown in Figures 7A-7I are only examples. In the embodiments of this application, the electronic device 100 may include more, fewer or different holding methods than the above embodiments in different device forms, and may also include the way the user holds it with both hands, etc. Since the setting position of each microphone may also be different from the above embodiments, the microphone blocking situation may also be different from the above embodiments in each holding method. This application does not limit it here.

[0210] For example, Figure 7J shows a schematic flowchart of another sound acquisition method provided in an embodiment of this application.

[0211] As shown in Figure 7J, the specific process of another sound acquisition method may include the following steps:

[0212] S701. Electronic device 100 monitors the device configuration of electronic device 100.

[0213] Electronic device 100 can monitor changes in its form using one or more devices such as an accelerometer or a gyroscope.

[0214] S702. Electronic device 100 detects operation 1 that enables microphone function.

[0215] For details of operation 1, please refer to the relevant description of step S601 shown in Figure 6C above, which will not be repeated here.

[0216] S703. Electronic device 100 detects the user's grip based on the device's form.

[0217] Electronic device 100 can detect the user's grip through one or more devices such as an accelerometer, gyroscope, gravity sensor, and low-power camera.

[0218] The user's holding posture can include: not holding and holding. Depending on the different form factors of the electronic device 100, various holding methods can be included when holding the device. For example, when the electronic device 100 is in an unfolded state, the user's holding posture can include holding it with one hand at the top, holding it with one hand at the bottom, and holding it with one hand on the side; when the electronic device 100 is in a semi-folded state, the user's holding posture can include holding it with one hand horizontally and holding it with one hand vertically; when the electronic device 100 is in a folded state, the user's holding posture can include holding it with side A horizontally, holding it with side A vertically, holding it with side B horizontally, and holding it with side B vertically, etc. Specific holding postures can be referred to the relevant descriptions in the embodiments shown in Figures 7A-7I above, and will not be repeated here. It is understood that the embodiments here are only examples. In the embodiments of this application, the user's holding posture can also include more, fewer, or different holding postures than the above embodiments, and this application does not limit this.

[0219] In some of the countries, when it is determined that the electronic device 100 is held by a user, the electronic device 100 can determine the user's holding method based on the device form.

[0220] S704. Electronic device 100 determines the microphone blockage status based on the user's grip.

[0221] If the electronic device 100 determines that the user is not holding the electronic device 100, the electronic device 100 can determine that no microphone is blocked.

[0222] If the electronic device 100 determines that a user is holding the electronic device 100, the electronic device 100 can determine the blockage status of each microphone in the electronic device 100 based on the way the user holds the electronic device 100. For example, the correspondence between different holding methods and the blockage status can be referred to the relevant descriptions in the embodiments shown in Figures 7A-7I above, and will not be repeated here.

[0223] S705. Electronic device 100 determines the microphone to be used based on the microphone's occlusion status.

[0224] In some embodiments, after determining whether the microphone is blocked, the electronic device 100 can select one or more microphones from the unblocked microphones as the microphones to be used in this instance.

[0225] In other embodiments, when the hole is determined to be blocked, the electronic device 100 can select one or more microphones as the main microphone and other microphones as auxiliary microphones from the microphones that are not blocked, to assist the main microphone in acquiring sound wave signals.

[0226] S706. Electronic device 100 acquires sound wave signals using the microphone used in this application.

[0227] It is understood that the embodiment shown in FIG7J is only an example. In the embodiments of this application, the electronic device 100 may also use more, fewer, or different steps than the embodiment shown in FIG7J to complete the sound acquisition. This application does not limit it here.

[0228] Using the sound acquisition method provided in this application, the electronic device 100 can determine the microphone to be used based on the microphone's hole blockage status, thereby reducing the risk of hole blockage and improving the quality of sound acquisition.

[0229] In some application scenarios, the electronic device 100 is in a folded state and in a call state. If the electronic device 100 receives a user's request to switch to hands-free calling mode, it can collect the user's sound wave signal through the side microphone 46 (left microphone 46b and / or right microphone 46a). Since the distance between the side microphone 46 and the speaker is greater than the distance between the top microphone 42 and the bottom microphone 43 and the speaker, collecting the user's sound wave signal through the side microphone 46 in hands-free calling mode can reduce echo interference and improve call quality.

[0230] In some embodiments, when the electronic device 100 is in a folded state and in a call state, upon receiving a user's request to switch to hands-free calling mode, in addition to acquiring the sound wave signal emitted by the user through the side microphone 46 (left microphone 46b and / or right microphone 46a), the electronic device 100 can also output a prompt 1 based on the device shape and holding method of the electronic device 100. The prompt 1 is used to prompt the user to point the side microphone 46 toward the user's mouth. This can shorten the distance between the side microphone 46 and the user's mouth and improve the quality of sound acquisition.

[0231] In other embodiments, the electronic device 100 is in a folded state and in a call state. In this case, when the electronic device 100 receives an operation from the user to switch to hands-free calling mode, it can construct a sound wave beam 4 based on the device's shape and how it is held. The sound wave beam 4 can be directed towards the user's mouth. Then, the electronic device 100 can collect the sound wave signal emitted by the user through the side microphones 46 (left microphone 46b and / or right microphone 46a) based on the sound wave beam 4. This enhances the sound in the direction the sound wave beam 4 is directed and suppresses sound from other directions, thereby improving call quality.

[0232] It is understood that the embodiments described herein are merely illustrative of the microphones used by the electronic device 100 in folded and hands-free calling modes. In the embodiments of this application, the electronic device 100 may also use more, fewer, or different microphones than those in the above embodiments in hands-free calling mode, depending on the device form (e.g., semi-folded, unfolded, etc.) and the way it is held. This application does not impose any limitations on this.

[0233] In some application scenarios, electronic device 100 can record spatial audio files using multiple microphones. These spatial audio files contain spatial information of the sound, providing users with a three-dimensional surround sound effect. As shown in the embodiment of Figure 3C above, recording spatial audio files requires at least the X-dimensional, Y-dimensional, and Z-dimensional components of the sound wave signal. The following describes the acquisition methods for the X-dimensional, Y-dimensional, and Z-dimensional components of the sound wave signal when electronic device 100 is in different device configurations.

[0234] Figure 8A shows a schematic diagram of a scenario where spatial audio is recorded when the electronic device 100 is in an unfolded state, according to an embodiment of this application.

[0235] As shown in Figure 8A, when the electronic device 100 is in the unfolded state, in some embodiments, a three-dimensional spatial coordinate system can be established with the line parallel to the upper edge of the foldable display screen 41 as the Y-axis, the line parallel to the side edge of the foldable display screen 41 as the X-axis, and the line perpendicular to the foldable display screen 41 as the Z-axis.

[0236] Therefore, in the embodiment shown in Figure 8A, if there is an acoustic signal As0 in the space where the electronic device 100 is located, the X-dimensional component can be determined based on the acoustic signal collected by the bottom microphone 43 and the acoustic signal collected by the top microphone 42 (or the left microphone 46b, or the right microphone 46a); the Y-dimensional component can be determined based on the acoustic signal collected by the right microphone 46a and the acoustic signal collected by the left microphone 46b; and the Z-dimensional component can be determined based on the acoustic signal collected by the rear microphone 47 and the acoustic signal collected by the top microphone 42 (or the left microphone 46b, or the right microphone 46a).

[0237] In summary, when the electronic device 100 is in the unfolded state, if it is necessary to record spatial audio signals, the electronic device 100 needs to use at least the bottom microphone 43, the right microphone 46a, the left microphone 46b, and the rear microphone 47 to collect sound wave signals.

[0238] Figure 8B shows a schematic diagram of a scenario where an electronic device 100 is in a semi-folded state, according to an embodiment of this application, to record spatial audio.

[0239] As shown in Figure 8B, when the electronic device 100 is in a semi-folded state, the foldable display screen 41 may include a display area 41A and a display area 41B. In some embodiments, a three-dimensional spatial coordinate system may be established with a line parallel to the upper edge of the foldable display screen 41 as the X-axis, a line parallel to the side edge of the display area 41A as the Y-axis, and a line perpendicular to the display area 41A as the Z-axis.

[0240] Therefore, in the embodiment shown in Figure 8B, if there is an acoustic signal As0 in the space where the electronic device 100 is located, the X-dimensional component can be determined based on the acoustic signal collected by the left microphone 46b and the acoustic signal collected by the right microphone 46a; the Y-dimensional component can be determined based on the acoustic signal collected by the bottom microphone 42 and the acoustic signal collected by the top microphone 42; and the Z-dimensional component can be determined based on the acoustic signal collected by the bottom microphone 43 and the acoustic signal collected by the rear microphone 47 (or the left microphone 46b, or the right microphone 46a).

[0241] In summary, when the electronic device 100 is in a semi-folded state, if it is necessary to record spatial audio signals, the electronic device 100 needs to use at least the top microphone 42, the bottom microphone 43, the right microphone 46a, and the left microphone 46b to collect sound wave signals.

[0242] Figure 8C shows a schematic diagram of a scenario where an electronic device 100 is in a folded state, according to an embodiment of this application, to record spatial audio.

[0243] As shown in Figure 8C, when the electronic device 100 is in a folded state, in some embodiments, a three-dimensional spatial coordinate system can be established with the line parallel to the upper edge of the foldable display screen 41 as the Y-axis, the line parallel to the side edge of the foldable display screen 41 as the X-axis, and the line perpendicular to the foldable display screen 41 as the Z-axis.

[0244] Therefore, in the embodiment shown in Figure 8C, if there is an acoustic signal As0 in the space where the electronic device 100 is located, the X-dimensional component can be determined based on the acoustic signal collected by the bottom microphone 43 and the acoustic signal collected by the side microphone 46 (left microphone 46b or right microphone 46a); the Y-dimensional component can be determined based on the acoustic signal collected by the left microphone 46b and the acoustic signal collected by the right microphone 46a; and the Z-dimensional component can be determined based on the acoustic signal collected by the rear microphone 47 and the acoustic signal collected by the bottom microphone 43.

[0245] In summary, when the electronic device 100 is in a folded state, if it is necessary to record spatial audio signals, the electronic device 100 needs to use at least the bottom microphone 43, the right microphone 46a, the left microphone 46b, and the rear microphone 47 to collect sound wave signals.

[0246] It is understood that the embodiments shown in Figures 8A-8C above are just some examples. In the embodiments of this application, the electronic device 100 may also include more, fewer or different microphones than the above embodiments. In different device forms, the electronic device 100 may also collect sound wave signals and determine sound components in different dimensions through microphones different from those in the above embodiments. This application does not limit this.

[0247] In some embodiments, in the scenario of spatial audio recording, the electronic device 100 can determine the microphone to be used based on the device configuration, and collect sound wave signals based on the determined microphone to generate a spatial audio file.

[0248] For example, Figure 8D shows a schematic flowchart of a sound acquisition method in a spatial audio recording scenario provided by an embodiment of this application.

[0249] As shown in Figure 8D, the specific process of sound acquisition in a spatial audio recording scenario may include the following steps:

[0250] S801. Electronic device 100 monitors the device configuration of electronic device 100.

[0251] The specific details of step S801 can be found in the relevant description in the embodiment shown in Figure 7J above.

[0252] S802. Electronic device 100 receives operation 2 of user recording spatial audio file.

[0253] S803. Electronic device 100 determines the microphone to be used based on the device configuration of electronic device 100.

[0254] For example, if the electronic device 100 is in an unfolded state, the electronic device 100 can determine that the microphones used this time include the bottom microphone 43, the right microphone 46a, the left microphone 46b, and the rear microphone 47; if the electronic device 100 is in a semi-folded state, the electronic device 100 can determine that the microphones used this time include the top microphone 42, the bottom microphone 43, the right microphone 46a, and the left microphone 46b; if the electronic device 100 is in a folded state, the electronic device 100 can determine that the microphones used this time include the bottom microphone 43, the right microphone 46a, the left microphone 46b, and the rear microphone 47.

[0255] It is understood that the embodiments described herein are merely illustrative. The microphone used in this instance can be determined based on the device configuration of the electronic device 100. In the embodiments of this application, the microphones used by the electronic device 100 may include more, fewer, or different microphones than those in the above embodiments under different device configurations. This application does not impose any limitations on these embodiments.

[0256] S804. Electronic device 100 acquires sound wave signals using the microphone used in this application.

[0257] S805. Electronic device 100 determines the X-dimensional component, Y-dimensional component, and Z-dimensional component based on the device shape and the acquired acoustic wave signal.

[0258] The electronic device 100, in different device configurations, can determine the X-dimensional, Y-dimensional, and Z-dimensional components based on the sound wave signals collected by different microphones. In some embodiments, the method for determining the sound components in each dimension can be compared with the relevant descriptions in the embodiments shown in Figures 8A-8C above, or can also refer to the relevant descriptions in step S907 shown in Figure 9 below.

[0259] S806. Electronic device 100 generates spatial audio file 1 based on X-dimensional components, Y-dimensional components and Z-dimensional components.

[0260] The details of step S806 can also be found in the description of step S907 shown in Figure 9 below, which will not be elaborated here.

[0261] It is understood that the embodiment shown in Figure 8D is only an example. In the embodiments of this application, the electronic device 100 may also use different methods than those shown in Figure 8D to complete the sound acquisition in the spatial audio recording scenario, and this application does not limit it here. For example, in some embodiments, the electronic device 100 may also determine the microphone used in the spatial audio recording scenario based on the user's holding method, etc., and this application does not limit it here.

[0262] Using the sound acquisition method provided in this application, the electronic device 100 can complete spatial audio recording in different device configurations, so as to provide users with richer recording effects.

[0263] In some embodiments, in the scenario of spatial audio recording, the electronic device 100 can use the top microphone 42, bottom microphone 43, side microphone 46 and rear microphone 47 to collect sound wave signals together, and determine the sound components in each dimension based on the device form to generate a spatial audio file.

[0264] For example, Figure 9 shows a schematic flowchart of another sound acquisition method in a spatial audio recording scenario provided by an embodiment of this application.

[0265] S901. Electronic device 100 receives operation 2 of user recording spatial audio file.

[0266] Electronic device 100 can receive and respond to operation 2, and synchronously execute the following steps S902 to S905.

[0267] In some embodiments, the electronic device 100 may also respond to operation 2 by simultaneously executing steps S906 and S902 to S905; in other embodiments, the electronic device 100 may respond to operation 2 by simultaneously executing steps S902 to S905 before executing step S906; in other embodiments, the electronic device 100 may respond to operation 2 by simultaneously executing steps S902 to S905 after executing step S906; the specific execution order of steps S906 and steps S902 to S905 is not limited in the embodiments of this application.

[0268] S902. Electronic device 100 acquires sound wave signal 1 via top microphone 42.

[0269] S903. Electronic device 100 acquires sound wave signal 2 through bottom microphone 43.

[0270] S904. Electronic device 100 acquires sound wave signal 3 via side microphone 46.

[0271] In this embodiment of the application, the side microphone 46 of the electronic device 100 may include a right microphone 46a and / or a left microphone 46b. The right microphone 46a may include one or more microphones disposed on the right edge of the electronic device 100, and the left microphone 46b may include one or more microphones disposed on the left edge of the electronic device 100.

[0272] In some embodiments, the sound wave signal 3 collected by the side microphone may include the sound wave signal 3a collected by the right microphone 46a and / or the sound wave signal 3b collected by the left microphone 46b.

[0273] S905. Electronic device 100 acquires sound wave signal 4 via rear microphone 47.

[0274] S906. Electronic device 100 acquires the device configuration of electronic device 100.

[0275] The specific method for electronic device 100 to obtain the device form of electronic device 100 can be referred to the relevant description in step S602 shown in Figure 6C above, and will not be repeated here.

[0276] In one possible implementation, after executing step S906, the electronic device 100 may determine the microphone to be used from among the multiple microphones of the electronic device 100 based on the device configuration of the electronic device 100 (for example, determining one or more microphones to be used from among the multiple side microphones 46), and then execute the above steps S902 to S905 based on the determined microphone.

[0277] S907. Electronic device 100 generates spatial audio file 1 based on device form, sound wave signal 1, sound wave signal 2, sound wave signal 3 and sound wave signal 4.

[0278] The electronic device 100 can determine the components of the sound wave signal in space in various dimensions based on the device form and the sound wave signals collected by each microphone.

[0279] In some embodiments, the electronic device 100 may store an acoustic signal processing model, which can determine the components of the acoustic signal in space in various dimensions based on the device form and the acoustic signals collected by each microphone.

[0280] For example, Figure 10A shows a schematic diagram of an acoustic signal processing model provided in an embodiment of the application.

[0281] As shown in Figure 10A, the input of the acoustic signal processing model can include the device shape, acoustic signal 1, acoustic signal 2, acoustic signal 3, and acoustic signal 4, etc.; the output of the acoustic signal processing model can include the X-axis component, Y-axis component, Z-axis component, and W-axis component of the acoustic signal in space.

[0282] It is understood that the embodiment shown in Figure 10A is only an example. In the embodiments of this application, the acoustic signal processing model may also include more, fewer, or different inputs and / or outputs than the above embodiments, and this application does not limit it.

[0283] For example, when the electronic device 100 is in different device forms, the correspondence between the sound wave signals collected by each microphone and the sound wave signals in the space can be referred to the relevant descriptions in the embodiments shown in Figures 8A-8C above, and will not be repeated here.

[0284] In some embodiments, the electronic device 100 may also store a spatial audio algorithm model, which can generate a spatial audio file 1 based on the components of sound wave signals in various dimensions in space.

[0285] For example, Figure 10B shows a schematic diagram of a spatial audio algorithm model provided in an embodiment of this application.

[0286] As shown in Figure 10B, the input of the spatial audio algorithm model may include X-axis components, Y-axis components, Z-axis components, and W-axis components, and the output of the spatial audio algorithm model may include spatial audio file 1.

[0287] It is understood that the embodiment shown in Figure 10B is only an example. In the embodiments of this application, the spatial audio algorithm model may also include more, fewer, or different inputs and / or outputs than the above embodiments, and this application does not limit it.

[0288] After determining the components of the sound wave signal in space in each dimension, the electronic device 100 can use the X-axis component, Y-axis component, Z-axis component and W-axis component of the sound wave signal as input to the spatial audio algorithm model, and generate a spatial audio file 1 through the spatial audio algorithm model.

[0289] It is understood that the embodiment shown in Figure 9 is only an example. In the embodiments of this application, the electronic device 100 may also use more, fewer, or different steps than the above embodiments to complete spatial audio recording. This application does not limit it here.

[0290] Using the sound acquisition method provided in this application, when recording spatial audio, the electronic device 100 can determine the components of the sound wave signal in various dimensions in the space based on the device shape of the electronic device 100 and the sound wave signals collected by multiple microphones, thereby completing the spatial audio recording and generating a spatial audio file 1. When playing the recorded spatial audio file 1, it can provide users with a stereo surround sound effect, giving users a better playback experience.

[0291] In some embodiments, the electronic device 100 can identify the direction of the sound source and construct a sound wave beam 1 based on the direction of the sound source and all microphones used in this operation. The main lobe range of the sound wave beam 1 can include the direction of all microphones used in this operation toward the sound source. When the electronic device 100 collects sound wave signals through one or more microphones based on the sound wave beam 1, it can amplify the sound within the main lobe range of the sound wave beam 1 and suppress the sound from other directions.

[0292] The following describes the direction of the sound wave beam constructed by the electronic device 100 when it is in different scenarios.

[0293] Figure 11A shows a schematic diagram of the direction of the sound wave beam when the electronic device 100 is in an interview scenario, according to an embodiment of this application.

[0294] As shown in Figure 11A, the electronic device 100 can be in an unfolded state, and the scenario in which the electronic device 100 is located is an interview scenario. That is, the user of the electronic device 100 simulates the sound wave beam 1 pointed at the interviewee through multiple microphones of the electronic device 100, and interviews the interviewee based on the sound wave beam 1, so as to enhance the interviewee's voice and suppress the sound from other directions.

[0295] In this interview scenario, if the microphones used by the electronic device 100 include a top microphone 42, a bottom microphone 43, and a right microphone 46a, then the main lobe range of the sound wave beam 1 constructed by the electronic device 100 can be a main lobe range Mv1. The main lobe range Mv1 can include the direction D1 of the interviewee relative to the top microphone 42, the direction D2 of the interviewee relative to the bottom microphone 43, and the direction D3 of the interviewee relative to the right microphone 46a, etc.

[0296] It is understood that the embodiment shown in Figure 11A is merely an illustrative example illustrating how directional sound acquisition can be achieved by constructing a sound wave beam, enhancing sound in a specified direction, and suppressing sound in other directions. In the embodiments of this application, the electronic device 100 may also use more, fewer, or different microphones than those in the above embodiments (e.g., left microphone 46b, rear microphone 47, etc.) to simulate a sound wave beam directed at the interviewee, which is not limited herein.

[0297] Figure 11B shows a schematic diagram of the direction of the sound wave beam when the electronic device 100 is in a call scenario, according to an embodiment of this application.

[0298] As shown in Figure 11B, the electronic device 100 can be in an unfolded state, and the scenario in which the electronic device 100 is in is a call scenario, that is, a scenario in which the user of the electronic device 100 uses the electronic device 100 to simulate a sound wave beam pointing towards the user to collect the sound emitted by the user and conduct a voice call with the user of the other end electronic device.

[0299] In this call scenario, if the microphones used by the electronic device 100 include a top microphone 42, a bottom microphone 43, and a right microphone 46a, then the main lobe range of the sound wave beam 1 constructed by the electronic device 100 can be the main lobe range Mv2. The main lobe range Mv2 can include the direction D4 of the user (e.g., the user's mouth) relative to the top microphone 42, the direction D5 of the user relative to the bottom microphone 43, and the direction D6 of the user relative to the right microphone 46a, etc.

[0300] It is understood that the embodiment shown in Figure 11B is merely an illustrative example of how directional sound acquisition can be achieved by constructing a sound wave beam. In this application embodiment, the electronic device 100 may also use more, fewer, or different microphones than those in the above embodiments (e.g., left microphone 46b, rear microphone 47, etc.) to simulate a sound wave beam pointing towards the user. This application does not limit this.

[0301] Figure 11C shows another schematic diagram of the direction of the sound wave beam when the electronic device 100 is in an interview scenario, as provided in an embodiment of this application.

[0302] As shown in Figure 11C, the electronic device 100 can be in a folded state, and the scenario in which the electronic device 100 is located is an interview scenario, that is, the user of the electronic device 100 uses the electronic device 100 to simulate a sound wave beam pointing at the interviewee, interview the interviewee, and collect the sound emitted by the interviewee.

[0303] In this interview scenario, if the microphones used by the electronic device 100 include a top microphone 42, a bottom microphone 43, a right microphone 46a, and a rear microphone 47, then the main lobe range of the sound wave beam 1 constructed by the electronic device 100 can be a main lobe range Mv1. The main lobe range Mv1 can include the direction D7 of the interviewee relative to the top microphone 42, the direction D8 of the interviewee relative to the bottom microphone 43, the direction D9 of the interviewee relative to the right microphone 46a, and the direction D10 of the interviewee relative to the rear microphone 47, etc.

[0304] It is understood that the embodiment shown in Figure 11C is merely an illustrative example of how directional sound acquisition can be achieved by constructing a sound wave beam. In this application embodiment, the electronic device 100 may also use more, fewer, or different microphones (e.g., left microphone 46b, etc.) than those in the above embodiments to simulate a sound wave beam, and this application does not limit this.

[0305] Figure 12 shows a schematic flowchart of another method for acquiring sound based on analog acoustic beams in an electronic device 100 provided in an embodiment of this application.

[0306] As shown in Figure 12, the specific process of electronic device 100 acquiring sound based on analog acoustic beams may include the following steps:

[0307] S1201. Electronic device 100 receives the user's operation to enable the directional sound acquisition function 3.

[0308] In some embodiments, operation 3 can be used to trigger the electronic device 100 to enable the directional sound acquisition function. Optionally, operation 3 can also be used to specify the direction of the sound wave beam to be constructed.

[0309] For example, operation 3 could be the user answering or making a phone call, the user starting an interview mode, or the user starting to record audio / video, etc.

[0310] S1202. Electronic device 100 determines the microphone to be used this time.

[0311] Electronic device 100 can respond to user operation 3 and determine the microphone to be used.

[0312] In one possible implementation, electronic device 100 may use all microphones by default. In this case, electronic device 100 can determine that the microphones used this time include all microphones of electronic device 100.

[0313] In another possible implementation, the electronic device 100 can store the microphone's activation status in different scenarios. In this case, the electronic device 100 needs to first obtain the scenario in which it is located, and then determine the microphone to be used based on the correspondence between the scenario and the microphone. The scenario in which the electronic device 100 is located can include, but is not limited to, any one or more of the following: interview scenario, live broadcast scenario, outdoor scenario, indoor scenario, work scenario, sports scenario, etc.

[0314] For example, Table 1 shows the microphone activation status in different scenarios provided by embodiments of this application.

[0315] As shown in Table 1, the electronic device 100 can store the microphone activation status for multiple scenarios. For example, in an interview scenario, the electronic device 100 can use the top microphone 42, bottom microphone 43, right microphone 46a, left microphone 46b, and rear microphone 47; in a live broadcast scenario, the electronic device 100 can use the top microphone 42, bottom microphone 43, right microphone 46a, and rear microphone 47; and in a call scenario, the electronic device 100 can use the top microphone 42, bottom microphone 43, right microphone 46a, and left microphone 46b.

[0316] It is understood that the embodiments shown in Table 1 are merely illustrative. In different scenarios, the electronic device 100 may have different preset microphone activation states. In the embodiments of this application, the electronic device 100 may also store more, fewer, or different scenarios than those in the above embodiments, as well as the microphone activation states corresponding to those scenarios. This application does not limit these scenarios.

[0317] The following describes how the scene in which electronic device 100 is located is determined.

[0318] In some embodiments, the electronic device 100 can determine its location using one or more devices such as an accelerometer, a gyroscope, a magnetic sensor, and a positioning device. For example, the electronic device 100 can determine that its environment is a company based on the positioning device, and determine that the current scene is a work scene based on the environment; the electronic device 100 can determine that its environment is outdoors based on the positioning device, and determine that the current scene is an outdoor scene based on the environment, and so on. As another example, the electronic device 100 can determine that the user is in motion using an accelerometer, and determine that the user is in a motion scene based on the user's motion state, and so on.

[0319] In other embodiments, the electronic device 100 can also determine the current scene through running application software. For example, if the electronic device 100 detects that the currently running application software includes live streaming software, it can determine that the current scene is a live streaming scene, etc.

[0320] It is understood that the embodiments described here are merely examples. In the embodiments of this application, the electronic device 100 may also determine the scene in a way different from the above embodiments. Alternatively, the scene in which the electronic device 100 is located may be determined by combining the operation of the application software and the detection results of sensors (such as accelerometers, magnetic sensors, positioning devices, etc.). This application does not limit this.

[0321] After determining the scene in which the electronic device 100 is located, the electronic device 100 can determine the microphone to be used this time based on the stored microphone activation status in that scene.

[0322] In other embodiments, if any microphone in the electronic device 100 malfunctions, a fault notification can be reported. The electronic device 100 can then adjust the microphone used based on the received fault notification to avoid the microphone malfunctioning and affecting sound acquisition.

[0323] S1203. Electronic device 100 acquires positioning sound wave signals through multiple microphones.

[0324] Steps S1203 to S1204 are optional steps.

[0325] In some embodiments, the electronic device 100 may respond to the user's operation 3 and determine the direction of the acoustic beam to be constructed based on the user's operation 3. In this case, the electronic device 100 may not perform steps S1203 to S1204.

[0326] In other embodiments, the electronic device 100 may, in response to user operation 3, acquire location acoustic signals via multiple microphones. The acquisition duration of the location acoustic signals may be a preset duration (e.g., 5 seconds, 10 seconds, 15 seconds, etc.). The location acoustic signals can be used to identify the location of a sound source.

[0327] It should be noted that, in some embodiments, the multiple microphones used by the electronic device 100 to collect the positioning sound wave signal may be the microphones determined in step S1202 for this use. In other embodiments, the multiple microphones used by the electronic device 100 to collect the positioning sound wave signal may also be all the microphones of the electronic device 100, which can improve the accuracy of sound source location identification. In still other embodiments, the multiple microphones used by the electronic device 100 to collect the positioning sound wave signal may also be multiple microphones located at different positions on the electronic device 100, which is not limited herein.

[0328] S1204. Electronic device 100 determines the location of a sound source based on a positioning acoustic wave signal.

[0329] Because the microphones on the electronic device 100 are located in different positions, the location and distance of the sound source relative to each microphone are also different. Therefore, there are certain differences between the positioning sound wave signals collected by each microphone. The electronic device 100 can determine the location of the sound source based on the positioning sound wave signals collected by different microphones.

[0330] In some embodiments, the electronic device 100 may store a sound source location identification model, which can determine the location of the sound source based on the location sound wave signals collected by multiple microphones.

[0331] For example, Figure 13 shows a schematic diagram of a sound source location identification model provided in an embodiment of this application.

[0332] As shown in Figure 13, the input of the sound source location recognition model can include location sound wave signal 1 and location sound wave signal 2, which are location sound wave signals collected by different microphones; the output of the sound source location recognition model can include the sound source location.

[0333] It is understood that the embodiment shown in Figure 13 is only an example. In the embodiments of this application, the sound source location identification model may also include more, fewer, or different inputs and / or outputs than the above embodiments, and this application does not limit it.

[0334] S1205. Electronic device 100 constructs a sound wave beam 1 based on the sound source location and the microphone used in this application.

[0335] Electronic device 100 can determine the microphone used in this operation and construct a sound wave beam 1 based on the microphone used in this operation. The main lobe range of the sound wave beam 1 can include the directions of all microphones used in this operation pointing to the sound source.

[0336] In some embodiments, the electronic device 100 may store an acoustic beamforming model, which can construct an acoustic beam 1 pointing to the location of the sound source based on the location of the sound source and the microphone used in this instance.

[0337] For example, Figure 14 shows a schematic diagram of an acoustic beamforming model provided in an embodiment of this application.

[0338] As shown in Figure 14, the input to the acoustic beamforming model can include the location of the sound source and the microphone used in this case, and the output can include acoustic beam 1.

[0339] It is understood that the embodiment shown in Figure 14 is only an example. In the embodiments of this application, the acoustic beamforming model may include more, fewer, or different inputs and / or outputs than the above embodiments, and this application does not limit it.

[0340] After determining the location of the sound source and the microphone used, the electronic device 100 can obtain the sound beam 1 based on the sound beam model.

[0341] S1206. Electronic device 100 acquires sound wave signal R1 through the microphone used in this application based on sound wave beam 1.

[0342] The electronic device 100 can set sound acquisition parameters for each microphone used in this application based on the sound wave beam 1. When the microphone acquires sound wave signals based on the set sound wave acquisition parameters, it can enhance the sound within the main lobe range of the sound wave beam 1 and suppress the sound in other directions.

[0343] For example, if the microphones used in this case include a top microphone 42, a bottom microphone 43, a side microphone 46, and a rear microphone 47, then the sound wave signal R1 can include the sound wave signal collected by the electronic device 100 based on the sound wave beam 1 through the top microphone 42, bottom microphone 43, side microphone 46, and rear microphone 47. It should be noted that the top microphone 42, bottom microphone 43, side microphone 46, and rear microphone 47 can collect sound wave signals simultaneously.

[0344] It is understood that the embodiments described herein are merely exemplary, and the electronic device 100 may also collect sound wave signals through more, fewer, or different microphones than those described in the above embodiments, which is not limited herein.

[0345] S1207. Electronic device 100 determines audio file 3 based on sound wave signal R1.

[0346] It is understood that the embodiment shown in Figure 12 is only an example. In the embodiments of this application, the electronic device 100 may also adopt more, fewer or different steps than the above embodiments to realize the directional sound acquisition function and acquire sound in a specified direction. This application does not limit this.

[0347] Using the sound acquisition method provided in this application, the electronic device 100 can construct a highly directional sound wave beam, acquire sound in the direction of the sound source, and suppress sound in other directions, thereby achieving noise suppression and providing users with a better user experience.

[0348] In some embodiments, the electronic device 100 may also implement an audio zoom function based on the sound acquisition method provided in the embodiments of this application. The audio zoom function can simulate the human brain's ability to focus attention on multiple sound sources, enhance sound in a specific area (or a specific direction), and suppress sound in areas outside the specific area (or in other directions besides the specific direction).

[0349] In some application scenarios, the electronic device 100 can also enable audio zoom when the user is recording video. After enabling audio zoom, the electronic device 100 can determine the sound source region 1 that needs to be focused based on the user's operation. After determining the sound source region 1, the electronic device 100 can construct a sound wave beam 2 based on the sound source region 1 and the microphone used. The direction of the sound wave beam 2 can be the location of the sound source region 1, and the main lobe range of the sound wave beam 2 can include the direction of all the microphones used relative to the sound source region 1. Subsequently, when the electronic device 100 collects sound wave signals through one or more microphones based on the sound wave beam 2, it can amplify the sound within the main lobe range of the sound wave beam 2 and suppress sound from other directions.

[0350] In this way, in the context of video recording, sound acquisition can be focused on a designated sound source area, reducing interference from other noises and helping video viewers concentrate.

[0351] The following describes the direction of the sound wave beam constructed by the electronic device 100 when recording video.

[0352] Figure 15A shows a schematic diagram of the direction of the sound wave beam when the electronic device 100 is recording video, according to an embodiment of this application.

[0353] As shown in Figure 15A, the electronic device 100 can be in a semi-folded state and in selfie mode, that is, the electronic device 100 captures the user's image through the front-facing camera. In some embodiments, in selfie mode, the sound source area that the electronic device 100 needs to focus on when recording video can be a default sound source area; in other embodiments, the electronic device 100 can also determine the sound source area that needs to be focused on based on the user's operation.

[0354] For example, if the electronic device 100 determines, based on the user's operation or selfie mode, that the sound source area to be focused is the sound source area 1 near the bottom microphone 43, then the electronic device 100 can construct a sound wave beam 2 based on the sound source area 1 and the microphone used this time. For example, if the microphone used this time includes the top microphone 42, the bottom microphone 43, the right microphone 46a, and the left microphone 46b, then the main lobe range of the sound wave beam 2 can include the direction Z1 of the top microphone 42 pointing to the sound source area 1, the direction Z2 of the bottom microphone 43 pointing to the sound source area 1, the direction Z3 of the right microphone 46a pointing to the sound source area 1, and the direction Z4 of the left microphone 46b pointing to the sound source area 1.

[0355] Subsequently, the electronic device 100 can collect sound wave signals based on the sound wave beam 2 through the top microphone 42, bottom microphone 43, right microphone 46a and left microphone 46b to enhance the sound in the sound source area 1 while suppressing the sound in other sound source areas.

[0356] Figure 15B shows a schematic diagram of the direction of the sound wave beam when the electronic device 100 is recording video, according to an embodiment of this application.

[0357] As shown in Figure 15B, the electronic device 100 can be in a semi-folded state and in camera mode, that is, the electronic device 100 captures images through a rear camera. In some embodiments, in camera mode, when the electronic device 100 records video, the sound source area that needs to be focused on can be a default sound source area; in other embodiments, the electronic device 100 can also determine the sound source area that needs to be focused on based on the user's operation.

[0358] For example, if the electronic device 100 determines, based on the user's operation or camera mode, that the sound source area to be focused is the sound source area 1 near the rear camera, then the electronic device 100 can construct a sound wave beam 2 based on the sound source area 1 and the microphone used this time. For example, if the microphone used this time includes a top microphone 42, a bottom microphone 43, a right microphone 46a, and a left microphone 46b, then the main lobe range of the sound wave beam 2 can include the direction Z5 of the top microphone 42 pointing to the sound source area 1, the direction Z6 of the bottom microphone 43 pointing to the sound source area 1, the direction Z7 of the right microphone 46a pointing to the sound source area 1, and the direction Z8 of the left microphone 46b pointing to the sound source area 1.

[0359] Subsequently, the electronic device 100 can collect sound wave signals based on the sound wave beam 2 through the top microphone 42, bottom microphone 43, right microphone 46a and left microphone 46b to enhance the sound in the sound source area 1 while suppressing the sound in other sound source areas.

[0360] It is understood that the embodiments shown in Figure 15B are only two examples. In the embodiments of this application, the electronic device 100 may also use more, fewer, or different microphones (e.g., rear microphone 47, etc.) than the above embodiments to collect sound during video recording to achieve audio zoom function. In this case, the main lobe range of the sound wave beam 1 may also be different from the above embodiments, and this application does not limit it here. In some other embodiments, the electronic device 100 may also use a device form different from the above embodiments (e.g., unfolded state, folded state, etc.) to record video and collect sound through audio zoom function, and this application does not limit it here.

[0361] The following describes a method flow for an electronic device 100 to collect sound during video recording, provided by an embodiment of this application.

[0362] As shown in Figure 16, the specific process of capturing sound during video recording may include the following steps:

[0363] S1601. Electronic device 100 receives user's video recording operation 4.

[0364] Operation 4 can be used to trigger electronic device 100 to enable the audio zoom function.

[0365] In this embodiment, operation 4 can be a user operation on the display screen, a button operation, or a user gesture operation, etc. For example, operation 4 can be a user clicking on a recording control in a camera application.

[0366] S1602. Electronic device 100 captures images 1 via a camera.

[0367] Electronic device 100 can respond to operation 4 by capturing image 1 via camera.

[0368] In some embodiments, if the electronic device 100 includes multiple cameras, in response to operation 4, the electronic device 100 can determine the camera to be used (e.g., a front camera or a rear camera) and capture image 1 using the camera to be used.

[0369] In some embodiments, the electronic device 100 may perform step S1603 during the execution of step S1602; in other embodiments, the electronic device 100 may also perform step S1606 during the execution of step S1602.

[0370] S1603. Electronic device 100 determines whether the camera mode is a selfie mode.

[0371] Steps S1603 to S1605 are optional.

[0372] In some embodiments, the camera mode may include a selfie mode and a camera mode. The selfie mode is a camera mode in which the electronic device 100 captures images using a front-facing camera; the camera mode refers to a mode in which the electronic device 100 uses a rear-facing camera to capture images.

[0373] Electronic device 100 can determine whether the camera mode is a selfie mode based on whether the camera used is a front-facing camera.

[0374] If the camera used this time is a front-facing camera, then the camera mode is determined to be the selfie camera mode. At this time, the electronic device 100 can perform the following step S1604.

[0375] If the camera used this time is a rear camera, then the camera mode is determined to be camera mode. At this time, the electronic device 100 can perform the following step S1605.

[0376] S1604. Electronic device 100 determines the sound source region 1 based on the selfie mode.

[0377] In some embodiments, in selfie mode, the electronic device 100 may default the sound source area 1 that needs to be focused on to be an area with a specific positional relationship to the electronic device 100, such as the sound source area near the bottom microphone 43. In this way, in selfie mode, the user is generally located within the sound source area 1, which can enhance the sound emitted by the user.

[0378] S1605. Electronic device 100 determines the sound source region 1 based on camera mode.

[0379] In some embodiments, in camera mode, the electronic device 100 may default the sound source area 1 that needs to be focused on to be an area with a specific positional relationship to the electronic device 100, such as the area directly facing the rear camera. In this way, in camera mode, the subject being filmed is generally located within the sound source area 1, which can enhance the sound emitted by the subject.

[0380] S1606. Electronic device 100 receives and responds to user operation 5 to determine sound source region 1.

[0381] In some embodiments, the electronic device 100 may also determine the sound source region 1 based on the user's operation 5 of selecting the sound source region to be focused.

[0382] In this embodiment, operation 5 can be a user operation on the display screen, a button operation, or a user gesture operation, etc. For example, when a real-time captured image is displayed on the display screen (e.g., display area 41A) of the electronic device 100, the electronic device 100 can receive and respond to the user's zoom-in operation on area a in the image to determine that the sound source area 1 is the area in actual space where the entity of the image in area a is located. That is, if an image of a puppy is displayed in area a, then by receiving and responding to the user's zoom-in operation on area a in the image, it can be determined that the sound source area 1 is the area where the puppy is located.

[0383] S1607. Electronic device 100 constructs a sound beam 2 based on the microphone and sound source region 1 used in this application.

[0384] The specific method by which electronic device 100 determines the microphone to be used can be referred to the relevant description in step S1202 shown in Figure 12 above.

[0385] The main lobe range of the sound beam 2 can include the direction in which all the microphones used in this operation are pointing towards the sound source.

[0386] The specific method by which electronic device 100 constructs sound wave beam 2 based on the microphone and sound source region 1 used in this application can also be referred to the relevant description of step S1205 shown in Figure 12 above, and will not be repeated here.

[0387] S1608. Electronic device 100 acquires sound wave signal R2 through the microphone used in this application based on sound wave beam 2.

[0388] The specific content of step S1608 can be compared with the relevant description of step S1206 shown in Figure 12 above, and will not be repeated here.

[0389] S1609. Electronic device 100 generates audio track 1 based on sound wave signal R2.

[0390] S1610. Electronic device 100 generates video file 1 based on image 1 and audio track 1.

[0391] It is understood that the embodiment shown in Figure 16 is only an example. In the embodiments of this application, the electronic device 100 may also adopt more, fewer or different steps than the above embodiments to implement the audio zoom function, enhance the sound in the specified sound source area and suppress the sound in other sound source areas. This application does not limit this.

[0392] Using the sound acquisition method provided in this application, the electronic device 100 can amplify the sound in a designated sound source area and suppress the sound in other sound source areas. Thus, in video recording scenarios, sound acquisition can be focused on the designated sound source area, reducing interference from other noises and helping video viewers concentrate.

[0393] In some application scenarios, during video recording, the electronic device 100 can switch the focus area of ​​the sound source from sound source area 1 to sound source area 2 based on user input. After determining sound source area 2, the electronic device 100 can construct a sound wave beam 3 based on sound source area 2 and the microphone used. The direction of the sound wave beam 3 can be the location of sound source area 2, and the main lobe range of the sound wave beam 3 can include the direction of all microphones used relative to sound source area 2. Subsequently, when the electronic device 100 collects sound wave signals through one or more microphones based on the sound wave beam 3, it can amplify the sound within the main lobe range of the sound wave beam 3 and suppress sound from other directions.

[0394] In this way, in video recording scenarios, the system can focus on capturing sound from different sound source areas at different times based on user actions, providing users with richer usage methods and a better user experience.

[0395] The functional modules of an electronic device 100 provided in the embodiments of this application are described below.

[0396] Figure 17 shows a schematic diagram of the functional modules of an electronic device 100 provided in an embodiment of this application.

[0397] As shown in Figure 17, the electronic device 100 may include a sound acquisition module 1701, a sound processing module 1702, an audio storage / playback module 1703, a device form monitoring module 1704, and a user interaction module 1705. Optionally, the electronic device 100 may also include any one or more of the following: a sound source localization module 1706, a beamforming module 1707, an image acquisition module 1708, a video storage / playback module 1709, etc. Wherein:

[0398] The sound acquisition module 1701 can acquire sound wave signals and send the acquired sound wave signals to the sound processing module 1702. In some embodiments, the sound acquisition module 1701 can receive a call command M1 sent by the user interaction module 1705, determine the microphone used for the call based on the device configuration sent by the device configuration monitoring module 1704, and acquire the sound wave signals emitted by the user based on the microphone used for the call. In some embodiments, the sound acquisition module 1701 can receive a spatial audio recording command sent by the user interaction module 1705, and acquire the X-dimensional, Y-dimensional, and Z-dimensional components of the sound wave signals in space. In some embodiments, the sound acquisition module 1701 can also receive a sound wave beam (e.g., sound wave beam 1, sound wave beam 2, etc.) sent by the beamforming module 1707, and adjust the sound acquisition parameters of each microphone based on the sound wave beam to enhance the sound in the direction the sound wave beam points and suppress the sound in other directions. In some embodiments, the sound acquisition module 1701 can receive a positioning command sent by the user interaction module 1705, acquire a positioning sound wave signal, and send the positioning sound wave signal to the sound source positioning module 1706 or the sound processing module 1702.

[0399] The sound processing module 1702 can receive the sound wave signal sent by the sound acquisition module 1701 and perform noise reduction, filtering, and other processing on the sound wave signal. In some embodiments, the sound processing module 1702 can send the processed sound wave signal to the audio storage / playback module 1703. In other embodiments, the sound processing module 1702 can send the processed sound wave signal to the video storage / playback module 1709. In other embodiments, the sound processing module 1702 can receive the positioning sound wave signal and send the processed positioning sound wave signal to the sound source positioning module 1706.

[0400] The audio storage / playback module 1703 can generate and store an audio file based on the processed sound wave signal. In some embodiments, the audio storage / playback module 1703 can receive and respond to an audio playback request sent by the user interaction module 1705, and play a specified audio file.

[0401] The device state monitoring module 1704 can monitor the device state of the electronic device 100. In some embodiments, the device state monitoring module 1704 can receive and respond to the call command M2 sent by the user interaction module 1705, and send the current device state of the electronic device 100 to the sound acquisition module 1701.

[0402] The user interaction module 1705 can interact with the user, receive user operations, and invoke various modules in the electronic device 100 to perform specified operations based on the user's operations. In some embodiments, the user interaction module 1705 can receive the user's operation of answering / making a phone call, send a call instruction M1 to the sound acquisition module 1701, and send a call instruction M2 to the device form monitoring module 1704. The call instruction M1 is used to instruct the sound acquisition module 1701 to acquire the sound wave signal emitted by the user, and the call instruction M2 is used to instruct the device form monitoring module 1704 to send the device form of the electronic device 100 to the sound acquisition module 1701. In some embodiments, the user interaction module 1705 can receive the user's operation of recording spatial audio, and send a spatial audio recording instruction to the sound acquisition module 1701. The spatial audio recording instruction is used to instruct the sound acquisition module 1701 to acquire the X-dimensional, Y-dimensional, and Z-dimensional components of the sound wave signal in space. In some embodiments, the user interaction module 1705 may receive and respond to a user's operation of enabling directional sound acquisition / enabling audio zoom function, and send a beamforming instruction to the beamforming module 1707. The beamforming instruction is used to instruct the beamforming module 1707 to construct a sound wave beam (e.g., sound wave beam 1 or sound wave beam 2, etc.). In some embodiments, the user interaction module 1705 may receive and respond to a user's operation of enabling directional sound acquisition / enabling audio zoom function, and send a positioning instruction to the sound acquisition module 1701. The positioning instruction is used to instruct the sound acquisition module 1701 to acquire a positioning sound wave signal. In some embodiments, the user interaction module 1705 may receive and respond to a user's operation of recording video / enabling audio zoom function, and send an image acquisition instruction to the image acquisition module 1708. The image acquisition instruction is used to instruct the image acquisition module 1708 to acquire an image. In some embodiments, the user interaction module 1705 may also receive and respond to a user's operation of playing video, and send a video playback request to the video storage / playback module 1709. The video playback request is used to request the video storage / playback module 1709 to play a specified video file.

[0403] The sound source localization module 1706 can receive the localization sound wave signal sent by the sound acquisition module 1701 or the sound processing module 1702, and determine the sound source location / sound source region based on the localization sound wave signal. The sound source localization module 1706 can send the sound source location / sound source region to the beamforming module 1707.

[0404] The beamforming module 1707 can receive the sound source location / region from the sound source localization module 1706. The beamforming module 1707 can also receive beamforming commands from the user interaction module 1705 to determine the microphone to be used. Then, based on the sound source location / region and the microphone used, the beamforming module 1707 can construct a sound wave beam (e.g., sound wave beam 1, sound wave beam 2, etc.). The beamforming module 1707 can then send the sound wave beam to the sound acquisition module 1701.

[0405] The image acquisition module 1708 can receive image acquisition commands sent by the user interaction module 1705 and acquire images. The image acquisition module 1708 can then send the acquired images to the video storage / playback module 1709.

[0406] The video storage / playback module 1709 can generate and store video based on the video sent by the image acquisition module 1708 and the audio signal sent by the audio processing module 1702. The video storage / playback module 1709 can also receive video playback requests and playback instructions for video files sent by the user interaction module 1705.

[0407] It is understood that the embodiment shown in FIG17 is only an example. In the embodiments of this application, the electronic device 100 may include more, fewer or different functional modules than the above embodiments. In addition, any of the above modules may be divided into multiple modules, and any multiple modules may be merged into one module. This application does not limit this.

[0408] For ease of subsequent description, the aforementioned electronic device 100 can be collectively referred to as a device. It should be understood that the division of units within this device is merely a logical functional division; in actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, the units within the device can be implemented by a processor calling software; for example, the device includes a processor connected to a memory containing instructions. The processor calls the instructions stored in the memory to implement any of the above methods or to realize the functions of each unit within the device. The processor can be, for example, a general-purpose processor, such as a central processing unit (CPU) or a microprocessor, and the memory can be internal or external to the device. Alternatively, the units in the device can be implemented as hardware circuits. The functionality of some or all units can be achieved through the design of these hardware circuits, which can be understood as one or more processors. For example, in one implementation, the hardware circuit is an application-specific integrated circuit (ASIC). The functionality of some or all of the above units is achieved through the design of the logical relationships between the components within the circuit. In another implementation, the hardware circuit can be implemented using a programmable logic device (PLD). Taking a field-programmable gate array (FPGA) as an example, it can include a large number of logic gates. The connection relationships between the logic gates are configured through a configuration file, thereby achieving the functionality of some or all of the above units. All units of the above device can be implemented entirely through processor-invoked software, entirely through hardware circuits, or partially through processor-invoked software with the remaining parts implemented through hardware circuits.

[0409] In this application embodiment, a processor is a circuit with data processing capabilities. In one implementation, the processor can be a circuit with instruction reading and execution capabilities, such as a CPU, microprocessor, graphics processing unit (GPU) (which can be understood as a type of microprocessor), or digital signal processor (DSP). In another implementation, the processor can implement certain functions through the logical relationships of hardware circuits. These logical relationships of hardware circuits are fixed or reconfigurable. For example, the processor is a hardware circuit implemented as an ASIC or PLD, such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document and configuring the hardware circuit can be understood as the process of the processor loading instructions to implement the functions of some or all of the above units. Furthermore, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as a Neural Network Processing Unit (NPU), Tensor Processing Unit (TPU), or Deep Learning Processing Unit (DPU).

[0410] As can be seen, each unit in the above device can be one or more processors (or processing circuits) configured to implement the above methods, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.

[0411] Furthermore, the units in the above devices can be integrated in whole or in part, or they can be implemented independently. In one implementation, these units are integrated together as a system-on-a-chip (SOC). The SOC may include at least one processor for implementing any of the above methods or implementing the functions of the units in the device. The at least one processor may be of different types, such as CPU and FPGA, CPU and artificial intelligence processor, CPU and GPU, etc.

[0412] The following describes a possible physical structure of the electronic device 100 provided in the embodiments of this application.

[0413] Figure 18 shows a schematic diagram of the physical structure of an electronic device 100 provided in an embodiment of this application.

[0414] As shown in Figure 18, the electronic device 100 may include a sound acquisition unit 1801, a sound processor 1802, an audio storage / player 1803, a device form factor monitor 1804, and a user interface 1805. Optionally, the electronic device 100 may also include any one or more of the following: a sound source locator 1806, a beamformer 1807, an image acquisition unit 1808, a video storage / player 1809, etc. Wherein:

[0415] The sound acquisition unit 1801 can acquire sound wave signals and send the acquired sound wave signals to the sound processor 1802. In some embodiments, the sound acquisition unit 1801 can receive a call command M1 sent by the user interface 1805, determine the microphone used for the call based on the device configuration sent by the device configuration monitor 1804, and acquire the sound wave signals emitted by the user based on the microphone used for the call. In some embodiments, the sound acquisition unit 1801 can receive a spatial audio recording command sent by the user interface 1805 and acquire the X-dimensional, Y-dimensional, and Z-dimensional components of the sound wave signals in space. In some embodiments, the sound acquisition unit 1801 can also receive sound wave beams (e.g., sound wave beam 1, sound wave beam 2, etc.) sent by the beambuilder 1807 and adjust the sound acquisition parameters of each microphone based on the sound wave beams to enhance the sound in the direction the sound wave beams point and suppress the sound in other directions. In some embodiments, the sound collector 1801 can receive a positioning command sent by the user interface 1805, collect positioning sound wave signals, and send the positioning sound wave signals to the sound source locator 1806 or the sound processor 1802.

[0416] The sound processor 1802 can receive sound wave signals sent by the sound acquisition unit 1801 and perform noise reduction, filtering, and other processing on the sound wave signals. In some embodiments, the sound processor 1802 can send the processed sound wave signals to the audio storage / player 1803. In other embodiments, the sound processor 1802 can send the processed sound wave signals to the video storage / player 1809. In still other embodiments, the sound processor 1802 can receive positioning sound wave signals and send the processed positioning sound wave signals to the sound source locator 1806.

[0417] The audio storage / player 1803 can generate and store an audio file based on the processed sound wave signal. In some embodiments, the audio storage / player 1803 can receive and respond to an audio playback request sent by the user interface 1805 to play a specified audio file.

[0418] Device status monitor 1804 can monitor the device status of electronic device 100. In some embodiments, device status monitor 1804 can receive and respond to call command M2 sent by user interface 1805, and send the current device status of electronic device 100 to sound collector 1801.

[0419] The user interface 1805 can interact with the user, receive user operations, and, based on the user's operations, invoke various devices in the electronic device 100 to perform specified operations. In some embodiments, the user interface 1805 can receive the user's operation of answering / making a phone call, send a call instruction M1 to the sound acquisition unit 1801, and send a call instruction M2 to the device form monitor 1804. The call instruction M1 is used to instruct the sound acquisition unit 1801 to acquire the sound wave signal emitted by the user, and the call instruction M2 is used to instruct the device form monitor 1804 to send the device form of the electronic device 100 to the sound acquisition unit 1801. In some embodiments, the user interface 1805 can receive the user's operation of recording spatial audio, and send a spatial audio recording instruction to the sound acquisition unit 1801. The spatial audio recording instruction is used to instruct the sound acquisition unit 1801 to acquire the X-dimensional, Y-dimensional, and Z-dimensional components of the sound wave signal in space. In some embodiments, the user interface 1805 may receive and respond to a user's operation of enabling directional sound acquisition / enabling audio zoom function, and send a beamforming instruction to the beambuilder 1807. The beamforming instruction instructs the beambuilder 1807 to construct an acoustic beam (e.g., acoustic beam 1 or acoustic beam 2). In some embodiments, the user interface 1805 may receive and respond to a user's operation of enabling directional sound acquisition / enabling audio zoom function, and send a positioning instruction to the sound collector 1801. The positioning instruction instructs the sound collector 1801 to acquire a positioning acoustic signal. In some embodiments, the user interface 1805 may receive and respond to a user's operation of recording video / enabling audio zoom function, and send an image acquisition instruction to the image collector 1808. The image acquisition instruction instructs the image collector 1808 to acquire an image. In some embodiments, the user interface 1805 may also receive and respond to a user's operation of playing video, and send a video playback request to the video storage / player 1809. The video playback request requests the video storage / player 1809 to play a specified video file.

[0420] The sound source locator 1806 can receive the location sound wave signal sent by the sound acquisition unit 1801 or the sound processor 1802, and determine the sound source location / sound source area based on the location sound wave signal. The sound source locator 1806 can send the sound source location / sound source area to the beambuilder 1807.

[0421] Beambuilder 1807 can receive the sound source location / sound source region sent by sound source locator 1806. Beambuilder 1807 can also receive beamforming instructions sent by user interface 1805 to determine the microphone to be used. Then, beambuilder 1807 can construct a sound beam (e.g., sound beam 1, sound beam 2, etc.) based on the sound source location / sound source region and the microphone used. Beambuilder 1807 can then send the sound beam to sound acquisition unit 1801.

[0422] The image capture device 1808 can receive image capture commands sent by the user interface 1805 and capture images. The image capture device 1808 can send the captured images to the video storage / player 1809.

[0423] The video storage / player 1809 can generate and store video based on the video sent by the image capture unit 1808 and the audio signal sent by the audio processor 1802. The video storage / player 1809 can also receive video playback requests and playback instructions for video files sent by the user interface unit 1805.

[0424] It is understood that the embodiment shown in FIG18 is only an example. In the embodiments of this application, the electronic device 100 may include more, fewer, or different devices than those in the above embodiments. In addition, any one of the above devices may be divided into multiple devices, and any multiple devices may be combined into one device. This application does not limit this.

[0425] Figure 19 shows a schematic flowchart of a sound acquisition method provided in an embodiment of this application.

[0426] As shown in Figure 19, the specific process of a sound acquisition method may include the following steps:

[0427] S1901. When the electronic device is in a folded state, the electronic device detects that the microphone function is enabled; the electronic device includes a display screen, a first microphone, a second microphone and a third microphone, wherein the first microphone is located at the top of the electronic device, the second microphone is located at the bottom of the electronic device and the third microphone is located on the side of the electronic device.

[0428] The electronic device can be a foldable electronic device (also called a foldable device). In some embodiments, the electronic device can be an electronic device with a foldable display screen; in other embodiments, the electronic device can also be an electronic device with a non-foldable display screen but a foldable device itself. For example, the electronic device can be the electronic device 100 in the above embodiments. The definitions of the top, bottom, and sides of the electronic device can be referred to the relevant descriptions in the embodiments shown in FIG4A above, and will not be repeated here.

[0429] For example, the display screen can be the foldable display screen 41 in the embodiment shown in FIG4A above, the first microphone can be the top microphone 42 in the above embodiment, the second microphone can be the bottom microphone 43 in the above embodiment, and the third microphone can be the side microphone 46 in the above embodiment.

[0430] S1902. Electronic devices acquire sound wave signals via a third microphone.

[0431] In this way, in the folded state, the electronic device can collect sound wave signals through the third microphone. Since the third microphone is located on the side of the electronic device, it is difficult for the microphone to be blocked when the user holds the folded electronic device. Therefore, the risk of the microphone being blocked can be reduced, and the quality of sound acquisition can be improved.

[0432] In one possible implementation, before acquiring the acoustic signal via the third microphone, the method further includes detecting that the first microphone and / or the second microphone is blocked.

[0433] In this way, if the first and / or second microphones are detected to be blocked, the electronic device can use the third microphone on the side to collect sound, avoiding poor sound pickup due to blockage and improving the quality of sound collection.

[0434] In one possible implementation, the electronic device further includes a gravity sensor and / or a gyroscope sensor; detecting that the first microphone and / or the second microphone is blocked specifically includes: detecting the way the electronic device is held using the gravity sensor and / or the gyroscope sensor; and determining that the first microphone and / or the second microphone is blocked based on the way the electronic device is held.

[0435] In this way, the electronic device can detect the way it is held using a gravity sensor and / or a gyroscope sensor, and determine whether the first and second microphones are blocked based on the way the device is held. If the device is detected to be held in a preset manner, and it is determined that the first and / or second microphones are blocked, then the third microphone on the side can be used to collect sound.

[0436] In one possible implementation, the electronic device includes a low-power camera; detecting that the first microphone and / or the second microphone is blocked specifically includes: detecting the way the electronic device is held via the low-power camera; and determining that the first microphone and / or the second microphone is blocked based on the way the electronic device is held.

[0437] In this way, images can be captured using a low-power camera, and the way the electronic device is held can be determined based on the positional relationships of objects in the captured images. This holding method can then be used to determine whether the first and second microphones are blocked. If it is determined that the first and / or second microphones are blocked, a third microphone can be used to capture sound.

[0438] In one possible implementation, detecting that the first microphone and / or the second microphone is blocked specifically includes: acquiring a first test signal through the first microphone; determining that the first microphone is blocked based on the first test signal; and / or acquiring a second test signal through the second microphone; determining that the second microphone is blocked based on the second test signal.

[0439] In this way, test signals can be directly acquired through the microphone under test (e.g., the first microphone and the second microphone), and it can be determined whether the microphone under test is blocked based on the relationship between the strength of the test signal and a preset strength threshold. If the strength of the test signal is less than or equal to the preset strength threshold, it is determined that the microphone acquiring the test signal is blocked; if the strength of the test signal is greater than the preset strength threshold, it is determined that the microphone acquiring the test signal is not blocked.

[0440] In one possible implementation, detecting that the electronic device has enabled the microphone function includes: detecting that the electronic device is making or receiving a phone call; or detecting that the electronic device has started recording.

[0441] In this way, in call or recording scenarios, the system can determine whether to use a third microphone to collect sound based on the device configuration of the electronic device, thus avoiding blocking the microphone.

[0442] In one possible implementation, the electronic device further includes a fourth microphone, which is disposed on the side of the electronic device, and the third and fourth microphones are disposed on different sides of the electronic device; when the electronic device is in a folded state, the activation of the microphone function of the electronic device is detected, specifically including: receiving a first operation to enable the directional sound acquisition function; the method further includes: in response to the first operation, determining the location of a second sound source; constructing a second beam based on the location of the second sound source; and acquiring sound wave signals through the third microphone, specifically including: acquiring sound wave signals through the third and fourth microphones based on the second beam.

[0443] Thus, in a directional sound acquisition scenario, if the electronic device is in a folded state, it can construct a second beam based on the direction of the second sound source, with the second beam pointing towards that direction. At this time, the electronic device uses multiple microphones to acquire sound wave signals based on the second beam, thereby enhancing the sound at the direction of the second sound source and suppressing sound from other directions.

[0444] In one possible implementation, in response to the first operation, determining the location of the second sound source specifically includes: determining the location of the second sound source based on the first operation; or, acquiring location sound wave signals through a third microphone and a fourth microphone; and determining the location of the second sound source based on the location sound wave signals.

[0445] In this way, the electronic device can determine the location of the second sound source based on the user's first operation, or it can collect the location sound wave signal on its own and determine the location of the second sound source based on the location sound wave signal.

[0446] In one possible implementation, when the electronic device is not in a folded state, the method further includes: receiving a second operation to enable directional sound acquisition; in response to the second operation, determining the location of a third sound source; constructing a third beam based on the location of the third sound source; and acquiring sound wave signals through a third microphone, specifically including: acquiring sound wave signals through a third microphone, a first microphone, and a second microphone based on the third beam.

[0447] Thus, in a directional sound acquisition scenario, if the electronic device is not in a folded state, it can construct a third beam based on the direction of the third sound source, with the third beam pointing towards that direction. At this point, the electronic device uses multiple microphones to acquire sound signals based on the third beam, thereby enhancing the sound from the direction of the third sound source and suppressing sound from other directions.

[0448] In one possible implementation, when the device form of the electronic device is not folded, the method further includes: receiving a third operation to record video; in response to the third operation, determining a first sound source region; constructing a fourth beam based on the first sound source region; and acquiring sound wave signals through a third microphone, specifically including: acquiring sound wave signals based on the fourth beam through the third microphone, the first microphone, and the second microphone.

[0449] Thus, in a video recording scenario, if the electronic device is not in a folded state, it can construct a fourth beam based on the first sound source region, with the fourth beam pointing towards the first sound source region. At this time, the electronic device uses multiple microphones to collect sound wave signals based on the fourth beam, thereby amplifying the sound from the first sound source region and suppressing sound from other regions.

[0450] In one possible implementation, when the electronic device is in a folded state, the method further includes: receiving a third operation to record video; in response to a fourth operation, determining a second sound source region; constructing a fifth beam based on the second sound source region; and acquiring sound wave signals through a third microphone, specifically including: acquiring sound wave signals based on the fifth beam through the third microphone, the first microphone, and the second microphone.

[0451] Thus, in a video recording scenario, if the electronic device is in a folded state, it can construct a fifth beam based on the second sound source region, with the fifth beam pointing towards that region. At this time, the electronic device uses multiple microphones to collect sound wave signals based on the fifth beam, thereby amplifying the sound from the second sound source region and suppressing sound from other regions.

[0452] Figure 20 shows a flowchart of another sound acquisition method provided in an embodiment of this application.

[0453] As shown in Figure 20, the specific process of another sound acquisition method may include the following steps:

[0454] S2001. When the electronic device is in a folded state, the electronic device detects that the call mode of the electronic device has been switched to hands-free mode; the electronic device includes a display screen, a first microphone, a second microphone and a third microphone, wherein the first microphone is located at the top of the electronic device, the second microphone is located at the bottom of the electronic device and the third microphone is located on the side of the electronic device.

[0455] S2002. Electronic devices acquire sound wave signals through a third microphone.

[0456] In this way, when the electronic device is folded and in hands-free calling mode, the electronic device can collect sound wave signals through the side microphone, increasing the distance between the microphone and the speaker, reducing echo interference, and improving call quality.

[0457] In one possible implementation, the method further includes: detecting the way the electronic device is held; after detecting that the electronic device's call mode has switched to hands-free mode, outputting a first prompt based on the way the electronic device is held, the first prompt being used to prompt the user to rotate the electronic device so that the third microphone is close to the user's mouth.

[0458] Thus, when an electronic device is folded and in hands-free calling mode, because the speaker is typically located at the top or bottom, the close proximity of the top microphone and the top speaker, as well as the bottom microphone and the bottom speaker microphone, can cause the sound wave signal emitted by the speaker to interfere with the sound pickup of the top and bottom microphones. When the electronic device detects that it is in hands-free calling mode, it can use the side microphones to collect sound wave signals, reducing echo interference and improving the quality of sound pickup.

[0459] In one possible implementation, the electronic device further includes a fourth microphone, which is disposed on the side of the electronic device, and the third microphone and the fourth microphone are disposed on different sides of the electronic device; after detecting that the electronic device has switched its call mode to hands-free mode, the method further includes: obtaining the location of a first sound source; constructing a first beam based on the location of the first sound source; and acquiring sound wave signals through the third microphone, specifically including: acquiring sound wave signals through the third microphone and the fourth microphone based on the first beam.

[0460] In this way, in hands-free calling mode, the electronic device can also determine the location of the first sound source and construct a first beam based on the location of the first sound source, so as to enhance the sound emitted by the user, suppress the sound from other directions, and improve the quality of the call.

[0461] In one possible implementation, obtaining the location of the first sound source specifically includes: detecting the way the electronic device is held; and determining the location of the first sound source based on the way the electronic device is held.

[0462] In this way, the electronic device can determine the location of the first sound source based on how the electronic device is held.

[0463] In one possible implementation, obtaining the location of the first sound source specifically includes: acquiring location sound wave signals through a third microphone and a fourth microphone; and determining the location of the first sound source based on the location sound wave signals.

[0464] In this way, electronic devices can determine the location of the first sound source based on the positioning sound wave signal.

[0465] Figure 21 shows a flowchart of another sound acquisition method provided in an embodiment of this application.

[0466] As shown in Figure 21, the specific process of another sound acquisition method may include the following steps:

[0467] S2101. When the electronic device is in a folded state, the electronic device receives an operation to record spatial audio; the electronic device includes a display screen, a first microphone, a second microphone, a third microphone, a fourth microphone and a fifth microphone, wherein the first microphone is located at the top of the electronic device, the second microphone is located at the bottom of the electronic device, the third microphone and the fourth microphone are respectively located on the two sides of the electronic device, and the fifth microphone is located on the back of the electronic device.

[0468] It should be noted that steps S2102 to S2105 below are steps executed synchronously.

[0469] S2102. The electronic device acquires the first sound wave signal of the first time period through the second microphone.

[0470] S2103. The electronic device acquires the second acoustic signal of the first time period through the third microphone.

[0471] S2104. The electronic device acquires the third sound wave signal of the first time period through the fourth microphone.

[0472] S2105. The electronic device acquires the fourth acoustic signal of the first time period through the fifth microphone.

[0473] S2106. An electronic device generates a first audio file based on a first sound wave signal, a second sound wave signal, a third sound wave signal, and a fourth sound wave signal.

[0474] Thus, in a spatial audio recording scenario, if the electronic device is in a folded state, it can determine whether to use the second, third, fourth, and fifth microphones to record spatial audio. At this time, the electronic device can use these microphones to acquire sound wave signals and then generate a spatial audio file based on the acquired sound wave signals, thereby providing users with richer recording effects.

[0475] It is understood that the specific process of recording spatial audio for electronic devices in different device forms can also refer to the relevant content in the embodiments shown in Figures 8A-9 above, and will not be repeated here.

[0476] In one possible implementation, generating a first audio file based on a first sound wave signal, a second sound wave signal, a third sound wave signal, and a fourth sound wave signal specifically includes: determining an X-dimensional component based on the first and second sound wave signals; determining a Y-dimensional component based on the second and third sound wave signals; determining a Z-dimensional component based on the second and fourth sound wave signals; and generating the first audio file based on the X-dimensional component, the Y-dimensional component, and the Z-dimensional component.

[0477] In this way, in the folded state, the components of sound in each dimension can be determined based on the collected sound wave signal, thereby generating a spatial audio file.

[0478] In one possible implementation, the method further includes: receiving an operation to record spatial audio when the electronic device is in an unfolded state; acquiring a fifth sound wave signal for a second time period via a second microphone; acquiring a sixth sound wave signal for a second time period via a third microphone; acquiring a seventh sound wave signal for a second time period via a fourth microphone; acquiring an eighth sound wave signal for a second time period via a fifth microphone; and generating a second audio file based on the fifth, sixth, seventh, and eighth sound wave signals.

[0479] Thus, in a spatial audio recording scenario, if the electronic device is in its deployed state, it can determine whether to use the second, third, fourth, and fifth microphones to record spatial audio. At this time, the electronic device can use these microphones to acquire sound wave signals and then generate a spatial audio file based on the acquired sound wave signals, thereby providing users with richer recording effects.

[0480] In one possible implementation, a second audio file is generated based on the fifth, sixth, seventh, and eighth sound wave signals, specifically including: determining the X-dimensional component based on the fifth and sixth sound wave signals; determining the Y-dimensional component based on the sixth and seventh sound wave signals; determining the Z-dimensional component based on the sixth and eighth sound wave signals; and generating the second audio file based on the X-dimensional, Y-dimensional, and Z-dimensional components.

[0481] In this way, in the unfolded state, the components of sound in each dimension can be determined based on the collected sound wave signal, thereby generating a spatial audio file.

[0482] In one possible implementation, the method further includes: receiving an operation to record spatial audio when the electronic device is in a semi-folded state; acquiring a ninth sound wave signal of a second time period through a first microphone; acquiring a tenth sound wave signal of a second time period through a second microphone; acquiring an eleventh sound wave signal of a second time period through a third microphone; acquiring a twelfth sound wave signal of a second time period through a fourth microphone; and generating a third audio file based on the ninth, tenth, eleventh, and twelfth sound wave signals.

[0483] Thus, in a spatial audio recording scenario, if the electronic device is in a semi-folded state, it can determine whether to use the first, second, third, and fourth microphones to record spatial audio. At this time, the electronic device can use the first, second, third, and fourth microphones to collect sound wave signals, and then generate a spatial audio file based on the collected sound wave signals, thereby providing users with richer recording effects.

[0484] In one possible implementation, a third audio file is generated based on the ninth, tenth, eleventh, and twelfth sound wave signals, specifically including: determining the X-dimensional component based on the eleventh and twelfth sound wave signals; determining the Y-dimensional component based on the ninth and tenth sound wave signals; determining the Z-dimensional component based on the tenth and eleventh sound wave signals; and generating the third audio file based on the X-dimensional, Y-dimensional, and Z-dimensional components.

[0485] In this way, in the semi-folded state, the components of sound in each dimension can be determined based on the collected sound wave signal, thereby generating a spatial audio file.

[0486] The following describes a chip system provided by an embodiment of this application.

[0487] This application also provides a chip system including at least one processor for implementing the functions involved in the electronic device 100 in any of the above embodiments.

[0488] In one possible design, the chip system also includes a memory for storing program instructions and data, which may be located within or outside the processor.

[0489] The chip system can consist of chips or include chips and other discrete components.

[0490] Optionally, the chip system may contain one or more processors. These processors can be implemented in hardware or software. When implemented in hardware, the processor can be a logic circuit, an integrated circuit, etc. When implemented in software, the processor can be a general-purpose processor, implemented by reading software code stored in memory.

[0491] Optionally, the chip system may contain one or more memories. The memory may be integrated with the processor or disposed separately from it; this application embodiment does not limit this. For example, the memory may be a non-transient processor, such as a read-only memory (ROM), which may be integrated with the processor on the same chip or disposed separately on different chips. This application embodiment does not specifically limit the type of memory or the arrangement of the memory and processor.

[0492] For example, the chip system may be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a micro controller unit (MCU), a programmable logic device (PLD), or other integrated chips.

[0493] It is understood that the above chip system is only an example. In the embodiments of this application, the chip system may also include more, fewer, or different devices than those in the above embodiments. This application does not limit the scope of the invention.

[0494] The various embodiments of this application can be combined arbitrarily to achieve different technical effects.

[0495] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0496] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

[0497] In summary, the above description is merely an embodiment of the technical solution of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made based on the disclosure of this application should be included within the scope of protection of this application.

Claims

1. A sound acquisition method, applied to a foldable electronic device, characterized in that, The electronic device includes a display screen, a first microphone, a second microphone, and a third microphone, wherein the first microphone is located at the top of the electronic device, the second microphone is located at the bottom of the electronic device, and the third microphone is located on the side of the electronic device; the method includes: When the electronic device is in a folded state, it is detected that the electronic device has enabled the microphone function; Sound wave signals are acquired through the third microphone.

2. The method according to claim 1, characterized in that, Before acquiring the sound wave signal through the third microphone, the method further includes: The first microphone and / or the second microphone were detected to be blocked.

3. The method according to claim 2, characterized in that, The electronic device also includes a gravity sensor and / or a gyroscope sensor; The detection that the first microphone and / or the second microphone is blocked specifically includes: The gripping method of the electronic device is detected by the gravity sensor and / or gyroscope sensor; The first microphone and / or the second microphone is determined to be blocked based on how the electronic device is held.

4. The method according to claim 2 or 3, characterized in that, The electronic device includes a low-power camera; The detection that the first microphone and / or the second microphone is blocked specifically includes: The low-power camera detects how the electronic device is held. Based on how the electronic device is held, it is determined that the first microphone and the second microphone are blocked.

5. The method according to any one of claims 2-4, characterized in that, The detection that the first microphone and / or the second microphone is blocked specifically includes: The first test signal is acquired through the first microphone; Based on the first test signal, it is determined that the first microphone is blocked. And / or, The second test signal is acquired through the second microphone; Based on the second test signal, it is determined that the second microphone is blocked.

6. A sound acquisition method, applied to a foldable electronic device, characterized in that, The electronic device includes a display screen, a first microphone, a second microphone, and a third microphone, wherein the first microphone is located at the top of the electronic device, the second microphone is located at the bottom of the electronic device, and the third microphone is located on the side of the electronic device; the method includes: When the electronic device is in a folded state, it is detected that the call mode of the electronic device has switched to hands-free mode; Sound wave signals are acquired through the third microphone.

7. The method according to claim 6, characterized in that, The method further includes: Detect the way the electronic device is held; After detecting that the electronic device has switched to hands-free mode, a first prompt is output based on the way the electronic device is held. The first prompt is used to prompt the user to rotate the electronic device so that the third microphone is close to the user's mouth.

8. The method according to claim 6, characterized in that, The electronic device further includes a fourth microphone, which is disposed on the side of the electronic device, and the third microphone and the fourth microphone are disposed on different sides of the electronic device; After detecting that the electronic device has switched its call mode to hands-free mode, the method further includes: Obtain the location of the first sound source; A first beam is constructed based on the orientation of the first sound source; The acquisition of sound wave signals through the third microphone specifically includes: Sound wave signals are acquired using the first beam through the third and fourth microphones.

9. The method according to claim 8, characterized in that, The acquisition of the first sound source location specifically includes: Detect the way the electronic device is held; The location of the first sound source is determined based on how the electronic device is held.

10. The method according to claim 8, characterized in that, The acquisition of the first sound source location specifically includes: Positioning sound wave signals are acquired through the third and fourth microphones; The location of the first sound source is determined based on the positioning acoustic signal.

11. The method according to any one of claims 1-5, characterized in that, The detection that the electronic device has enabled the microphone function specifically includes: The electronic device was detected making or receiving a phone call. or, The electronic device was detected to have started recording.

12. The method according to any one of claims 1-5, characterized in that, The electronic device further includes a fourth microphone, which is disposed on the side of the electronic device, and the third microphone and the fourth microphone are disposed on different sides of the electronic device; When the electronic device is in a folded state, detecting that the electronic device has enabled the microphone function specifically includes: The first operation to enable directional sound acquisition function has been received; The method further includes: In response to the first operation, the location of the second sound source is determined; A second beam is constructed based on the orientation of the second sound source; The acquisition of sound wave signals through the third microphone specifically includes: Sound wave signals are acquired using the second beam through the third and fourth microphones.

13. The method according to claim 12, characterized in that, The step of determining the location of the second sound source in response to the first operation specifically includes: The location of the second sound source is determined based on the first operation; or, Positioning sound wave signals are acquired through the third and fourth microphones; The location of the second sound source is determined based on the positioning acoustic signal.

14. The method according to any one of claims 1-5, 12-13, characterized in that, When the electronic device is not in a folded state, the method further includes: The second operation to enable directional sound acquisition function has been received; In response to the second operation, the location of the third sound source is determined; A third beam is constructed based on the location of the third sound source; The acquisition of sound wave signals through the third microphone specifically includes: Sound wave signals are acquired using the third beam through the third microphone, the first microphone, and the second microphone.

15. The method according to any one of claims 1-5, characterized in that, When the electronic device is not in a folded state, the method further includes: The third operation to record video has been received; In response to the third operation, the first sound source region is determined; A fourth beam is constructed based on the first sound source region; The acquisition of sound wave signals through the third microphone specifically includes: Based on the fourth beam, sound wave signals are collected through the third microphone, the first microphone, and the second microphone.

16. A sound acquisition method, applied to a foldable electronic device, characterized in that, The electronic device includes a display screen, a first microphone, a second microphone, a third microphone, a fourth microphone, and a fifth microphone. The first microphone is located at the top of the electronic device, the second microphone is located at the bottom of the electronic device, the third and fourth microphones are respectively located on two sides of the electronic device, and the fifth microphone is located on the back of the electronic device. The method includes: When the electronic device is in a folded state, an operation to record spatial audio is received. The first sound wave signal of the first time period is acquired through the second microphone; The second sound wave signal of the first time period is acquired through the third microphone; The third sound wave signal of the first time period is acquired through the fourth microphone; The fourth sound wave signal of the first time period is acquired through the fifth microphone; A first audio file is generated based on the first sound wave signal, the second sound wave signal, the third sound wave signal, and the fourth sound wave signal.

17. The method according to claim 16, characterized in that, The method further includes: When the electronic device is in the unfolded state, an operation to record spatial audio is received. The fifth sound wave signal of the second time period was acquired through the second microphone; The sixth sound wave signal of the second time period is acquired through the third microphone; The seventh sound wave signal of the second time period is acquired through the fourth microphone; The eighth sound wave signal of the second time period is acquired through the fifth microphone; A second audio file is generated based on the fifth, sixth, seventh, and eighth sound wave signals.

18. The method according to claim 16 or 17, characterized in that, The method further includes: When the electronic device is in a semi-folded state, an operation to record spatial audio is received. The ninth sound wave signal of the second time period was acquired through the first microphone; The tenth sound wave signal of the second time period is acquired through the second microphone; The eleventh sound wave signal of the second time period was acquired through the third microphone; The twelfth sound wave signal of the second time period was acquired through the fourth microphone; A third audio file is generated based on the ninth, tenth, eleventh, and twelfth sound wave signals.

19. An electronic device, characterized in that, The device includes one or more processors, one or more memories, a foldable display screen, a first microphone, a second microphone, and a third microphone; wherein the one or more memories are coupled to the one or more processors, the one or more memories being used to store computer instructions that, when the one or more processors execute the computer instructions, implement the sound acquisition method according to any one of claims 1-18.

20. A chip system, characterized in that, It includes a processing circuit and an interface circuit, wherein the interface circuit is used to receive code instructions and transmit them to the processing circuit, and the processing circuit is used to execute the code instructions to perform the sound acquisition method according to any one of claims 1-18.

21. A readable storage medium, characterized in that, The device stores computer instructions that, when executed by a processor, implement the sound acquisition method according to any one of claims 1-18.

22. A computer program product, characterized in that, It includes computer instructions, which, when executed by a processor, implement the sound acquisition method according to any one of claims 1-18.

Images