Sound leakage cancellation method, device and storage medium

Abstract

The application provides a sound leakage cancellation method and device and a storage medium, and relates to the technical field of terminals.In the scheme, the electronic device can at least include a first sound generating device and a second sound generating device, the second sound generating device is mainly used for sound generation to the ear area of a user, and the first sound generating device is mainly used for sound leakage cancellation of the second sound generating device in a sound leakage area.In the process in which the user uses the electronic device to listen to voice content, if the sound hole is blocked, the impedance data of the first sound generating device will change, the electronic device can collect the impedance data of the first sound generating device in real time, and then adjust the audio optimization parameter corresponding to the first sound generating device according to the impedance data of the first sound generating device.Through the adjustment of the audio optimization parameter corresponding to the first sound generating device, the audio signal output by the first sound generating device can be adjusted, so that the cancellation effect of the sound waves of the two sound generating devices in the sound leakage area is optimized, the sound leakage risk is reduced, and the use experience of the user is improved.

Description

Technical Field

[0001] This application relates to the field of audio technology, and in particular to a method, device and storage medium for canceling sound leakage. Background Technology

[0002] Currently, voice communication has become an important function of mobile phones and other electronic devices. In the process of a mobile phone interacting with other devices to provide voice communication, the phone converts audio electrical signals from other devices into sound signals and outputs these signals through a sound-generating device. However, the sound-generating device is usually located at the edge of the phone, which may lead to sound leakage, causing voice communication content from other devices to be heard by people nearby, thus compromising user privacy. Summary of the Invention

[0003] This application provides a method, apparatus, and storage medium for canceling sound leakage, used to solve the sound leakage problem of electronic devices.

[0004] To achieve the above objectives, this application adopts the following technical solution:

[0005] Firstly, embodiments of this application provide a method for canceling sound leakage. This method can be applied to an electronic device, which may include at least a first sound-emitting device and a second sound-emitting device. The second sound-emitting device primarily emits sound into the user's inner ear region, and the first sound-emitting device is primarily used to cancel the sound leakage from the second sound-emitting device in the leakage area. The first sound-emitting device includes at least one sound outlet; for example, the first sound-emitting device may be a receiver. The method may include:

[0006] A first audio signal is output through a first sound-emitting device, and a second audio signal is output through a second sound-emitting device. During the process of outputting the first audio signal through the first sound-emitting device, the impedance data of the first sound-emitting device is acquired. The impedance data of the first sound-emitting device is input into an artificial intelligence model to obtain first audio optimization parameters. The audio optimization parameters of the first filter corresponding to the first sound-emitting device are updated to the first audio optimization parameters to obtain a third audio signal. The third audio signal is output through the first sound-emitting device, and the second audio signal is output through the second sound-emitting device.

[0007] In the above solution, if the sound outlet is blocked while the user is listening to voice content on the electronic device, the impedance data of the first sound-emitting device will change. The electronic device can collect the impedance data of the first sound-emitting device in real time, and then adjust the corresponding audio optimization parameters of the first sound-emitting device based on the impedance data. By adjusting the corresponding audio optimization parameters of the first sound-emitting device, the audio signal output by the first sound-emitting device can be adjusted, thereby optimizing the cancellation effect of the sound waves of the two sound-emitting devices in the sound leakage area, reducing the risk of sound leakage, preventing the leakage of user privacy information, improving user privacy and security, and enhancing user experience.

[0008] In one possible implementation, the first audio signal is the audio signal that was not adjusted when the first sound-emitting device experienced a blockage. The sound leakage cancellation depth after the first and second audio signals are superimposed in the sound leakage region is the first sound leakage cancellation depth. The sound leakage cancellation depth after the third and second audio signals are superimposed in the sound leakage region is the second sound leakage cancellation depth, which is less than the first sound leakage cancellation depth.

[0009] It should be noted that the sound leakage cancellation depth, also known as sound leakage cancellation capability or sound leakage cancellation effect, describes the extent to which an electronic device cancels sound leakage in a leaky area when the sound leakage cancellation function is enabled. It is usually measured in dB. For example, if the sound leakage cancellation depth is 0dB, it means that it has no effect on the sound in the leaky area; if the sound leakage cancellation depth is -30dB, it means that it can reduce the sound leakage in the leaky area by 30dB; if the sound leakage cancellation depth is negative infinity, it means that the sound leakage cancellation effect is the strongest; if the sound leakage cancellation depth is positive, it means that not only does it not cancel the sound leakage, but it also increases the sound leakage.

[0010] In the above scheme, when the first sound-generating device becomes blocked, the sound leakage cancellation effect of the electronic device deteriorates significantly. However, by adjusting the audio signal output by the first sound-generating device, the sound leakage cancellation effect can be improved to some extent.

[0011] In one possible implementation, the method may further include: acquiring impedance data of the second sound-emitting device during the output of the second audio signal through the second sound-emitting device; inputting the impedance data of the second sound-emitting device into an artificial intelligence model to obtain second audio optimization parameters; updating the audio optimization parameters of the second filter corresponding to the second sound-emitting device to the second audio optimization parameters to obtain a fourth audio signal; outputting a third audio signal through the first sound-emitting device, and continuing to output a fourth audio signal through the second sound-emitting device; wherein the sound leakage cancellation depth after the third audio signal and the fourth audio signal are superimposed in the sound leakage region is the third sound leakage cancellation depth, and the third sound leakage cancellation depth is less than the second sound leakage cancellation depth.

[0012] In the above scheme, the electronic device may include at least a first sound-emitting device and a second sound-emitting device. The second sound-emitting device is mainly used to emit sound into the user's inner ear region, while the first sound-emitting device is mainly used to cancel the sound leakage of the second sound-emitting device in the sound leakage area. Each of the first and second sound-emitting devices may include at least one sound outlet. During the user's use of the electronic device to listen to voice content, the electronic device can collect the impedance data of the first and second sound-emitting devices in real time. Then, it adjusts the audio optimization parameters of the first and second sound-emitting devices based on their impedance data. By adjusting the audio optimization parameters of the first and second sound-emitting devices, the cancellation effect of the sound waves from the two sound-emitting devices in the sound leakage area can be further optimized, thereby further reducing sound leakage.

[0013] In one possible implementation, during the output of the first audio signal through the first sound-emitting device, the impedance data of the first sound-emitting device is the first impedance data. Accordingly, the method may further include: during the output of the third audio signal through the first sound-emitting device, acquiring the second impedance data of the first sound-emitting device; if the second impedance data and the first impedance data satisfy a preset condition, inputting the second impedance data into an artificial intelligence model to obtain third audio optimization parameters; updating the audio optimization parameters of the first filter to the third audio optimization parameters to obtain a fifth audio signal; outputting the fifth audio signal through the first sound-emitting device, and continuing to output the second audio signal through the second sound-emitting device, wherein the sound leakage cancellation depth after the fifth audio signal and the second audio signal are superimposed in the sound leakage region is a fourth sound leakage cancellation depth, and the fourth sound leakage cancellation depth is less than the first sound leakage cancellation depth. Furthermore, after acquiring the second impedance data of the first sound-emitting device, the method may further include: if the second impedance data and the first impedance data do not meet a preset condition, continuing to output a third audio signal through the first sound-emitting device; during the process of outputting the third audio signal through the first sound-emitting device, continuing to acquire the impedance data of the first sound-emitting device until the impedance data of the first sound-emitting device acquired in the latest acquisition meets the preset condition with the impedance data of the first sound-emitting device acquired in the previous acquisition.

[0014] For example, the above preset conditions may include at least one:

[0015] (1) The difference between the frequency of the first resonance peak in the impedance data of the first sound-emitting device acquired in the latest acquisition and the frequency of the first resonance peak in the impedance data of the first sound-emitting device acquired in the previous acquisition is greater than or equal to the first threshold.

[0016] (2) The difference between the impedance of the first resonance peak in the latest acquired impedance data of the first sound-emitting device and the impedance of the first resonance peak in the previous acquired impedance data of the first sound-emitting device is greater than or equal to the second threshold.

[0017] (3) The difference between the frequency of the second resonance peak in the impedance data of the first sound-emitting device acquired in the latest acquisition and the frequency of the second resonance peak in the impedance data of the first sound-emitting device acquired in the previous acquisition is greater than or equal to the third threshold.

[0018] (4) The difference between the impedance of the second resonance peak in the latest acquired impedance data of the first sound-emitting device and the impedance of the second resonance peak in the previous acquired impedance data of the first sound-emitting device is greater than or equal to the fourth threshold.

[0019] In the above scheme, the electronic device can detect the impedance data of the sound-generating device in real time and determine whether the difference between this impedance data and the previously detected impedance data exceeds a preset threshold. Only when the preset threshold is exceeded will the audio optimization parameter adjustment mechanism be activated. In this way, the power consumption of the electronic device can be saved while ensuring that the change in the cancellation effect is not significant.

[0020] In one possible implementation, the impedance data of the first sound-emitting device may include: the impedance and frequency of the first resonance peak, and the impedance and frequency of the second resonance peak. Accordingly, acquiring the impedance data of the first sound-emitting device during the output of the first audio signal through the first sound-emitting device may include: acquiring the voltage and current values ​​of the first sound-emitting device during the output of the first audio signal through the first sound-emitting device; and determining the impedance and frequency of the first resonance peak, and the impedance and frequency of the second resonance peak, based on the voltage and current values ​​of the first sound-emitting device.

[0021] In the above scheme, under different hole-clogging ratios, the first and second resonance peaks in the impedance curve of the first sound-generating device exhibit the following variation: as the hole-clogging ratio increases, the first resonance peak decreases significantly, and its frequency shifts to lower frequencies; as the hole-clogging ratio increases, the second resonance peak increases significantly, and its frequency also shifts to lower frequencies. In other words, the first and second resonance peaks are closely related to the hole-clogging ratio of the first sound-generating device. Therefore, the impedance data of the first sound-generating device can at least include: the impedance and frequency of the first resonance peak in the impedance curve of the first sound-generating device, and the impedance and frequency of the second resonance peak in the impedance curve of the first sound-generating device. This facilitates establishing a relationship with the transfer function under the same hole-clogging ratio based on the impedance and frequency of the first and / or second resonance peaks, and then predicting the audio optimization parameters for optimizing the first filter corresponding to the first sound-generating device based on the transfer function.

[0022] In one possible implementation, before inputting the impedance data of the first sound-emitting device into the artificial intelligence model, the method may further include: acquiring the temperature of the first sound-emitting device during the output of the first audio signal through the first sound-emitting device; and correcting the impedance data of the first sound-emitting device based on the temperature of the first sound-emitting device.

[0023] In the above scheme, the electronic device can detect the impedance data of the sound-generating device in real time and determine whether the difference between this impedance data and the previously detected impedance data exceeds a preset threshold. Only when the preset threshold is exceeded will the audio optimization parameter adjustment mechanism be activated. In this way, the power consumption of the electronic device can be saved while ensuring that the change in the cancellation effect is not significant.

[0024] In one possible implementation, the aforementioned artificial intelligence model is obtained by training multiple impedance data points and corresponding audio optimization parameters. The target impedance data refers to the impedance data of the first sound-emitting device collected when the electronic device is in the target holding posture. The audio optimization parameters corresponding to the target impedance data are calculated based on a target transfer function, which is obtained after testing the acoustic energy of the leakage area when the electronic device is in the target holding posture. The target impedance data is any set of data from the multiple impedance data points.

[0025] For example, if the above-mentioned artificial intelligence model is trained by an electronic device, the method may further include: acquiring target impedance data and audio optimization parameters corresponding to the target impedance data when the electronic device is in a target holding posture; after acquiring multiple impedance data and audio optimization parameters corresponding to the multiple impedance data, training the multiple impedance data and audio optimization parameters corresponding to the multiple impedance data to obtain an artificial intelligence model.

[0026] In the above scheme, by establishing an AI model, the audio optimization parameters corresponding to the impedance data can be obtained based solely on the current impedance data during the user's use, without the need to use a comprehensive tester to test the sound energy of multiple sampling points in the sound leakage area under the current holding posture, thereby improving the feasibility of the sound leakage cancellation method provided in this application.

[0027] Secondly, this application provides an apparatus comprising units for performing the method described in the first aspect above. This apparatus corresponds to performing the sound leakage cancellation method described in the first aspect above. For a detailed description of the units within this apparatus, please refer to the description in the first aspect above; for brevity, it will not be repeated here.

[0028] Thirdly, this application provides an electronic device that includes at least a first sound-generating device, a second sound-generating device, one or more processors, and a memory. The memory is coupled to one or more processors and is used to store computer program code, which includes computer instructions. The one or more processors invoke the computer instructions to cause the electronic device to perform the sound leakage cancellation method provided by the first aspect and any possible implementation thereof.

[0029] Fourthly, this application provides a computer-readable storage medium. The computer-readable storage medium includes instructions that, when executed on an electronic device, cause the electronic device to perform the sound leakage cancellation method provided by the first aspect and any possible implementation thereof.

[0030] Fifthly, this application provides a computer program product. When the computer program product is run on a computer, it causes the computer to perform the sound leakage cancellation method provided by the first aspect and any possible implementation thereof.

[0031] Sixthly, this application provides a chip system. The chip system is applied to an electronic device and includes one or more processors. These processors invoke computer instructions to cause the electronic device to perform the sound leakage cancellation method provided by the first aspect and any possible implementation thereof.

[0032] It is understood that the beneficial effects achieved by the apparatus of the second aspect, the electronic device of the third aspect, the computer-readable storage medium of the fourth aspect, the computer program product of the fifth aspect, and the chip of the sixth aspect can be referred to the beneficial effects of the first aspect and any possible implementation thereof, which will not be repeated here. Attached Figure Description

[0033] Figure 1 A schematic diagram illustrating a scenario where a user uses a mobile phone to answer voice messages, provided as an embodiment of this application;

[0034] Figure 2 This is a schematic diagram illustrating another scenario where a user uses a mobile phone to answer voice messages, provided as an embodiment of this application.

[0035] Figure 3 This is a schematic diagram illustrating another scenario where a user uses a mobile phone to answer voice messages, provided as an embodiment of this application.

[0036] Figure 4 This is a schematic diagram illustrating another scenario where a user uses a mobile phone to answer voice messages, provided as an embodiment of this application.

[0037] Figure 5 A schematic diagram of the hardware structure of a mobile phone provided in an embodiment of this application;

[0038] Figure 6 A schematic diagram of the architecture of an electronic device provided in an embodiment of this application;

[0039] Figure 7 A schematic diagram of the transfer function corresponding to the sound leakage region under different plugging rates, provided in an embodiment of this application;

[0040] Figure 8 A front view showing the effect of phase difference and amplitude difference on the cancellation capability provided in an embodiment of this application;

[0041] Figure 9 The embodiments provided in this application are related to Figure 8 The corresponding left view;

[0042] Figure 10 The embodiments provided in this application are related to Figure 8 The corresponding top view;

[0043] Figure 11 A schematic diagram of the impedance curves of the sound-generating device under different plugging rates provided in the embodiments of this application;

[0044] Figure 12 A flowchart illustrating a sound leakage cancellation method provided in an embodiment of this application;

[0045] Figure 13 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0046] Figure 14 This is a schematic diagram of the cancellation effect curves in different scenarios provided in the embodiments of this application;

[0047] Figure 15 A flowchart illustrating another sound leakage cancellation method provided in an embodiment of this application;

[0048] Figure 16 This is a schematic diagram of the structure of another electronic device provided in an embodiment of this application;

[0049] Figure 17 A flowchart illustrating another sound leakage cancellation method provided in an embodiment of this application;

[0050] Figure 18 A schematic diagram of the resonance peaks in the impedance curves obtained in two separate acquisitions provided in this application embodiment;

[0051] Figure 19 This is a schematic diagram illustrating the creation of an AI model as provided in an embodiment of this application. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of this application, but not all embodiments.

[0053] In the description of this application, unless otherwise stated, " / " means "or". For example, A / B can mean A or B. In the description of this application, "and / or" is merely a way of describing the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone.

[0054] In this application, the terms "first" and "second," etc., are used to distinguish different objects or to distinguish different processing of the same object, rather than to describe a specific order of objects. For example, "first operation" and "second operation," etc., are used to distinguish different operations, rather than to describe a specific order of operations. In the embodiments of this application, "multiple" refers to two or more.

[0055] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0056] References to "some embodiments" and the like in this specification mean that one or more embodiments of this application include the specific features, structures, or characteristics described in connection with that embodiment. Therefore, phrases such as "in some embodiments," "in other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiments, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0057] Currently, more and more electronic devices have voice communication functions, such as voice call functions based on telephone applications, audio and video communication functions based on conferencing applications, and voice messaging functions based on instant messaging applications.

[0058] Taking voice call functionality based on a telephone application as an example, during the real-time interaction of voice information between the mobile phone and other devices, the phone can convert audio electrical signals from other devices into sound signals and output these sound signals through a sound-generating device located on the phone. Common sound-generating devices can include the following categories: The first category is the receiver, also known as the "earpiece," which is usually located at the top of the phone. When answering calls or voice messages, the user can bring the receiver close to their ear to hear the voice. The second category is the speaker, also known as the "horn," which is usually located at the bottom of the phone. Users can listen to music or hands-free calls through the speaker. The third category is the screen sound device, which is usually located below the phone's screen and uses the mechanical vibration of an exciter to drive the screen to produce sound. It should be understood that mobile phones can also use other types of sound-generating devices. Regardless of the type of sound-generating device used, these devices can all convert audio electrical signals into sound signals.

[0059] However, when users use their mobile phones to listen to voice content, there is a certain distance between the human ear and the sound-producing device, and the sound from the sound-producing device will inevitably spread to the surroundings.

[0060] For example, such as Figure 1 As shown, a receiver is located at the top of the mobile phone, and the receiver includes multiple sound outlets used to play sound to the outside of the phone. Experiments have shown that the sound propagation path from the sound outlets is multi-directional, forming a sound field. This sound field can be divided into the intra-ear sound field and the leakage sound field. The intra-ear sound field, also known as the ear canal sound field or target sound field, refers to the sound field within the user's ear canal and its surrounding area within a predetermined range. The leakage sound field, also known as the extra-ear sound field, refers to the sound field outside the intra-ear sound field (away from the target audio playback device). The sound in the intra-ear sound field is the sound that the user needs to hear, i.e., the sound that the user is interested in; that is, the speech intelligibility of the intra-ear sound field should be as high as possible. The sound in the leakage sound field is leakage sound, which is the sound that the user does not want to be heard by those around them; that is, the speech intelligibility of the leakage sound field should be as low as possible. This application divides the area around the mobile phone into an intra-ear region (also known as the first spatial region) and a leakage region (also known as the second spatial region). The inner ear region corresponds to the sound field within the ear. The leakage region corresponds to the sound field that leaks sound. When multiple sampling points are taken between the ear and the sound outlet, the sound emitted from the sound outlet can travel to the ear after passing through the sampling points in the ear region. However, the sound emitted from the sound outlet cannot travel to the ear after passing through the sampling points in the leakage region and instead diffuses into the surrounding area of ​​the phone. This results in the voice content being heard by people nearby, thus leaking the user's privacy information.

[0061] To prevent the leakage of personal privacy information, a related technology provides a sound leakage cancellation scheme based on multiple sound-emitting devices. In this scheme, the electronic device includes at least two sound-emitting devices, such as a first sound-emitting device and a second sound-emitting device. The second sound-emitting device mainly emits sound into the inner ear area, allowing the user to hear the sound, but some sound from the second sound-emitting device still diffuses into the sound leakage area; the sound emitted by the first sound-emitting device and the sound diffused by the second sound-emitting device are superimposed in the sound leakage area, thereby reducing sound leakage.

[0062] As an example, the first sound-generating device and the second sound-generating device can be different types of sound-generating devices.

[0063] For example, such as Figure 2 As shown in (a), a candybar phone can include two sound-emitting devices: a receiver located at the top of the phone and a screen sound-emitting device located below the screen. Through algorithm design, when a user holds the candybar phone to make a call, sound wave 1 emitted by the screen sound-emitting device mainly propagates towards the inner ear area, allowing the user to hear the voice content. Sound wave 2 emitted by the screen sound-emitting device and the sound wave emitted by the receiver superimpose in the sound leakage area (e.g., sound leakage point 1). Figure 2 As shown in (b), the sound wave 2 emitted by the screen sound device and the sound wave emitted by the receiver have the same amplitude and opposite phase. In this way, the sound wave 2 emitted by the screen sound device and the sound wave emitted by the receiver can cancel each other out, thereby effectively eliminating the sound leakage of the candybar phone.

[0064] As another example, the first sound-producing device and the second sound-producing device can be the same type of sound-producing device.

[0065] For example, such as Figure 3 As shown in (a), a foldable phone can include two sound-emitting devices: receiver 1 located on the top bezel of the main screen, and receiver 2 located on the top bezel of the back screen. Through algorithm design, when a user holds the foldable phone to make a call, sound waves 1 emitted by receiver 1 and 2 primarily propagate towards the inner ear, allowing the user to hear the call. Sound waves 2 emitted by receiver 1 and 2 overlap in the sound leakage area (e.g., sound leakage point 1). Figure 3 As shown in (b), the sound wave 2 emitted by receiver 1 and the sound wave 2 emitted by receiver 2 have the same amplitude and opposite phase. In this way, the sound wave 2 emitted by receiver 1 and the sound wave 2 emitted by receiver 2 can cancel each other out, thereby effectively eliminating the sound leakage of the foldable screen phone.

[0066] It should be noted that the algorithm design in the two examples above is based on a standard grip posture. A standard grip posture can be interpreted as the angle of the electronic device being held. The standard grip angle is the angle at which a user typically holds the electronic device when making a call. This angle can be the angle between the electronic device and the user's face when making a call. Since the contrast between the sound energy in the inner ear region and the sound energy in the leakage region is at its maximum under the standard grip posture, when a user uses the electronic device for a call with the standard grip posture, the sound energy in the leakage region of the sound-emitting device is reduced, while the sound energy in the inner ear region remains unchanged.

[0067] in addition, Figure 2 and Figure 3 The sound-generating devices described are merely illustrative and do not limit the scope of this application. In actual implementation, the types of the first and second sound-generating devices, and their specific placement on the electronic device, can be adjusted according to actual usage requirements. For example, a candybar phone may include three sound-generating devices: receiver 1 and receiver 2 located on the upper bezel, and a speaker located on the lower bezel. Receiver 1 is mainly used to emit sound into the ear area, receiver 2 is mainly used to cancel the sound leaked from receiver 1 into the sound leakage area, and the speaker is used to play audio in external playback mode.

[0068] In the aforementioned sound leakage cancellation scheme based on multiple sound-emitting devices, the transmission paths of both sound-emitting devices are fixed from the initial design stage. Researchers can use a comprehensive testing instrument to collect sound leakage data from the leakage area when a user holds the phone in a standard grip posture, and design algorithms to cancel the sounds from the two sound-emitting devices in the leakage area, thus reducing sound leakage. However, each user's grip posture and position on the phone may differ. For example, some users' faces, ears, or hands may come into contact with the sound outlet of a certain sound-emitting device, causing changes in the frequency response or sound field of that device. In this case, the cancellation effect based on the previously designed algorithm deteriorates, not only failing to effectively eliminate sound leakage but potentially even exacerbating it.

[0069] For example, such as Figure 4 As shown in (a), a mobile phone may include two sound-emitting devices: a receiver located at the top of the phone and a screen-emitting device located below the screen. When a user answers a call, the user may, based on personal habit, place their ear against the top of the phone (i.e., the user is not using a standard holding posture), thus blocking one or more sound outlets of the receiver, thereby changing the receiver's frequency response or sound field. Figure 4As shown in (b), the phase and amplitude of the sound wave emitted by the screen sound device remain unchanged, while the phase of the sound wave emitted by the receiver remains unchanged but the amplitude is halved. Thus, although the phase of the sound wave 2 emitted by receiver 1 is opposite to that of the sound wave emitted by receiver 2, the amplitudes of the sound wave 2 emitted by receiver 1 and the sound wave emitted by receiver 2 are different. Therefore, the two sound waves cannot completely cancel each other out, resulting in a worse sound leakage elimination effect.

[0070] In a sound leakage cancellation scheme based on multiple sound-emitting devices, the electronic device includes at least two sound-emitting devices: a second sound-emitting device primarily emits sound into the inner ear region, and a first sound-emitting device is mainly used to cancel the sound leakage of the second sound-emitting device in the leakage area. The first and / or second sound-emitting devices may or may not have sound outlets. For sound-emitting devices that output sound through sound outlets, when a user uses the electronic device to receive voice calls, audio communication content, or voice messages, in any of the following scenarios, if the user's body parts block the sound outlet of the sound-emitting device, it will cause a change in the sound field emitted by the sound-emitting device.

[0071] In the first scenario, when the sound outlet of the second sound-generating device is not blocked, but the sound outlet of the first sound-generating device is blocked to a certain extent, the sound field in the inner ear region remains basically unchanged. However, due to the blockage of the sound outlet of the first sound-generating device, the phase and / or amplitude of the sound wave of the first sound-generating device changes, causing the sound field of the first sound-generating device in the sound leakage area to change, thereby reducing the sound leakage elimination effect, or even aggravating the sound leakage effect.

[0072] The second scenario is that when the sound outlets of both the second and first sound-generating devices are blocked to a certain extent, the change in the sound field in the ear region is small and has little impact on the user's listening experience. However, due to the blockage of the sound outlets of the second and first sound-generating devices, the phase and / or amplitude of the leakage sound wave from the second sound-generating device and the canceling sound wave from the first sound-generating device change. This causes the sound field of the first and second sound-generating devices in the leakage region to change, thereby reducing the effectiveness of eliminating leakage and potentially even aggravating it.

[0073] The third scenario involves a situation where the sound outlet of the first sound-emitting device is not blocked, but the sound outlet of the second sound-emitting device is blocked. This results in a significant change in the sound field within the ear, while the sound field changes less in the leakage area between the first and second sound-emitting devices. Since users can clearly perceive a reduction in sound within their ear, such as difficulty hearing conversations, they can adjust their grip on the electronic device to better hear the conversation.

[0074] In the first and second scenarios mentioned above, since users cannot perceive the deterioration in sound leakage from electronic devices, user privacy may be compromised. Research has found that when users use electronic devices to answer voice calls, audio communications, or voice messages, if their body parts block the sound outlet of the sound-emitting device, the impedance data of the device changes. Therefore, to address the problem of deteriorated sound leakage cancellation caused by blocking the sound outlet of the sound-emitting device, this application provides an improved sound leakage cancellation scheme: during the user's use of the electronic device to answer voice content, the electronic device can collect the impedance data of the sound-emitting device in real time, and then adjust the audio optimization parameters of the sound-emitting device based on the impedance data. By adjusting the audio optimization parameters of the sound-emitting device, the cancellation effect of the sound waves from the two sound-emitting devices in the sound leakage area can be optimized, thereby reducing the risk of sound leakage and improving the user experience.

[0075] The aforementioned electronic devices are also referred to as terminals or user equipment (UE). For example, electronic devices can be personal computers (PCs), mobile phones, tablets, wearable devices, computers with wireless transceiver capabilities, virtual reality (VR) devices, augmented reality (AR) devices, wireless terminals in smart cities or smart homes, etc., or they can be other devices or apparatuses with audio output capabilities.

[0076] The following is combined with Figures 5 to 19 An example is provided to illustrate the improved sound leakage cancellation scheme provided in this application.

[0077] Taking mobile phones as an example, Figure 5 This is a schematic diagram of the hardware structure of a mobile phone provided in an embodiment of this application.

[0078] like Figure 5As shown, the mobile phone 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0079] Processor 110 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc.

[0080] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0081] The charging management module 140 is used to receive charging input from the charger.

[0082] The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to power the processor 110, internal memory 121, external memory, display 194, camera 193, and wireless communication module 160, etc.

[0083] The wireless communication function of mobile phone 100 can be implemented through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor, and baseband processor. Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Mobile communication module 150 can provide solutions for wireless communication applications including 2G / 3G / 4G / 5G on mobile phone 100. Mobile communication module 150 may include at least one filter, switch, power amplifier, low-noise amplifier (LNA), etc. Wireless communication module 160 may be one or more devices integrating at least one communication processing module. Wireless communication module 160 receives electromagnetic waves via antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signal to processor 110. Wireless communication module 160 can also receive signals to be transmitted from processor 110, frequency modulate and amplify them, and convert them into electromagnetic waves for radiation via antenna 2.

[0084] The mobile phone 100 uses a GPU, a display screen 194, and an application processor to achieve its display function.

[0085] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel.

[0086] The mobile phone 100 can achieve shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.

[0087] The external storage interface 120 can be used to connect an external storage card, such as a Micro SD card, to expand the storage capacity of the mobile phone 100. The external storage card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, music, video, and other files can be saved on the external storage card.

[0088] Internal memory 121 can be used to store computer executable program code, which includes instructions. Processor 110 executes various functional applications and data processing of mobile phone 100 by running the instructions stored in internal memory 121. Internal memory 121 may include a program storage area and a data storage area.

[0089] Mobile phone 100 can implement audio functions through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor, such as music playback and recording. The audio module 170 is used to convert digital audio information into analog audio signals for output, and also to convert analog audio input into digital audio signals. The audio module 170 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 170 can be located in processor 110, or some functional modules of the audio module 170 can be located in processor 110.

[0090] The speaker 170A, also known as a "loudspeaker," is used to convert audio electrical signals into sound signals. Mobile phone 100 can listen to music or make hands-free calls through the speaker 170A.

[0091] The receiver 170B, also known as the "earpiece," is used to convert audio electrical signals into sound signals. When the mobile phone 100 answers a call or voice message, the receiver 170B can be brought close to the user's ear to listen to the voice.

[0092] Microphone 170C, also known as a "microphone" or "voice transducer," is used to convert sound signals into electrical signals. When making a phone call or sending a voice message, the user can speak by bringing their mouth close to microphone 170C, inputting the sound signal into microphone 170C. Mobile phone 100 may have at least one microphone 170C. In some embodiments, mobile phone 100 may have two microphones 170C, which, in addition to collecting sound signals, can also implement functions such as sound leakage cancellation and noise reduction.

[0093] The 170D headphone jack is used to connect wired headphones.

[0094] In the embodiments of this application, the electronic device is provided with at least two sound-emitting devices, and at least one sound-emitting device has a sound outlet.

[0095] Keypad 190 includes a power button, volume buttons, etc. Mobile phone 100 can receive keypad input and generate key signal inputs related to user settings and function control of mobile phone 100.

[0096] Motor 191 can generate vibration alerts.

[0097] Indicator 192 can be an indicator light, used to indicate charging status, power changes, or to indicate messages.

[0098] The SIM card interface 195 is used to connect the SIM card.

[0099] Understandable, Figure 5 The illustrated structure does not constitute a specific limitation on the mobile phone 100. In other embodiments, the mobile phone 100 may include... Figure 5 The diagram shows more or fewer components, or combinations of components, or separate components, or different arrangements of components. The components shown can be implemented in hardware, software, or a combination of both.

[0100] The software system of the aforementioned electronic device can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This embodiment of the invention uses the layered architecture of the Android system as an example to exemplify the architecture of the electronic device. Figure 6 This is a schematic diagram of the architecture of an electronic device provided in an embodiment of this application.

[0101] like Figure 6 As shown, electronic devices can adopt a layered architecture, dividing the software into several layers, each with a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments, the software layers of the software architecture are divided from top to bottom as follows: application (APP) layer, application framework (FWK) layer, hardware abstraction layer (HAL), and kernel layer. This software architecture runs on top of the hardware layers, which may include receivers, microphones, speakers, and screen sound-emitting devices, etc.

[0102] In this embodiment, the audio architecture based on the Android system is referred to as the "Android audio system architecture" or "Android audio framework," or simply the "audio system." Specifically, this audio system can realize functions such as audio data acquisition and output, audio stream control, audio device management, and volume adjustment. It should be understood that electronic devices can play audio from different applications based on the audio system. The external playback interface of this audio system is an audio track. Each application creates a corresponding audio track at the application framework layer, and each audio track can further call the audio system engine (Audio Flinger) to achieve audio playback.

[0103] The application layer, also known as the application package layer, can include a series of application packages. Examples include calling applications, instant messaging applications, and conferencing applications. These applications can be system applications or third-party applications. When these application packages are run, they can access the various service modules provided by the application framework layer through application programming interfaces (APIs) and execute corresponding intelligent business logic.

[0104] The application framework layer provides APIs and programming frameworks for applications in the application layer.

[0105] In this embodiment, the application framework layer may include APIs for various applications to control audio stream playback, such as APIs for call applications, instant messaging applications, and conferencing applications. The application framework layer may also include Audiotracks for each application. Figure 6 (Not shown), Audio System Engine, and Audio Policy Service. The Audio System Engine and Audio Policy Service are the two fundamental services of an audio system. The Audio Policy Service is responsible for setting audio system policies, including decisions on audio device switching, volume adjustment, etc. For example, the Audio Policy Service can act as a router, determining which audio playback device (such as a receiver, speaker, or headphones) will ultimately receive the audio stream generated by a particular Audiotrack.

[0106] For example, when a user answers a call, an audio service (Audioserver) of the audio system is triggered and started. The audio service file is audioserver.rc. The application framework layer can parse the audioserver.rc file and compile it into an audio executable program. Each application can create a corresponding Audiotrack at the application framework layer, which can output decoded audio data. When the audio system starts, the audio service can load the audio system engine. During this process, the audio service can call the audio system engine and the audio policy service respectively, and initialize the audio system engine and the audio policy service.

[0107] The audio policy service, based on the user's choice of the call application, determines the call mode to be handheld mode. When this is the case, it selects the speaker and / or receiver (the target audio playback device) for playing audio in handheld mode. The audio stream generated by the Audiotrack corresponding to the call application is then transmitted to the target audio playback device. The audio system engine is the executor of the audio system's playback policy, responsible for managing the input and output audio stream devices, switching audio modes, loading audio parameters, and processing and transmitting audio stream data. During audio system engine initialization, it can obtain the initial default volume parameters of the audio system, such as the volume levels for call stream, system stream, ringtone stream, alarm stream (alarm), notification stream, and dial key stream. It should be understood that during the audio policy service initialization, it can parse the audio configuration settings and utilize the interfaces provided by the audio system engine to complete the deployment of the entire audio system, thus providing underlying support for subsequent upper-layer applications to use audio devices. The audio policy service is the place that actually calls the audio system engine. It can call functions within the audio system engine through a binder mechanism, i.e., it calls the audio system engine interface.

[0108] The hardware abstraction layer has standard interfaces implemented by hardware vendors. For example, the hardware abstraction layer may include audio HAL, etc.

[0109] The kernel layer is the layer between hardware and software, belonging to the lower layers of the Android system. The kernel layer can contain various audio driver interfaces, such as earpiece drivers, speaker drivers, and headphone jack drivers. There can be one or more earpiece drivers. When there is one earpiece driver, that driver is responsible for driving at least two earpieces. When there are multiple earpiece drivers, each earpiece driver is responsible for driving one earpiece.

[0110] It should be noted that although the embodiments of this application are described using the Android system as an example, the basic principles are also applicable to electronic devices based on operating systems such as iOS or Windows.

[0111] It is understood that, in order to implement the sound leakage cancellation scheme in the embodiments of this application, the electronic device includes hardware and / or software modules that perform various functions. Based on the algorithm steps of the examples described in the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is implemented in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in conjunction with the embodiments.

[0112] Based on the description of the above embodiments, in the sound leakage cancellation scheme based on multiple sound-emitting devices, the electronic device includes at least two sound-emitting devices, such as a first sound-emitting device and a second sound-emitting device. The second sound-emitting device mainly emits sound into the inner ear region, while the first sound-emitting device is mainly used to cancel the sound leakage of the second sound-emitting device in the sound leakage region. When a user uses the electronic device to listen to voice calls, audio communication content, or voice messages, if the sound outlet of the first sound-emitting device is blocked, or if the sound outlets of both the first and second sound-emitting devices are blocked, it may cause a change in the sound field emitted by the sound-emitting devices. The degree to which the sound outlets of the sound-emitting devices are blocked also affects the sound leakage cancellation effect.

[0113] To characterize the degree to which the sound outlet of a sound-generating device is blocked, this application proposes the concept of "blockage rate." In one possible implementation, the blockage rate is equal to the number of blocked sound outlets of a sound-generating device divided by the total number of sound outlets of that device, or an equivalent degree of blockage. For example, as... Figure 4 As shown, the receiver of a certain model of mobile phone includes four sound outlets. When the user does not block any of the sound outlets, the blockage rate is 0%; when the user blocks one sound outlet, the blockage rate is 25%; when the user blocks two sound outlets, the blockage rate is 50%; when the user blocks three sound outlets, the blockage rate is 75%; and when the user blocks four sound outlets, the blockage rate is 100%. It can be understood that in actual implementation, the user may not completely block a sound outlet, but may only block a portion of it, so that sound can only be emitted from the unblocked area of ​​that sound outlet. In this case, the blockage rate is equal to the total area of ​​the blocked sound outlets divided by the total area of ​​all sound outlets. That is to say, the blockage rate from 0% to 100% is a smooth transition. Research has found that under different blockage rates, the transfer function corresponding to the sound leakage area exhibits a certain variation pattern.

[0114] To better understand the relationship between the plugging rate and the transfer function, the concept of "transfer function" will be explained below.

[0115] Typically, under zero initial conditions, the ratio of the Laplace transform of the system output to the Laplace transform of the input that caused the output is called the system's transfer function. In the embodiments of this application, the transfer function refers to the relationship between the sound energy of the audio signal at point A and point B as the signal is transmitted from point A to point B.

[0116] For example, in a scenario where a user uses an electronic device to listen to voice content, the area surrounding the sound-emitting device is divided into an in-ear region and a sound leakage region. Researchers can set multiple first sampling points in the in-ear region and multiple second sampling points in the sound leakage region. The electronic device can output a second audio signal through a second sound-emitting device and a first audio signal through a first sound-emitting device. Researchers can use a comprehensive testing instrument to sample the multiple first sampling points and multiple second sampling points to obtain the sound energy (i.e., the transfer function of the first sampling point) and the sound energy (i.e., the transfer function of the second sampling point) of each first sampling point. The sound energy corresponding to the in-ear region can include the sound energy of the first audio signal and the second audio signal in the in-ear region; the sound energy corresponding to the sound leakage region can include the sound energy of the first audio signal and the second audio signal in the sound leakage region. Researchers can determine the average sound energy (i.e., the transfer function of the in-ear region) corresponding to the in-ear region based on the sound energy of the multiple first sampling points; and determine the average sound energy (i.e., the transfer function of the sound leakage region) corresponding to the sound leakage region based on the sound energy of the multiple second sampling points.

[0117] Thus, we can obtain two transfer functions: one is the transfer function corresponding to the intraocular region, i.e., the transfer function from the sound-producing device to the intraocular region; the other is the transfer function corresponding to the sound leakage region, i.e., the transfer function from the sound-producing device to the sound leakage region. The study found that the transfer function corresponding to the intraocular region has a relatively small impact on the sound leakage effect, while the transfer function corresponding to the sound leakage region has a larger impact on the sound leakage effect.

[0118] Based on the description of the above embodiments, if the sound outlet of the first sound-emitting device is blocked, or if the sound outlets of both the first and second sound-emitting devices are blocked, it may cause a change in the sound field emitted by the sound-emitting devices. The following example illustrates this with the sound outlet of the first sound-emitting device being blocked. Figure 7 The variation of the transfer function from the first sound-generating device to the sound leakage region under different pore blockage ratios is illustrated with examples.

[0119] For example, taking a certain model of mobile phone whose receiver includes 4 sound holes as an example, Figure 7 The transfer function is shown for orifice plugging rates of 0% (no plugging), 25% (1 plugging), 50% (2 plugging), 75% (3 plugging), and 100% (4 plugging). The horizontal axis represents frequency in Hz (Hertz) or kHz (kilohertz), and the scale is logarithmically uniform. The vertical axis represents audio signal gain in dB (decibels), which refers to the change in amplitude or phase of the audio signal from point A (e.g., the location of the sound-emitting device) to point b (e.g., a sound leakage area).

[0120] like Figure 7 As shown, at a frequency of 2.5 kHz, the gain of the transfer function (i.e., the gain at point P1) is -21 dB when no via is blocked, -20 dB when one via is blocked, -18 dB when two vias are blocked, -15 dB when three vias are blocked, and -11 dB when four vias are blocked. At a frequency of 4.5 kHz, the gain of the transfer function (i.e., the gain at point P3) is 6 dB when no via is blocked, 0 dB when one via is blocked, -8 dB when two vias are blocked, -12 dB when three vias are blocked, and -19 dB when four vias are blocked.

[0121] It can be seen that, under different hole-closing ratios, the transfer function from the first sound-emitting device to the leakage area follows the following variation pattern: (1) As the hole-closing ratio increases, the gain of the transfer function increases significantly at 2.5KHz, and increases by about 10dB from point P1 to point P2, that is, the amplitude difference or phase difference between point P1 and point P2 is 10dB. (2) As the hole-closing ratio increases, the gain of the transfer function decreases significantly at 4.5KHz, and decreases by about 25dB from point P3 to point P4, that is, the amplitude difference or phase difference between point P1 and point P2 is 25dB. In other words, as the hole-closing ratio increases, at certain frequencies (such as... Figure 7 The amplitude or phase difference shown at 2.5kHz and 4.5kHz will exhibit significant changes.

[0122] The following is combined with Figures 8 to 10 The relationship between the changes in amplitude or phase difference and the sound leakage cancellation effect at a frequency of 2.5 kHz is explained.

[0123] For example, Figure 8 A front view showing the effect of phase and amplitude differences in sound waves on the cancellation capability is shown. Figure 9 It shows the relationship with Figure 8 The corresponding left view, Figure 10 It shows the relationship with Figure 8 The corresponding top view.

[0124] These three figures involve three concepts: amplitude difference, phase difference, and leakage cancellation depth.

[0125] Amplitude difference refers to the magnitude of the change in amplitude of an audio signal at point A (such as the location of the sound-emitting device) versus point B (such as a sound leakage area) within a specific frequency band. For example, if the amplitude difference is 0 dB, it means that the amplitude does not change when the audio signal is transmitted from point A to point B; if the amplitude difference gradually increases from 0 dB, or gradually decreases from 0 dB, it means that the magnitude of the amplitude change also gradually increases.

[0126] Phase difference refers to the magnitude of the phase change of an audio signal at point A (such as the location of the sound-emitting device) versus point B (such as a sound leakage area) within a specific frequency band. For example, if the phase difference is 0 dB, it means that the phase of the audio signal does not change when it travels from point A to point B; if the phase difference gradually increases from 0 dB, or the amplitude difference gradually decreases from 0 dB, it means that the phase change is greater.

[0127] Sound leakage cancellation depth, also known as sound leakage cancellation capability, describes the extent to which an electronic device reduces sound leakage in a leaky area when sound leakage cancellation is enabled. It is usually measured in dB. For example, a sound leakage cancellation depth of 0dB means no effect on the sound in the leaky area; a depth of -30dB means a reduction of 30dB in sound leakage; negative infinity means the strongest sound leakage cancellation effect; and a positive depth means that sound leakage is not only not canceled but actually increased. It should be noted that the sound leakage cancellation depth specified by electronic device manufacturers is usually the peak value, i.e., the maximum sound leakage cancellation level achieved in a specific frequency band, and does not represent the average sound leakage cancellation effect across the entire frequency band. The sound leakage cancellation depth of electronic devices varies across different frequency bands.

[0128] like Figures 8 to 10 As shown, in an ideal situation, when an audio signal travels from point A to point B, if both the amplitude difference and phase difference are 0dB, the sound leakage cancellation depth is negative infinity, meaning the sound leakage cancellation effect is optimal. When at least one of the amplitude difference and phase difference of the audio signal changes, it will affect the specific value of the sound leakage cancellation depth. Taking a phase difference of 0dB as an example, if the amplitude difference is 0dB, the sound leakage cancellation effect is negative infinity; if the amplitude difference is 2dB, the sound leakage cancellation effect is -12dB; if the amplitude difference is 4dB, the sound leakage cancellation effect is -4dB; if the amplitude difference is 6dB, the sound leakage cancellation effect is 0dB; if the amplitude difference is 8dB, the sound leakage cancellation effect is 3dB; and if the amplitude difference is 10dB, the sound leakage cancellation effect is 6dB.

[0129] It can be seen that the sound leakage cancellation effect gradually deteriorates as the amplitude difference gradually increases. Taking a frequency of 2.5kHz as an example, from... Figure 7 Point P1 as shown to... Figure 7 At point P4, the gain of the transfer function increases by approximately 10 dB. However, as the hole-clogging ratio increases, it not only fails to cancel out sound leakage but also increases it. Therefore, it is necessary to adjust the cancellation algorithm based on changes in the hole-clogging ratio to reduce the impact of hole-clogging on sound leakage cancellation.

[0130] The above embodiments illustrate the variation of the transfer function from the sound-generating device to the sound leakage region under different plugging rates, and the relationship between the changes in amplitude or phase differences and the sound leakage cancellation effect. Theoretically, when the transfer function from the sound-generating device to the sound leakage region is obtained, the corresponding audio optimization parameters can be calculated based on the transfer function, and the sound-generating device can be adjusted according to these parameters to optimize the cancellation effect of the sound waves from the two sound-generating devices in the sound leakage region, thereby reducing the risk of sound leakage. However, as described in the above embodiments, the transfer function corresponding to the sound leakage region needs to be determined based on the acoustic energy of multiple sampling points in the sound leakage region. In the usage phase of electronic devices, using a comprehensive testing instrument to collect the acoustic energy of multiple sampling points in the sound leakage region presents certain implementation difficulties.

[0131] Studies have found that the impedance curves of sound-generating devices follow certain variation patterns under different pore-blocking ratios.

[0132] For example, taking a certain model of mobile phone whose receiver includes 4 sound holes as an example, Figure 11 The impedance curves of the sound-generating device are shown when the plugging rate is 0% (no plugging), 25% (1 plugging), 50% (2 plugging), 75% (3 plugging), and 100% (4 plugging). The horizontal axis represents frequency in Hz (Hertz) or kHz (kilohertz); the vertical axis represents the impedance of the sound-generating device in ohms (Ω).

[0133] like Figure 11 As shown, when the holes are not blocked, the impedance curve of the sound-generating device has the first resonance peak at 800Hz, and the peak value of the first resonance peak is 11.5Ω; when one hole is blocked, the impedance curve of the sound-generating device has the first resonance peak at 750Hz, and the peak value of the first resonance peak is 11Ω; when two holes are blocked, the impedance curve of the sound-generating device has the first resonance peak at 725Hz, and the peak value of the first resonance peak is 10.9Ω; when three holes are blocked, the impedance curve of the sound-generating device has the first resonance peak at 700Hz, and the peak value of the first resonance peak is 10.8Ω; when four holes are blocked, the impedance curve of the sound-generating device has the first resonance peak at 650Hz, and the peak value of the first resonance peak is 10.5Ω.

[0134] When the holes are not blocked, the impedance curve of the sound-generating device exhibits a second resonance peak at 4200Hz, with a peak value of 7.9Ω; when one hole is blocked, the impedance curve exhibits a second resonance peak at 3900Hz, with a peak value of 8.1Ω; when two holes are blocked, the impedance curve exhibits a second resonance peak at 2400Hz, with a peak value of 8.4Ω; when three holes are blocked, the impedance curve exhibits a second resonance peak at 2000Hz, with a peak value of 8.5Ω; and when four holes are blocked, the impedance curve exhibits a second resonance peak at 1150Hz, with a peak value of 9.1Ω.

[0135] It can be seen that under different porosity, the impedance curve of the sound-generating device conforms to the following variation law: (1) As the porosity increases, the first resonance peak decreases significantly and the frequency shifts to lower frequencies; (2) As the porosity increases, the second resonance peak increases significantly and the frequency also shifts to lower frequencies. Based on this, the electronic device can collect the impedance curve of the sound-generating device and calculate the porosity of the sound-generating device according to the variation law of the impedance curve (such as the frequency and impedance of the two resonance peaks). Then, according to the audio optimization parameters corresponding to the porosity, the sound-generating device is adjusted to achieve the purpose of optimizing the cancellation effect of the sound waves of the two sound-generating devices in the sound leakage area and reducing the risk of sound leakage. It can be understood that compared with calculating the porosity of the sound-generating device based on the variation law of the transfer function corresponding to the sound leakage area, since the electronic device can collect the impedance curve of its own sound-generating device in real time during the use stage of the electronic device without the need to use a comprehensive tester, the feasibility of calculating the porosity of the sound-generating device based on the variation law of the impedance curve is higher.

[0136] Thus far, the above embodiments have completed the introduction of the basic principles involved in the sound leakage cancellation method provided in this application. The specific implementation process of the sound leakage cancellation method provided in this application will be illustrated below.

[0137] For example, Figure 12 An impedance curve-based sound leakage cancellation method is illustrated. This method can be applied to electronic devices such as mobile phones. The electronic device can include at least a first sound-emitting device and a second sound-emitting device. The second sound-emitting device is mainly used to emit sound into the user's inner ear region, and the first sound-emitting device is mainly used to cancel the sound leakage of the second sound-emitting device in the leakage area. The first sound-emitting device includes at least one sound outlet; for example, the first sound-emitting device can be a receiver. When a user uses the electronic device to receive voice calls, audio communication content, or voice messages, one or more sound outlets of the first sound-emitting device may be blocked, resulting in a deterioration in the sound leakage cancellation effect. In this scenario, a method can be adopted... Figure 12 The provided sound leakage cancellation method adjusts the audio optimization parameters corresponding to the first sound-generating device.

[0138] It should be noted that, as Figure 12 , Figure 15 as well as Figure 17 The example given is an electronic device, which does not limit the scope of this application. It is understood that the implementing entity of these methods can also be a functional module within an electronic device, such as... Figure 5 The processor shown or such Figure 6 The audio system engine, etc., are shown.

[0139] like Figure 12 As shown, the method may include the following steps S11 to S15.

[0140] S11. Output a first audio signal through a first sound-generating device and output a second audio signal through a second sound-generating device.

[0141] Take, for example, a user answering a voice call using an electronic device. Figure 13 As shown, the electronic device may include an antenna, a signal processing module, a first filter and a second filter connected to the signal processing module, a first sound-emitting device connected to the first filter, and a second sound-emitting device connected to the second filter. After the electronic device receives a downlink call signal through the antenna, the signal processing module processes the downlink call signal to obtain two signals. One signal passes through the first filter to obtain a first audio signal, which is then output by the first sound-emitting device. The first audio signal mainly reaches the leakage area. The other signal passes through the second filter to obtain a second audio signal, which is then output by the second sound-emitting device. A portion of the second audio signal reaches the ear area, and the other portion reaches the leakage area. Thus, the first and second audio signals are superimposed in the leakage area. That is, in the leakage area, the first audio signal is used to cancel the leakage of the second audio signal. However, in this process, if one or more of the sound outlets of the first sound-emitting device are blocked, it may cause a large gain change in the transfer function of the first sound-emitting device at certain frequencies (mainly referring to a large amplitude difference and / or phase difference), thereby causing a large change in the leakage cancellation depth and affecting the leakage cancellation effect of the electronic device.

[0142] To solve this problem, during the process of outputting the first audio signal through the first sound-emitting device, the electronic device can adjust the audio signal output by the first sound-emitting device in real time based on the hole blockage rate of the first sound-emitting device.

[0143] Specifically, referring to the above embodiments... Figure 7 and Figure 11The description states that, under different porosity ratios, the transfer function from the first sound-generating device to the sound leakage region and the impedance curve of the first sound-generating device each conform to their own variation laws, and these variation laws are all related to the porosity ratio of the first sound-generating device. In other words, under various porosity ratios, the transfer function and impedance curve of the first sound-generating device have a one-to-one mapping relationship, for example, as... Figure 7 The transfer function shown is as follows when the orifice is not plugged. Figure 11 The impedance curve shown is a mapping relationship when the orifice is not plugged, as follows: Figure 7 The transfer function shown when plugging hole 1 is as follows: Figure 11 The impedance curve shown is a mapping relationship when hole 1 is plugged, as follows: Figure 7 The transfer function shown is as follows when plugging 2 holes. Figure 11 The impedance curves shown for blocking two holes represent a mapping relationship. Furthermore, transfer functions and impedance curves each have their own characteristics: For transfer functions, during the product development phase of electronic devices, a comprehensive testing instrument can be used to collect the acoustic energy of multiple sampling points in the sound leakage area to obtain the transfer function corresponding to the sound leakage area, and then calculate the corresponding audio optimization parameters. However, during the usage phase of electronic devices, using a comprehensive testing instrument to collect the acoustic energy of multiple sampling points in the sound leakage area presents certain implementation difficulties. For impedance curves, during the usage phase of electronic devices, without the need for a comprehensive testing instrument, the electronic device can collect the impedance curve of its own sound-generating device in real time, but the audio optimization parameters cannot be directly calculated using the impedance curve. Based on the characteristics of transfer functions and impedance curves, this application proposes a technical solution to adjust the cancellation effect of two audio signals in the sound leakage area using transfer functions and impedance curves, as shown in S12 to S15 below.

[0144] S12. During the process of outputting the first audio signal through the first sound-emitting device, the impedance data of the first sound-emitting device is obtained.

[0145] The impedance data of the first sound-generating device can be as follows: Figure 11 The figure shows an impedance curve. This impedance curve consists of multiple sampling points, each of which includes a frequency value and the corresponding impedance value.

[0146] Referring to the above embodiments Figure 11According to the description, under different hole-clogging ratios, the first and second resonance peaks in the impedance curve of the first sound-generating device conform to the following variation law: (1) As the hole-clogging ratio increases, the first resonance peak decreases significantly, and the frequency shifts to lower frequencies; (2) As the hole-clogging ratio increases, the second resonance peak increases significantly, and the frequency also shifts to lower frequencies. That is to say, the first and second resonance peaks are closely related to the hole-clogging ratio of the first sound-generating device. Therefore, the impedance data of the first sound-generating device can at least include: the impedance and frequency of the first resonance peak in the impedance curve of the first sound-generating device, and the impedance and frequency of the second resonance peak in the impedance curve of the first sound-generating device. This facilitates the establishment of a relationship with the transfer function under the same hole-clogging ratio based on the impedance and frequency of the first and / or second resonance peaks, and then predicts the audio optimization parameters for optimizing the first filter corresponding to the first sound-generating device based on the transfer function.

[0147] In some embodiments, such as Figure 13 As shown, the electronic device may further include an impedance acquisition module connected to the first sound-emitting device, and an artificial intelligence (AI) model connected to the impedance acquisition module. During the process of the electronic device outputting a first audio signal through the first sound-emitting device, the impedance acquisition module can collect the voltage and current values ​​of the first sound-emitting device in real time. Then, the impedance acquisition module can calculate the impedance data of the first sound-emitting device based on the voltage and current values. For example, the impedance data of the first sound-emitting device may include the impedance and frequency of the first resonance peak, and the impedance and frequency of the second resonance peak. The impedance acquisition module can then input the impedance data of the first sound-emitting device into the AI ​​model.

[0148] In some embodiments, temperature variations may cause the impedance data of the first sound-generating device, calculated based on its voltage and current values, to be inaccurate. To address this issue, the electronic device can also correct the impedance data of the first sound-generating device. For example... Figure 13 As shown, the electronic device may further include a temperature acquisition module connected to the first sound-generating device and the impedance acquisition module. For example, the temperature acquisition module may be mounted on the coil of the first sound-generating device. During the output of the first audio signal through the first sound-generating device, the temperature acquisition module can acquire the temperature of the first sound-generating device in real time and send the temperature to the impedance acquisition module. In this way, the impedance acquisition module can correct the impedance data of the first sound-generating device based on the temperature, and then input the corrected impedance data into the AI ​​model. It can be understood that correcting the impedance data based on the temperature of the first sound-generating device can make the impedance data finally input into the AI ​​model more accurate, thereby making the audio optimization parameters predicted by the AI ​​model more accurate.

[0149] For example, manufacturers of electronic devices can calibrate the first sound-generating device of the electronic device before it leaves the factory. Specifically, they obtain the temperature of the first sound-generating device after calibration begins (referred to as the calibration temperature) and the impedance of the first sound-generating device after calibration begins (referred to as the calibration impedance); calculate the impedance offset based on the calibration temperature and the reference temperature; calculate the reference impedance based on the calibration impedance and the impedance offset; and write the reference impedance into the power amplifier. During the use of the electronic device, the electronic device can detect the current impedance of the first sound-generating device; and obtain the corrected impedance based on the current impedance, the reference impedance, and the reference temperature.

[0150] S13. Input the impedance data of the first sound-generating device into the AI ​​model to obtain the first audio optimization parameters.

[0151] In some embodiments, such as Figure 13 As shown, the input to the AI ​​model is the impedance data of the first sound-generating device obtained through S12, and the output of the AI ​​model is the first audio optimization parameter corresponding to the impedance data. The first audio optimization parameter is used to adjust the first filter corresponding to the first sound-generating device, causing a change in the audio signal output by the first sound-generating device, which is equivalent to adjusting the sound leakage cancellation algorithm of the electronic device. It can be understood that when different impedance data values ​​are input to the AI ​​model, the AI ​​model may output different audio optimization parameters.

[0152] The aforementioned AI model was trained during the product development phase using multiple impedance data points and corresponding audio optimization parameters. The target impedance data refers to the impedance data of the first sound-emitting device collected when the electronic device is in the target holding posture. The audio optimization parameters corresponding to the target impedance data are calculated based on the target transfer function. The target transfer function is obtained after testing the acoustic energy of the leakage area while the electronic device is in the target holding posture. The target impedance data can be any one of the multiple impedance data points.

[0153] For example, during the product development phase, the acoustic energy of multiple sampling points in a sound leakage area is tested under a certain grip posture to obtain the acoustic energy of multiple sampling points. Based on the tested acoustic energy of multiple sampling points, the average acoustic energy corresponding to the sound leakage area is determined, i.e., the transfer function corresponding to the sound leakage area. According to the transfer function, audio optimization parameters corresponding to the grip posture are calculated, and a correspondence between impedance data and audio optimization parameters under this grip posture is established. Then, the grip posture of the electronic device or the person holding the electronic device is adjusted to obtain multiple impedance data. The above steps are repeated to obtain multiple audio optimization parameters corresponding to the multiple impedance data. Then, an AI model can be built based on these data. It can be understood that since the AI ​​model is built based on impedance data, transfer function, and audio optimization parameters during the product development phase, during the user usage phase, the current impedance data can be directly input into the AI ​​model to obtain the audio optimization parameters corresponding to the current impedance data. For the specific implementation of building the AI ​​model, please refer to the following embodiments. Figure 19 The relevant descriptions are not elaborated here.

[0154] S14. Update the audio optimization parameters of the first filter corresponding to the first sound-emitting device to the first audio optimization parameters to obtain the third audio signal.

[0155] During the process of the electronic device outputting a first audio signal through the first sound-emitting device, the electronic device can periodically acquire the impedance data of the first sound-emitting device. For example, in the (i-1)th cycle, the audio optimization parameter corresponding to the impedance data is acquired as audio optimization parameter a, and the first filter transmits the first audio signal corresponding to audio optimization parameter a to the first sound-emitting device; in the ith cycle, the audio optimization parameter corresponding to the impedance data is acquired as audio optimization parameter b, and the electronic device can update the audio optimization parameter of the first sound-emitting device from audio optimization parameter a to audio optimization parameter b (i.e., the first audio optimization parameter), and the first filter transmits the third audio signal corresponding to audio optimization parameter b to the first sound-emitting device. Here, i is a positive integer.

[0156] S15. Output a third audio signal through the first sound-generating device, and continue to output a second audio signal through the second sound-generating device.

[0157] The third audio signal differs in frequency and / or phase from the first audio signal, resulting in different sound fields for the third and first audio signals in the sound leakage region. After acquiring the third audio signal through steps S12 to S14, the electronic device can output the third audio signal through the first sound-emitting device and continue to output the second audio signal through the second sound-emitting device. Thus, the sound leakage cancellation depth after the third and second audio signals are superimposed in the sound leakage region changes compared to the sound leakage cancellation depth after the first and second audio signals are superimposed in the same region.

[0158] As an example, the first audio signal in S11 is the audio signal that was not adjusted when the first sound-generating device experienced blockage. At this moment, the sound leakage cancellation depth after the first and second audio signals were superimposed in the sound leakage area is the first sound leakage cancellation depth. The third audio signal in S15 is the audio signal adjusted according to audio optimization parameters after the first sound-generating device experienced blockage. At this moment, the sound leakage cancellation depth after the third and second audio signals were superimposed in the sound leakage area is the second sound leakage cancellation depth. The second sound leakage cancellation depth is less than the first sound leakage cancellation depth.

[0159] As another example, the first audio signal in S11 is the audio signal output when the first sound-generating device is not blocked. At this moment, the sound leakage cancellation depth after the first and second audio signals are superimposed in the sound leakage area is the target sound leakage cancellation depth. The third audio signal in S15 is the audio signal adjusted according to audio optimization parameters after the first sound-generating device is blocked. At this moment, the sound leakage cancellation depth after the third and second audio signals are superimposed in the sound leakage area is the second sound leakage cancellation depth. The second sound leakage cancellation depth is greater than or equal to the target sound leakage cancellation depth.

[0160] For example, such as Figure 14 As shown, the blue line represents the target sound leakage cancellation depth, the red line represents the first sound leakage cancellation depth, and the yellow line represents the second sound leakage cancellation depth. For most frequencies, the second sound leakage cancellation depth is less than the first sound leakage cancellation depth, and the second sound leakage cancellation depth is greater than or equal to the target sound leakage cancellation depth. Taking a frequency of 1000Hz as an example, the sound leakage cancellation depth at point P11 is -29dB, at point P12 it is -16dB, and at point P13 it is -23dB. It can be seen that when the first sound-emitting device is blocked, the sound leakage cancellation effect of the electronic device deteriorates significantly, while adjusting the audio signal output by the first sound-emitting device can improve the sound leakage cancellation effect to some extent.

[0161] It should be noted that, as Figure 14 As shown, when the first sound leakage cancellation depth, the second sound leakage cancellation depth, and the target sound leakage cancellation depth are all negative, the target sound leakage cancellation depth is less than the second sound leakage cancellation depth, and the second sound leakage cancellation depth is less than the first sound leakage cancellation depth. In reality, the absolute value of the target sound leakage cancellation depth is greater than the absolute value of the second sound leakage cancellation depth, and the absolute value of the second sound leakage cancellation depth is greater than the absolute value of the first sound leakage cancellation depth. That is to say, the sound leakage cancellation effect when the first sound-emitting device is not blocked is better than the sound leakage cancellation effect after adjusting the audio signal output of the first sound-emitting device, and the sound leakage cancellation effect after adjusting the audio signal output of the first sound-emitting device is better than the sound leakage cancellation effect when the audio signal output of the first sound-emitting device is not adjusted.

[0162] In the sound leakage cancellation method provided in this application, the electronic device may include at least a first sound-emitting device and a second sound-emitting device. The second sound-emitting device is mainly used to emit sound into the user's inner ear region, and the first sound-emitting device is mainly used to cancel the sound leakage of the second sound-emitting device in the sound leakage region. The first sound-emitting device may include at least one sound outlet. During the user's use of the electronic device to listen to voice content, the electronic device can collect the impedance data of the first sound-emitting device in real time, and then adjust the audio optimization parameters of the first sound-emitting device based on the impedance data. By adjusting the audio optimization parameters of the first sound-emitting device, the cancellation effect of the sound waves from the two sound-emitting devices in the sound leakage region can be optimized, thereby reducing the risk of sound leakage and improving the user experience.

[0163] The above Figure 12 and Figure 13 This explanation uses the example of an electronic device adjusting only the audio signal output by the first sound-generating device, and does not limit the scope of this application. In actual implementation, such as Figure 3 As shown, the first and second sound-generating devices of the electronic device may each include at least one sound outlet. During the process of a user listening to voice content using the electronic device, when both the sound outlets of the second and first sound-generating devices are blocked to a certain extent, if the change in the sound field in the ear region is small, it basically does not affect the user's listening experience. However, due to the blockage of the sound outlets of the second and first sound-generating devices, the phase and / or amplitude of the leaking sound wave from the second sound-generating device and the canceling sound wave from the first sound-generating device both change, causing a change in the sound field of the first and second sound-generating devices in the leaking area, thus worsening the sound leakage elimination effect, and may even aggravate the sound leakage effect. To solve this problem, in Figure 12 and Figure 13 Based on this, this application provides another method for sound leakage cancellation based on impedance curves.

[0164] For example, Figure 15 Another method for sound leakage cancellation based on impedance curves is shown. This method can be applied to electronic devices such as mobile phones. The electronic device can include at least a first sound-emitting device and a second sound-emitting device. The second sound-emitting device is mainly used to emit sound into the user's inner ear region, and the first sound-emitting device is mainly used to cancel the sound leakage of the second sound-emitting device in the leakage area. The first and second sound-emitting devices can each include at least one sound outlet; for example, both the first and second sound-emitting devices can be receivers. When a user uses the electronic device to answer voice calls, audio communication content, or voice messages, the sound outlets of the first and second sound-emitting devices may be blocked, resulting in a deterioration in the sound leakage cancellation effect. In this scenario, a method can be used... Figure 15The provided sound leakage cancellation method adjusts the audio optimization parameters corresponding to the first and second sound-generating devices.

[0165] like Figure 15 As shown, the method may include the following steps S21 to S25.

[0166] S21. Output a first audio signal through a first sound-emitting device and output a second audio signal through a second sound-emitting device.

[0167] For the specific implementation of S21, please refer to the description of S11 in the above embodiments, which will not be repeated here.

[0168] S22. During the process of outputting a first audio signal through a first sound-emitting device and outputting a second audio signal through a second sound-emitting device, the impedance data of the first sound-emitting device and the impedance data of the second sound-emitting device are acquired.

[0169] The difference from S12 in the above embodiment is that, during the process of outputting a first audio signal through the first sound-emitting device and a second audio signal through the second sound-emitting device, the electronic device can acquire not only the impedance data of the first sound-emitting device but also the impedance data of the second sound-emitting device. Similar to the impedance data of the first sound-emitting device, the impedance data of the second sound-emitting device can be an impedance curve, which consists of multiple sampling points. Each sampling point includes a frequency value and the impedance value corresponding to that frequency value.

[0170] For example, the impedance data of the first sound-generating device may include: the impedance and frequency of the first resonance peak in the impedance curve of the first sound-generating device, and the impedance and frequency of the second resonance peak in the impedance curve of the first sound-generating device. The impedance data of the second sound-generating device may include: the impedance and frequency of the first resonance peak in the impedance curve of the second sound-generating device, and the impedance and frequency of the second resonance peak in the impedance curve of the second sound-generating device.

[0171] S23. Input the impedance data of the first sound-generating device and the impedance data of the second sound-generating device into the AI ​​model to obtain the first audio optimization parameters and the second audio optimization parameters.

[0172] For example, such as Figure 16As shown, the electronic device may include an impedance acquisition module connected to a first sound-emitting device and a second sound-emitting device, a temperature acquisition module connected to the first sound-emitting device, the second sound-emitting device, and the impedance acquisition module, and an AI model connected to the impedance acquisition module, a first filter, and a second filter. During the process of outputting a first audio signal through the first sound-emitting device and a second audio signal through the second sound-emitting device, the impedance acquisition module can acquire the voltage and current values ​​of the first sound-emitting device and the second sound-emitting device in real time. Then, the impedance acquisition module can calculate the impedance data of the first sound-emitting device based on its voltage and current values; and calculate the impedance data of the second sound-emitting device based on its voltage and current values. Next, the impedance acquisition module can input the impedance data of the first and second sound-emitting devices into the AI ​​model. Then, the AI ​​model can obtain first audio optimization parameters based on the impedance data of the first sound-emitting device and pass these parameters to the first filter; and obtain second audio optimization parameters based on the impedance data of the second sound-emitting device and pass these parameters to the second filter. For the training method of AI models, please refer to the example. Figure 19 The relevant descriptions are not elaborated here.

[0173] It should be noted that, Figure 16 This explanation uses the example of the first and second sound-generating devices sharing a common AI model, impedance acquisition module, and temperature acquisition module, and does not limit the scope of this application. In actual implementation, the first and second sound-generating devices can use their respective corresponding AI models, impedance acquisition modules, and temperature acquisition modules. For example, the first sound-generating device is connected to the first impedance acquisition module and the first temperature acquisition module, and the first impedance acquisition module is connected to the first filter through the first AI model; the second sound-generating device is connected to the second impedance acquisition module and the second temperature acquisition module, and the second impedance acquisition module is connected to the second filter through the second AI model.

[0174] S24. Update the audio optimization parameters of the first filter corresponding to the first sound-emitting device to the first audio optimization parameters to obtain the third audio signal, and update the audio optimization parameters of the second filter corresponding to the second sound-emitting device to the second audio optimization parameters to obtain the fourth audio signal.

[0175] The difference between this embodiment and S14 is that the electronic device updates not only the audio optimization parameters of the first filter but also the audio optimization parameters of the second filter. For the specific implementation of updating the audio optimization parameters of the second filter, please refer to the relevant description of S14 in the above embodiment; it will not be repeated here.

[0176] S25. Output a third audio signal through the first sound-generating device and output a fourth audio signal through the second sound-generating device.

[0177] Among them, the sound leakage cancellation depth after the third audio signal and the fourth audio signal are superimposed in the sound leakage area is the third sound leakage cancellation depth.

[0178] As an example, if the electronic device adjusts only the audio signal output by the first sound-emitting device and not the audio signal output by the second sound-emitting device, then, referring to the description of embodiment S15 above, the sound leakage cancellation depth after the third audio signal and the second audio signal are superimposed in the sound leakage region is the second sound leakage cancellation depth. If the electronic device adjusts the audio signals output by both the first and second sound-emitting devices, then the sound leakage cancellation depth after the third and fourth audio signals are superimposed in the sound leakage region is the third sound leakage cancellation depth. It can be understood that compared to adjusting the sound field of only one sound-emitting device in the sound leakage region, adjusting the sound field of both sound-emitting devices in the sound leakage region simultaneously can achieve a better sound leakage cancellation effect, that is, the third sound leakage cancellation depth is less than the second sound leakage cancellation depth.

[0179] In the sound leakage cancellation method provided in this application, the electronic device may include at least a first sound-emitting device and a second sound-emitting device. The second sound-emitting device is mainly used to emit sound into the user's inner ear region, and the first sound-emitting device is mainly used to cancel the sound leakage of the second sound-emitting device in the sound leakage region. Each of the first and second sound-emitting devices may include at least one sound outlet. During the user's use of the electronic device to listen to voice content, the electronic device can collect the impedance data of the first and second sound-emitting devices in real time. Then, it adjusts the audio optimization parameters of the first and second sound-emitting devices based on their impedance data. By adjusting the audio optimization parameters of the first and second sound-emitting devices, the cancellation effect of the sound waves from the two sound-emitting devices in the sound leakage region can be further optimized, thereby further reducing sound leakage.

[0180] The above embodiments describe how an electronic device can detect the impedance data of a sound-generating device in real time and adjust the audio optimization parameters of the sound-generating device based on the detected impedance data. However, if the audio optimization parameters of the sound-generating device are adjusted based solely on the detected impedance data, it may lead to excessive power consumption of the electronic device, and the improvement in sound leakage cancellation effect may not be significant. To address this issue, this application also provides another sound leakage cancellation method based on impedance curves. In this method, after the electronic device detects impedance data, it first determines whether the difference between the detected impedance data and the previously detected impedance data exceeds a preset threshold. If it exceeds the preset threshold, the audio optimization parameter adjustment mechanism is activated; otherwise, the audio optimization parameters are not adjusted.

[0181] For example, Figure 15 Another method for sound leakage cancellation based on impedance curves is shown. Taking the process of outputting a first audio signal through a first sound-emitting device, where the impedance data of the first sound-emitting device is the first impedance data, as an example, after S11 to S15 in the above embodiment, the method may include the following S31 to S37.

[0182] S31. During the process of outputting the third audio signal through the first sound-generating device, the second impedance data of the first sound-generating device is obtained.

[0183] Referring to the description of embodiment S12 above, the second impedance data can be an impedance curve, which consists of multiple sampling points. Each sampling point includes a frequency value and the impedance value corresponding to that frequency value. For example, the second impedance data may include: the impedance and frequency of the first resonance peak in the impedance curve of the first sound-emitting device during the output of the third audio signal through the first sound-emitting device, and the impedance and frequency of the second resonance peak in the impedance curve of the first sound-emitting device.

[0184] S32. Determine whether the impedance data of the first sound-emitting device acquired in the latest instance (such as the second impedance data) and the impedance data of the first sound-emitting device acquired in the previous instance (such as the first impedance data) meet the preset conditions.

[0185] In some embodiments, the above preset conditions may include at least one:

[0186] (1) The difference between the frequency of the first resonance peak in the impedance data of the first sound-emitting device acquired in the latest acquisition and the frequency of the first resonance peak in the impedance data of the first sound-emitting device acquired in the previous acquisition is greater than or equal to the first threshold.

[0187] (2) The difference between the impedance of the first resonance peak in the latest acquired impedance data of the first sound-emitting device and the impedance of the first resonance peak in the previous acquired impedance data of the first sound-emitting device is greater than or equal to the second threshold.

[0188] (3) The difference between the frequency of the second resonance peak in the impedance data of the first sound-emitting device acquired in the latest acquisition and the frequency of the second resonance peak in the impedance data of the first sound-emitting device acquired in the previous acquisition is greater than or equal to the third threshold.

[0189] (4) The difference between the impedance of the second resonance peak in the latest impedance data of the first sound-emitting device and the impedance of the second resonance peak in the previous impedance data of the first sound-emitting device is greater than or equal to the fourth threshold.

[0190] For example, Figure 18 This is a schematic diagram of the resonance peaks in the impedance curves obtained in two separate acquisitions, as provided in the embodiments of this application. The impedance curve of the first sound-emitting device obtained in the previous acquisition is shown below. Figure 18 The solid line shown represents the impedance curve of the first sound-generating device obtained in the latest acquisition. Figure 18 The dashed lines indicate the first resonance peak in the impedance curve of the first sound-generating device obtained in the previous acquisition. The first resonance peak in the impedance curve of the first sound-generating device obtained in the latest acquisition is represented by P23(f3,i3). The second resonance peak in the impedance curve of the first sound-generating device obtained in the previous acquisition is represented by P22(f21,i2). The second resonance peak in the impedance curve of the first sound-generating device obtained in the latest acquisition is represented by P24(f4,i4). Where f1 represents the frequency of the first resonance peak in the impedance data of the first sound-emitting device acquired in the previous acquisition, f2 represents the frequency of the second resonance peak in the impedance data of the first sound-emitting device acquired in the previous acquisition, f3 represents the frequency of the first resonance peak in the impedance data of the first sound-emitting device acquired in the latest acquisition, f4 represents the frequency of the second resonance peak in the impedance data of the first sound-emitting device acquired in the latest acquisition, i1 represents the impedance of the first resonance peak in the impedance data of the first sound-emitting device acquired in the previous acquisition, i2 represents the impedance of the second resonance peak in the impedance data of the first sound-emitting device acquired in the previous acquisition, i3 represents the impedance of the first resonance peak in the impedance data of the first sound-emitting device acquired in the latest acquisition, and i4 represents the impedance of the second resonance peak in the impedance data of the first sound-emitting device acquired in the latest acquisition. If at least one of the following is satisfied: |f3-f1|≥k1, |i3-i1|≥k2, |f4-f2|≥k3, |i4-i2|≥k4, then the preset condition is satisfied. Where k1, k2, k3, and k4 represent the first threshold, the second threshold, the third threshold, and the fourth threshold, respectively.

[0191] S33. Under the condition that the preset conditions are met, the second impedance data is input into the AI ​​model to obtain the third audio optimization parameters.

[0192] S34. Update the audio optimization parameters of the first filter corresponding to the first sound-emitting device from the first audio optimization parameters to the third audio optimization parameters to obtain the fifth audio signal.

[0193] S35. Output the fifth audio signal through the first sound-emitting device, and continue to output the second audio signal through the second sound-emitting device.

[0194] Among them, the sound leakage cancellation depth after the fifth audio signal and the second audio signal are superimposed in the sound leakage area is the fourth sound leakage cancellation depth.

[0195] As an example, the first audio signal in S11 is the audio signal that was not adjusted when the first sound-generating device experienced blockage. At this moment, the sound leakage cancellation depth after the first and second audio signals are superimposed in the sound leakage area is the first sound leakage cancellation depth. The fifth audio signal in S35 is the audio signal that has been adjusted again according to the audio optimization parameters after the first sound-generating device experienced blockage. At this moment, the sound leakage cancellation depth after the fifth and second audio signals are superimposed in the sound leakage area is the fourth sound leakage cancellation depth. The fourth sound leakage cancellation depth is less than the first sound leakage cancellation depth.

[0196] S36. If the preset conditions are not met, continue to output the third audio signal through the first sound-generating device.

[0197] S37. While continuing to output the third audio signal through the first sound-emitting device, continue to acquire the impedance data of the first sound-emitting device. Then, return to execute S32 above and determine whether the impedance data of the first sound-emitting device acquired in the latest instance meets the preset condition compared with the impedance data of the first sound-emitting device acquired in the previous instance (such as the second impedance data), until the impedance data of the first sound-emitting device acquired in the latest instance meets the preset condition compared with the impedance data of the first sound-emitting device acquired in the previous instance.

[0198] In the sound leakage cancellation method provided in this application, the electronic device can detect the impedance data of the sound-generating device in real time and determine whether the difference between the impedance data and the previously detected impedance data exceeds a preset threshold; only when the preset threshold is exceeded will the audio optimization parameter adjustment mechanism be activated. In this way, the power consumption of the electronic device can be saved while ensuring that the change in the cancellation effect is not significant.

[0199] The above embodiments illustrate how electronic devices can acquire audio optimization parameters corresponding to impedance data based on AI models. This AI model can be trained during the product development phase using multiple impedance data points and the corresponding audio optimization parameters. The following section will combine... Figure 19 The process of building an AI model is illustrated with an example.

[0200] For example, Figure 19 This is a schematic diagram illustrating the creation of an AI model as provided in an embodiment of this application.

[0201] like Figure 19 As shown, during the product development phase, to ensure the accuracy of the AI ​​model, different users can use the electronic device to answer voice calls, audio communications, or voice messages under different gripping postures. This allows the first electronic device to collect impedance data under different orifice coverage ratios. Taking the first electronic device in a target gripping posture as an example, it can acquire the target impedance data corresponding to the sound-emitting device under that posture. Furthermore, testers can use a comprehensive testing instrument to test the acoustic energy at multiple sampling points in the leakage area under the target gripping posture, obtaining the acoustic energy at multiple sampling points. Based on the measured acoustic energy at these multiple sampling points, the first electronic device can calculate the average acoustic energy corresponding to the leakage area, i.e., the transfer function corresponding to the leakage area. Then, based on the transfer function, it can calculate the audio optimization parameters corresponding to that gripping posture, thus establishing a one-to-one correspondence between impedance data, transfer function, and audio optimization parameters under the target gripping posture. Repeating these steps allows for the acquisition of a one-to-one correspondence between impedance data, transfer function, and audio optimization parameters under different gripping postures. For example, in the first holding posture, impedance 1 corresponds to transfer function 1 and audio optimization parameter 1; in the second holding posture, impedance 2 corresponds to transfer function 2 and audio optimization parameter 2; in the third holding posture, impedance 3 corresponds to transfer function 3 and audio optimization parameter 3... Based on this data, the first electronic device can build an AI model.

[0202] During the user operation phase, as the second electronic device outputs an audio signal through the sound-generating device, it can acquire the impedance data of the sound-generating device and then determine whether the latest acquired impedance data meets the preset conditions compared to the previously acquired impedance data. If the preset conditions are met, the impedance data is input into the established AI model to obtain the audio optimization parameters corresponding to the impedance data. The audio optimization parameters of the filter corresponding to the sound-generating device are then updated, and finally, the optimized audio signal is output.

[0203] It should be noted that the first electronic device and the second electronic device mentioned above can be the same electronic device or different electronic devices; the sound-generating device mentioned above can be the first sound-generating device or the second sound-generating device.

[0204] It is understandable that by establishing an AI model, the audio optimization parameters corresponding to the current impedance data can be obtained during the user's use, without the need to use a comprehensive tester to test the sound energy of multiple sampling points in the sound leakage area under the current holding posture, thereby improving the feasibility of the sound leakage cancellation method provided in this application.

[0205] This application also provides a chip coupled to a memory, which is used to read and execute computer programs or instructions stored in the memory to perform the methods in the above embodiments.

[0206] This application also provides an electronic device including a chip for reading and executing computer programs or instructions stored in a memory, causing the methods in the various embodiments to be performed.

[0207] This embodiment also provides a computer-readable storage medium storing computer instructions. When the computer instructions are executed on an electronic device, the electronic device performs the aforementioned method steps to implement the sound leakage cancellation method in the above embodiment.

[0208] This embodiment also provides a computer program product. The computer-readable storage medium stores program code. When the computer program product is run on a computer, the computer performs the above-mentioned related steps to implement the sound leakage cancellation method in the above embodiment.

[0209] In addition, embodiments of this application also provide an apparatus, which may specifically be a chip, component or module. The apparatus may include a connected processor and a memory; wherein the memory is used to store computer execution instructions, and when the apparatus is running, the processor may execute the computer execution instructions stored in the memory to cause the chip to execute the sound leakage cancellation method in the above method embodiments.

[0210] In this embodiment, the electronic device, computer-readable storage medium, computer program product or chip are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects of the corresponding methods provided above, and will not be repeated here.

[0211] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another apparatus, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0212] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for canceling sound leakage, characterized in that, The method is applied to an electronic device, the electronic device including at least a first sound-generating device and a second sound-generating device, the method comprising: The first audio signal is output through the first sound-generating device, and the second audio signal is output through the second sound-generating device; During the process of outputting the first audio signal through the first sound-generating device, the impedance data of the first sound-generating device is acquired; The impedance data of the first sound-generating device is input into the artificial intelligence model to obtain the first audio optimization parameters; The audio optimization parameters of the first filter corresponding to the first sound-generating device are updated to the first audio optimization parameters to obtain the third audio signal; The third audio signal is output through the first sound-generating device, and the second audio signal continues to be output through the second sound-generating device; Wherein, the sound leakage cancellation depth after the first audio signal and the second audio signal are superimposed in the sound leakage region is the first sound leakage cancellation depth, and the sound leakage cancellation depth after the third audio signal and the second audio signal are superimposed in the sound leakage region is the second sound leakage cancellation depth, and the second sound leakage cancellation depth is less than the first sound leakage cancellation depth.

2. The method according to claim 1, characterized in that, The method further includes: During the process of outputting the second audio signal through the second sound-generating device, the impedance data of the second sound-generating device is acquired; The impedance data of the second sound-generating device is input into the artificial intelligence model to obtain the second audio optimization parameters; The audio optimization parameters of the second filter corresponding to the second sound-generating device are updated to the second audio optimization parameters to obtain the fourth audio signal; The first sound-generating device outputs a third audio signal, and the second sound-generating device outputs the fourth audio signal; wherein, the sound leakage cancellation depth after the third audio signal and the fourth audio signal are superimposed in the sound leakage region is the third sound leakage cancellation depth, and the third sound leakage cancellation depth is less than the second sound leakage cancellation depth.

3. The method according to claim 1, characterized in that, During the process of outputting the first audio signal through the first sound-generating device, the impedance data of the first sound-generating device is the first impedance data; the method further includes: During the process of outputting the third audio signal through the first sound-generating device, the second impedance data of the first sound-generating device is acquired; If the second impedance data and the first impedance data meet the preset conditions, the second impedance data is input into the artificial intelligence model to obtain the third audio optimization parameters; The audio optimization parameters of the first filter are updated to the third audio optimization parameters to obtain the fifth audio signal; The first sound-generating device outputs a fifth audio signal, and the second sound-generating device continues to output a second audio signal. The sound leakage cancellation depth after the fifth audio signal and the second audio signal are superimposed in the sound leakage area is a fourth sound leakage cancellation depth, which is less than the first sound leakage cancellation depth.

4. The method according to claim 3, characterized in that, After obtaining the second impedance data of the first sound-generating device, the method further includes: If the second impedance data and the first impedance data do not meet the preset condition, the third audio signal continues to be output through the first sound-emitting device; while continuing to output the third audio signal through the first sound-emitting device, the impedance data of the first sound-emitting device continues to be acquired until the latest acquired impedance data of the first sound-emitting device meets the preset condition with the previously acquired impedance data of the first sound-emitting device.

5. The method according to claim 3 or 4, characterized in that, The preset conditions include at least one of the following: The difference between the frequency of the first resonance peak in the latest acquired impedance data of the first sound-generating device and the frequency of the first resonance peak in the previously acquired impedance data of the first sound-generating device is greater than or equal to a first threshold. The difference between the impedance of the first resonance peak in the latest acquired impedance data of the first sound-generating device and the impedance of the first resonance peak in the previous acquired impedance data of the first sound-generating device is greater than or equal to the second threshold. The difference between the frequency of the second resonance peak in the latest acquired impedance data of the first sound-generating device and the frequency of the second resonance peak in the previously acquired impedance data of the first sound-generating device is greater than or equal to the third threshold. The difference between the impedance of the second resonance peak in the latest acquired impedance data of the first sound-generating device and the impedance of the second resonance peak in the previously acquired impedance data of the first sound-generating device is greater than or equal to the fourth threshold.

6. The method according to any one of claims 1 to 4, characterized in that, The impedance data of the first sound-generating device includes: the impedance and frequency of the first resonance peak, and the impedance and frequency of the second resonance peak; The step of acquiring impedance data of the first sound-generating device during the process of outputting the first audio signal through the first sound-generating device includes: During the process of outputting the first audio signal through the first sound-generating device, the voltage and current values ​​of the first sound-generating device are obtained; Based on the voltage and current values ​​of the first sound-generating device, the impedance and frequency of the first resonance peak, as well as the impedance and frequency of the second resonance peak, are determined.

7. The method according to any one of claims 1 to 4, characterized in that, Before inputting the impedance data of the first sound-generating device into the artificial intelligence model, the method further includes: During the process of outputting the first audio signal through the first sound-generating device, the temperature of the first sound-generating device is obtained; based on the temperature of the first sound-generating device, the impedance data of the first sound-generating device is corrected.

8. The method according to any one of claims 1 to 4, characterized in that, The artificial intelligence model is obtained by training multiple impedance data and audio optimization parameters corresponding to the multiple impedance data. The target impedance data is the impedance data of the first sound-emitting device collected when the electronic device is in the target holding posture. The audio optimization parameters corresponding to the target impedance data are calculated based on the target transfer function, which is obtained after testing the sound energy of the sound leakage area when the electronic device is in the target holding posture. The target impedance data is any one of the plurality of impedance data.

9. The method according to claim 8, characterized in that, The method further includes: With the electronic device in the target holding posture, acquire the target impedance data and the audio optimization parameters corresponding to the target impedance data; After obtaining the multiple impedance data and the corresponding audio optimization parameters, the multiple impedance data and the corresponding audio optimization parameters are trained to obtain the artificial intelligence model.

10. An electronic device, characterized in that, The electronic device includes at least a first sound-generating device, a second sound-generating device, one or more processors, and a memory; The memory is coupled to one or more processors and is used to store computer program code, the computer program code including computer instructions, which the one or more processors invoke to cause the electronic device to perform the method as described in any one of claims 1 to 9.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes instructions that, when executed on an electronic device, cause the electronic device to perform the method as described in any one of claims 1 to 9.

Images