Audio signal processing method, electronic device, and storage medium

Abstract

Embodiments of the present application provide an audio signal processing method, an electronic device and a storage medium, wherein the method specifically comprises: obtaining a first signal collected by a sound collection device and a second signal played by a sound playing device; filtering, according to the first signal and the second signal, a signal related to the second signal from the first signal to obtain a third signal; filtering, according to the third signal and an output signal of a previous processing period, a signal related to the output signal of the previous processing period from the third signal to obtain an output signal of a next processing period. The embodiments of the present application can suppress howling to a certain extent, and further improve voice quality.

Description

Technical Field

[0001] This application relates to the field of communication technology, and in particular to an audio signal processing method, an electronic device, and a storage medium. Background Technology

[0002] With the development of communication technology, more and more users are using network communication to communicate. Currently, a common implementation method is: multiple clients establish connections with a server; audio signals sent by any client are transmitted to other clients via the server, and the client also plays the audio signals from other clients locally.

[0003] When multiple clients are physically close together, the interaction between the microphones and speakers used by the multiple clients can easily form a positive feedback loop, resulting in howling.

[0004] For example, if client A and client B are in the same room, the audio played by client A's speaker is picked up by client B's microphone, then sent to client A via the network, and finally played back by client A's speaker. A positive feedback loop with a gain greater than 1 will produce a piercing howling sound and may even cause network communication to be interrupted. Summary of the Invention

[0005] This application provides an audio signal processing method that can suppress feedback to a certain extent, thereby improving voice quality.

[0006] Correspondingly, embodiments of this application also provide an audio signal processing device, an electronic device, and a storage medium to implement and apply the above-described method.

[0007] To address the aforementioned problems, this application discloses an audio signal processing method, the method comprising:

[0008] Acquire the first signal collected by the sound acquisition device and the second signal played by the sound playback device;

[0009] Based on the first signal and the second signal, the signal related to the second signal is filtered out from the first signal to obtain the third signal;

[0010] Based on the third signal and the output signal of the previous processing cycle, signals related to the output signal of the previous processing cycle are filtered out from the third signal to obtain the output signal of the next processing cycle.

[0011] To address the aforementioned problems, this application discloses an audio signal processing apparatus, the apparatus comprising:

[0012] The signal acquisition module is used to acquire the first signal acquired by the sound acquisition device and the second signal played by the sound playback device.

[0013] The first filtering module is used to filter out signals related to the second signal from the first signal based on the first signal and the second signal to obtain a third signal;

[0014] The second filtering module is used to filter out signals related to the output signal of the previous processing cycle from the third signal based on the third signal and the output signal of the previous processing cycle, so as to obtain the output signal of the next processing cycle.

[0015] Optionally, the second filtering module includes:

[0016] The second related signal determination module is used to determine the second related signal corresponding to the output signal of the previous processing cycle based on the third signal and the output signal of the previous processing cycle.

[0017] The second correlation signal removal module is used to remove the second correlation signal from the third signal.

[0018] Optionally, the second related signal determination module is specifically used to process the output signal of the previous processing cycle according to the mapping relationship between the second related signal, the second weight parameter, and the output signal of the previous processing cycle, so as to obtain the second related signal corresponding to the output signal of the previous processing cycle; wherein, the second weight parameter of the subsequent processing cycle is obtained by updating the second weight parameter of the previous processing cycle according to the output signal of the previous processing cycle and the difference signal between the third signal and the second related signal sample.

[0019] Optionally, the second related signal is removed from the third signal to obtain the second residual signal;

[0020] The second filtering module further includes:

[0021] The second filtering module is used to filter the second residual signal according to the voice state corresponding to the first signal and the second signal to obtain the output signal of the next processing cycle.

[0022] Optionally, the device further includes:

[0023] The second alignment processing module is used to align the third signal and the output signal of the previous processing cycle before filtering out the signal related to the output signal of the previous processing cycle from the third signal.

[0024] Optionally, the first filtering module includes:

[0025] The first correlation signal determination module is used to determine the first correlation signal corresponding to the second signal based on the first signal and the second signal;

[0026] The first correlation signal removal module is used to remove the first correlation signal from the first signal.

[0027] Optionally, after removing the first related signal from the first signal, a first residual signal is obtained;

[0028] The first filtering module further includes:

[0029] The first filtering module is used to filter the first residual signal according to the voice states corresponding to the first signal and the second signal respectively, so as to obtain the third signal.

[0030] Optionally, the device further includes a transmitting module for transmitting the output signal of the next processing cycle to the communication peer.

[0031] To address the aforementioned problems, this application discloses an electronic device, including: a processor; and a memory storing executable code thereon, wherein when the executable code is executed, the processor performs the method as described in any of the above embodiments.

[0032] To address the aforementioned issues, embodiments of this application disclose one or more machine-readable media storing executable code thereon, which, when executed, causes a processor to perform the method as described in any of the above embodiments.

[0033] Compared with the prior art, the embodiments of this application have the following advantages:

[0034] In the technical solution of this application embodiment, the first signal may be a signal collected by the sound acquisition device of the first client, and the second signal may be a signal played by the sound playback device of the first client. Based on the first signal and the second signal, the first client in this application embodiment filters out signals related to the second signal from the first signal, and can filter out at least a portion of the signals played by the first client itself from the first signal.

[0035] Furthermore, in this embodiment, based on the third signal and the output signal of the previous processing cycle, signals related to the output signal of the previous processing cycle are filtered out from the third signal. According to the principle of howling generation, the output signal of the first client is transmitted to the second client via a network transmission path, played by the second client's speaker, and then transmitted to the first client via a spatial acoustic path, where it is picked up by the first client's microphone, thus generating howling; thus, the output signal of the first client can serve as the source signal of the howling phenomenon. Based on this, this embodiment filters out signals related to the output signal of the previous processing cycle from the third signal, thereby filtering out at least a portion of the source signals of the howling phenomenon from the third signal.

[0036] In summary, the embodiments of this application first filter out at least a portion of the signal played by the first client itself from the first signal, and then filter out at least a portion of the source signal of the howling phenomenon from the third signal, which can suppress howling to a certain extent and thus improve voice quality. Attached Figure Description

[0037] Figure 1 This is a schematic diagram of an audio signal processing scenario according to an embodiment of this application;

[0038] Figure 2 This is a schematic diagram of an audio signal processing scenario according to an embodiment of this application;

[0039] Figure 3 This is a flowchart of the steps of an audio signal processing method according to an embodiment of this application;

[0040] Figure 4 This is a schematic diagram of a process for filtering out signals related to a second signal from a first signal according to an embodiment of this application;

[0041] Figure 5 This is a schematic flowchart of an embodiment of the present application for filtering out signals related to the output signal of the previous processing cycle from a third signal;

[0042] Figure 6 This is a flowchart of the steps of an audio signal processing method according to an embodiment of this application;

[0043] Figure 7 This is a schematic diagram of the structure of an audio signal processing apparatus according to an embodiment of this application;

[0044] Figure 8 This is a schematic diagram of the structure of an exemplary device provided in one embodiment of this application. Detailed Implementation

[0045] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0046] In a sound processing system, an audio signal is amplified and emitted by a sound playback device such as a loudspeaker, and then captured by a sound acquisition device such as a microphone, forming a closed loop. When the gain of the closed loop is greater than or equal to 1, the closed loop is a positive feedback loop, which will produce a howling phenomenon.

[0047] With the development of mobile terminal technology, applications such as online conferencing and team-based gaming are becoming increasingly common. Feedback can severely impact voice quality, and excessive gain in the closed-loop circuit can damage audio processing devices such as sound playback and capture equipment. For example, in real-time gaming scenarios, when multiple users are playing together, a piercing feedback sound will occur if any two users are physically close together, degrading the gaming experience. Similarly, in real-time conferencing scenarios, if multiple users join the meeting simultaneously and any two users are physically close together, a piercing feedback sound will also occur.

[0048] The audio signal processing in this application embodiment is used to suppress howling to a certain extent, which is of great significance in sound processing systems and communication systems.

[0049] Reference Figure 1 The diagram illustrates an audio signal processing scenario according to an embodiment of this application, wherein client A and client B are in the same room, and the audio signal played by the speaker of client A is x. a (n), the audio signal collected by the microphone of client A is d a (n), d a (n) After being processed by the sound processing system A of client A, the output signal out is obtained. a (n); The audio signal played by the speaker of client B is x b (n), the audio signal collected by the microphone of client B is d b (n), d b (n) After being processed by the sound processing system B of client B, the output signal out is obtained. b (n).

[0050] An example of a positive feedback loop is: the output signal out of client B. b (n) is transmitted to client A via the network transmission path, played by client A's speaker, and then transmitted to client B via the spatial acoustic path; wherein, the spatial acoustic path can be the sound propagation path within the space where client A and client B are located.

[0051] This application provides an audio signal processing method, which specifically includes: acquiring a first signal acquired by a sound acquisition device and a second signal played by a sound playback device; filtering out signals related to the second signal from the first signal based on the first signal and the second signal to obtain a third signal; and filtering out signals related to the output signal of the previous processing cycle from the third signal based on the third signal and the output signal of the previous processing cycle to obtain the output signal of the next processing cycle.

[0052] This application embodiment can be applied to a first client, wherein the first signal may be a signal collected by the first client's sound acquisition device, and the second signal may be a signal played by the first client's sound playback device. Based on the first signal and the second signal, the first client in this application embodiment filters out signals related to the second signal from the first signal, and can filter out at least a portion of the signals played by the first client itself from the first signal.

[0053] Furthermore, in this embodiment, based on the third signal and the output signal of the previous processing cycle, signals related to the output signal of the previous processing cycle are filtered out from the third signal. According to the principle of howling generation, the output signal of the first client is transmitted to the second client via a network transmission path, played by the second client's speaker, and then transmitted to the first client via a spatial acoustic path, where it is picked up by the first client's microphone, thus generating howling; thus, the output signal of the first client can serve as the source signal of the howling phenomenon. Based on this, this embodiment filters out signals related to the output signal of the previous processing cycle from the third signal, thereby filtering out at least a portion of the source signals of the howling phenomenon from the third signal.

[0054] In summary, the embodiments of this application first filter out at least a portion of the signal played by the first client itself from the first signal, and then filter out at least a portion of the source signal of the howling phenomenon from the third signal, which can suppress howling to a certain extent and improve voice quality.

[0055] Reference Figure 2 This diagram illustrates an audio signal processing scenario according to an embodiment of the present application. Figure 2 Based on the audio signal processing scenario shown, the sound processing system A specifically includes: a first filtering module A and a second filtering module A. The first filtering module A can be used to process audio signals from d... a In (n), at least a portion of the signal played by client A itself is filtered out to obtain e. a (n); The second filtering module A can be used to filter from e a In (n), at least part of the source signal of the howling phenomenon is filtered out to obtain out. a (n).

[0056] exist Figure 2 Based on the audio signal processing scenario shown, the sound processing system B specifically includes: a first filtering module B and a second filtering module B. The first filtering module B can be used to process audio signals from d... b In (n), at least a portion of the signal played by client B itself is filtered out to obtain e. b (n); The second filtering module B can be used to filter from e b In (n), at least part of the source signal of the howling phenomenon is filtered out to obtain out. b (n).

[0057] Method Example 1

[0058] Reference Figure 3 The diagram illustrates a flowchart of an audio signal processing method according to an embodiment of this application, which may specifically include the following steps:

[0059] Step 301: Acquire the first signal acquired by the sound acquisition device and the second signal played by the sound playback device;

[0060] Step 302: Based on the first signal and the second signal, filter out the signal related to the second signal from the first signal to obtain the third signal;

[0061] Step 303: Based on the third signal and the output signal of the previous processing cycle, filter out the signals related to the output signal of the previous processing cycle from the third signal to obtain the output signal of the next processing cycle.

[0062] The audio signal processing method of this application embodiment can run on a first client. The first client can be any client. When the physical distance between the first client and the second client is relatively short, this application embodiment can suppress the feedback corresponding to the first client to a certain extent. It is understood that this application embodiment does not limit the specific entity executing the audio signal processing method.

[0063] In step 301, a first signal can be acquired using a sound acquisition device such as a microphone. A second signal can also be played using a sound playback device such as a speaker. The second signal can be an audio signal sent by the server or other clients. For example, in Figure 1 In the audio signal processing scenario shown, assuming the first client is client B and the second client is client A, the second signal can be the output signal "out" of client A. a (n). It is understood that the embodiments of this application do not limit the specific source of the second signal.

[0064] In step 302, based on the first signal and the second signal, signals related to the second signal are filtered out from the first signal, and at least a portion of the signals played by the first client itself can be filtered out from the first signal.

[0065] In a specific implementation, the process of filtering out signals related to the second signal from the first signal may include: determining a first related signal corresponding to the second signal based on the first signal and the second signal; and removing the first related signal from the first signal.

[0066] In this embodiment of the application, a first related signal y1(n) contained in the first signal d(n) can be determined based on the second signal x(n), and the first related signal y1(n) can be removed from the first signal d(n). After removing the first related signal y1(n) from the first signal d(n), a first residual signal e1(n) can be obtained, where e1(n) = d(n) - y1(n).

[0067] In practical applications, the process of determining the first related signal corresponding to the second signal may specifically include: processing the second signal according to the mapping relationship between the first related signal, the first weight parameter, and the second signal to obtain the first related signal corresponding to the second signal.

[0068] In this embodiment, the processing period can be the period during which the sound processing system of the first client processes the first signal and the second signal. The time length corresponding to the processing period can be determined by those skilled in the art according to actual application requirements. For example, the time length corresponding to the processing period can be the time length corresponding to M frames of the first signal, where M can be a positive integer.

[0069] This application embodiment can provide a mapping relationship between a first correlated signal, a first weighting parameter, and a second signal. Specifically, this mapping relationship can be a mapping relationship between the frequency domain feature Y1 of the first correlated signal y1(n), the first weighting parameter W1, and the frequency domain feature X of the second signal x(n). For example, the frequency domain feature Y1 of the first correlated signal y1(n) is obtained by multiplying the first weighting parameter W1 by the frequency domain feature X of the second signal x(n).

[0070] In practical applications, the value of the first weight parameter can be fixed. However, in this case, the matching degree between the first related signal and the first client's own playback signal may be low, resulting in the first residual signal still containing a large amount of the first client's own playback signal. In other words, the filtering effect of the first client's own playback signal is not good.

[0071] The value of the first weight parameter in this application embodiment can be continuously updated. Specifically, the first weight parameter of the subsequent processing cycle can be obtained by updating the first weight parameter of the previous processing cycle based on the second signal of the subsequent processing cycle and the difference signal between the first signal and the first related signal sample of the subsequent processing cycle. By updating the first weight parameter of the previous processing cycle based on the second signal of the subsequent processing cycle and the difference signal between the first signal and the first related signal sample, this application embodiment can make the first related signal y1(n) gradually approach the self-playing signal of the first client, thereby improving the filtering effect of the self-playing signal of the first client.

[0072] In practical applications, the difference between the first signal and the first related signal sample in the next processing cycle can be denoted as the first difference signal, which can be expressed as: e'1(n) = d(n) - y'1(n), where y'1(n) represents the first related signal sample in the nth processing cycle, which can be used in the update process of the first weight parameter. The value of the first weight parameter in the first processing cycle can be a preset value, and the first related signal sample in the first processing cycle can be a preset signal. The preset value and preset signal can be determined by those skilled in the art according to the actual application requirements. For example, the preset value can be zero or a historical value, and the preset signal can be a zero signal or a historical first related signal, etc. When the processing cycle is greater than 1, the first related signal sample in the nth processing cycle can be a historical first related signal, which can be provided by the second calculation unit.

[0073] In this embodiment, the first weight parameter W1(n-1) of the previous processing cycle can be updated based on the frequency domain features X of the second signal and the frequency domain features E1' of the first difference signal in the subsequent processing cycle, to obtain the first weight parameter W1(n) of the subsequent processing cycle. For example, the frequency domain features X of the second signal in the subsequent processing cycle can be multiplied by the frequency domain features E1' of the first difference signal and a preset parameter, and the corresponding first product result can be summed with the first weight parameter of the previous processing cycle to obtain the first weight parameter of the subsequent processing cycle; wherein, the first product result can represent the update amount of the first weight parameter.

[0074] The process of processing the second signal according to the mapping relationship between the first correlation signal, the first weight parameter, and the second signal in this embodiment is as follows: First, the frequency domain feature Y1 of the first correlation signal y1(n) of the next processing cycle is obtained by multiplying the first weight parameter W1(n) of the next processing cycle with the frequency domain feature X of the second signal x(n) of the next processing cycle; then, the frequency domain feature Y1 of the first correlation signal y1(n) of the next processing cycle is subjected to an inverse Fourier transform to obtain the first correlation signal y1(n) corresponding to the second signal.

[0075] In this embodiment of the application, after eliminating the first related signal from the first signal, a first residual signal can be obtained; then the process of filtering out the signal related to the second signal from the first signal may further include: filtering the first residual signal according to the voice state corresponding to the first signal and the second signal to obtain a third signal.

[0076] The voice state in this application embodiment can characterize whether the first signal and the second signal are simultaneously in a speaking state, that is, the user of the first signal is speaking while the second signal is in a playback state. The voice state can include a first voice state or a second voice state, where the first voice state indicates that the first signal and the second signal are simultaneously in a speaking state, and the second voice state indicates that the first signal and the second signal are not simultaneously in a speaking state. Based on the voice state, this application embodiment filters the first residual signal, which can further filter out its own playback signal from the first residual signal.

[0077] In a practical implementation, the first residual signal can be filtered according to a first suppression parameter. The first suppression parameter can be a filtering parameter of a first nonlinear filter, used to control the filtering strength of the first nonlinear filter.

[0078] Different speech states can correspond to different first suppression parameters. Specifically, the filtering strength of the first suppression parameter corresponding to the first speech state can be less than the filtering strength of the first suppression parameter corresponding to the second speech state.

[0079] When the first voice state indicates that both the first signal and the second signal are in a speaking state, this embodiment can use a smaller filtering strength to reduce the suppression of the user's speaking signal from the first client. Conversely, when the first voice state indicates that both the first signal and the second signal are in a speaking state, this embodiment can use a larger filtering strength to increase the suppression of its own playback signal. Since this embodiment can dynamically adjust the first suppression parameter according to the voice state, it can suppress its own playback signal as much as possible while preserving the user's speaking signal from the first client.

[0080] Reference Figure 4This diagram illustrates a flowchart of filtering signals related to a second signal from a first signal according to an embodiment of this application. The first and second calculation units perform linear filtering on the first signal. The first calculation unit updates the first weight parameter W1(n-1) of the previous processing cycle based on the first signal d(n), the second signal x(n), and the first related signal sample y'1(n) to obtain the first weight parameter W1(n) of the next processing cycle. The first weight parameter W1(n) of the next processing cycle is provided to the second calculation unit. The second calculation unit determines the first related signal y1(n) corresponding to the second signal based on the first weight parameter W1(n) of the next processing cycle and the second signal x(n). The first correlation signal y1(n) determined by the second calculation unit can be provided to the first calculation unit as the first correlation signal sample y'1(n) for the next processing cycle, so that the first calculation unit can iteratively update the first weight parameter; or, the first correlation signal y1(n) determined by the second calculation unit can be used to determine the first residual signal e1(n), which can be the difference signal between the first signal d(n) and the first correlation signal y1(n).

[0081] In practical applications, the process of updating the first weight parameter of the previous processing cycle can be an iterative update process. The first calculation unit can send the first weight parameter of the next processing cycle to the second calculation unit when the sending conditions are met. While sending the first weight parameter of the next processing cycle to the second calculation unit, the first calculation unit can continuously update the first weight parameter of the previous processing cycle. The sending conditions can be: the number of iterations reaches a first preset number, or the energy of the first difference signal meets a first preset condition, etc. The first preset condition can include: the energy of the first difference signal is less than the energy of the first signal d(n), or the difference between the energy of the first signal d(n) and the energy of the first difference signal is greater than a first difference threshold, etc.

[0082] The iterative update of the first weight parameter gives it the advantages of high real-time performance and high speed, but it is prone to divergence. To alleviate the divergence problem of the first weight parameter, the second computing unit in this embodiment can save the target first weight parameter sent by the first computing unit, and send a first reset instruction to the first computing unit when the energy of the first difference signal meets the first reset condition. The first reset instruction may include: the target first weight parameter saved by the second computing unit, so that the first computing unit resets the first weight parameter to the target first weight parameter, and iteratively updating the first weight parameter based on the target first weight parameter. The first reset condition may be: the energy of the first difference signal is greater than the energy of the first signal d(n), etc.

[0083] Figure 4 In the process, the first residual signal e1(n) can enter the first detection unit and the first filtering unit. The first detection unit can detect the speech state corresponding to the first signal and the second signal, and determine the first suppression parameter according to the speech state. The first filtering unit can filter the first residual signal e1(n) according to the first suppression parameter to obtain the third signal c(n).

[0084] In one implementation of this application, before filtering out signals related to the second signal from the first signal, the first signal and the second signal may be aligned. Assuming a filter is used to filter out signals related to the second signal from the first signal, the alignment process can reduce the filter order and improve the filter's convergence speed.

[0085] In practical applications, the start time of the first signal can be adjusted based on the start time of the second signal to align with the start time of the first signal. Alternatively, the start time of the second signal can be adjusted based on the start time of the first signal to align with the start time of the first signal.

[0086] In practical implementation, audio fingerprint matching algorithms or correlation algorithms can be used to align the first and second signals. Here, an audio fingerprint is a content-based digital signature representing the important acoustic features of an audio segment.

[0087] In step 303, signals related to the output signal of the previous processing cycle can be filtered out from the third signal based on the third signal and the output signal of the previous processing cycle. According to the principle of howling generation, the output signal of the first client is transmitted to the second client via a network transmission path, played by the second client's speaker, and then transmitted to the first client via a spatial acoustic path, where it is picked up by the first client's microphone, thus generating howling. Therefore, the output signal of the first client can serve as the source signal of the howling phenomenon. Based on this, this embodiment of the application filters out signals related to the output signal of the previous processing cycle from the third signal, thereby filtering out at least a portion of the source signal of the howling phenomenon from the third signal.

[0088] In a specific implementation, the process of filtering out signals related to the output signal of the previous processing cycle from the third signal may include: determining a second related signal corresponding to the output signal of the previous processing cycle based on the third signal and the output signal of the previous processing cycle; and removing the second related signal from the third signal.

[0089] In this embodiment of the application, the second related signal y2(n) contained in the third signal c(n) can be determined based on the output signal out(n-1) of the previous processing cycle, and the second related signal y2(n) can be removed from the third signal c(n). After removing the second related signal y2(n) from the third signal c(n), the second residual signal e2(n) can be obtained, e2(n) = c(n) - y2(n).

[0090] The process of determining the second related signal corresponding to the output signal of the previous processing cycle can specifically include: processing the output signal of the previous processing cycle according to the mapping relationship between the second related signal, the second weight parameter, and the output signal of the previous processing cycle to obtain the second related signal corresponding to the output signal of the previous processing cycle.

[0091] This application embodiment can provide a mapping relationship between the second correlated signal, the second weighting parameter, and the output signal of the previous processing cycle. Specifically, this mapping relationship can be a mapping relationship between the frequency domain feature Y2 of the second correlated signal y2(n), the second weighting parameter W2, and the frequency domain feature O of the output signal out(n-1) of the previous processing cycle. For example, the frequency domain feature Y2 of the second correlated signal y2(n) is obtained by multiplying the second weighting parameter W2 with the frequency domain feature O of the output signal out(n-1) of the previous processing cycle.

[0092] In practical applications, the value of the second weight parameter can be fixed. However, in this case, the matching degree between the second correlation signal and the output signal of the previous processing cycle may be low, resulting in the second residual signal still containing a large amount of the output signal of the previous processing cycle. In other words, the filtering effect of the output signal of the previous processing cycle (i.e., the suppression effect of howling) is not good.

[0093] The value of the second weight parameter in this application embodiment can be continuously updated. Specifically, the second weight parameter of the subsequent processing cycle can be obtained by updating the second weight parameter of the previous processing cycle based on the output signal of the subsequent processing cycle and the difference signal between the third signal and the second related signal sample of the subsequent processing cycle. By updating the second weight parameter of the previous processing cycle based on the output signal of the subsequent processing cycle and the difference signal between the third signal and the second related signal sample of the subsequent processing cycle, this application embodiment can make the second related signal y2(n) gradually approach the output signal of the previous processing cycle, thereby improving the filtering effect of the output signal of the previous processing cycle (i.e., the suppression effect of howling).

[0094] In practical applications, the difference between the third signal and the second related signal sample in the next processing cycle can be denoted as the second difference signal, which can be expressed as: e'2(n) = c(n) - y'2(n), where the value of the second weight parameter in the first processing cycle can be a preset value, and the second related signal in the first processing cycle can be a preset signal. The preset value and preset signal can be determined by those skilled in the art according to the actual application requirements; for example, the preset value can be zero or a historical value, and the preset signal can be a zero signal or a historical second related signal. y'2(n) represents the second related signal sample in the nth processing cycle, which can be used to update the second weight parameter. When the processing cycle is greater than 1, the second related signal sample in the nth processing cycle can be a historical second related signal, which can be provided by the fourth calculation unit.

[0095] In this embodiment, the second weighting parameter W2(n-1) of the previous processing cycle can be updated based on the frequency domain characteristics O of the output signal out(n-1) and the frequency domain characteristics E2' of the second difference signal in the previous processing cycle, to obtain the second weighting parameter W2(n) of the next processing cycle. For example, the frequency domain characteristics X of the second signal in the next processing cycle can be multiplied by the frequency domain characteristics E2' of the first difference signal and a preset parameter, and the corresponding second product result can be summed with the second weighting parameter of the previous processing cycle to obtain the second weighting parameter of the next processing cycle; wherein, the second product result can represent the update amount of the second weighting parameter.

[0096] The process of processing the output signal of the previous processing cycle based on the mapping relationship between the second correlation signal, the second weight parameter, and the output signal of the previous processing cycle in this embodiment is as follows: First, the frequency domain feature Y2 of the second correlation signal y2(n) of the next processing cycle is obtained by multiplying the second weight parameter W2(n) of the next processing cycle with the frequency domain feature O of the output signal of the previous processing cycle; then, the frequency domain feature Y2 of the second correlation signal y2(n) of the next processing cycle is subjected to an inverse Fourier transform to obtain the second correlation signal y2(n) corresponding to the output signal of the previous processing cycle.

[0097] In this embodiment of the application, after removing the second related signal from the third signal, a second residual signal can be obtained; then the process of filtering out the signal related to the output signal of the previous processing cycle from the third signal may further include: filtering the second residual signal according to the voice state corresponding to the first signal and the second signal to obtain the output signal of the next processing cycle.

[0098] The voice state in this application embodiment can characterize whether the first signal and the second signal are simultaneously in a speaking state, that is, the user of the first signal is speaking while the second signal is in a playback state. The voice state can include a first voice state or a second voice state, where the first voice state indicates that the first signal and the second signal are simultaneously in a speaking state, and the second voice state indicates that the first signal and the second signal are not simultaneously in a speaking state. Based on the voice state, this application embodiment filters the first residual signal, which can further filter out its own playback signal from the first residual signal.

[0099] In a practical implementation, the second residual signal can be filtered according to the second suppression parameter. The second suppression parameter can be a filtering parameter of the second nonlinear filter, used to control the filtering strength of the second nonlinear filter.

[0100] Different speech states can correspond to different second suppression parameters. Specifically, the filtering strength of the second suppression parameter corresponding to the first speech state can be less than the filtering strength of the second suppression parameter corresponding to the second speech state.

[0101] When the first voice state indicates that both the first and second signals are in a speaking state, this embodiment can use a smaller filtering strength to reduce the suppression strength of the user's speaking signal to the first client; while when the first voice state indicates that both the first and second signals are in a speaking state, this embodiment can use a larger filtering strength to increase the suppression strength of howling. Since this embodiment can dynamically adjust the second suppression parameter according to the voice state, howling can be suppressed as much as possible while preserving the user's speaking signal to the first client.

[0102] Reference Figure 5This diagram illustrates a flowchart of an embodiment of the present application for filtering signals related to the output signal of the previous processing cycle from a third signal. The third and fourth calculation units perform linear filtering on the third signal. The third calculation unit updates the second weight parameter W2(n-1) of the previous processing cycle based on the third signal c(n), the output signal out(n-1) of the previous processing cycle, and the second related signal sample y'2(n), to obtain the second weight parameter W2(n) of the next processing cycle. The second weight parameter W2(n) of the next processing cycle is then provided to the fourth calculation unit. The fourth calculation unit determines the second related signal y2(n) corresponding to the output signal out(n) of the previous processing cycle based on the second weight parameter W2(n) of the next processing cycle and the output signal out(n-1) of the previous processing cycle. The second correlation signal y2(n) determined by the fourth calculation unit can be used as the second correlation signal sample y'2(n) of the next processing cycle and provided to the third calculation unit so that the third calculation unit can perform iterative updates of the second weight parameter W2(n); or, the second correlation signal y2(n) determined by the fourth calculation unit can be used to determine the second residual signal e2(n), which can be the difference signal between the third signal c(n) and the second correlation signal y2(n).

[0103] In practical applications, the process of updating the second weight parameter of the previous processing cycle can be an iterative update process. The third calculation unit can send the second weight parameter of the next processing cycle to the fourth calculation unit when the sending conditions are met. While sending the second weight parameter of the next processing cycle to the fourth calculation unit, the third calculation unit can continuously update the second weight parameter of the previous processing cycle. The sending conditions can be: the number of iterations reaches a second preset number, or the energy of the second difference signal meets a second preset condition, etc. The second preset condition can include: the energy of the second difference signal is less than the energy of the third signal c(n), or the difference between the energy of the third signal c(n) and the energy of the second difference signal is greater than a second difference threshold, etc.

[0104] The iterative update of the second weight parameter gives it the advantages of high real-time performance and high speed, but it is prone to divergence. To alleviate the divergence problem, the fourth calculation unit in this embodiment can save the target second weight parameter sent by the third calculation unit, and send a second reset instruction to the third calculation unit when the energy of the second difference signal meets the second reset condition. The second reset instruction may include: the target second weight parameter saved by the fourth calculation unit, so that the third calculation unit resets the second weight parameter to the target second weight parameter, and iteratively updating the second weight parameter based on the target second weight parameter. The second reset condition may be: the energy of the second difference signal is greater than the energy of the third signal c(n), etc.

[0105] Figure 5 In the process, the second residual signal e2(n) can enter the detection unit and the second filtering unit. The second detection unit can detect the speech state corresponding to the first signal and the second signal, and determine the second suppression parameter according to the speech state. The second filtering unit can filter the second residual signal e2(n) according to the second suppression parameter to obtain the output signal out(n) of the next processing cycle.

[0106] In one implementation of this application, before filtering out signals related to the output signal of the previous processing cycle from the third signal, the third signal and the output signal of the previous processing cycle may be aligned. Assuming that a filter is used to filter out signals related to the output signal of the previous processing cycle from the third signal, the above alignment process can reduce the order of the filter and improve the convergence speed of the filter.

[0107] In practical applications, the start time of the third signal can be used as a reference to adjust the start time of the output signal of the previous processing cycle, so that the start time of the third signal is aligned with the start time of the output signal of the previous processing cycle. Alternatively, the start time of the output signal of the previous processing cycle can be used as a reference to adjust the start time of the third signal, so that the start time of the third signal is aligned with the start time of the output signal of the previous processing cycle.

[0108] After determining the output signal for the next processing cycle, the output signal for the next processing cycle can be sent to the communication peer. The communication peer can be a server or other clients. For example, in a web conferencing scenario, the first client can send the output signal for the next processing cycle to the server, so that the server can send the output signal provided by the first client to other clients, and the other clients can play the output signal provided by the first client. Since the embodiments of this application perform audio signal processing on the output signal provided by the first client, the audio quality of the playback signal on the other client side can be improved.

[0109] In summary, the audio signal processing method of this application embodiment allows the first client to filter out signals related to the second signal from the first signal based on the first signal and the second signal, thereby filtering out at least a portion of the signals played by the first client itself from the first signal.

[0110] Furthermore, in this embodiment, based on the third signal and the output signal of the previous processing cycle, signals related to the output signal of the previous processing cycle are filtered out from the third signal. According to the principle of howling generation, the output signal of the first client is transmitted to the second client via a network transmission path, played by the second client's speaker, and then transmitted to the first client via a spatial acoustic path, where it is picked up by the first client's microphone, thus generating howling; thus, the output signal of the first client can serve as the source signal of the howling phenomenon. Based on this, this embodiment filters out signals related to the output signal of the previous processing cycle from the third signal, thereby filtering out at least a portion of the source signals of the howling phenomenon from the third signal.

[0111] In summary, the embodiments of this application first filter out at least a portion of the signal played by the first client itself from the first signal, and then filter out at least a portion of the source signal of the howling phenomenon from the third signal, which can suppress howling to a certain extent and thus improve voice quality.

[0112] Method Example 2

[0113] Reference Figure 6 The diagram illustrates a flowchart of an audio signal processing method according to an embodiment of this application, which may specifically include the following steps:

[0114] Step 601: Acquire the first signal acquired by the sound acquisition device and the second signal played by the sound playback device;

[0115] Step 602: Align the first signal and the second signal;

[0116] Step 603: Based on the first signal and the second signal, determine the first related signal corresponding to the second signal, and remove the first related signal from the first signal to obtain the first residual signal;

[0117] Step 604: Based on the speech states corresponding to the first signal and the second signal, filter the first residual signal to obtain the third signal;

[0118] Step 605: Align the third signal with the output signal of the previous processing cycle;

[0119] Step 606: Based on the third signal and the output signal of the previous processing cycle, determine the second related signal corresponding to the output signal of the previous processing cycle, and remove the second related signal from the third signal to obtain the second residual signal;

[0120] Step 607: Based on the speech state corresponding to the first signal and the second signal, filter the second residual signal to obtain the output signal of the next processing cycle.

[0121] In the audio signal processing method of this application embodiment, step 603 can perform linear filtering on the first signal, step 604 can perform nonlinear filtering on the first residual signal, step 606 can perform linear filtering on the third signal, and step 607 can perform nonlinear filtering on the second residual signal, thereby achieving howling suppression.

[0122] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of this application are not limited to the described order of actions, because according to the embodiments of this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of this application.

[0123] Based on the above embodiments, this application also provides an audio signal processing apparatus, referring to... Figure 7 The device may include: a signal acquisition module 701, a first filtering module 702, and a second filtering module 703.

[0124] Among them, the signal acquisition module 701 is used to acquire the first signal acquired by the sound acquisition device and the second signal played by the sound playback device;

[0125] The first filtering module 702 is used to filter out signals related to the second signal from the first signal based on the first signal and the second signal to obtain a third signal;

[0126] The second filtering module 703 is used to filter out signals related to the output signal of the previous processing cycle from the third signal based on the third signal and the output signal of the previous processing cycle, so as to obtain the output signal of the next processing cycle.

[0127] Optionally, the second filtering module may include:

[0128] The second related signal determination module is used to determine the second related signal corresponding to the output signal of the previous processing cycle based on the third signal and the output signal of the previous processing cycle.

[0129] The second correlation signal removal module is used to remove the second correlation signal from the third signal.

[0130] Optionally, the second related signal determination module is specifically used to process the output signal of the previous processing cycle according to the mapping relationship between the second related signal, the second weight parameter, and the output signal of the previous processing cycle, so as to obtain the second related signal corresponding to the output signal of the previous processing cycle; wherein, the second weight parameter of the subsequent processing cycle is obtained by updating the second weight parameter of the previous processing cycle according to the output signal of the previous processing cycle and the difference signal between the third signal and the second related signal sample.

[0131] Optionally, the second related signal is removed from the third signal to obtain the second residual signal;

[0132] The second filtering module may further include:

[0133] The second filtering module is used to filter the second residual signal according to the voice state corresponding to the first signal and the second signal to obtain the output signal of the next processing cycle.

[0134] Optionally, the device may further include:

[0135] The second alignment processing module is used to align the third signal and the output signal of the previous processing cycle before filtering out the signal related to the output signal of the previous processing cycle from the third signal.

[0136] Optionally, the first filtering module may include:

[0137] The first correlation signal determination module is used to determine the first correlation signal corresponding to the second signal based on the first signal and the second signal;

[0138] The first correlation signal removal module is used to remove the first correlation signal from the first signal.

[0139] Optionally, after removing the first related signal from the first signal, a first residual signal is obtained;

[0140] The first filtering module may further include:

[0141] The first filtering module is used to filter the first residual signal according to the voice states corresponding to the first signal and the second signal respectively, so as to obtain the third signal.

[0142] Optionally, the device may further include: a transmitting module for transmitting the output signal of the next processing cycle to the communication peer.

[0143] In summary, the audio signal processing apparatus of this application embodiment allows the first client to filter out signals related to the second signal from the first signal based on the first signal and the second signal, thereby filtering out at least a portion of the signals played by the first client itself from the first signal.

[0144] Furthermore, in this embodiment, based on the third signal and the output signal of the previous processing cycle, signals related to the output signal of the previous processing cycle are filtered out from the third signal. According to the principle of howling generation, the output signal of the first client is transmitted to the second client via a network transmission path, played by the second client's speaker, and then transmitted to the first client via a spatial acoustic path, where it is picked up by the first client's microphone, thus generating howling; thus, the output signal of the first client can serve as the source signal of the howling phenomenon. Based on this, this embodiment filters out signals related to the output signal of the previous processing cycle from the third signal, thereby filtering out at least a portion of the source signals of the howling phenomenon from the third signal.

[0145] In summary, the embodiments of this application first filter out at least a portion of the signal played by the first client itself from the first signal, and then filter out at least a portion of the source signal of the howling phenomenon from the third signal, which can suppress howling to a certain extent and thus improve voice quality.

[0146] This application also provides a non-volatile readable storage medium storing one or more modules (programs). When these modules are applied to a device, they enable the device to execute the instructions for the method steps in this application.

[0147] This application provides one or more machine-readable media storing instructions that, when executed by one or more processors, cause an electronic device to perform one or more of the methods described in the above embodiments. In this application, the electronic device includes devices such as servers and terminal devices.

[0148] Embodiments of this disclosure can be implemented as an apparatus with any suitable hardware, firmware, software, or any combination thereof, configured as desired, and the apparatus may include electronic devices such as servers (clusters) and terminals. Figure 8An exemplary apparatus 1700 is schematically shown that can be used to implement the various embodiments described in this application.

[0149] In one embodiment, Figure 8 An exemplary device 1700 is shown, which includes one or more processors 1702, a control module (chipset) 1704 coupled to at least one of the processors 1702, a memory 1706 coupled to the control module 1704, a non-volatile memory (NVM) / storage device 1708 coupled to the control module 1704, one or more input / output devices 1710 coupled to the control module 1704, and a network interface 1712 coupled to the control module 1704.

[0150] Processor 1702 may include one or more single-core or multi-core processors, and processor 1702 may include any combination of general-purpose processors or special-purpose processors (e.g., graphics processors, application processors, baseband processors, etc.). In some embodiments, device 1700 can serve as a server, terminal, or other device as described in the embodiments of this application.

[0151] In some embodiments, apparatus 1700 may include one or more computer-readable media (e.g., memory 1706 or NVM / storage device 1708) having instructions 1714 and one or more processors 1702 that are combined with the one or more computer-readable media and configured to execute instructions 1714 to implement a module thereby performing the actions described in this disclosure.

[0152] In one embodiment, the control module 1704 may include any suitable interface controller to provide any suitable interface to at least one of the processors 1702 and / or any suitable device or component communicating with the control module 1704.

[0153] The control module 1704 may include a memory controller module to provide an interface to the memory 1706. The memory controller module may be a hardware module, a software module, and / or a firmware module.

[0154] Memory 1706 may be used, for example, to load and store data and / or instructions 1714 for device 1700. In one embodiment, memory 1706 may include any suitable volatile memory, such as suitable DRAM. In some embodiments, memory 1706 may include double data rate type quad synchronous dynamic random access memory (DDR4 SDRAM).

[0155] In one embodiment, the control module 1704 may include one or more input / output controllers to provide an interface to the NVM / storage device 1708 and (one or more) input / output devices 1710.

[0156] For example, NVM / storage device 1708 may be used to store data and / or instructions 1714. NVM / storage device 1708 may include any suitable non-volatile memory (e.g., flash memory) and / or may include any suitable (one or more) non-volatile storage devices (e.g., one or more hard disk drives (HDDs), one or more optical disc drives (CDs), and / or one or more digital universal optical disc (DVD) drives).

[0157] NVM / storage device 1708 may include storage resources that are part of a device on which device 1700 is mounted, or that are accessible by the device but do not necessarily have to be part of the device. For example, NVM / storage device 1708 may be accessed via a network via one or more input / output devices 1710.

[0158] One or more input / output devices 1710 may provide an interface for device 1700 to communicate with any other suitable device. Input / output devices 1710 may include communication components, audio components, sensor components, etc. Network interface 1712 may provide an interface for device 1700 to communicate via one or more networks. Device 1700 may wirelessly communicate with one or more components of a wireless network according to any of one or more wireless network standards and / or protocols, such as accessing a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G, 5G, etc., or a combination thereof.

[0159] In one embodiment, at least one of the processors 1702 may be logically packaged with one or more controllers (e.g., memory controller modules) of the control module 1704. In one embodiment, at least one of the processors 1702 may be logically packaged with one or more controllers of the control module 1704 to form a system-in-package (SiP). In one embodiment, at least one of the processors 1702 may be integrated with the logic of one or more controllers of the control module 1704 on the same die. In one embodiment, at least one of the processors 1702 may be integrated with the logic of one or more controllers of the control module 1704 on the same die to form a system-on-a-chip (SoC).

[0160] In various embodiments, device 1700 may be, but is not limited to, a terminal device such as a server, desktop computing device, or mobile computing device (e.g., laptop computing device, handheld computing device, tablet computer, netbook, etc.). In various embodiments, device 1700 may have more or fewer components and / or different architectures. For example, in some embodiments, device 1700 includes one or more cameras, a keyboard, a liquid crystal display (LCD) screen (including a touch screen display), a non-volatile memory port, multiple antennas, a graphics chip, an application-specific integrated circuit (ASIC), and a speaker.

[0161] In the device 1700, a main control chip can be used as a processor or control module, sensor data, position information, etc. are stored in a memory or NVM / storage device, the sensor group can be used as an input / output device, and the communication interface can include a network interface.

[0162] This application also provides an electronic device, including: a processor; and a memory storing executable code thereon, which, when executed, causes the processor to perform one or more methods as described in this application.

[0163] This application also provides one or more machine-readable media having executable code stored thereon, which, when executed, causes a processor to perform one or more of the methods described in this application.

[0164] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0165] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0166] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more blocks of the flowchart illustrations and / or one or more blocks of the block diagrams.

[0167] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more blocks of a block diagram.

[0168] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable terminal apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable terminal apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more blocks of a block diagram.

[0169] Although preferred embodiments of the present application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present application.

[0170] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0171] The above provides a detailed description of an audio signal processing method, an audio signal processing device, an electronic device, and a storage medium provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. An audio signal processing method, characterized in that, The method includes: Acquire the first signal collected by the sound acquisition device and the second signal played by the sound playback device; Based on the first signal and the second signal, the signal related to the second signal is filtered out from the first signal to obtain the third signal; Based on the third signal and the output signal of the previous processing cycle, signals related to the output signal of the previous processing cycle are filtered out from the third signal to obtain the output signal of the next processing cycle; the processing cycle is the cycle in which the sound processing system of the first client processes the first signal and the second signal; after determining the output signal of the next processing cycle, the output signal of the next processing cycle can be sent to the communication peer. The step of filtering out signals related to the output signal of the previous processing cycle from the third signal includes: Based on the mapping relationship between the second correlation signal, the second weight parameter, and the output signal of the previous processing cycle, the output signal of the previous processing cycle is processed to obtain the second correlation signal corresponding to the output signal of the previous processing cycle; wherein, the second weight parameter of the subsequent processing cycle is obtained by updating the second weight parameter of the previous processing cycle based on the output signal of the previous processing cycle and the difference signal between the third signal and the second correlation signal sample. Remove the second related signal from the third signal.

2. The method according to claim 1, characterized in that, The second related signal is removed from the third signal to obtain the second residual signal; The step of filtering out signals related to the output signal of the previous processing cycle from the third signal further includes: Based on the speech states corresponding to the first and second signals, the second residual signal is filtered to obtain the output signal of the next processing cycle.

3. The method according to claim 1, characterized in that, The method further includes: Before filtering out signals related to the output signal of the previous processing cycle from the third signal, the third signal and the output signal of the previous processing cycle are aligned.

4. The method according to any one of claims 1 to 3, characterized in that, The step of filtering out signals related to the second signal from the first signal includes: Based on the first signal and the second signal, determine the first related signal corresponding to the second signal; Remove the first related signal from the first signal.

5. The method according to claim 4, characterized in that, After removing the first related signal from the first signal, the first residual signal is obtained; The step of filtering out signals related to the second signal from the first signal further includes: Based on the speech states corresponding to the first signal and the second signal respectively, the first residual signal is filtered to obtain the third signal.

6. The method according to any one of claims 1 to 3, characterized in that, The method further includes: sending the output signal of the next processing cycle to the communication peer.

7. An audio signal processing device, characterized in that, The device includes: The signal acquisition module is used to acquire the first signal acquired by the sound acquisition device and the second signal played by the sound playback device. The first filtering module is used to filter out signals related to the second signal from the first signal based on the first signal and the second signal to obtain a third signal; The second filtering module is used to filter out signals related to the output signal of the previous processing cycle from the third signal based on the third signal and the output signal of the previous processing cycle, so as to obtain the output signal of the next processing cycle; the processing cycle is the cycle in which the sound processing system of the first client processes the first signal and the second signal; after determining the output signal of the next processing cycle, the output signal of the next processing cycle can be sent to the communication peer. The step of filtering out signals related to the output signal of the previous processing cycle from the third signal includes: Based on the mapping relationship between the second correlation signal, the second weight parameter, and the output signal of the previous processing cycle, the output signal of the previous processing cycle is processed to obtain the second correlation signal corresponding to the output signal of the previous processing cycle; wherein, the second weight parameter of the subsequent processing cycle is obtained by updating the second weight parameter of the previous processing cycle based on the output signal of the previous processing cycle and the difference signal between the third signal and the second correlation signal sample. Remove the second related signal from the third signal.

8. An electronic device, characterized in that, include: processor; and A memory having executable code stored thereon, which, when executed, causes the processor to perform the method as described in any one of claims 1-6.

9. One or more machine-readable media having executable code stored thereon, which, when executed, causes a processor to perform the method as described in any one of claims 1-6.

Images