Evaluation Method, Device and Electronic Device for Pickup Function of Terminal

Abstract

Embodiments of the present application provide a method, device, and electronic device for evaluating the voice pickup function of a terminal, which relate to the technical field of voice processing. The method includes: obtaining a playback signal of the terminal and a voice pickup signal of the playback signal, determining a target signal sequence of at least one frame of first signal according to the voice pickup signal, determining an associated signal of the first signal according to the frame position of the first signal in the voice pickup signal, determining the correlation between each frame of the first signal in the voice pickup signal and the associated signal of the first signal in the playback signal, and evaluating the voice pickup function of the terminal according to the correlation corresponding to each frame of the first signal to determine whether the voice pickup function of the terminal fails. Based on the solution provided by the embodiments of the present application, the voice pickup function of the terminal can be evaluated simply and quickly, the accuracy and robustness of the evaluation of the voice pickup function of the terminal can be effectively improved, and the practical requirements can be better met.

Description

Technical Field

[0001] The present application relates to the field of voice technology. Specifically, the present application relates to a method, device, electronic device, computer-readable storage medium, and computer program product for evaluating the voice pickup function of a terminal. Background Art

[0002] As the usage time of the terminal increases, the microphones of some terminals may malfunction, resulting in the inability to collect normal sound signals, that is, the voice pickup function of the terminal fails. Among them, the reason for the failure of the voice pickup function of the terminal may be hardware damage to the microphone, or the underlying audio algorithm design of the terminal may fail, resulting in the inability to obtain real sound signals, causing problems such as the user being unable to make normal calls.

[0003] Currently, in the related art, for the evaluation of the voice pickup function of a terminal, usually on the premise that the reason for the failure of the voice pickup function of the terminal is that the energy of the voice pickup signal is very small, the energy of the voice pickup signal is used to determine whether the voice pickup function of the terminal fails. However, the accuracy of this method is relatively low, and it is easy to produce false positives and false negatives, which cannot meet the practical needs. Summary of the Invention

[0004] Embodiments of the present application provide a method, device, electronic device, computer-readable storage medium, and computer program product for evaluating the voice pickup function of a terminal, which can simply and quickly evaluate the voice pickup function of the terminal, and can effectively improve the accuracy and robustness of the evaluation of the voice pickup function of the terminal, better meeting the practical needs.

[0005] According to one aspect of the embodiments of the present application, there is provided a method for evaluating the voice pickup function of a terminal, the method including:

[0006] Obtain the playback signal of the terminal and the voice pickup signal of the playback signal;

[0007] Determine a target signal sequence according to the voice pickup signal, the target signal sequence including at least one frame of first signal in the voice pickup signal;

[0008] According to the frame position of the first signal in the voice pickup signal, determine the associated signal of the first signal. The associated signal of one first signal includes M frames of second signals in the playback signal that are before the signal corresponding to the first signal, and the signal in the playback signal that is at the same frame position as the first signal corresponding to the first signal, where M≥1;

[0009] For each frame of first signal in the voice pickup signal, determine the correlation degree between the first signal and the associated signal of the first signal in the playback signal;

[0010] Determine whether the sound pickup function of the terminal fails according to the correlation corresponding to the first signal of each frame.

[0011] According to another aspect of the embodiments of the present application, there is provided an evaluation device for the sound pickup function of a terminal, the device includes a signal acquisition module, a correlation determination module, and a failure determination module, wherein:

[0012] The signal acquisition module is used to acquire the playback signal of the terminal and the sound pickup signal of the playback signal;

[0013] The target signal sequence determination module is used to determine a target signal sequence according to the sound pickup signal, and the target signal sequence includes at least one frame of the first signal in the sound pickup signal;

[0014] The correlation determination module is used to determine the associated signal of the first signal according to the frame position of the first signal in the sound pickup signal. The associated signal of a first signal includes M frames of second signals in the playback signal that are before the signal corresponding to the first signal, and the signal in the playback signal corresponding to the first signal is the signal in the playback signal with the same frame position as the first signal, where M≥1;

[0015] For each frame of the first signal in the sound pickup signal, determine the correlation between the first signal and the associated signal of the first signal in the playback signal;

[0016] The failure determination module is used to determine whether the sound pickup function of the terminal is normal according to the correlation corresponding to each frame of the first signal.

[0017] Optionally, for each frame of the first signal in the sound pickup signal, when the correlation determination module determines the correlation between the first signal and the associated signal of the first signal in the playback signal, it specifically is used for:

[0018] Determine the correlation between the first signal and each frame of the second signal in the associated signal of the first signal;

[0019] When the failure determination module determines whether the sound pickup function of the terminal is normal according to the correlation corresponding to each frame of the first signal, it specifically is used for:

[0020] According to the correlations corresponding to each frame of the first signal, determine the correlations corresponding to each frame position offset in the correlations corresponding to each frame of the first signal, where the frame position offset refers to the offset between a frame of the first signal and a frame of the second signal in the associated signal of the first signal;

[0021] For each frame position offset among the M frame position offsets, determine the first quantity of the correlations greater than or equal to the first threshold in the correlations corresponding to the frame position offset;

[0022] Determine whether the sound pickup function of the terminal is normal according to the first quantity corresponding to each frame position offset among the M frame position offsets.

[0023] Optionally, when determining whether the sound pickup function of the terminal is normal according to the first quantity corresponding to each frame position offset among the M frame position offsets, the failure determination module is used for:

[0024] If there is at least one first quantity among the first quantities corresponding to each frame position offset among the M frame position offsets that satisfies the preset condition, the sound pickup function of the terminal is normal;

[0025] If none of the first quantities corresponding to each frame position offset among the M frame position offsets satisfies the preset condition, the sound pickup function of the terminal fails;

[0026] Among them, the preset condition includes:

[0027] The first quantity is greater than or equal to the second threshold, and the ratio of the first quantity to the reference value is greater than or equal to the third threshold;

[0028] Among them, the reference value represents the degree of interference of the first signal.

[0029] Optionally, the correlation determination module is further used for: determining the mean value of the first quantities corresponding to each frame position offset among the M frame position offsets, and using the mean value as the reference value.

[0030] Optionally, for each frame of the first signal in the sound pickup signal, the correlation between the first signal and the associated signal of the first signal in the playback signal includes the correlation between the first signal and each frame of the second signal in the associated signal;

[0031] For each frame of the first signal in the sound pickup signal, when the correlation determination module determines the correlation between the first signal and the associated signal of the first signal in the playback signal, it is used for:

[0032] Determine the first frequency domain expression information of the first signal and the second frequency domain expression information of each frame of the second signal in the associated signal of the first signal;

[0033] For each frame of the second signal in the associated signal, based on the first frequency domain expression information and the second frequency domain expression information of the second signal, obtain the correlation between the first signal and the second signal.

[0034] Optionally, when the correlation determination module determines the first frequency domain expression information of the first signal and the second frequency domain expression information of each frame of the second signal in the associated signal of the first signal, it includes:

[0035] Obtain the first spectrum of the first signal and the second spectra of each frame of the second signal, and both the first spectrum and the second spectrum include the amplitude values of multiple frequency points;

[0036] Based on the first spectrum, determine the frequency-domain expression information of each frequency point of the first signal, and based on the second spectrum of each frame of the second signal, determine the frequency-domain expression information of each frequency point of each frame of the second signal;

[0037] Wherein, the first frequency-domain expression information includes the frequency-domain expression information of multiple frequency points of the first signal, and the second frequency-domain expression information of one frame of the second signal includes the frequency-domain expression information of multiple frequency points of the second signal.

[0038] Optionally, for each frame of the second signal in the associated signals, when the correlation determination module obtains the correlation between the first signal and the second signal based on the first frequency-domain expression information and the second frequency-domain expression information of the second signal, it is used for:

[0039] Determine the information dispersion degree of the first signal according to the frequency-domain expression information of multiple frequency points of the first signal;

[0040] Determine the information dispersion degree of the second signal according to the frequency-domain expression information of multiple frequency points of the second signal;

[0041] Determine the overall deviation between the first signal and the second signal according to the frequency-domain expression information of multiple frequency points of the first signal and the frequency-domain expression information of multiple frequency points of the second signal;

[0042] Determine the correlation between the first signal and the second signal according to the information dispersion degree of the first signal, the information dispersion degree of the second signal, and the overall deviation between the first signal and the second signal.

[0043] Optionally, when the correlation determination module determines the information dispersion degree of the first signal according to the frequency-domain expression information of multiple frequency points of the first signal, it is specifically used for:

[0044] Determine the first mean value of the spectral expression information of multiple frequency points in the first signal;

[0045] Determine the information dispersion degree of the first signal according to the spectral expression information of multiple frequency points in the first signal and the first mean value;

[0046] When the correlation determination module determines the information dispersion degree of the second signal according to the frequency-domain expression information of multiple frequency points of the second signal, it is specifically used for:

[0047] Determine the second mean value of the spectral expression information of multiple frequency points in the second signal;

[0048] Determine the information dispersion degree of the second signal according to the spectral expression information of multiple frequency points in the second signal and the second mean value;

[0049] When determining the overall deviation between the first signal and the second signal based on the frequency-domain expression information of multiple frequency points of the first signal and the frequency-domain expression information of multiple frequency points of the second signal, the relevance determination module is specifically configured to:

[0050] Determine the difference between the spectral expression information of each frequency point in the first signal and the first mean value;

[0051] Determine the difference between the spectral expression information of each frequency point in the second signal and the second mean value;

[0052] Determine the overall deviation between the first signal and the second signal according to the difference between the spectral expression information of each frequency point in the first signal and the first mean value and the difference between the spectral expression information of each frequency point in the second signal and the second mean value.

[0053] Optionally, both the first frequency-domain expression information and the second frequency-domain information expression include at least one of the power spectrum, amplitude spectrum, logarithmic power spectrum, or logarithmic amplitude spectrum of the signal.

[0054] Optionally, the playback signal is any one of the following: a test signal; a specified signal; a call signal of the terminal.

[0055] According to another aspect of the embodiments of the present application, an electronic device is provided. The electronic device includes: a memory, a processor, and a computer program stored on the memory. The processor executes the computer program to implement the steps of the method for evaluating the voice pickup function of the above terminal.

[0056] According to still another aspect of the embodiments of the present application, a computer-readable storage medium is provided. A computer program is stored on the computer-readable storage medium. When the computer program is executed by a processor, the steps of the method for evaluating the voice pickup function of the above terminal are implemented.

[0057] According to still another aspect of the embodiments of the present application, a computer program product is provided. The computer program product includes a computer program. When the computer program is executed by a processor, the steps of the method for evaluating the voice pickup function of the above terminal are implemented.

[0058] The beneficial effects brought by the technical solutions provided in the embodiments of the present application are:

[0059] The evaluation method provided in the embodiments of the present application uses the correlation between each frame signal in the voice pickup signal of the playback signal and the associated signal in the playback signal corresponding to each frame signal to evaluate whether the voice pickup function of the terminal is normal. Since the voice pickup signal is collected according to the playback signal, therefore, by analyzing the correlation between the playback signal and the voice pickup signal of the playback signal, the evaluation of the voice pickup function of the terminal can be realized, and the accuracy and robustness of the evaluation of the voice pickup function of the terminal can be effectively improved. In addition, the method is convenient and fast to implement, and can better meet the actual application requirements. Brief Description of the Drawings

[0060] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments of the present application.

[0061] Figure 1 The figure shows a schematic diagram of a scenario embodiment applicable to the present application.

[0062] Figure 2 The figure shows a flowchart of a method for evaluating the voice pickup function of a terminal in a scenario embodiment applicable to the present application.

[0063] Figure 3 The figure shows a flowchart of a method for evaluating the voice pickup function of a terminal provided in an embodiment of the present application.

[0064] Figure 4 The figure shows a schematic diagram of a playback signal and a voice pickup signal in an embodiment of the present application.

[0065] Figure 5 The figure shows a schematic diagram of an apparatus for evaluating the voice pickup function of a terminal provided in an embodiment of the present application.

[0066] Figure 6 The figure shows a schematic diagram of an electronic device provided in an embodiment of the present application. Detailed Description of the Embodiments

[0067] The following describes the embodiments of the present application in conjunction with the drawings in the present application. It should be understood that the embodiments described below in conjunction with the drawings are exemplary descriptions for explaining the technical solutions of the embodiments of the present application, and do not constitute a limitation to the technical solutions of the embodiments of the present application.

[0068] Those skilled in the art of the present technology can understand that, unless specifically stated otherwise, the singular forms "a", "an", "the" and "said" used herein may also include the plural forms. It should be further understood that the terms "comprise" and "include" used in the embodiments of the present application mean that the corresponding features can be implemented as the presented features, information, data, steps, operations, elements and / or components, but do not exclude the implementation of other features, information, data, steps, operations, elements, components and / or their combinations supported by the art of the present technology. It should be understood that when we say that an element is "connected" or "coupled" to another element, the one element can be directly connected or coupled to the other element, or it can mean that the one element and the other element establish a connection relationship through an intermediate element. In addition, the "connection" or "coupling" used herein may include a wireless connection or a wireless coupling. The term "and / or" used herein indicates at least one of the items defined by the term, for example, "A and / or B" indicates being implemented as "A", or being implemented as "B", or being implemented as "A and B".

[0069] The evaluation method for the voice pickup function of the terminal provided by the embodiments of the present application can be implemented based on speech technology. Among them, the voice pickup function of the terminal is the ability of the terminal to collect sounds, and the collected sounds can be voice pickup signals. The present application does not limit the specific method of collecting sounds. For example, the collection of sounds can be achieved through the recording module of the terminal itself. For example, for each frame of the first signal in the voice pickup signal of the playback signal of the terminal, determine the correlation between the first signal and the associated signal of the first signal in the playback signal, and evaluate the voice pickup function of the terminal according to the correlation corresponding to each frame of the first signal to determine whether the voice pickup function of the terminal fails. Among them, the key technologies of speech technology include automatic speech recognition technology, speech synthesis technology, and voiceprint recognition technology. Enabling the computer to listen, see, speak, and feel is the future development direction of human-computer interaction, and voice has become one of the most promising human-computer interaction methods in the future. With the research and progress of speech technology, speech technology has been widely studied and applied in many fields. It is believed that with the development of technology, speech technology will be applied in more fields and play an increasingly important role.

[0070] In the embodiments of the present application, the specific forms of the voice pickup module (i.e., the voice signal collection module) and the playback module (i.e., the voice signal playback module) in the terminal are not limited. The voice pickup module may include, but is not limited to, a microphone, and may also be other types of signal collection modules. The playback module may include, but is not limited to, a speaker. For the convenience of description, in the following, the voice pickup module is a microphone and the playback module is a speaker as an example for illustration.

[0071] The solution provided by the embodiments of the present application can be applied to any terminal with the functions of voice signal collection and voice signal playback (i.e., having a voice pickup module and a playback module), and can collect voices through the voice pickup module while playing voices through the playback module. The terminal may include, but is not limited to, a computer, an ipad, a mobile phone, a recording pen, an MP3, a smart voice interaction device, a vehicle-mounted terminal, etc. The embodiments of the present invention can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, intelligent transportation, assisted driving, etc. Moreover, the evaluation of the voice pickup function of the terminal provided by the embodiments of the present application as a parameter for whether the terminal meets the user's usage requirements can be applied to the terminal factory inspection process to ensure that the voice pickup functions of all factory-produced terminals are normal. It can also be applied during the user's use of the terminal. By triggering the relevant plug-ins in the used terminal, the voice pickup function of the terminal can be evaluated to detect whether the voice pickup function of the used terminal fails.

[0072] In practical applications, as the service life of a terminal increases, the microphones of some terminals may malfunction, resulting in the inability of the microphones to collect voice signals normally. Or during the factory inspection of a terminal, the microphone of the terminal is often also inspected to prevent any abnormalities in the microphone of the terminal from affecting the user experience. Generally, to prevent the user from being unable to make a normal call, after detecting the current actual state of the microphone, if the diagnosis result is that the microphone is abnormal or an abnormal voice pickup result occurs, the detection result will be reported for troubleshooting or fault transfer, so as to ensure that the user can continue to use the terminal for calls or clearly inform the user of the cause of the fault for subsequent repair. For example, fault transfer can be achieved by activating other backup microphones in the terminal.

[0073] However, in related technologies, based on the premise that the assumed cause of the terminal voice pickup failure is that the energy of the voice pickup signal is very small, the method of determining whether the terminal voice pickup fails according to the energy of the voice pickup signal has a low accuracy of the result of whether the terminal voice pickup fails, and this method is prone to misjudgment, missed judgment, etc., and cannot meet the practical requirements.

[0074] Based on this, the present application provides an evaluation method for the voice pickup function of a terminal, which is implemented based on the correlation between the playback signal and the voice pickup signal. By playing a sound signal (hereinafter also referred to as the playback signal) using the terminal's own speaker while using the terminal's own microphone to collect the sound signal played by the speaker (hereinafter also referred to as the voice pickup signal), the method determines the M frames of the second signals before the signal corresponding to each frame of the first signal in the playback signal, and evaluates the voice pickup function of the terminal according to the correlation between the first signal and each frame of the second signals, so as to determine whether the voice pickup function of the terminal fails, improving the accuracy and robustness of the evaluation of the voice pickup function of the terminal. In addition, this method is convenient and fast to implement and can better meet the actual application requirements.

[0075] The technical solutions of the embodiments of the present application and the technical effects produced by the technical solutions of the present application will be described below through the description of several exemplary embodiments. It should be noted that the following embodiments can be referenced, borrowed, or combined with each other. For the same terms, similar features, and similar implementation steps in different embodiments, they will not be described repeatedly.

[0076] To better illustrate the practicality of the solution of the embodiments of the present application, the solution of the embodiments of the present application will be described below in combination with the specific application scenario of the evaluation method for the voice pickup function of the terminal. This application scenario can be to collect the sound (i.e., the playback signal) played by the terminal's speaker using the terminal's microphone to obtain the voice pickup signal, and then evaluate the voice pickup function of the terminal according to the played sound and the voice pickup signal to determine whether the voice pickup function of the terminal is normal. Specifically:

[0077] Figure 1 A schematic diagram showing a scenario embodiment applicable to the present application is shown. Figure 2 A flowchart of an evaluation method for the sound pickup function of a terminal showing a scenario embodiment applicable to the present application is shown. As Figure 1 shown, taking the terminal 10 as a mobile phone and the playback signal as a test signal as an example, a plugin 11 for evaluating the sound pickup function of the terminal is provided on the terminal 10. By triggering this plugin, the terminal can be controlled to play the test signal through the speaker 111, collect the sound pickup signal through the microphone 112, and evaluate the sound pickup function of the terminal. The following will specifically describe the execution process of the terminal for evaluating the sound pickup function of the terminal of the present application, where this execution process can be implemented by a processor in the terminal. Figure 2 Specifically describe the execution process of the terminal for evaluating the sound pickup function of the terminal of the present application. Among them, this execution process can be implemented by a processor in the terminal.

[0078] Step S1: After detecting a triggering operation by the user on the plugin 11 of the terminal, play a test signal through the speaker 111 and collect a sound pickup signal through the microphone 112.

[0079] Step S2: Calculate, for each frequency point j within the preset frequency point range N1 to N2, the power spectrum X(i, j) of each frame (i.e., the i-th frame) of the test signal at this frequency point, and the first power spectrum D(i, j) of each frame (i.e., the i-th frame) of the first signal in the sound pickup signal at this frequency point. Among them, a corresponding frequency range can be selected as multiple frequency points with a starting frequency of 800 hz (hertz) to an ending frequency of 3.5 khz. Among these multiple frequency points, the smallest frequency point serial number is N1, and the largest frequency point serial number is N2.

[0080] Optionally, before detecting a sound pickup signal corresponding to each frame of the test signal, the power spectra of each frame of the test signal at multiple frequency points can be cached in a buffer area.

[0081] Step S3: Set M to 100, k ∈ [1, 100], determine the signal in the playback signal that has the same frame position as each frame of the first signal in the sound pickup signal. Assume that a certain frame of the first signal in the sound pickup signal is associated with the first k frames of the second signal (hereinafter referred to as the corresponding second signal) in the playback signal that has the same frame position as this first signal, then the second power spectrum X(i - k, j) of the corresponding second signal in the playback signal at this frequency point can be read from the buffer area.

[0082] According to the first power spectrum D(i, j) of each frame of the first signal in the sound pickup signal at this frequency point j and the second power spectrum X(i - k, j) of each frame of the second signal in the playback signal at this frequency point j, calculate the first mean value of the corresponding first power spectra within the multiple frequency points N1 to N2 in each frame of the first signal in the sound pickup signal. The second mean value of the second power spectra corresponding to multiple frequency points N1 to N2 in the corresponding second signal

[0083] Determine the information dispersion degree of the first signal according to the spectral expression information of multiple frequency points in each frame of the first signal and the above first mean value Determine the information dispersion degree of the second signal according to the spectral expression information of multiple frequency points in each frame of the second signal and the above second mean value

[0084] Determine the difference between the spectral expression information of each frequency point in each frame of the first signal and the first mean value according to the frequency-domain expression information of multiple frequency points of each frame of the first signal and the frequency-domain expression information of multiple frequency points of each frame of the second signal The difference between the spectral expression information of each frequency point in each frame of the second signal and the second mean value

[0085] Determine the overall deviation between the first signal and the second signal according to the difference between the spectral expression information of each frequency point in each frame of the first signal and the first mean value and the difference between the spectral expression information of each frequency point in each frame of the second signal and the second mean value

[0086] Starting from k = 1, for each frame of the first signal and each frame of the second signal corresponding to this frame of the first signal, combine the following formula, and calculate the cross-correlation degree P(i,k) (also called the correlation degree in this article) between the first signal and the corresponding second signal according to the information dispersion degree of the first signal, the information dispersion degree of the second signal, and the overall deviation between the first signal and the second signal

[0087]

[0088] Step S4: Assume that there are 5000 frames of the second signal in the test signal, and there are 5000 frames of the first signal in the corresponding picked-up signal. Set the first threshold to 0.5, and determine the first quantity of the correlation degrees greater than 0.5 among the correlation degrees corresponding to each frame position offset among the M frame position offsets. Among them, taking the first frame position offset as an example, determine that the first quantity of P(1,1) to P(5000,1) greater than or equal to 0.5 is 3200. According to this method, calculate the first quantity corresponding to each frame position offset in turn. The frame position offset refers to the offset between one frame of the first signal and one frame of the second signal in the associated signal of the first signal

[0089] Step S5: Determine the peak value 2300 in the first quantity corresponding to each frame position offset among the M frame position offsets, and calculate the third mean value 173 of the first quantity corresponding to each frame position offset among the M frame position offsets. Based on this, if it can be determined that the peak value 50 is less than 250 and the ratio of the peak value to the third mean value is less than 200, it can be determined that the voice pickup function of the terminal fails, and the evaluation of the voice pickup function of the terminal is completed.

[0090] Among them, in the case where it is determined that the voice pickup function of the terminal fails, the result of the failure of the voice pickup function of the terminal can be reported for troubleshooting or fault transfer, so as to ensure that the user can continue to use the terminal for calls or clearly inform the user of the cause of the fault for subsequent repair of the fault.

[0091] Figure 3 The flowchart of the method for evaluating the voice pickup function of the terminal provided by the embodiment of the present application is shown. As Figure 3 shown, the method includes steps S11 to S14. The method is implemented by the terminal.

[0092] Step S11: Obtain the playback signal of the terminal and the voice pickup signal of the playback signal.

[0093] The specific source and form of the playback signal are not limited in the embodiment of the present application and can be configured according to actual application requirements. For example, the playback signal can be any voice signal played by the terminal. For example, it can be the signal played by the user of the terminal during a voice call. Optionally, the playback module and the voice pickup module of the terminal can be opened simultaneously. While playing the playback signal through the playback module, the voice pickup signal of the playback signal (hereinafter referred to as the voice pickup signal) is collected through the voice pickup module. Among them, it is preferably to play the playback signal in the external speaker mode.

[0094] Since the voice pickup signal is an echo signal that is collected by the microphone after being conducted through the air after the playback signal is played through the speaker, there is an echo time delay (hereinafter referred to as the time delay) between the voice pickup signal and the playback signal. Among them, in addition to including the echo signal of the playback signal, the voice pickup signal may also include noise signals in the detected environment (when collecting the voice pickup signal), such as human voices near the microphone, other sounds in the collection environment, etc. Among them, in an ideal situation, the voice pickup signal only includes the echo signal of the playback signal.

[0095] Optionally, the voice pickup signal of the playback signal can be a signal directly collected by the microphone or a signal obtained by performing noise reduction processing on the signal collected by the microphone. The noise reduction processing may include: wavelet threshold denoising, band-pass filter denoising, etc.

[0096] Figure 4Schematic diagram showing the playback signal and the pickup signal of an embodiment of the present application. In Figure 4 , the solid circles represent signals related to the actual playback content, and the hollow circles represent signals without signals or signals unrelated to the actual playback content. As Figure 4 shown, in the time domain, assuming that the time delay between the pickup signal and the playback signal is the duration corresponding to k (k≤M) frames of signals, the i-th frame signal in the pickup signal is actually obtained by collecting the (i-k)-th frame signal of the playback signal. The i-th frame signal in the pickup signal is only related to the (i-k)-th frame signal of the playback signal and is unrelated to other signals in the playback signal except the (i-k)-th frame. Assuming that the frame position of the first frame signal related to the actual playback content in the pickup signal is i, then before the i-th frame signal of the pickup signal, there are k frames of signals unrelated to the actual playback content ( Figure 4 represented as k frames of unrelated signals in ). Before obtaining the pickup signal corresponding to a certain frame of the playback signal, the playback signals of each frame before that frame of the playback signal can be cached in a buffer area.

[0097] Optionally, the buffer area can be any buffer area in the terminal or a buffer area external to the terminal. Among them, the buffer area can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disc.

[0098] Optionally, the playback signal is any one of the following: a test signal; a specified signal; a call signal of the terminal.

[0099] Optionally, the test signal can be made into a program or a plug-in and installed in the terminal. After receiving a trigger operation on the program or plug-in by the user on the terminal, the test signal can be automatically or manually played by the user, and the method can be executed to evaluate the pickup function of the terminal.

[0100] Among them, the test signal can be any sound signal. For example, it can be a sound signal pre-stored in the terminal or a certain sound signal played online. Among them, the sound signal can be music, telephone recording pickup, sound in a video, a ringtone, etc. The specified signal can also be any sound signal. The call signal of the terminal can be a sound signal generated by the user during a call through the terminal.

[0101] Step S12: Determine a target signal sequence according to the pickup signal, where the target signal sequence includes at least one frame of first signal in the pickup signal.

[0102] Optionally, the target signal sequence can be obtained by determining the energy of each frame of the pick-up signal in the pick-up signal, for example, the rms (root mean square) value of each frame of the pick-up signal, determining the frames of the pick-up signal with rms values greater than a preset value in the pick-up signal, and extracting frames from the frames of the pick-up signal with rms values greater than the preset value according to the following frame extraction method, or directly extracting frames from the pick-up signal according to the following frame extraction method. This application does not limit this. Among them, the preset value can be selected according to actual needs (such as an empirical value or an experimental value), and this application does not limit this. For example, the preset value can be set to 300.

[0103] Optionally, the frame extraction method can be to extract frames at a preset time interval. Among them, the preset time interval can be determined according to the total duration of the pick-up signal or the number of frames of the pick-up signal included in the pick-up signal, and this application does not limit this. For example, the preset time interval can be positively correlated with the total duration of the pick-up signal or the number of frames of the pick-up signal included in the pick-up signal. For example, when the total duration of the pick-up signal is 1 hour, the preset time interval can be set to 1 minute. When the total duration of the pick-up signal is 15 minutes, the preset time interval can be 5 seconds. Of course, the preset time intervals corresponding to different pick-up signals can also be the same.

[0104] Step S13: Determine the associated signal of the first signal according to the frame position of the first signal in the pick-up signal. The associated signal of a first signal includes M frames of second signals in the playback signal that are before the signal corresponding to the first signal, where M≥1.

[0105] For each frame of the first signal in the pick-up signal, determine the correlation between the first signal and the associated signal of the first signal in the playback signal.

[0106] Optionally, the M frames of second signals can be continuous signals in the playback signal or discontinuous signals. For example, the second signal is obtained every 5 frames, and a total of M times are obtained to obtain M frames of second signals. This application does not limit this.

[0107] Optionally, the signal in the playback signal corresponding to the first signal can be the signal in the playback signal with the same frame position as the first signal.

[0108] Optionally, as described above, due to the time delay between the pickup signal and the playback signal, the associated signal of a first signal should be the M-frame second signals in the playback signal that are before the signal corresponding to the first signal. Wherein, the value of M can be determined according to the time delay between the playback signal and the pickup signal of the playback signal. The time delay between the playback signal and the pickup signal of the playback signal can vary due to different terminals or can be a fixed value. Optionally, the method may further include:

[0109] Obtain the device information of the terminal, and determine the value of M according to the device information.

[0110] Wherein, the device information of the terminal may include the device configuration information of the terminal, and the device configuration information can be used to evaluate the time delay situation between the playback and pickup of the voice signal of the terminal. Generally, the higher the configuration of the terminal, the relatively shorter the time delay. Based on this solution, a more reasonable value of M can be determined. For a terminal with a higher configuration, a smaller value of M can be selected, which can effectively reduce the amount of data processing.

[0111] Optionally, the time delay between the playback signal and the pickup signal of the playback signal may include the time for the speaker to convert the playback signal from an electrical signal to a sound signal, the time for the playback signal converted into a sound signal to be collected by the microphone after air conduction, and the time for the microphone to convert the playback signal converted into a sound signal into a pickup signal. The value of M can be configured according to actual needs (such as an empirical value or an experimental value), and this application does not limit it. For example, M can be set to 200.

[0112] Since the pickup signal is collected for the playback signal of the terminal, if the pickup function of the terminal is normal, there should be a high correlation between the i-th frame signal in the pickup signal and the (i-k)-th frame signal in the playback signal (k≤M), and the correlation between the i-th frame signal in the pickup signal and other signals in the playback signal except the (i-k)-th frame signal is low. Therefore, the correlation between each frame of the first signal in the pickup signal and the associated signal of the first signal in the playback signal can be calculated to evaluate the pickup function of the terminal and determine whether the pickup function of the terminal is normal.

[0113] Step S14: Determine whether the pickup function of the terminal is normal according to the correlation corresponding to each frame of the first signal.

[0114] By determining the signals in the playback signal that have the same frame positions as the first signals in each frame of the pickup signal, determining the M frames of second signals in the playback signal that are before such signals, determining the correlation between each frame of the first signals in the pickup signal and the corresponding each frame of the second signals, and evaluating the pickup function of the terminal based on the correlations corresponding to each frame of the first signals, it is determined whether the pickup function of the terminal is normal, which improves the accuracy and robustness of evaluating the pickup function of the terminal. In addition, this method is convenient and fast to implement and can better meet the actual application requirements.

[0115] Optionally, for each frame of the first signals in the pickup signal, determining the correlation between the first signal and the associated signal of the first signal in the playback signal includes:

[0116] Determining the correlation between the first signal and each frame of the second signals in the associated signal of the first signal;

[0117] Determining whether the pickup function of the terminal is normal according to the correlations corresponding to each frame of the first signals includes:

[0118] According to the correlations corresponding to each frame of the first signals, determining the correlations corresponding to each frame position offset among the correlations corresponding to each frame of the first signals, where the frame position offset refers to the offset between a frame of the first signal and a frame of the second signal in the associated signal of the first signal;

[0119] For each frame position offset among the M frame position offsets, determining the first quantity of the correlations greater than or equal to the first threshold among the correlations corresponding to the frame position offset;

[0120] Determining whether the pickup function of the terminal is normal according to the first quantity corresponding to each frame position offset among the M frame position offsets.

[0121] Optionally, the frame position offset can represent the offset of the frame position of a frame of the first signal relative to the frame position of a frame of the second signal in the associated signal of the first signal. For example, when the frame position of the first signal is 58 and the frame position of a frame of the second signal in the associated signal of the first signal is 39, the frame position of the first signal is offset by -19 relative to the frame position of the second signal, and the corresponding frame position offset is 19.

[0122] As described above, if the voice pickup function of the terminal is normal, there should be a correlation between each frame of the first signal in the voice pickup signal and the associated signal of the first signal in the playback signal. Therefore, in order to more accurately evaluate the voice pickup function of the terminal and determine whether the voice pickup function of the terminal is normal, a first threshold can be set, and by determining whether the correlation corresponding to each frame of the first signal is greater than or equal to the first threshold, it can be determined whether the voice pickup function of the terminal fails. Optionally, the first threshold can be configured according to actual requirements (such as an empirical value or an experimental value), and this application does not limit it. For example, the first threshold can be set to 0.5.

[0123] Due to the continuity of the sound signal, under ideal conditions, for each frame of the first signal in the voice pickup signal, it should be correlated with one of the first M frames of the second signal corresponding to the first signal in the playback signal (hereinafter referred to as "the relevant second signal"), and the frame position of the relevant second signal should be fixed relative to the frame position of the first signal, that is, if the difference in frame positions between the relevant second signal and the first signal is k frames, and the frame position of the relevant second signal is before the frame position of the first signal, then each frame of the first signal in the voice pickup signal has the highest correlation with the second signal in the playback signal whose frame position is k frames before the frame position of the first signal. Therefore, by calculating the first quantity of the correlations greater than or equal to the first threshold among the correlations corresponding to each frame position offset among the M frame position offsets, the accuracy and robustness of evaluating the voice pickup function of the terminal are improved.

[0124] Optionally, determining whether the voice pickup function of the terminal is normal according to the first quantity corresponding to each frame position offset among the M frame position offsets may include:

[0125] If at least one of the first quantities corresponding to each frame position offset among the M frame position offsets satisfies a preset condition, the voice pickup function of the terminal is normal;

[0126] If none of the first quantities corresponding to each frame position offset among the M frame position offsets satisfies the preset condition, the voice pickup function of the terminal fails;

[0127] Wherein, the preset condition includes: the first quantity is greater than or equal to a second threshold, and the ratio of the first quantity to a reference value is greater than or equal to a third threshold; wherein, the reference value characterizes the degree of interference of the first signal.

[0128] Wherein, under ideal conditions, only one of the first quantities corresponding to each frame position offset among the M frame position offsets satisfies the preset condition.

[0129] Optionally, none of the first quantities corresponding to each frame position offset among the M frame position offsets satisfying the preset condition includes at least one of the following:

[0130] The first quantity corresponding to each of the M frame position offsets is zero;

[0131] The first quantity corresponding to each of the M frame position offsets is greater than zero but less than a second threshold;

[0132] The ratio of the first quantity corresponding to each of the M frame position offsets to a reference value is less than a third threshold.

[0133] Optionally, the second threshold can be configured according to actual needs (such as an empirical value or an experimental value), and the present application does not limit this. For example, the second threshold can be set to 200.

[0134] Optionally, the third threshold can be related to the total number of frames of the first signal in the pick-up signal (i.e., the total number of frames of the second signal in the playback signal), and can be configured according to actual needs (such as an empirical value or an experimental value), and the present application does not limit this. For example, when the total number of frames of the first signal is 5000 frames, the third threshold can be set to 200.

[0135] Optionally, the degree of interference of the first signal can indicate the degree of interference of the noise signal in the detection environment of the method on the first signal. For example, the degree of interference of the first signal can refer to the degree of interference of the human voice near the microphone in the above text, other sounds in the collection environment, etc. on the echo signal of the playback signal.

[0136] Optionally, the reference value is configured according to actual needs (such as an empirical value or an experimental value), and the present application does not limit this. For example, the reference value is determined according to the collection environment of the first signal, the service life of the terminal, etc. Among them, when the noise in the collection environment of the first signal is relatively large, a higher reference value can be set. The longer the service life of the terminal, the larger the reference value that can be set.

[0137] As an example of the present application, the maximum value of the first quantity corresponding to each frame position among the M frame position offsets can be determined first, that is, the peak value. At the frame position offset corresponding to the peak value, the correlation between the first signal and the second signal is the highest, and the first signal is obtained by collecting the second signal at this frame position offset. If the peak value meets the preset conditions, it is determined that the pick-up function of the terminal is normal. If the peak value does not meet the preset conditions, it is determined that the pick-up function of the terminal fails.

[0138] By evaluating the pick-up function of the terminal according to the relationship between the first quantity corresponding to each frame position offset among the M frame position offsets and the second threshold, and the relationship between the ratio of the first quantity and the reference value and the third threshold, it is determined whether the pick-up function of the terminal fails, which improves the accuracy and robustness of the evaluation of the pick-up function of the terminal, and avoids the adverse consequences of misjudgment and missed judgment on whether the pick-up function of the terminal fails due to the interference of the first signal.

[0139] Optionally, the method may further include: determining the mean value of the first quantities corresponding to each frame position offset among the M frame position offsets, and using the mean value as the reference value.

[0140] In this implementation manner, the preset conditions include: the first quantity is greater than or equal to the second threshold, and the ratio of the first quantity to the mean value is greater than or equal to the third threshold.

[0141] Optionally, the mean value of the first quantities corresponding to each frame position offset among the M frame position offsets can reflect the same influence on the correlation degree of each frame of the second signal in the associated signal of each frame of the first signal and the first signal, that is, it can reflect the interference degree of each frame of the first signal in the pick-up signal.

[0142] Among them, when the first signal is subject to greater interference, the gap between the peak value and other values other than the peak value among the first quantities corresponding to each frame position offset among the M frame position offsets will become relatively smaller, and the mean value will also increase accordingly; when the first signal is subject to smaller interference, the gap between the peak value and other values other than the peak value among the first quantities corresponding to each frame position offset among the M frame position offsets will become relatively larger, and the mean value may decrease.

[0143] By according to the relationship between the first quantity corresponding to each frame position offset among the M frame position offsets and the second threshold, and the relationship between the ratio of the first quantity and the mean value and the third threshold, it is possible to combine the influence of the actual detection environment of the method on the first quantity corresponding to each frame position offset, evaluate the pick-up function of the terminal, determine whether the pick-up function of the terminal fails, improve the accuracy and robustness of the evaluation of the pick-up of the terminal, and avoid the occurrence of adverse consequences of misjudgment and missed judgment on the pick-up function of the terminal during the evaluation process of the pick-up function of the terminal due to the interference of the first signal.

[0144] Optionally, the correlation degree between the first signal and the associated signal of the first signal in the playback signal includes the correlation degree between the first signal and each frame of the second signal in the associated signal;

[0145] For each frame of the first signal in the pick-up signal, determining the correlation degree between the first signal and the associated signal of the first signal in the playback signal includes:

[0146] Determine the first frequency-domain expression information of the first signal and the second frequency-domain expression information of each frame of the second signal in the associated signal of the first signal;

[0147] For each frame of the second signal in the associated signal, based on the first frequency-domain expression information and the second frequency-domain expression information of the second signal, obtain the correlation between the first signal and the second signal.

[0148] As described above, since there is a time delay between the i-th frame in the pickup signal and the (i - k)-th frame signal of the corresponding playback signal, the i-th frame signal in the pickup signal is actually obtained by collecting the (i - k)-th frame signal of the playback signal. Therefore, before obtaining the pickup signal corresponding to a certain frame of the playback signal, the second frequency-domain expression information of each frame of the second signal before this frame of the playback signal can also be cached in the buffer. After obtaining the first signal corresponding to the second signal, then determine the first frequency-domain expression information of the first signal, and read the second frequency-domain expression information of each frame of the second signal in the associated signal of the first signal from the buffer, so as to obtain the correlation between the first signal and the second signal.

[0149] Optionally, both the first frequency-domain expression information and the second frequency-domain information expression include at least one of the power spectrum, amplitude spectrum, logarithmic power spectrum, or logarithmic amplitude spectrum of the signal.

[0150] By determining the first frequency-domain expression information of the first signal and the second frequency-domain expression information of the second signal, and then determining the correlation between the first signal and the second signal from the frequency domain, the correlation between the first signal and the second signal can be determined more intuitively.

[0151] Optionally, determining the first frequency-domain expression information of the first signal and the second frequency-domain expression information of each frame of the second signal in the associated signal of the first signal includes:

[0152] Obtain the first spectrum of the first signal and the second spectrum of each frame of the second signal, and both the first spectrum and the second spectrum include amplitude values at multiple frequency points;

[0153] Based on the first spectrum, determine the frequency-domain expression information of each frequency point of the first signal, and based on the second spectrum of each frame of the second signal, determine the frequency-domain expression information of each frequency point of each frame of the second signal;

[0154] Among them, the first frequency-domain expression information includes the frequency-domain expression information of multiple frequency points of the first signal, and the second frequency-domain expression information of one frame of the second signal includes the frequency-domain expression information of multiple frequency points of the second signal.

[0155] In this implementation manner, the multiple frequency points can be all the frequency points of the corresponding signal or partial frequency points selected according to application requirements. The specific number of frequency points to be used and the selection range of the frequency points can be configured according to actual needs (such as empirical values or experimental values). For example, the range of multiple frequency points can be set as N1 to N2. Among them, the corresponding frequency range can be multiple frequency points with a starting frequency of 800 hz to an ending frequency of 3.5 khz. Among these multiple frequency points, the smallest frequency point number is N1, and the largest frequency point number is N2.

[0156] By obtaining the amplitude values of multiple frequency points in the first signal and the amplitude values of multiple frequency points in the second signal, the frequency-domain expression information of each frequency point of the first signal is determined according to the amplitude value of each frequency point of the first signal, and the frequency-domain expression information of each frequency point of the second signal is determined according to the amplitude value of each frequency point of the second signal. Furthermore, the first frequency-domain expression information and the second frequency-domain expression information can be more accurately obtained.

[0157] Optionally, for each frame of the second signal in the associated signals, based on the first frequency-domain expression information and the second frequency-domain expression information of the second signal, the correlation degree between the first signal and the second signal is obtained, including:

[0158] Determine the information dispersion degree of the first signal according to the frequency-domain expression information of multiple frequency points of the first signal;

[0159] Determine the information dispersion degree of the second signal according to the frequency-domain expression information of multiple frequency points of the second signal;

[0160] Determine the overall deviation between the first signal and the second signal according to the frequency-domain expression information of multiple frequency points of the first signal and the frequency-domain expression information of multiple frequency points of the second signal;

[0161] Determine the correlation degree between the first signal and the second signal according to the information dispersion degree of the first signal, the information dispersion degree of the second signal, and the overall deviation between the first signal and the second signal.

[0162] Optionally, the overall deviation between the first signal and the second signal can determine the covariance between the first signal and the second signal. The information dispersion degree of the first signal can be the first standard deviation of the spectral expression information of multiple frequency points in the first signal, and the information dispersion degree of the second signal can be the second standard deviation of the spectral expression information of the second signal at multiple frequency points.

[0163] Optionally, determining the information dispersion degree of the first signal according to the frequency-domain expression information of multiple frequency points of the first signal includes:

[0164] Determine the first mean value of the spectral expression information of multiple frequency points in the first signal;

[0165] Determine the information dispersion degree of the first signal according to the spectral expression information of multiple frequency points in the first signal and the first mean value.

[0166] Determine the information dispersion degree of the second signal according to the frequency-domain expression information of multiple frequency points of the second signal, including:

[0167] Determine the second mean value of the spectral expression information of multiple frequency points in the second signal;

[0168] Determine the information dispersion degree of the second signal according to the spectral expression information of multiple frequency points in the second signal and the second mean value.

[0169] Determine the overall deviation between the first signal and the second signal according to the frequency-domain expression information of multiple frequency points of the first signal and the frequency-domain expression information of multiple frequency points of the second signal, including:

[0170] Determine the difference between the spectral expression information of each frequency point in the first signal and the first mean value;

[0171] Determine the difference between the spectral expression information of each frequency point in the second signal and the second mean value;

[0172] Determine the overall deviation between the first signal and the second signal according to the difference between the spectral expression information of each frequency point in the first signal and the first mean value and the difference between the spectral expression information of each frequency point in the second signal and the second mean value.

[0173] Specifically, taking the case where both the first frequency-domain expression information and the second frequency-domain information include the power spectrum of the signal, the correlation degree between the first signal and the second signal can be obtained through the following formula:

[0174]

[0175] Among them, P(i,k) represents the correlation degree between the first signal of the i-th frame and the second signal of the associated signal, X(i-k,j) represents the power spectrum of the second signal at the j-th frequency point, represents the second mean value. D(i,j) represents the power spectrum of the first signal at the j-th frequency point, represents the first mean value. represents the covariance between the first signal and the second signal, j = N1N2(Xi-k,j - X(i-k))2 represents the standard deviation of the power spectra of multiple frequency points in the second signal, j = N1N2(Di,j - D(i))2 represents the standard deviation of the power spectra of multiple frequency points in the first signal. Among them, i represents the frame number of each frame of the first signal, j ∈ (N1, N2), N1, N2 represent the frequency point numbers corresponding to the starting and ending frequencies, and k ∈ M.

[0176] For each of multiple frequency points, by determining the correlation between the first signal and the second signal according to the spectral expression information of this frequency point in the first signal and the spectral expression information of this frequency point in the second signal, the accuracy of the correlation between the obtained first signal and the second signal is improved. Furthermore, according to the correlation between the first signal and the second signal, the sound pickup function of the terminal is evaluated to determine whether the sound pickup function of the terminal fails, improving the accuracy and robustness of the evaluation of the sound pickup function of the terminal.

[0177] Figure 5 FIG. shows a schematic diagram of an evaluation device for the sound pickup function of a terminal provided in an embodiment of the present application. As Figure 5 shown, the device 50 includes a signal acquisition module 510, a target signal sequence determination module 520, a correlation determination module 530, and a failure determination module 540, where:

[0178] The signal acquisition module 510 is configured to acquire the playback signal of the terminal and the sound pickup signal of the playback signal;

[0179] The target signal sequence determination module 520 is configured to determine a target signal sequence according to the sound pickup signal, and the target signal sequence includes at least one frame of the first signal in the sound pickup signal;

[0180] The correlation determination module 530 is configured to determine the associated signal of the first signal according to the frame position of the first signal in the sound pickup signal. The associated signal of one first signal includes M frames of the second signal in the playback signal that are before the signal corresponding to the first signal in the playback signal, and the signal in the playback signal corresponding to the first signal is the signal in the playback signal with the same frame position as the first signal, where M≥1;

[0181] For each frame of the first signal in the sound pickup signal, determine the correlation between the first signal and the associated signal of the first signal in the playback signal;

[0182] The failure determination module 540 is configured to determine whether the sound pickup function of the terminal is normal according to the correlation corresponding to each frame of the first signal.

[0183] Optionally, for each frame of the first signal in the sound pickup signal, when the correlation determination module 530 determines the correlation between the first signal and the associated signal of the first signal in the playback signal, it is configured to:

[0184] Determine the correlation between the first signal and each frame of the second signal in the associated signal of the first signal;

[0185] When the failure determination module 540 determines whether the sound pickup function of the terminal is normal according to the correlation corresponding to each frame of the first signal, it specifically is configured to:

[0186] Determine, according to the correlations corresponding to the first signals of each frame, the correlations corresponding to each frame position offset among the correlations corresponding to the first signals of each frame, where the frame position offset refers to the offset between a frame of the first signal and a frame of the second signal in the associated signal of the first signal;

[0187] For each of the M frame position offsets, determine a first quantity of the correlations corresponding to the frame position offset that is greater than or equal to a first threshold;

[0188] Determine whether the sound pickup function of the terminal is normal according to the first quantity corresponding to each of the M frame position offsets.

[0189] Optionally, when determining whether the sound pickup function of the terminal is normal according to the first quantity corresponding to each of the M frame positions, the failure determination module 540 is configured to:

[0190] If there is at least one first quantity among the first quantities corresponding to each of the M frame position offsets that satisfies a preset condition, the sound pickup function of the terminal is normal;

[0191] If none of the first quantities corresponding to each of the M frame position offsets satisfies the preset condition, the sound pickup function of the terminal fails;

[0192] Wherein, the preset condition includes:

[0193] The first quantity is greater than or equal to a second threshold, and the ratio of the first quantity to a reference value is greater than or equal to a third threshold;

[0194] Wherein, the reference value characterizes the degree of interference of the first signal.

[0195] Optionally, the correlation determination module 530 is further configured to: determine the mean value of the first quantities corresponding to each of the M frame position offsets, and use the mean value as the reference value.

[0196] Optionally, for each frame of the first signal in the sound pickup signal, the correlation between the first signal and the associated signal of the first signal in the playback signal includes the correlation between the first signal and each frame of the second signal in the associated signal;

[0197] For each frame of the first signal in the sound pickup signal, when the correlation determination module 530 determines the correlation between the first signal and the associated signal of the first signal in the playback signal, it is configured to:

[0198] Determine the first frequency domain expression information of the first signal and the second frequency domain expression information of each frame of the second signal in the associated signal of the first signal;

[0199] For each frame of the second signal in the associated signals, based on the first frequency-domain expression information and the second frequency-domain expression information of the second signal, obtain the correlation degree between the first signal and the second signal.

[0200] Optionally, when determining the first frequency-domain expression information of the first signal and the second frequency-domain expression information of each frame of the second signal in the associated signals of the first signal, the correlation degree determination module 530 includes:

[0201] Obtain the first spectrum of the first signal and the second spectra of each frame of the second signal, where the first spectrum and the second spectra both include amplitude values of multiple frequency points;

[0202] Based on the first spectrum, determine the frequency-domain expression information of each frequency point of the first signal, and based on the second spectrum of each frame of the second signal, determine the frequency-domain expression information of each frequency point of each frame of the second signal;

[0203] Wherein, the first frequency-domain expression information includes the frequency-domain expression information of multiple frequency points of the first signal, and the second frequency-domain expression information of one frame of the second signal includes the frequency-domain expression information of multiple frequency points of the second signal.

[0204] Optionally, for each frame of the second signal in the associated signals, when the correlation degree determination module 530 obtains the correlation degree between the first signal and the second signal based on the first frequency-domain expression information and the second frequency-domain expression information of the second signal, it is used for:

[0205] Determine the information dispersion degree of the first signal according to the frequency-domain expression information of multiple frequency points of the first signal;

[0206] Determine the information dispersion degree of the second signal according to the frequency-domain expression information of multiple frequency points of the second signal;

[0207] Determine the overall deviation between the first signal and the second signal according to the frequency-domain expression information of multiple frequency points of the first signal and the frequency-domain expression information of multiple frequency points of the second signal;

[0208] Determine the correlation degree between the first signal and the second signal according to the information dispersion degree of the first signal, the information dispersion degree of the second signal, and the overall deviation between the first signal and the second signal.

[0209] Optionally, when the correlation degree determination module determines the information dispersion degree of the first signal according to the frequency-domain expression information of multiple frequency points of the first signal, it is specifically used for:

[0210] Determine the first mean value of the spectrum expression information of multiple frequency points in the first signal;

[0211] Determine the information dispersion degree of the first signal according to the spectrum expression information of multiple frequency points in the first signal and the first mean value;

[0212] When determining the information dispersion degree of the second signal according to the frequency-domain expression information of multiple frequency points of the second signal, the relevance determination module is specifically configured to:

[0213] Determine a second mean value of the spectral expression information of multiple frequency points in the second signal;

[0214] Determine the information dispersion degree of the second signal according to the spectral expression information of multiple frequency points in the second signal and the second mean value;

[0215] When determining the overall deviation between the first signal and the second signal according to the frequency-domain expression information of multiple frequency points of the first signal and the frequency-domain expression information of multiple frequency points of the second signal, the relevance determination module is specifically configured to:

[0216] Determine the difference between the spectral expression information of each frequency point in the first signal and the first mean value;

[0217] Determine the difference between the spectral expression information of each frequency point in the second signal and the second mean value;

[0218] Determine the overall deviation between the first signal and the second signal according to the difference between the spectral expression information of each frequency point in the first signal and the first mean value and the difference between the spectral expression information of each frequency point in the second signal and the second mean value.

[0219] Optionally, both the first frequency-domain expression information and the second frequency-domain information expression include at least one of the power spectrum, amplitude spectrum, logarithmic power spectrum, or logarithmic amplitude spectrum of the signal.

[0220] Optionally, the playback signal is any one of the following: a test signal; a specified signal; a call signal of the terminal.

[0221] The device according to the embodiment of the present application can execute the method provided by the embodiment of the present application, and its implementation principle is similar. The actions performed by each module in the device according to the embodiment of the present application correspond to the steps in the method according to the embodiment of the present application. For the detailed function description of each module of the device, reference can be specifically made to the description in the corresponding method shown above, and details are not described herein again.

[0222] The embodiment of the present application further provides an electronic device, which includes a memory, a processor, and a computer program stored on the memory. The processor executes the computer program to implement the steps of the method for evaluating the failure of the sound pickup function of the above terminal.

[0223] Figure 6 Shows a schematic diagram of an electronic device provided by an embodiment of the present application, as Figure 6 shown, Figure 6The electronic device 4000 shown includes: a processor 4001 and a memory 4003. Among them, the processor 4001 and the memory 4003 are connected, such as through a bus 4002. Optionally, the electronic device 4000 may further include a transceiver 4004, and the transceiver 4004 can be used for data interaction between this electronic device and other electronic devices, such as data sending and / or data receiving, etc. It should be noted that in practical applications, the transceiver 4004 is not limited to one, and the structure of the electronic device 4000 does not constitute a limitation to the embodiments of the present application.

[0224] The processor 4001 can be a CPU (Central Processing Unit, central processor), a general-purpose processor, a DSP (Digital Signal Processor, data signal processor), an ASIC (Application Specific Integrated Circuit, application-specific integrated circuit), an FPGA (Field Programmable Gate Array, field programmable gate array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It can implement or execute various exemplary logical blocks, modules and circuits described in combination with the disclosure of the present application. The processor 4001 can also be a combination that realizes computing functions, such as a combination including one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0225] The bus 4002 may include a path for transmitting information between the above components. The bus 4002 can be a PCI (Peripheral Component Interconnect, peripheral component interconnect standard) bus or an EISA (Extended Industry Standard Architecture, extended industry standard architecture) bus, etc. The bus 4002 can be divided into an address bus, a data bus, a control bus, etc. For the sake of representation, Figure 6 only a thick line is used to represent it in the figure, but it does not mean that there is only one bus or one type of bus.

[0226] The memory 4003 can be a ROM (Read Only Memory), or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory), or other types of dynamic storage devices that can store information and instructions. It can also be an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory), or other optical disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium that can be used to carry or store computer programs and can be read by a computer, which is not limited herein.

[0227] The memory 4003 is used to store the computer program for implementing the embodiments of the present application and is controlled by the processor 4001 for execution. The processor 4001 is used to execute the computer program stored in the memory 4003 to implement the steps shown in the foregoing method embodiments.

[0228] Based on the same principle as the method provided in the embodiments of the present application, the embodiments of the present application also provide a computer program product or a computer program. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method provided in any optional embodiment of the present application above.

[0229] The embodiments of the present application also provide a computer-readable storage medium. A computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the method for evaluating the voice pickup function of the foregoing terminal are implemented.

[0230] The embodiments of the present application also provide a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the steps of the method for evaluating the voice pickup function of the foregoing terminal are implemented.

[0231] It should be understood that although the flowchart of the embodiments of the present application indicates various operation steps by arrows, the execution order of these steps is not limited to the order indicated by the arrows. Unless there is a clear description in this article, in some implementation scenarios of the embodiments of the present application, the implementation steps in each flowchart can be executed in other orders according to requirements. In addition, some or all of the steps in each flowchart may include multiple sub-steps or multiple stages based on the actual implementation scenario. Some or all of these sub-steps or stages can be executed at the same time, and each sub-step or stage among these sub-steps or stages can also be executed at different times respectively. In the scenario where the execution times are different, the execution order of these sub-steps or stages can be flexibly configured according to requirements, and the embodiments of the present application do not limit this.

[0232] The above are only optional implementation manners of some implementation scenarios of the present application. It should be noted that for those of ordinary skill in the art, without departing from the technical concept of the solution of the present application, adopting other similar implementation means based on the technical idea of the present application also belongs to the protection scope of the embodiments of the present application.

Claims

1. An evaluation method for the sound pickup function of a terminal, characterized in that, Including: Obtaining a playback signal of a terminal and a sound pickup signal of the playback signal; the playback signal is a signal played through a speaker of the terminal; The sound pickup signal is a signal obtained by collecting the playback signal through a microphone of the terminal; Determining a target signal sequence according to the sound pickup signal, where the target signal sequence includes at least one frame of first signal in the sound pickup signal; Determining an associated signal of the first signal according to a frame position of the first signal in the sound pickup signal, where an associated signal of one first signal includes M frames of second signals in the playback signal that are before a signal corresponding to the first signal, and the signal in the playback signal corresponding to the first signal is a signal in the playback signal that has the same frame position as the first signal, where M≥1; For each frame of first signal in the sound pickup signal, determining a correlation degree between the first signal and the associated signal of the first signal in the playback signal; the correlation degree indicates a correlation between content information in the first signal and content information in the associated signal of the first signal in the playback signal; Determining whether a sound pickup function of the terminal is normal according to the correlation degrees corresponding to each frame of first signal.

2. The method according to claim 1, wherein For each frame of first signal in the sound pickup signal, the determining the correlation degree between the first signal and the associated signal of the first signal in the playback signal includes: Determining a correlation degree between the first signal and each frame of second signal in the associated signal of the first signal in the playback signal; The determining whether the sound pickup function of the terminal is normal according to the correlation degrees corresponding to each frame of first signal includes: Determining, according to the correlation degrees corresponding to each frame of first signal, correlation degrees corresponding to each frame position offset in the correlation degrees corresponding to each frame of first signal, where the frame position offset refers to an offset between a frame of first signal and a frame of second signal in the associated signal of the first signal in the playback signal; For each frame position offset among M frame position offsets, determining a first quantity of correlation degrees greater than or equal to a first threshold in the correlation degrees corresponding to the frame position offset; Determining whether the sound pickup function of the terminal is normal according to the first quantity corresponding to each frame position offset among the M frame position offsets.

3. The method according to claim 2, wherein The determining whether the sound pickup function of the terminal is normal according to the first quantity corresponding to each frame position offset among the M frame position offsets includes: If there is at least one first quantity that satisfies a preset condition among the first quantities corresponding to each frame position offset among the M frame position offsets, the sound pickup function of the terminal is normal; If the first quantities corresponding to each frame position offset among the M frame position offsets do not satisfy the preset condition, the sound pickup function of the terminal fails; Wherein, the preset condition includes: The first quantity is greater than or equal to a second threshold, and a ratio of the first quantity to a reference value is greater than or equal to a third threshold; Wherein, the reference value characterizes an interference degree of the first signal.

4. The method according to claim 3, characterized in that, The method further includes: Determining an average value of the first quantities corresponding to each frame position offset among the M frame position offsets, and using the average value as the reference value.

5. The method according to any one of claims 1 to 4, characterized in that For each frame of the first signal in the pickup signal, the correlation degree between the first signal and the associated signal of the first signal in the playback signal includes the correlation degree between the first signal and each frame of the second signal in the associated signal; For each frame of the first signal in the pickup signal, determining the correlation degree between the first signal and the associated signal of the first signal in the playback signal includes: Determining the first frequency-domain expression information of the first signal and the second frequency-domain expression information of each frame of the second signal in the associated signal of the first signal in the playback signal; For each frame of the second signal in the associated signal, based on the first frequency-domain expression information and the second frequency-domain expression information of the second signal, obtaining the correlation degree between the first signal and the second signal.

6. The method according to claim 5, wherein The determining the first frequency-domain expression information of the first signal and the second frequency-domain expression information of each frame of the second signal in the associated signal of the first signal in the playback signal includes: Obtaining the first spectrum of the first signal and the second spectra of each frame of the second signal, where both the first spectrum and the second spectra include amplitude values of multiple frequency points; Based on the first spectrum, determining the frequency-domain expression information of each frequency point of the first signal, and based on the second spectrum of each frame of the second signal, determining the frequency-domain expression information of each frequency point of each frame of the second signal; Wherein, the first frequency-domain expression information includes the frequency-domain expression information of multiple frequency points of the first signal, and the second frequency-domain expression information of one frame of the second signal includes the frequency-domain expression information of multiple frequency points of the second signal.

7. The method according to claim 6, characterized in that, For each frame of the second signal in the associated signal, the obtaining the correlation degree between the first signal and the second signal based on the first frequency-domain expression information and the second frequency-domain expression information of the second signal includes: Determining the information dispersion degree of the first signal according to the frequency-domain expression information of multiple frequency points of the first signal; Determining the information dispersion degree of the second signal according to the frequency-domain expression information of multiple frequency points of the second signal; Determining the overall deviation between the first signal and the second signal according to the frequency-domain expression information of multiple frequency points of the first signal and the frequency-domain expression information of multiple frequency points of the second signal; Determining the correlation degree between the first signal and the second signal according to the information dispersion degree of the first signal, the information dispersion degree of the second signal, and the overall deviation between the first signal and the second signal.

8. The method according to claim 7, wherein The determining the information dispersion degree of the first signal according to the frequency-domain expression information of multiple frequency points of the first signal includes: Determining the first mean value of the spectral expression information of multiple frequency points in the first signal; According to the spectral expression information of multiple frequency points in the first signal and the first mean value, determining the information dispersion degree of the first signal; The determining the information dispersion degree of the second signal according to the frequency-domain expression information of multiple frequency points of the second signal includes: Determining the second mean value of the spectral expression information of multiple frequency points in the second signal; According to the spectral expression information of multiple frequency points in the second signal and the second mean value, determining the information dispersion degree of the second signal; Determining an overall deviation between the first signal and the second signal based on frequency-domain expression information of a plurality of frequency points of the first signal and frequency-domain expression information of a plurality of frequency points of the second signal includes: Determining a difference between the spectral expression information of each frequency point in the first signal and a first mean value; Determining a difference between the spectral expression information of each frequency point in the second signal and a second mean value; Determining an overall deviation between the first signal and the second signal based on the difference between the spectral expression information of each frequency point in the first signal and the first mean value and the difference between the spectral expression information of each frequency point in the second signal and the second mean value.

9. The method according to claim 5, wherein The first frequency-domain expression information and the second frequency-domain expression information each include at least one of a power spectrum, an amplitude spectrum, a logarithmic power spectrum, or a logarithmic amplitude spectrum of a signal.

10. The method according to any one of claims 1 to 4, characterized in that, The playback signal is any one of the following: A test signal; a specified signal; a call signal of a terminal.

11. An evaluation device for the sound pickup function of a terminal, characterized in that, Including: A signal acquisition module, configured to acquire a playback signal of a terminal and a picked-up signal of the playback signal; the playback signal is a signal played through a speaker of the terminal; the picked-up signal is a signal collected by a microphone of the terminal for the playback signal; A target signal sequence determination module, configured to determine a target signal sequence according to the picked-up signal, where the target signal sequence includes at least one frame of a first signal in the picked-up signal; A correlation determination module, configured to determine an associated signal of the first signal according to a frame position of the first signal in the picked-up signal, where an associated signal of a first signal includes M frames of a second signal in the playback signal before a signal corresponding to the first signal in the playback signal, and the signal in the playback signal corresponding to the first signal is a signal in the playback signal having the same frame position as the first signal, where M≥1; For each frame of the first signal in the picked-up signal, determining a correlation between the first signal and the associated signal of the first signal in the playback signal; the correlation indicates a correlation between content information in the first signal and content information in the associated signal of the first signal in the playback signal; A failure determination module, configured to determine whether a picked-up function of the terminal is normal according to correlations corresponding to each frame of the first signal.

12. The device according to claim 11, characterized in that, For each frame of the first signal in the picked-up signal, when the correlation determination module determines a correlation between the first signal and the associated signal of the first signal in the playback signal, it is configured to: Determine a correlation between the first signal and each frame of the second signal in the associated signal of the first signal in the playback signal; Determining whether the picked-up function of the terminal is normal according to correlations corresponding to each frame of the first signal includes: According to correlations corresponding to each frame of the first signal, determining correlations corresponding to each frame position offset in the correlations corresponding to each frame of the first signal, where the frame position offset refers to an offset between a frame of the first signal and a frame of the second signal in the associated signal of the first signal in the playback signal; For each frame position offset among M frame position offsets, determining a first quantity of correlations greater than or equal to a first threshold in the correlations corresponding to the frame position offset; Determine whether the sound pickup function of the terminal is normal according to the first quantity corresponding to each frame position offset among the M frame position offsets.

13. The device according to claim 12, characterized in that, When determining whether the sound pickup function of the terminal is normal according to the first quantity corresponding to each frame position offset among the M frame position offsets, the failure determination module is used for: If there is at least one first quantity among the first quantities corresponding to each frame position offset among the M frame position offsets that satisfies a preset condition, the sound pickup function of the terminal is normal; If the first quantities corresponding to each frame position offset among the M frame position offsets do not satisfy the preset condition, the sound pickup function of the terminal fails; Wherein, the preset condition includes: The first quantity is greater than or equal to a second threshold, and the ratio of the first quantity to a reference value is greater than or equal to a third threshold; Wherein, the reference value characterizes the degree of interference of the first signal.

14. The device according to claim 13, characterized in that, The correlation determination module is further used for: Determine the mean value of the first quantities corresponding to each frame position offset among the M frame position offsets, and use the mean value as the reference value.

15. The device according to any one of claims 11 to 14, characterized in that, For each frame of the first signal in the sound pickup signal, the correlation between the first signal and the associated signal of the first signal in the playback signal includes the correlation between the first signal and each frame of the second signal in the associated signal; For each frame of the first signal in the sound pickup signal, when the correlation determination module determines the correlation between the first signal and the associated signal of the first signal in the playback signal, it is used for: Determine the first frequency domain expression information of the first signal and the second frequency domain expression information of each frame of the second signal in the associated signal of the first signal in the playback signal; For each frame of the second signal in the associated signal, based on the first frequency domain expression information and the second frequency domain expression information of the second signal, obtain the correlation between the first signal and the second signal.

16. The device according to claim 15, characterized in that, When the correlation determination module determines the first frequency domain expression information of the first signal and the second frequency domain expression information of each frame of the second signal in the associated signal of the first signal in the playback signal, it is used for: Obtain the first frequency spectrum of the first signal and the second frequency spectra of each frame of the second signal, and both the first frequency spectrum and the second frequency spectra include amplitude values of multiple frequency points; Based on the first frequency spectrum, determine the frequency domain expression information of each frequency point of the first signal, and based on the second frequency spectrum of each frame of the second signal, determine the frequency domain expression information of each frequency point of each frame of the second signal; Wherein, the first frequency domain expression information includes the frequency domain expression information of multiple frequency points of the first signal, and the second frequency domain expression information of one frame of the second signal includes the frequency domain expression information of multiple frequency points of the second signal.

17. The device according to claim 16, characterized in that, For each frame of the second signal in the associated signal, when the correlation determination module obtains the correlation between the first signal and the second signal based on the first frequency domain expression information and the second frequency domain expression information of the second signal, it is used for: Determine the information dispersion degree of the first signal according to the frequency domain expression information of multiple frequency points of the first signal; Determine the information dispersion degree of the second signal according to the frequency domain expression information of multiple frequency points of the second signal; Determine the overall deviation between the first signal and the second signal according to the frequency-domain expression information of multiple frequency points of the first signal and the frequency-domain expression information of multiple frequency points of the second signal; Determine the correlation between the first signal and the second signal according to the information dispersion degree of the first signal, the information dispersion degree of the second signal, and the overall deviation between the first signal and the second signal.

18. The device according to claim 17, wherein When determining the information dispersion degree of the first signal according to the frequency-domain expression information of multiple frequency points of the first signal, the correlation determination module is configured to: Determine a first mean value of the spectral expression information of multiple frequency points in the first signal; Determine the information dispersion degree of the first signal according to the spectral expression information of multiple frequency points in the first signal and the first mean value; When determining the information dispersion degree of the second signal according to the frequency-domain expression information of multiple frequency points of the second signal, the correlation determination module is configured to: Determine a second mean value of the spectral expression information of multiple frequency points in the second signal; Determine the information dispersion degree of the second signal according to the spectral expression information of multiple frequency points in the second signal and the second mean value; When determining the overall deviation between the first signal and the second signal according to the frequency-domain expression information of multiple frequency points of the first signal and the frequency-domain expression information of multiple frequency points of the second signal, the correlation determination module is configured to: Determine the difference between the spectral expression information of each frequency point in the first signal and the first mean value; Determine the difference between the spectral expression information of each frequency point in the second signal and the second mean value; Determine the overall deviation between the first signal and the second signal according to the difference between the spectral expression information of each frequency point in the first signal and the first mean value and the difference between the spectral expression information of each frequency point in the second signal and the second mean value.

19. The device according to claim 15, characterized in that, Both the first frequency-domain expression information and the second frequency-domain expression information include at least one of the power spectrum, amplitude spectrum, logarithmic power spectrum, or logarithmic amplitude spectrum of the signal.

20. The device according to any one of claims 11 to 14, characterized in that The playback signal is any one of the following: Test signal; specified signal; call signal of the terminal.

21. An electronic device, comprising a memory, a processor, and a computer program stored on the memory, characterized in that, The processor executes the computer program to implement the steps of the method for evaluating the sound pickup function of the terminal according to any one of claims 1-10.

22. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the steps of the method for evaluating the sound pickup function of the terminal according to any one of claims 1-10 are implemented.

23. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, the steps of the method for evaluating the sound pickup function of the terminal according to any one of claims 1-10 are implemented.

Images