Voice signal acquisition method and device, electronic equipment and storage medium

Abstract

The invention discloses a voice signal acquisition method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring at least one sound source in a pickup range of a microphone array, wherein the microphone array comprises at least two sub-arrays; for each sound source, determining a subarray of which the pickup range is closest to the position of the sound source; adjusting the pickup direction of the sub-array to enable the center line of the sub-array to point to the sound source; and taking the output of the subarray with the adjusted pickup direction as a voice signal of a sound source. According to the method, for each sound source in the pickup range of the microphone array, by adjusting the pickup direction of the sub-array of which the pickup range is closest to the position of the sound source, the sub-array can perform voice signal acquisition on each sound source in the best pickup direction; the overall directional pickup effect of the microphone array is improved; meanwhile, the pickup direction of the sub-array can be changed, so that the flexibility of directional pickup through the microphone array is improved.

Description

Technical Field

[0001] The present application relates to the field of audio acquisition technology, and more specifically, to a voice signal acquisition method, device, electronic device and storage medium. Background Art

[0002] Microphone array refers to a sound collection technology that uses a combination of multiple microphones to achieve sound positioning and direction recognition. Microphone arrays are needed to achieve directional sound pickup in scenarios such as smart conferences. However, the spatial arrangement of microphones in the current microphone array is fixed, and the directional sound pickup method used by the microphone array is almost a fixed traditional signal processing method, or it uses a trained neural network model to achieve directional sound pickup, which makes the pickup range of the microphone array relatively limited, greatly affecting the flexibility of directional sound pickup through the microphone array. Summary of the invention

[0003] The present application proposes a speech signal acquisition method, device, electronic device and storage medium to improve the above-mentioned problems.

[0004] In a first aspect, an embodiment of the present application provides a method for collecting speech signals, the method comprising: acquiring at least one sound source within a pickup range of a microphone array, the microphone array comprising at least two sub-arrays; for each sound source, determining a sub-array whose pickup range is closest to the position of the sound source; adjusting the pickup direction of the sub-array so that a center line of the sub-array points to the sound source; and using the output of the sub-array after the pickup direction is adjusted as the speech signal of the sound source.

[0005] In a second aspect, an embodiment of the present application provides a speech signal acquisition device, the device comprising: a sound source acquisition module, used to acquire at least one sound source located within the pickup range of a microphone array, the microphone array comprising at least two sub-arrays; a sub-array determination module, used to determine, for each sound source, a sub-array whose pickup range is closest to the position of the sound source; a pickup direction adjustment module, used to adjust the pickup direction of the sub-array so that the center line of the sub-array points to the sound source; and a speech signal acquisition module, used to use the output of the sub-array after the pickup direction is adjusted as the speech signal of the sound source.

[0006] In a third aspect, an embodiment of the present application provides an electronic device, comprising: one or more processors; a memory; one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, and the one or more applications are configured to execute the voice signal acquisition method provided in the first aspect above.

[0007] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a program code is stored. The program code can be called by a processor to execute the voice signal acquisition method provided in the first aspect above.

[0008] The present application provides a method, device, electronic device and storage medium for collecting speech signals. The method comprises obtaining at least one sound source within the pickup range of a microphone array, wherein the microphone array includes at least two sub-arrays. For each sound source, the sub-array whose pickup range is closest to the location of the sound source is determined. The pickup direction of the sub-array is adjusted so that the center line of the sub-array points to the sound source. The output of the sub-array after the pickup direction is adjusted is used as the speech signal of the sound source. Thus, for each sound source within the pickup range of the microphone array, the pickup direction of the sub-array whose pickup range is closest to the location of the sound source is adjusted so that the center line of the sub-array points to the sound source. For each sound source within the pickup range of the microphone array, the sub-array can collect speech signals from the sound source in the best pickup direction, thereby improving the overall directional pickup effect of the microphone array. At the same time, since the pickup direction of the sub-array can be changed, the pickup range of the microphone array can be flexibly adjusted, thereby improving the flexibility of directional pickup through the microphone array. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0010] Figure 1 A flow chart of a voice signal acquisition method provided in an embodiment of the present application is shown.

[0011] Figure 2 Shows Figure 1 Flow chart of the method of step S120 in .

[0012] Figure 3 A flow chart of a voice signal acquisition method provided by another embodiment of the present application is shown.

[0013] Figure 4 A structural block diagram of a speech signal acquisition device provided in an embodiment of the present application is shown.

[0014] Figure 5 A structural block diagram of an electronic device provided in an embodiment of the present application is shown.

[0015] Figure 6A storage unit for storing or carrying program codes for implementing the voice signal acquisition method according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0016] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0017] Directional sound pickup is one of the many methods of speech enhancement. It refers to picking up the target signal from the mixed signal according to the direction of the sound source, that is, only picking up the sound signal propagating from a specific direction, while the noise and interference signals from other directions are not picked up but attenuated or shielded, so as to achieve the effect of target speech enhancement. Smart conferences and other scenarios require the use of microphone arrays to achieve directional sound pickup. Usually, the spatial arrangement of microphones in the microphone array is fixed. The directional sound pickup method behind it is usually implemented through a fixed traditional signal processing method, or with the help of a trained neural network model. Regardless of which method is used, the parameters have been fixed according to the microphone arrangement of the microphone array before it is put into use, and it can only be adapted to fixed usage scenarios, such as conference rooms of a specific size range, or fixed sound pickup areas, which greatly limits the flexibility of directional sound pickup of microphone arrays.

[0018] After long-term research, the inventor found that at least one sound source located within the pickup range of the microphone array can be obtained, and the microphone array includes at least two sub-arrays; for each sound source, determine the sub-array whose pickup range is closest to the location of the sound source; adjust the pickup direction of the sub-array so that the center line of the sub-array points to the sound source; and use the output of the sub-array after the pickup direction is adjusted as the voice signal of the sound source. Therefore, for each sound source within the pickup range of the microphone array, by adjusting the pickup direction of the sub-array whose pickup range is closest to the location of the sound source so that the center line direction of the sub-array points to the sound source, the sub-array can collect the voice signal of each sound source in the best pickup direction, thereby improving the overall directional pickup effect of the microphone array; at the same time, since the pickup direction of the sub-array can be changed, the pickup range of the microphone array can be flexibly adjusted, thereby improving the flexibility of directional pickup through the microphone array.

[0019] Therefore, in order to improve the above-mentioned problems, the inventors have proposed the voice signal collection method, device, electronic device and storage medium provided in the present application, which can enable the sub-array to collect voice signals in the best pickup direction for each sound source, thereby improving the overall directional pickup effect of the microphone array; at the same time, since the pickup direction of the sub-array can be changed, the pickup range of the microphone array can be flexibly adjusted, thereby improving the flexibility of directional pickup through the microphone array.

[0020] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0021] See also Figure 1 , shows a flow chart of a voice signal collection method provided by an embodiment of the present application. This embodiment provides a voice signal collection method, which can be applied to an electronic device equipped with a microphone array, wherein the electronic device in this embodiment can be a mobile communication device with a network connection function such as a smart phone and a computer, and the specific device type may not be limited. The method includes:

[0022] Step S110: Acquire at least one sound source within the sound pickup range of a microphone array, where the microphone array includes at least two sub-arrays.

[0023] In the implementation manner of the present application, the microphone array may include at least two sub-arrays (the specific number of sub-arrays may not be limited). Each sub-array may be composed of m (m ≥ 2, the specific value of m may not be limited) microphones in a fixed arrangement, the directional sound pickup algorithms of all sub-arrays may be the same, and the size of the sound pickup area (which can be understood as the sound pickup range) of each sub-array may be expressed as θ (the specific value range of θ may not be limited, preferably, θ may be: 0≤θ≤5°).

[0024] It should be noted that the sound pickup area of ​​the subarray can be a sound pickup area in the horizontal direction (for example, in a meeting scenario, users are sitting and only need to pay attention to the voice signals in the left and right directions), a sound pickup area in the vertical direction, or a sound pickup area in the horizontal and vertical directions (for example, when singing in a chorus, the heights of the standing people are uneven, and both the horizontal and vertical directions need to be considered). Optionally, the horizontal direction can be understood as the left and right direction, and the vertical direction can be understood as the height direction.

[0025] The sound pickup direction of the sub-array may be directly in front of the sub-array. In this embodiment, the center line direction of the sound pickup area of ​​the sub-array may be used as the optimal sound pickup direction of the sub-array.

[0026] As an implementation method, the sound pickup ranges of all sub-arrays together constitute the sound pickup range of the microphone array. When the microphone array is turned on, all sub-arrays will receive voice signals in real time. When a voice signal is received, the location where the voice signal is emitted, that is, the sound source location, can be calculated according to the sound source localization algorithm, so that at least one sound source located within the sound pickup range of the microphone array can be obtained.

[0027] Among them, in order to facilitate more accurate distinction of the object that is currently making sound, the sound source at each calculated position can be stored in a sound source list (the sound source and the sound source position can be stored in the sound source list in association with each other, or the sound source can be directly stored at the position where it is located). Optionally, before storing, you can first check whether the sound source to be stored already exists in the sound source list. If not, you can add the sound source to the sound source list. If it already exists, there is no need to add it again.

[0028] As an implementation method, the active state of all sound sources in the sound source list can also be detected in real time. Optionally, if there is a sound source that is inactive for more than a specified time, the sound source that is inactive for more than the specified time can be removed (deleted) from the sound source list. If the time interval between two voice signals sent by a sound source is greater than the specified time, it can be determined that the sound source is in an inactive state; and if the time interval between two voice signals sent by a sound source is less than the specified time, it can be determined that the sound source is in an active state.

[0029] Step S120: For each sound source, determine a subarray whose sound pickup range is closest to the location of the sound source.

[0030] In this embodiment, regardless of whether the number of sound sources is one or more, for each sound source, in order to enhance the effect of directional sound pickup, a sub-array whose pickup range is closest to the location of the sound source can be determined, and then the sub-array can be used to collect the voice signal emitted by the sound source in a targeted manner, thereby improving the effectiveness and accuracy of voice signal collection.

[0031] The specific process of determining the subarray whose pickup range is closest to the location of the sound source is described as follows:

[0032] See also Figure 2 As an implementation, step S120 may include:

[0033] Step S121: Acquire a first azimuth angle of the sound source relative to the microphone array.

[0034] As an implementation mode, during the process of detecting sound sources by the microphone array, the azimuth of each sound source relative to the microphone array may be determined. For ease of description and distinction, the azimuth of the sound source relative to the microphone array is referred to as a first azimuth.

[0035] The first azimuth angle of the sound source relative to the microphone array can be understood as the azimuth angle of the sound source relative to the center point of the microphone array. As an implementation method, a coordinate system can be constructed based on the microphone array, and each sound source has a coordinate corresponding to the coordinate system. The azimuth angle of the sound source relative to the microphone array can be obtained based on the coordinate and the coordinate of the center point of the microphone array.

[0036] Step S122: Obtain a second azimuth angle of each sub-array relative to the microphone array.

[0037] In this embodiment, the structural arrangement of the microphone array and the multiple sub-arrays it includes is relatively fixed, so that the azimuth angle of each sub-array relative to the microphone array can be determined. For the sake of convenience in description and distinction, the azimuth angle of the sub-array relative to the microphone array is referred to as the second azimuth angle.

[0038] As an implementation method, taking the above example, the coordinates of each subarray in the coordinate system of the microphone array can be determined, and then based on the relative position relationship between the coordinates of the subarray and the coordinates of the center point of the microphone array, the second azimuth angle of each subarray relative to the microphone array can be obtained.

[0039] Step S123: Calculate the difference between the first azimuth angle and the second azimuth angle.

[0040] As an implementation method, in order to determine the angular distance between the sound source and the subarray (which can be referred to as angular moment), the difference between the first azimuth angle and the second azimuth angle can be calculated. In this implementation method, the difference is an angle value. Optionally, since the size relationship between the first azimuth angle and the second azimuth angle can be unlimited, the difference can represent an absolute value.

[0041] Step S124: Determine the subarray corresponding to the second azimuth angle when the difference is the smallest as the subarray whose sound pickup range is closest to the position of the sound source.

[0042] The smaller the difference, the closer the location of the sound source is to the sound pickup range of the subarray, and the larger the difference, the farther the location of the sound source is from the sound pickup range of the subarray. Optionally, when the difference is small enough, it can be considered that the location of the sound source is within the sound pickup range of the subarray.

[0043] As an implementation, the subarray corresponding to the second azimuth angle when the difference is the smallest can be determined as the subarray whose sound pickup range is closest to the location of the sound source. Optionally, if there are multiple subarrays corresponding to the second azimuth angle when the difference is the smallest, any (i.e., randomly determined) subarray can be determined as the subarray whose sound pickup range is closest to the location of the sound source.

[0044] Step S130: adjusting the sound pickup direction of the sub-array so that the center line of the sub-array points to the sound source.

[0045] It is understandable that, when the above difference is small enough, even if the location of the sound source is within the sound pickup range of the sub-array, it is difficult to avoid interference from other sound signals when the sub-array collects the voice signal, or it is impossible to ensure the most accurate collection of the voice signal emitted by the sound source.

[0046] In order to overcome the above problems, the present embodiment can adjust the pickup direction of the sub-array whose pickup range is closest to the location of the sound source, so that the center line of the sub-array points to the sound source, thereby ensuring that the sub-array can collect the voice signal emitted by the sound source in the best pickup direction, thereby improving the effectiveness and accuracy of voice signal collection.

[0047] The sound pickup direction of the subarray can be adjusted by rotating the subarray. The specific rotation direction and angle can be determined according to the relative position relationship between the location of the sound source and the subarray, which is not limited here.

[0048] Step S140: using the output of the subarray after the sound pickup direction is adjusted as the speech signal of the sound source.

[0049] After the sound pickup direction is adjusted, the output of the subarray after the sound pickup direction is adjusted can be used as the voice signal of the sound source. Therefore, in this embodiment, no matter whether the number of sound sources is one or more, each sound source is aimed at the optimal sound pickup direction of the subarray (sound sources at different positions are aimed at different subarrays, and each sound source is pointed at by a dedicated subarray), so that effective directional sound pickup can be performed for each sound source. Therefore, the microphone array as a whole can also complete directional sound pickup better, thereby improving the overall voice signal collection accuracy.

[0050] The voice signal collection method provided in this embodiment obtains at least one sound source within the pickup range of the microphone array, wherein the microphone array includes at least two sub-arrays; for each sound source, determines the sub-array whose pickup range is closest to the location of the sound source; adjusts the pickup direction of the sub-array so that the center line of the sub-array points to the sound source; and uses the output of the sub-array after the pickup direction is adjusted as the voice signal of the sound source. Thus, for each sound source within the pickup range of the microphone array, by adjusting the pickup direction of the sub-array whose pickup range is closest to the location of the sound source so that the center line of the sub-array points to the sound source, the sub-array can collect the voice signal of each sound source in the best pickup direction, thereby improving the overall directional pickup effect of the microphone array; at the same time, since the pickup direction of the sub-array can be changed, the pickup range of the microphone array can be flexibly adjusted, thereby improving the flexibility of directional pickup through the microphone array.

[0051] See also Figure 3 , shows a flow chart of a voice signal collection method provided by another embodiment of the present application. This embodiment provides a voice signal collection method that can be applied to electronic equipment. The method includes:

[0052] Step S210: Acquire at least one sound source within a sound pickup range of a microphone array, where the microphone array includes at least two sub-arrays.

[0053] The specific implementation of step S210 may refer to the relevant description of step S110 in the aforementioned embodiment, which will not be repeated here.

[0054] Step S220: For each sound source, determine a subarray whose sound pickup range is closest to the location of the sound source.

[0055] The specific implementation of step S220 may refer to the relevant description of step S120 in the aforementioned embodiment, which will not be repeated here.

[0056] Step S230: adjusting the sound pickup direction of the sub-array so that the center line of the sub-array points to the sound source.

[0057] The specific implementation of step S230 may refer to the relevant description of step S130 in the aforementioned embodiment, which will not be repeated here.

[0058] Step S240: using the subarray after the sound pickup direction is adjusted as the subarray corresponding to the sound source, and adding an identifier of the subarray to the sound source.

[0059] As an implementation method, in order to facilitate the distinction of the correspondence between different sound sources and different sub-arrays, the sub-array after the sound pickup direction is adjusted can be used as the sub-array corresponding to the sound source, and the identifier of the sub-array is added to the sound source. Therefore, whether it is one sound source or multiple sound sources, it is possible to quickly distinguish which sub-arrays are aimed at different sound sources, and it is also convenient to know which sub-arrays are in an occupied state and which sub-arrays are in an idle state.

[0060] The specific form and content of the subarray identification may not be limited.

[0061] In this embodiment, if there are multiple sound sources, and the number of sound sources is less than or equal to the number of subarrays, for each sound source, it can be detected whether there is at least one sound source without a subarray identifier in the sound pickup range of the subarray after the sound pickup direction is adjusted; if there is, the angle bisector of the two sound sources with the farthest distance between them in the sound pickup range of the subarray can be obtained; then the sound pickup direction of the subarray is adjusted so that the center line of the subarray points to the angle bisector, and the subarray identifier is added to the at least one sound source without a subarray identifier, until there are no new sound sources without a subarray identifier in the sound pickup range of all subarrays. Thus, each subarray can include at least two sound sources in its sound pickup range, and by taking the angle bisector of the two sound sources as the new direction of the subarray, the sound sources in the sound pickup range can be made as close to the subarray as possible to avoid missing the sound sources.

[0062] It is understandable that each sound source may be within the pickup range of multiple different sub-arrays at the same time, that is, the pickup ranges of multiple different sub-arrays may overlap, which will affect the accuracy and effect of directional pickup.

[0063] In order to improve the above problem, as an implementation method, if there is an overlap in the pickup ranges between different subarrays, and there is a sound source in the overlapping area of ​​the pickup ranges, the subarray with the overlapping pickup ranges can be used as the subarray corresponding to the sound source. In this way, since the angular moments between different subarrays and the location of the sound source are different, in order to more accurately collect the voice signal of the sound source, in the process of using the output of the subarray after adjusting the pickup direction as the voice signal of the sound source, the weight parameters of each subarray corresponding to the sound source can be obtained first, and then the voice signal of the sound source can be determined based on the output and weight parameters of all subarrays corresponding to the sound source. It should be noted that the weight parameters of different subarrays are different.

[0064] Wherein, obtaining the weight parameter of each subarray corresponding to the sound source may specifically include: for each subarray corresponding to the sound source, obtaining the angle between the center line of the subarray and the center line connecting the sound source and the microphone array; and obtaining the weight parameter of the subarray according to a specified formula based on the angle. Wherein, if γ is used to represent the angle, the specified formula can be expressed as:

[0065] ω=1-γ / π

[0066] Among them, ω represents the weight parameter, and γ represents the angle.

[0067] In the above manner, the method of determining the speech signal of the sound source based on the output of all subarrays corresponding to the sound source and the weight parameter may specifically include: obtaining the product of the output of each subarray corresponding to the sound source and the weight parameter; obtaining the sum of the products of the output of all subarrays corresponding to the sound source and the weight parameter; obtaining the sum of the products and the entropy of the sum of the weight parameters of all subarrays corresponding to the sound source as the speech signal of the sound source. Exemplarily, assuming that the result of directional sound pickup of the subarray (i.e., the output of the subarray) is Ri, and the number of subarrays is k, then the calculation formula of the speech signal of a sound source can be expressed as:

[0068] (ω1*R1+...+ωi*Ri+...ωk*Rk) / (ω1+...+ωi+...+ωk)

[0069] Among them, R1 represents the directional sound pickup result of the first subarray, ω1 represents the weight parameter of the first subarray; Ri represents the directional sound pickup result of the i-th subarray, ωi represents the weight parameter of the i-th subarray; Rk represents the directional sound pickup result of the k-th subarray, ωk represents the weight parameter of the k-th subarray, i<k.

[0070] In this implementation, the number of sound sources may be greater than the number of subarrays. As an implementation, if the number of sound sources is greater than the number of subarrays, each sound source may be first assigned a subarray according to the method described in the above embodiment. Optionally, if there are still sound sources without subarray identifiers after the subarrays are assigned, a pop-up window prompt may be displayed to prompt or warn the user that the current directional sound pickup effect will deteriorate (because there are voice signals that have not been picked up).

[0071] As an implementation mode, while displaying the pop-up window prompt information, it is possible to detect whether the sound source corresponding to each subarray exists in the sound source list (as can be seen from the above description, a sound source will be kicked out if it is inactive for a long time). Optionally, if different sound sources with the same subarray identifier have not sent out voice signals within a specified time, the different sound sources can be deleted; and the occupation of the subarray can be released. That is, if all the sound sources bound to a subarray do not exist in the sound source list, the subarray can be released and marked as unallocated (i.e., unoccupied and idle).

[0072] Step S250: using the output of the subarray after the sound pickup direction is adjusted as the speech signal of the sound source.

[0073] The specific implementation of step S250 may refer to the relevant description of step S140 in the aforementioned embodiment, which will not be repeated here.

[0074] The voice signal collection method provided in this embodiment obtains at least one sound source within the pickup range of the microphone array, wherein the microphone array includes at least two sub-arrays; for each sound source, determines the sub-array whose pickup range is closest to the location of the sound source; adjusts the pickup direction of the sub-array so that the center line of the sub-array points to the sound source; uses the sub-array after the pickup direction is adjusted as the sub-array corresponding to the sound source, and adds the identifier of the sub-array to the sound source; and uses the output of the sub-array after the pickup direction is adjusted as the voice signal of the sound source. Thus, for each sound source within the pickup range of the microphone array, by adjusting the pickup direction of the sub-array whose pickup range is closest to the location of the sound source, so that the center line direction of the sub-array points to the sound source, the sub-array can collect the voice signal of each sound source in the best pickup direction, thereby improving the overall directional pickup effect of the microphone array; at the same time, since the pickup direction of the sub-array can be changed, the pickup range of the microphone array can be flexibly adjusted, thereby improving the flexibility of directional pickup through the microphone array.

[0075] At the same time, when a sound source has multiple corresponding sub-arrays, accurate voice signal collection can be achieved by determining the final voice signal of the sound source based on weight parameters of different sub-arrays, thereby improving the accuracy and effect of directional pickup.

[0076] See also Figure 4 , is a structural block diagram of a speech signal acquisition device provided in an embodiment of the present application. This embodiment provides a speech signal acquisition device 300, which can be operated on an electronic device. The device 300 includes: a sound source acquisition module 310, a subarray determination module 320, a sound pickup direction adjustment module 330 and a speech signal acquisition module 340:

[0077] The sound source acquisition module 310 is used to acquire at least one sound source located within the sound pickup range of the microphone array, where the microphone array includes at least two sub-arrays.

[0078] The sub-array determination module 320 is used to determine, for each sound source, a sub-array whose sound pickup range is closest to the location of the sound source.

[0079] As an implementation mode, the subarray determination module 320 can be specifically used to obtain a first azimuth angle of the sound source relative to the microphone array; obtain a second azimuth angle of each subarray relative to the microphone array; calculate the difference between the first azimuth angle and the second azimuth angle; and determine the subarray corresponding to the second azimuth angle when the difference is minimum as the subarray whose pickup range is closest to the position of the sound source.

[0080] The sound pickup direction adjustment module 330 is used to adjust the sound pickup direction of the sub-array so that the center line of the sub-array points to the sound source.

[0081] The voice signal collection module 340 is used to use the output of the subarray after the sound pickup direction is adjusted as the voice signal of the sound source.

[0082] Optionally, the device 300 may further include a marking module, which is used to use the subarray after the sound pickup direction is adjusted as the subarray corresponding to the sound source before using the output of the subarray after the sound pickup direction is adjusted as the voice signal of the sound source, and add the identification of the subarray to the sound source.

[0083] In the implementation manner of the present application, the number of sound sources may be one or more. Optionally, if the number of the sound sources is more than one, and the number of the sound sources is less than or equal to the number of the subarrays, the sound pickup direction adjustment module 330 may also be used to detect whether there is at least one sound source without a subarray identifier in the sound pickup range of the subarray after the sound pickup direction is adjusted; if so, obtain the angular bisector of the two sound sources with the farthest interval in the sound pickup range of the subarray; adjust the sound pickup direction of the subarray so that the center line of the subarray points to the angular bisector. In this case, the marking module may also be used to add the identifier of the subarray to the at least one sound source without a subarray identifier.

[0084] Optionally, in this embodiment, if there is an overlap in the pickup ranges between different subarrays, and there is a sound source in the overlapping area of ​​the pickup ranges, the subarrays with overlapping pickup ranges can be used as the subarrays corresponding to the sound source (in this case, the sound source located in the overlapping pickup range). In this way, the voice signal acquisition module 340 can be specifically used to obtain the weight parameter of each subarray corresponding to the sound source; and determine the voice signal of the sound source based on the output of all subarrays corresponding to the sound source and the weight parameter.

[0085] Among them, obtaining the weight parameters of each subarray corresponding to the sound source may include: for each subarray corresponding to the sound source, obtaining the angle between the center line of the subarray and the center line connecting the sound source and the microphone array; and obtaining the weight parameters of the subarray according to a specified formula based on the angle.

[0086] Among them, determining the speech signal of the sound source based on the outputs of all subarrays corresponding to the sound source and the weight parameters may include: obtaining the product of the output of each subarray corresponding to the sound source and the weight parameter; obtaining the sum of the products of the outputs of all subarrays corresponding to the sound source and the weight parameters; obtaining the sum of the products and the entropy of the sum of the weight parameters of all subarrays corresponding to the sound source as the speech signal of the sound source.

[0087] Optionally, the device 300 may further include a prompt module, configured to display a pop-up prompt message if the number of the sound sources is greater than the number of the sub-arrays and there are sound sources without a sub-array identifier.

[0088] Optionally, the device 300 may further include a subarray management module, which is used to delete the different sound sources added with the same subarray identifier if the different sound sources do not send out a voice signal within a specified time period; and release the occupancy of the subarray.

[0089] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and modules can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here.

[0090] In several embodiments provided in the present application, the coupling between modules may be electrical, mechanical or other forms of coupling.

[0091] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or software functional modules.

[0092] See also Figure 5 Based on the above-mentioned speech signal collection method and device, the embodiment of the present application also provides an electronic device 100 that can execute the above-mentioned speech signal collection method. The electronic device 100 includes a memory 102 and one or more (only one is shown in the figure) processors 104 coupled to each other, and the memory 102 and the processor 104 are connected by a communication line. The memory 102 stores a program that can execute the content of the above-mentioned embodiment, and the processor 104 can execute the program stored in the memory 102.

[0093] Among them, the processor 104 may include one or more processing cores. The processor 104 uses various interfaces and lines to connect various parts of the entire electronic device 100, and executes various functions and processes data of the electronic device 100 by running or executing instructions, programs, code sets or instruction sets stored in the memory 102, and calling data stored in the memory 102. Optionally, the processor 104 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 104 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 104, but may be implemented separately through a communication chip.

[0094] The memory 102 may include a random access memory (RAM) or a read-only memory (ROM). The memory 102 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 102 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the aforementioned embodiments, etc. The data storage area may also store data (such as a phone book, audio and video data, chat record data) created by the electronic device 100 during use.

[0095] Please refer to Figure 6, which shows a structural block diagram of a computer-readable storage medium provided in an embodiment of the present application. The computer-readable medium 400 stores program codes, which can be called by a processor to execute the method described in the above method embodiment.

[0096] The computer readable storage medium 400 may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read-only memory), an EPROM, a hard disk, or a ROM. Optionally, the computer readable storage medium 400 includes a non-transitory computer-readable storage medium. The computer readable storage medium 400 has storage space for program code 410 that performs any method steps in the above method. These program codes can be read from or written to one or more computer program products. The program code 410 can be compressed, for example, in an appropriate form.

[0097] In summary, the embodiments of the present application provide a method, device, electronic device and storage medium for collecting voice signals, by acquiring at least one sound source located within the pickup range of a microphone array, wherein the microphone array includes at least two sub-arrays; for each sound source, determining the sub-array whose pickup range is closest to the location of the sound source; adjusting the pickup direction of the sub-array so that the center line of the sub-array points to the sound source; and using the output of the sub-array after the pickup direction is adjusted as the voice signal of the sound source. Thus, for each sound source within the pickup range of the microphone array, by adjusting the pickup direction of the sub-array whose pickup range is closest to the location of the sound source so that the center line direction of the sub-array points to the sound source, the sub-array can collect voice signals from each sound source in the best pickup direction, thereby improving the overall directional pickup effect of the microphone array; at the same time, since the pickup direction of the sub-array can be changed, the pickup range of the microphone array can be flexibly adjusted, thereby improving the flexibility of directional pickup through the microphone array.

[0098] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for collecting speech signals, characterized in that: The method comprises: Acquire at least one sound source within a sound pickup range of a microphone array, wherein the microphone array includes at least two sub-arrays; For each sound source, determining a subarray whose pickup range is closest to the location of the sound source; Adjusting the sound pickup direction of the subarray so that the center line of the subarray points to the sound source; The output of the subarray after the sound pickup direction is adjusted is used as the speech signal of the sound source.

2. The method according to claim 1, characterized in that The determining of the subarray whose sound pickup range is closest to the location of the sound source includes: Acquire a first azimuth angle of the sound source relative to the microphone array; Obtaining a second azimuth angle of each sub-array relative to the microphone array; Calculating a difference between the first azimuth angle and the second azimuth angle; The subarray corresponding to the second azimuth angle when the difference is the smallest is determined as the subarray whose sound pickup range is closest to the position of the sound source.

3. The method according to claim 1, characterized in that Before using the output of the subarray after the sound pickup direction is adjusted as the voice signal of the sound source, the method further includes: The subarray after the sound pickup direction is adjusted is used as the subarray corresponding to the sound source, and an identifier of the subarray is added to the sound source.

4. The method according to claim 3, characterized in that If there are multiple sound sources, and the number of the sound sources is less than or equal to the number of the sub-arrays, the method further includes: Detecting whether there is at least one sound source without a subarray identifier within the sound pickup range of the subarray after the sound pickup direction is adjusted; If so, obtaining the angular bisector of the two sound sources with the farthest interval within the pickup range of the subarray; The sound pickup direction of the subarray is adjusted so that the center line of the subarray points to the angle bisector, and the identification of the subarray is added to at least one sound source to which the subarray identification is not added.

5. The method according to claim 3 or 4, characterized in that: The method further comprises: If there is overlap in sound pickup ranges between different sub-arrays, and there is a sound source in the overlapping area of ​​the sound pickup ranges, the sub-arrays with overlapping sound pickup ranges are used as the sub-arrays corresponding to the sound source; The step of using the output of the subarray after adjusting the sound pickup direction as the speech signal of the sound source comprises: Obtaining a weight parameter of each subarray corresponding to the sound source; The speech signal of the sound source is determined based on the outputs of all sub-arrays corresponding to the sound source and the weight parameters.

6. The method according to claim 5, characterized in that The obtaining of the weight parameter of each subarray corresponding to the sound source comprises: For each subarray corresponding to the sound source, obtaining an angle between a center line of the subarray and a line connecting the centers of the sound source and the microphone array; A weight parameter of the subarray is obtained based on the included angle according to a specified formula.

7. The method according to claim 5, characterized in that The method of determining the speech signal of the sound source based on the outputs of all sub-arrays corresponding to the sound source and weight parameters includes: Obtaining the product of the output of each sub-array corresponding to the sound source and a weight parameter; Obtaining the sum of the products of the outputs of all sub-arrays corresponding to the sound source and the weight parameter; The sum of the products and the entropy of the sum of weight parameters of all sub-arrays corresponding to the sound source are obtained as the speech signal of the sound source.

8. The method according to claim 5, characterized in that If the number of the sound sources is greater than the number of the sub-arrays, the method further includes: If there is a sound source without a subarray identifier, a pop-up window prompt is displayed.

9. The method according to claim 8, characterized in that The method further comprises: If a different sound source with the same subarray identifier added does not send a voice signal within a specified time period, delete the different sound source; The subarray is released from occupation.

10. A voice signal collection device, characterized in that: The device comprises: A sound source acquisition module, used to acquire at least one sound source within the sound pickup range of a microphone array, wherein the microphone array includes at least two sub-arrays; A subarray determination module, used to determine, for each sound source, a subarray whose pickup range is closest to the location of the sound source; A sound pickup direction adjustment module, used to adjust the sound pickup direction of the subarray so that the center line of the subarray points to the sound source; The voice signal acquisition module is used to use the output of the subarray after the sound pickup direction is adjusted as the voice signal of the sound source.

11. An electronic device, characterized in that: including one or more processors and memory; One or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program code, wherein when the program code is executed by a processor, the method according to any one of claims 1 to 9 is executed.

Images