Sound source positioning method, related method, device, equipment and storage medium

Abstract

The invention discloses a sound source positioning method, a related method, a related device, equipment and a storage medium, and the method comprises the steps: obtaining a to-be-processed audio collected by a pickup array comprising two microphones; the to-be-processed audio is processed based on the sound source positioning model, and the relative position from the target sound source to the pickup array is predicted; wherein the sound source positioning model is obtained based on sample audio training, the sample audio is marked with an actual area where a sample sound source is located, the sample audio at least comprises a simulation audio, and the simulation audio is obtained by superposing collected audios of a sample array when the audios are played in a plurality of in-vehicle areas; and before the sound source positioning model is trained, the loudspeaking mechanism plays audio in each in-vehicle area in sequence. According to the scheme, the hardware cost, the operation and maintenance difficulty and the performance requirement on the vehicle-mounted audio processing system can be reduced as much as possible on the premise of realizing multiple audio regions, and meanwhile, the influence of hardware reasons on the multi-audio region partitioning effect is reduced as much as possible.

Description

Technical Field

[0001] The present application relates to the field of audio processing technology, and in particular to a sound source localization method and related methods, devices, equipment and storage media. Background Art

[0002] The automotive industry is at a critical stage of intelligent transformation. As an intuitive and natural way of human-computer interaction, voice technology has become an important part of the development of automotive intelligence. In the process of automotive intelligence development, multi-audio zone interaction in the car has become a key factor in improving user experience.

[0003] At present, the existing technology either uses two microphones to meet the left and right sound zones, but cannot achieve multiple sound zones, that is, the front and rear sound zones on the same side cannot be isolated from each other, or uses four microphones to meet the four sound zones of the main driver, co-driver, left rear row, and right rear row, but the cost is high and the operation and maintenance are difficult. In addition, since the audio signals collected by multiple microphones need to be processed and analyzed in a complex manner, the performance requirements of the vehicle audio processing system are high. At the same time, the frequency response and phase consistency between multiple microphones are difficult to ensure, which may have a serious impact on the zoning effect of multiple sound zones. In view of this, how to reduce the hardware cost, operation and maintenance difficulty and the performance requirements of the vehicle audio processing system as much as possible under the premise of achieving multiple sound zones, while minimizing the impact of hardware reasons on the zoning effect of multiple sound zones, has become an urgent problem to be solved. Summary of the invention

[0004] The main technical problem solved by the present application is to provide a sound source localization method and related methods, devices, equipment and storage media, which can reduce the hardware cost, operation and maintenance difficulty and performance requirements for the vehicle audio processing system as much as possible under the premise of realizing multiple sound zones, and at the same time minimize the impact of hardware reasons on the multi-tone zone zoning effect.

[0005] In order to solve the above technical problems, the first aspect of the present application provides a sound source localization method, including: obtaining the audio to be processed collected by a sound pickup array including dual microphones; processing the audio to be processed based on a sound source localization model, and predicting the relative position of the target sound source to the sound pickup array; wherein the relative position includes a relative direction and a relative distance, and the sound source localization model is obtained based on sample audio training, and the sample audio is marked with the actual area where the sample sound source is located, and the sample audio at least includes simulated audio, and the simulated audio is obtained by superimposing the collected audio of the sample array when the audio is played in multiple in-vehicle areas respectively, and before the sound source localization model is trained, the sound speaker mechanism plays the audio in each in-vehicle area in turn.

[0006] In order to solve the above technical problems, the second aspect of the present application provides a sound enhancement method, including: predicting the relative position of the target sound source to the sound pickup array; wherein the relative position is determined based on the sound source localization method in the above first aspect; based on the relative position of the target sound source to the sound pickup array, obtaining the area inside the vehicle where the target sound source is located as the target area; adjusting the sound pickup area of ​​the sound pickup array based on beamforming to cover the target area, and obtaining the target audio.

[0007] In order to solve the above technical problems, the third aspect of the present application provides a speech recognition method, including: obtaining target audio; wherein the target audio is obtained based on the sound enhancement method in the above second aspect; performing speech recognition based on the target audio to obtain recognized text.

[0008] In order to solve the above technical problems, the fourth aspect of the present application provides a sound source localization device, including: an audio acquisition module and a position prediction module, the audio acquisition module is used to acquire the audio to be processed collected by a sound pickup array including dual microphones; the position prediction module is used to process the audio to be processed based on a sound source localization model, and predict the relative position of the target sound source to the sound pickup array; wherein the relative position includes a relative direction and a relative distance, the sound source localization model is obtained based on sample audio training, the sample audio is marked with the actual area where the sample sound source is located, the sample audio at least includes simulated audio, the simulated audio is obtained by superimposing the collected audio of the sample array when the audio is played in multiple in-vehicle areas respectively, and before the sound source localization model is trained, the sound speaker mechanism plays the audio in each in-vehicle area in turn.

[0009] In order to solve the above technical problems, the fifth aspect of the present application provides a sound enhancement device, including: a position prediction module, an area determination module and a beamforming module, the position prediction module is used to predict the relative position of the target sound source to the sound pickup array; wherein the relative position is determined based on the sound source positioning device in the above fourth aspect; the area determination module is used to obtain the in-vehicle area where the target sound source is located as the target area based on the relative position of the target sound source to the sound pickup array; the beamforming module is used to adjust the sound pickup area of ​​the sound pickup array based on beamforming to cover the target area and obtain the target audio.

[0010] In order to solve the above technical problems, the sixth aspect of the present application provides a speech recognition device, including: an audio acquisition module and an audio recognition module, the audio acquisition module is used to acquire target audio; wherein the target audio is obtained based on the sound enhancement device in the above fifth aspect; the audio recognition module is used to perform speech recognition based on the target audio to obtain recognized text.

[0011] In order to solve the above technical problems, the seventh aspect of the present application provides an electronic device, comprising at least a memory and a processor coupled to each other, wherein the memory at least stores program instructions, and the processor is used to execute the program instructions to implement the sound source localization method in the above first aspect, or to implement the sound enhancement method in the above second aspect, or to implement the speech recognition method in the above third aspect.

[0012] In order to solve the above technical problems, the eighth aspect of the present application provides a computer-readable storage medium, which stores program instructions that can be executed by a processor, and the program instructions are used to implement the sound source localization method of the first aspect, or to implement the sound enhancement method of the second aspect, or to implement the speech recognition method of the third aspect.

[0013] The above scheme obtains the audio to be processed collected by the sound pickup array including dual microphones, and then processes the audio to be processed based on the sound source localization module, and predicts the relative position of the target sound source compared to the sound pickup array, the relative position includes the relative direction and the relative distance, and the sound source localization model is obtained based on sample audio training, and the sample audio is marked with the actual area where the sample sound source is located. The sample audio at least includes simulated audio, and the simulated audio is obtained by superimposing the collected audio of the sample array when the audio is played in multiple areas inside the car respectively, and before the sound source localization model is trained, the speaker mechanism plays the audio in each area inside the car in turn. Therefore, on the one hand, since the sample audio includes the simulated audio, and the simulated audio is obtained by superimposing the collected audio of the sample array when the audio is played in multiple areas inside the car respectively. The collected audio of the frequency-time sample array is superimposed, and the speaker mechanism plays the audio in each area of ​​the car in turn before the sound source localization model is trained. Therefore, by playing the collected audio in each area of ​​the car in turn, especially the simulated audio formed by superimposing the collected audio, the audio data of human-computer interaction in different areas in the real environment can be simulated as much as possible. Then the model is trained based on this, which enables the model to model and distinguish the sound representation of different areas during the training process. Even in complex scenarios such as multiple people talking at the same time, accurate sound zone separation can be achieved. On the other hand, there is no need for multiple microphones such as four microphones, but only two microphones are needed to support multiple areas to meet the isolation between different areas of the front and rear rows on the same side. Therefore, under the premise of realizing multiple sound zones, the hardware cost, operation and maintenance difficulty and performance requirements for the vehicle audio processing system can be reduced as much as possible, while minimizing the impact of hardware reasons on the effect of multiple sound zone zoning. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a flow chart of an embodiment of a sound source localization method of the present application;

[0015] Figure 2 It is a flowchart of an embodiment of the sound enhancement method of the present application;

[0016] Figure 3It is a flowchart of an embodiment of the speech recognition method of the present application;

[0017] Figure 4 It is a schematic diagram of the framework of an embodiment of a speech decoding device of the present application;

[0018] Figure 5 It is a schematic diagram of the framework of an embodiment of the sound enhancement device of the present application;

[0019] Figure 6 It is a schematic diagram of the framework of an embodiment of the speech recognition device of the present application;

[0020] Figure 7 It is a schematic diagram of the framework of an embodiment of the electronic device of the present application;

[0021] Figure 8 It is a schematic diagram of a framework of an embodiment of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0022] The scheme of the embodiment of the present application is described in detail below in conjunction with the drawings of the specification.

[0023] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.

[0024] The terms "system" and "network" are often used interchangeably in this article. The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the fragment " / " in this article generally indicates that the associated objects before and after are in an "or" relationship. In addition, "many" in this article means two or more than two.

[0025] See also Figure 1 , Figure 1 It is a flow chart of an embodiment of a sound source localization method of the present application.

[0026] Specifically, the following steps may be included:

[0027] Step S11: obtaining the audio to be processed collected by the sound pickup array including two microphones.

[0028] In one implementation scenario, the sound pickup array can be arranged in the front row of the vehicle. For example, the dual microphones in the sound pickup array can be located on both sides of the front ceiling light or sun visor, and the specific position of the sound pickup array in the front row of the vehicle is not limited. In addition, as a special example, the dual microphones can be symmetrical about the middle axis of the vehicle.

[0029] In one implementation scenario, the microphone in the sound pickup array can be a microphone with high sensitivity and wide frequency response range to ensure accurate collection of various sound signals from low-frequency human voice to high-frequency noise as much as possible. At the same time, the microphone can have good anti-interference performance and can effectively resist electromagnetic interference generated by equipment such as automobile engines and air conditioners.

[0030] In one implementation scenario, the audio to be processed may specifically include a first audio and a second audio collected by two microphones respectively, that is, the audio to be processed may specifically include two sound channels.

[0031] Step S12: Process the audio to be processed based on the sound source localization model to predict the relative position of the target sound source to the sound pickup array.

[0032] In the disclosed embodiment, the relative position may include a relative direction and a relative distance. For example, the center of the pickup array may be used as a reference, and a straight line passing through the center and parallel to the center of the vehicle may be made, denoted as L1, and the straight line formed by connecting the target sound source with the center may be denoted as L2, and the relative direction may represent the deviation direction of L2 relative to L2 (e.g., α degrees to the left, β degrees to the right, etc. on the horizontal plane; or, the deviation angle on each plane in the physical space). In addition, the relative distance may include the straight-line distance from the target sound source on the horizontal plane to the aforementioned center, or the relative distance may also include the straight-line distance from the target sound source in the physical space to the aforementioned center. In actual application, the relative direction and relative distance on the horizontal plane may be selected for measurement according to the actual application needs, or the relative direction and relative distance in the physical space may be selected for measurement. The above-mentioned measurement methods are only several possible measurement methods for relative direction and relative distance in actual application, and the measurement methods for relative direction and relative distance are not limited here.

[0033] In the disclosed embodiments, the sound source localization model can be obtained based on sample audio training, and the sample audio can be labeled with the actual area where the sample sound source is located. Exemplarily, taking the distinction between the four areas of the main driver's seat, the co-driver's seat, the left rear seat, and the right rear seat as an example, the sample audio can be labeled with any of the above-mentioned in-car areas according to the actual position of the sample sound source in the above four areas. For example, if the sample sound source is in the main driver's seat, the sample audio can be labeled with the actual area "main driver's seat"; or, if the sample sound source is in the left rear seat, the sample audio can be labeled with the actual area "left rear seat". Of course, the above examples are only a few possible examples of sample audio in actual applications, and there is no limitation on the sample audio and its labeled actual areas.

[0034] In the disclosed embodiment, the sample audio may include at least simulated audio, which may be obtained by superimposing the collected audio of the sample array when the audio is played in multiple in-car areas respectively, and the speaker mechanism plays the audio in each in-car area in turn before the sound source localization model is trained. In addition, as a possible example, the sample audio may also include collected audio.

[0035] In one implementation scenario, when the sample audio includes simulated audio, each in-car area can be selected as the current area, and the speaker mechanism can be controlled to play audio in the current area, and then the sample array can be obtained when the speaker mechanism plays audio. The collected audio. As a possible example, the sample array and the pickup array can have the same microphone layout, such as the sample array and the pickup array are both set at the front ceiling light position, and are symmetrical about the center axis of the vehicle. On this basis, at least one area combination consisting of multiple in-car areas can be obtained, and then the collected audio when the speaker mechanism plays audio in each in-car area in the area group in turn can be obtained and superimposed to obtain simulated audio. The above method controls the speaker mechanism to play audio in each in-car area, and obtains the collected audio during playback, so as to combine and superimpose the collected audio corresponding to multiple in-car areas to obtain simulated audio, which can simulate multi-sound source scenes and help improve the support of the sound source localization model for sound source localization in complex scenes such as multiple sound sources.

[0036] In a specific implementation scenario, as a possible example, after selecting the current area, the current area can also be divided into multiple sub-areas. For example, the current area can be divided into nine sub-areas in the form of a nine-square layout. Of course, the above example is only a possible example of dividing the current area, and does not limit the division method of the current area. The division method of the current area is no longer given one by one. On this basis, the speaker mechanism can be controlled to play audio in the sub-areas in the current area in sequence. For example, still taking the current area divided in the form of a nine-square grid as an example, the speaker mechanism can be controlled to play audio in sequence on the nine sub-areas obtained by dividing the current area. On this basis, when playing audio on the first sub-area, the sample array can be obtained. The collected audio at this time can be obtained, and when playing audio on the second sub-area, the sample array can be obtained. The collected audio at this time can be obtained, and so on, until when playing audio on the ninth sub-area, the sample array can be obtained. The collected audio at this time. Of course, when the current area is divided into other forms, it can also be inferred by analogy that several collected audios when playing audio on the current area can be obtained, and no one is given here. The above method divides the current area into multiple sub-areas and controls the speaker mechanism to play audio in the sub-areas in the current area in turn. This can simulate possible situations where human-computer interaction occurs when the sound source is at different positions in a certain area within the vehicle, and helps to reflect various scenarios of real interaction as comprehensively as possible.

[0037] In a specific implementation scenario, still taking the distinction of four areas, namely, the main driver's seat, the co-driver's seat, the left rear seat, and the right rear seat, as an example, at least two of the in-car areas can be arbitrarily selected to form an area combination. For example, the in-car area "main driver's seat" and the in-car area "co-driver's seat" can be selected to form an area combination; or, the in-car area "co-driver's seat" and the in-car area "left rear seat" can be selected to form another area combination; or, the in-car area "left rear seat" and the in-car area "right rear seat" can be selected to form another area combination. Of course, the above examples are only a few possible scenarios of area combinations, and the specific scenarios of area combinations will not be given examples one by one here.

[0038] In a specific implementation scenario, still taking the distinction of four areas, namely, the main driver's seat, the front passenger's seat, the left rear seat, and the right rear seat, as an example, when the interior area "front passenger's seat" and the interior area "left rear seat" are selected to form an area combination, the sample array collected audio when the speaker mechanism plays audio in the interior area "front passenger's seat" (e.g., the "first audio 1" of the left microphone and the "second audio 1" of the right microphone in the dual microphones) can be obtained, and the sample array collected audio when the speaker mechanism plays audio in the interior area "left rear seat" (e.g., the "first audio 2" and the "second audio 3" of the left microphone in the dual microphones) can be obtained. The "second audio 2" of the right microphone) is then superimposed on the above-collected audio. Specifically, the collected audio of the same microphone in the dual microphones can be superimposed, such as superimposing the "first audio 1" and "first audio 2" of the left microphone to obtain "first audio 12", and superimposing the "second audio 1" and "second audio 2" of the right microphone to obtain "second audio 12". At this time, the simulated audio includes the superimposed audio after the collected audio of the same microphone in the dual microphones is superimposed, such as "first audio 12" and "second audio 12". Of course, the above example is only a possible example of audio superposition in actual application, and other possible situations are not given examples one by one here.

[0039] In a specific implementation scenario, in the process of obtaining the collected audio, the in-car area where the audio is collected can also be recorded, and the in-car area where the audio is collected represents the current area where the speaker mechanism is located when the sample array obtains the collected audio. Still taking the distinction of the four areas of the main driver, the co-driver, the left rear seat, and the right rear seat as an example, when the speaker mechanism is located at any position on the in-car area "main driver" to play the audio, the in-car area "main driver" can be recorded for the collected audio of the sample array to represent that the sample audio is collected when the speaker mechanism is located in the in-car area "main driver", and other situations can be deduced by analogy, and no examples are given here. On this basis, the actual area can be marked for the simulated audio based on the speaker mechanism in each in-car area in the area combination. As a possible example, the in-car area in the target direction can be selected in the area combination and marked as the actual area of ​​the simulated audio. Still taking the in-car area "co-driver" and the in-car area "left rear seat" as an example of a regional combination, the in-car area "co-driver" or "left rear seat" in the target direction can be selected in the regional combination and marked as the actual area of ​​the simulated audio. It should be noted that the target direction can be set according to the actual application situation. Exemplarily, the target direction can be defined by distinguishing whether the audio played in each in-car area in the area combination has practical significance. In the aforementioned example, if the audio played in the in-car area "co-pilot" is a voice command (such as "turn on the air conditioner", etc.), and the audio played in the in-car area "left rear seat" is chatting, then the target direction can be determined as the direction of the in-car area where the played audio has practical significance, such as the target direction in the aforementioned example is the direction of the in-car area "co-pilot"; or, the target direction can be defined by distinguishing the relative sound levels of the audio played in each in-car area in the area combination. In the aforementioned example, if the audio played in the in-car area "co-pilot" has a higher volume than the audio played in the in-car area "left rear seat", then the direction of the in-car area where the audio played has a higher volume can be determined as the target direction, such as the target direction in the aforementioned example is the in-car area "co-pilot". Of course, the above examples are only a few possible scenarios of the target direction, and the specific setting of the target direction is not limited here. In addition, the target direction can be determined by combining multiple setting methods, such as combining whether the audio played in each in-car area in the area combination has practical significance and the relative sound level of the audio played in each in-car area in the area combination to distinguish and define the target direction. In the above method, the in-car area where the audio is collected is recorded, and the in-car area representation sample array of the collected audio obtains the current area where the speaker mechanism is located when the audio is collected, and then based on each in-car area in the area combination corresponding to the simulated audio, the actual area is marked for the simulated audio. Specifically, the in-car area in the target direction is selected in the area combination and marked as the actual area of ​​the simulated audio, which can determine the actual area to be marked for the simulated audio.

[0040] In another implementation scenario, as mentioned above, the sample audio may also include collected audio. In this case, the collected audio when the speaker mechanism plays audio in each in-car area can be directly obtained and used as sample audio respectively. In addition, it should be noted that the actual area marked with the sample audio in this case is the in-car area where the speaker mechanism is located when the collected audio is obtained. For example, when the speaker mechanism plays audio in the "main driving" area in the car, the collected audio of the sample array can be obtained, and when the collected audio is used as the sample audio, the in-car area "main driving" can be marked as the actual area of ​​the sample audio. Of course, the above example is only a possible example in the actual application process, and other possible situations will not be given examples one by one here.

[0041] In an implementation scenario, after obtaining the sample audio, the sound source localization model can be trained based on the sample audio. Specifically, the sample audio can be processed based on the sound source localization model to predict the predicted position of the sample sound source relative to the sample array, and the predicted position includes the predicted direction and predicted distance of the sample sound source relative to the sample array. It should be noted that the specific acquisition process of the predicted position can refer to the relevant description of the sound source localization model processing the audio to be processed, which will not be repeated here. On this basis, the in-car area can be selected as the predicted area of ​​the sample sound source based on the predicted direction, predicted distance and the position layout of each in-car area, and then the network parameters of the sound source localization model can be adjusted based on the difference between the predicted area and the actual area. In the above method, the predicted position of the sample sound source relative to the sample array is first predicted during the training process, and then the in-car area is selected as the predicted area of ​​the sample sound source based on the position layout of each in-car area, and then the network parameters of the sound source localization model are adjusted based on the difference between the predicted area and the actual area, which can force the sound source localization model to learn the acoustic representation of each in-car area during the training process with the actual area as the target.

[0042] In a specific implementation scenario, similar to the aforementioned sound pickup array, in actual application, a straight line passing through the center of the sample array and parallel to the center of the vehicle can be used as a reference line, and the deviation angle between the straight line connecting the center and the sample sound source and the reference line is the predicted direction of the sample sound source relative to the sample array, and the specific distance from the sample sound source to the center is the predicted distance of the sample sound source relative to the sample array. In this case, after obtaining the predicted direction and predicted distance, a circle can be drawn with the center as the center and the predicted distance as the radius, and then a point can be taken on the circle, and the line between the point and the center of the circle deviates from the predicted direction of the aforementioned reference line (e.g., a certain angle to the left, a certain angle to the right, etc.).

[0043] In a specific implementation scenario, after the location of the sample sound source is determined by combining the predicted direction and the predicted distance through an example, the location layout of each in-car area can be further combined to select the in-car area as the predicted area of ​​the sample sound source. Exemplarily, the location layout of each in-car area can be represented by a coordinate range, and the location of the sample sound source can also be represented by a coordinate value, so the in-car area can be selected as the predicted area of ​​the sample sound source based on the coordinate value of the sample sound source and the coordinate range of each in-car area.

[0044] In a specific implementation scenario, in order to further improve the model performance of the sound source localization model, the sample audio can cover different car models with the same microphone layout, different driving speeds, voice characteristics of different male and female people, and sound data under various environmental noise conditions.

[0045] In an implementation scenario, after the sound source localization model is trained through the aforementioned process, the performance of the sound source localization model can be further tested using a test set. For example, test audio can be recorded in each area of ​​the vehicle in postures such as sitting upright, leaning forward, tilting the head 45 degrees, and tilting the head 90 degrees, and the actual area of ​​each test audio (for details, refer to the actual area of ​​the aforementioned sample audio) is recorded as a data label to perform a performance test on the sound source localization model.

[0046] In an implementation scenario, after the sound source localization model is obtained through the training of the above process, the sound source localization model can be used to localize the sound source of the audio to be processed. Specifically, as mentioned above, the audio to be processed may include the first audio and the second audio respectively collected by the dual microphones, then the arrival time difference between the first audio and the second audio can be obtained, and the sound intensity difference between the first audio and the second audio can be obtained, and then the arrival time difference and the sound intensity difference are processed based on the sound source localization model to predict the relative position. Of course, the above example is only a possible example in the actual application process, and the specific process of the sound source localization model processing audio is not limited here. For example, the audio can also be directly input into the sound source localization model to obtain the relative position of the target sound source to the pickup array. In the above method, the arrival time difference between the first audio and the second audio is obtained, and the sound intensity difference between the first audio and the second audio is obtained, and then the arrival time difference and the sound intensity difference are processed based on the sound source localization model to predict the relative position, that is, the arrival time difference and the sound intensity difference are used as model features so that the model can learn differentiated feature representations in different pronunciation areas accordingly.

[0047] The above scheme obtains the audio to be processed collected by the sound pickup array including dual microphones, and then processes the audio to be processed based on the sound source localization module, and predicts the relative position of the target sound source compared to the sound pickup array, the relative position includes the relative direction and the relative distance, and the sound source localization model is obtained based on sample audio training, and the sample audio is marked with the actual area where the sample sound source is located. The sample audio at least includes simulated audio, and the simulated audio is obtained by superimposing the collected audio of the sample array when the audio is played in multiple areas inside the car respectively, and before the sound source localization model is trained, the speaker mechanism plays the audio in each area inside the car in turn. Therefore, on the one hand, since the sample audio includes the simulated audio, and the simulated audio is obtained by superimposing the collected audio of the sample array when the audio is played in multiple areas inside the car respectively. The collected audio of the frequency-time sample array is superimposed, and the speaker mechanism plays the audio in each area of ​​the car in turn before the sound source localization model is trained. Therefore, by playing the collected audio in each area of ​​the car in turn, especially the simulated audio formed by superimposing the collected audio, the audio data of human-computer interaction in different areas in the real environment can be simulated as much as possible. Then the model is trained based on this, which enables the model to model and distinguish the sound representation of different areas during the training process. Even in complex scenarios such as multiple people talking at the same time, accurate sound zone separation can be achieved. On the other hand, there is no need for multiple microphones such as four microphones, but only two microphones are needed to support multiple areas to meet the isolation between different areas of the front and rear rows on the same side. Therefore, under the premise of realizing multiple sound zones, the hardware cost, operation and maintenance difficulty and performance requirements for the vehicle audio processing system can be reduced as much as possible, while minimizing the impact of hardware reasons on the effect of multiple sound zone zoning.

[0048] See also Figure 2 , Figure 2 It is a flow chart of an embodiment of a speech enhancement method of the present application.

[0049] Specifically, the following steps may be included:

[0050] Step S21: predicting the relative position of the target sound source to the sound pickup array.

[0051] In the embodiment of the present disclosure, the relative position is determined based on the process steps in any of the above-mentioned sound source localization method embodiments. For details, please refer to the process steps in the above-mentioned sound source localization method embodiments, which will not be repeated here.

[0052] Step S22: Based on the relative position of the target sound source to the sound pickup array, the vehicle interior area where the target sound source is located is obtained as the target area.

[0053] Specifically, as in the aforementioned disclosed embodiments, the relative position may include a relative direction and a relative distance. Then, when the center point of the dual microphones in the sound pickup array is used as a reference, a circle may be drawn with the center point as the center and the relative distance as the radius. Then, a point may be taken on the circle, and the deviation between the line connecting the point and the center point relative to the reference line passing through the center point and parallel to the middle of the vehicle satisfies the "relative direction" (e.g., a specific angle to the left, a specific angle to the right, etc.). Then, the point may be regarded as the location of the target sound source, and the location layout of each in-car area may be further combined to select the in-car area as the target area where the target sound source is located. For details, please refer to the relevant description of "selecting the in-car area as the predicted area of ​​the sample sound source based on the predicted direction, predicted distance and the location layout of each in-car area" in the aforementioned disclosed embodiments, which will not be repeated here.

[0054] Step S23: adjusting the sound pickup area of ​​the sound pickup array based on beam forming to cover the target area and obtain the target audio.

[0055] Specifically, after determining the target area, the pickup area of ​​the pickup array can be adjusted based on beamforming so that the adjusted pickup area can cover the target area. In this case, the collected audio signal can be used as the target audio for subsequent tasks (such as speech recognition, etc.). It should be noted that the specific process of adjusting the pickup area can refer to the technical details of beamforming, which will not be repeated here.

[0056] The above scheme predicts the relative position of the target sound source to the sound pickup array, and the relative position is obtained based on any of the above-mentioned sound source localization method embodiments, and based on the relative position of the target sound source to the sound pickup array, the area inside the vehicle where the target sound source is located is obtained as the target area, and then the sound pickup area of ​​the sound pickup array is adjusted based on beamforming to cover the target area to obtain the target audio. Therefore, after determining the relative position of the target sound source to the sound pickup array through any of the above-mentioned sound source localization method embodiments, the target area where the target sound source is located is determined accordingly, and the sound pickup area is adjusted based on beamforming, so that the sound signal of the target sound source can be enhanced as much as possible, and the sound signals of other irrelevant sound sources can be suppressed, which helps to improve the audio quality of the target audio.

[0057] See also Figure 3 , Figure 3 It is a flow chart of an embodiment of the speech recognition method of the present application.

[0058] Specifically, the following steps may be included:

[0059] Step S31: Acquire target audio.

[0060] In the embodiment of the present disclosure, the target audio is obtained based on the process steps in any of the above-mentioned sound enhancement method embodiments. For details, please refer to the process steps in the above-mentioned sound enhancement method embodiments, which will not be repeated here.

[0061] Step S32: Perform speech recognition based on the target audio to obtain recognized text.

[0062] Specifically, various speech recognition technologies such as deep learning-based speech recognition technology and Markov-based speech recognition technology can be used to recognize the target audio to obtain the recognized text. For the specific recognition process, please refer to the technical details of deep learning-based speech recognition technology and Markov-based speech recognition technology, which will not be repeated here. Of course, in the embodiment of the present disclosure, speech recognition of the target audio is only one possible scenario in practical applications, and other possible scenarios are not limited here. For example, it can also be applied to speech tasks such as speech translation, which will not be given one by one here.

[0063] The above scheme obtains the target audio, which is obtained based on any of the above-mentioned sound enhancement method embodiments, and performs speech recognition based on the target audio to obtain a recognized text. Therefore, after the target audio is obtained through any of the above-mentioned sound enhancement method embodiments, speech recognition is performed on it, that is, speech recognition can be performed on the basis of the target audio that has enhanced the target sound source and suppressed other irrelevant sound sources, which can improve the accuracy of speech recognition.

[0064] See also Figure 4 , Figure 4 It is a schematic diagram of the framework of an embodiment of the sound source localization device of the present application. The sound source localization device 40 includes: an audio acquisition module 41 and a position prediction module 42, the audio acquisition module 41 is used to acquire the audio to be processed collected by the sound pickup array including dual microphones; the position prediction module 42 is used to process the audio to be processed based on the sound source localization model, and predict the relative position of the target sound source to the sound pickup array; wherein the relative position includes the relative direction and the relative distance, the sound source localization model is obtained based on the sample audio training, the sample audio is marked with the actual area where the sample sound source is located, the sample audio at least includes the simulated audio, the simulated audio is obtained by superimposing the collected audio of the sample array when the audio is played in multiple in-car areas respectively, and the speaker mechanism plays the audio in each in-car area in turn before the sound source localization model is trained.

[0065] In the above scheme, the sound source localization device 40 obtains the audio to be processed collected by the sound pickup array including dual microphones, and then processes the audio to be processed based on the sound source localization module, and predicts the relative position of the target sound source compared to the sound pickup array, the relative position includes the relative direction and the relative distance, and the sound source localization model is obtained based on the sample audio training, the sample audio is marked with the actual area where the sample sound source is located, the sample audio at least includes the simulated audio, and the simulated audio is obtained by superimposing the collected audio of the sample array when the audio is played in multiple in-car areas respectively, and before the sound source localization model is trained, the speaker mechanism plays the audio in each in-car area in turn, so on the one hand, because the sample audio includes the simulated audio, and the simulated audio is played in multiple in-car areas separately. The collected audio of the sample array is superimposed when the audio is played separately, and the speaker mechanism plays the audio in each area of ​​the car in turn before the sound source localization model is trained. Therefore, by collecting the audio when playing the audio in each area of ​​the car in turn, especially the simulated audio formed by superimposing the collected audio, it is possible to simulate the audio data of human-computer interaction in different areas in the real environment as much as possible, and then train the model based on this, so that the model can model and distinguish the sound representation of different areas during the training process, and can achieve accurate sound zone separation even in complex scenarios such as multiple people talking at the same time. On the other hand, there is no need for multiple microphones such as four microphones, but only two microphones can support multiple areas to meet the isolation between different areas of the front and rear rows on the same side. Therefore, under the premise of realizing multiple sound zones, it is possible to reduce the hardware cost, operation and maintenance difficulty and performance requirements for the vehicle audio processing system as much as possible, and at the same time minimize the impact of hardware reasons on the effect of multiple sound zone zoning.

[0066] In some disclosed embodiments, the sound source localization device 40 includes a selection control module, which is used to select each in-vehicle area as the current area and control the speaker mechanism to play audio in the current area; the sound source localization device 40 includes an audio acquisition module, which is used to obtain the collected audio of the sample array when the speaker mechanism plays audio; the sound source localization device 40 includes an area combination module, which is used to obtain at least one area combination composed of multiple in-vehicle areas; the sound source localization device 40 includes an audio superposition module, which is used to obtain the collected audio when the speaker mechanism plays audio in each in-vehicle area in the area combination in turn, and superimpose it to obtain simulated audio.

[0067] In some disclosed embodiments, the sound source localization device 40 includes an area recording module for recording the in-vehicle area where the audio is collected; wherein the in-vehicle area representation sample array for collecting the audio obtains the current area where the speaker mechanism is located when the audio is collected; the sound source localization device 40 includes an area labeling module for labeling the actual area for the simulated audio based on each in-vehicle area in the area combination.

[0068] In some disclosed embodiments, the area marking module is specifically used to select an area inside the vehicle in a target direction within the area combination and mark it as an actual area of ​​the simulated audio.

[0069] In some disclosed embodiments, the sound source localization device 40 includes a region division module for dividing the current region into multiple sub-regions; the selection control module is specifically used to control the speaker mechanism to play audio in the sub-regions in the current region in turn.

[0070] In some disclosed embodiments, the sound source localization device 40 includes a sample prediction module, which is used to process sample audio based on a sound source localization model to predict a predicted position of the sample sound source relative to a sample array; wherein the predicted position includes a predicted direction and a predicted distance of the sample sound source relative to the sample array; the sound source localization device 40 includes an area selection module, which is used to select an in-vehicle area as a predicted area of ​​the sample sound source based on the predicted direction, the predicted distance and the position layout of each in-vehicle area; the sound source localization device 40 includes a parameter adjustment module, which is used to adjust the network parameters of the sound source localization model based on the difference between the predicted area and the actual area.

[0071] In some disclosed embodiments, the audio to be processed includes a first audio and a second audio collected by two microphones respectively, and the sound source localization device 40 includes a difference acquisition module for acquiring an arrival time difference between the first audio and the second audio, and acquiring a sound intensity difference between the first audio and the second audio; the position prediction module 42 is specifically used to process the arrival time difference and the sound intensity difference based on the sound source localization model to predict the relative position.

[0072] In some disclosed embodiments, the sound pickup array is arranged in the front row of the vehicle, and the multiple vehicle interior areas include at least the rear area; and / or the sample array and the sound pickup array have the same microphone layout.

[0073] See also Figure 5 , Figure 5 Schematic diagram of the framework of an embodiment of the sound enhancement device of the present application. The sound enhancement device 50 includes: a position prediction module 51, an area determination module 52 and a beam forming module 53. The position prediction module 51 is used to predict the relative position of the target sound source to the sound pickup array; wherein the relative position is determined based on the sound source localization device in the above-mentioned sound source localization device embodiment; the area determination module 52 is used to obtain the in-vehicle area where the target sound source is located as the target area based on the relative position of the target sound source to the sound pickup array; the beam forming module 53 is used to adjust the sound pickup area of ​​the sound pickup array based on beam forming to cover the target area and obtain the target audio.

[0074] In the above scheme, the sound enhancement device 50 predicts the relative position of the target sound source to the sound pickup array, and the relative position is obtained based on any of the above-mentioned sound source localization device embodiments, and based on the relative position of the target sound source to the sound pickup array, the vehicle interior area where the target sound source is located is obtained as the target area, and then the sound pickup area of ​​the sound pickup array is adjusted based on beamforming to cover the target area to obtain the target audio. Therefore, after determining the relative position of the target sound source to the sound pickup array through any of the above-mentioned sound source localization device embodiments, the target area where the target sound source is located is determined accordingly, and the sound pickup area is adjusted based on beamforming, so that the sound signal of the target sound source can be enhanced as much as possible, and the sound signals of other irrelevant sound sources can be suppressed, which helps to improve the audio quality of the target audio.

[0075] See also Figure 6 , Figure 6 Schematic diagram of the framework of an embodiment of the speech recognition device of the present application. The speech recognition device 60 includes: an audio acquisition module 61 and an audio recognition module 62. The audio acquisition module 61 is used to acquire target audio; wherein the target audio is obtained based on the above-mentioned sound enhancement device; and the audio recognition module 62 is used to perform speech recognition based on the target audio to obtain a recognized text.

[0076] In the above scheme, the speech recognition device 60 obtains the target audio, which is obtained based on any of the above-mentioned sound enhancement device embodiments, and performs speech recognition based on the target audio to obtain a recognized text. Therefore, after the target audio is obtained through any of the above-mentioned sound enhancement device embodiments, speech recognition is performed on it. That is, speech recognition can be performed on the basis of the target audio that has enhanced the target sound source and suppressed other irrelevant sound sources, which can improve the accuracy of speech recognition.

[0077] See also Figure 7 , Figure 7 It is a schematic diagram of the framework of an embodiment of an electronic device of the present application. The electronic device 70 includes at least a memory 71 and a processor 72 coupled to each other, the memory 71 at least stores program instructions, and the processor 72 is used to execute the program instructions to implement the steps in any of the above-mentioned sound source localization method embodiments, or to implement the steps in any of the above-mentioned sound enhancement method embodiments, or to implement the steps in any of the above-mentioned speech recognition method embodiments. For details, please refer to the aforementioned disclosed embodiments, which will not be repeated here. As a possible example, the electronic device 70 may include but is not limited to a server, etc., and the specific type of the electronic device 70 is not limited here.

[0078] Specifically, the processor 72 is used to control itself and the memory 71 to implement the steps in any of the above-mentioned sound source localization method embodiments, or to implement the steps in any of the above-mentioned sound enhancement method embodiments, or to implement the steps in any of the above-mentioned speech recognition method embodiments. The processor 72 can also be called a CPU (Central Processing Unit). The processor 72 may be an integrated circuit chip with signal processing capabilities. The processor 72 can also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field-programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 72 can be implemented by an integrated circuit chip.

[0079] In the above scheme, the electronic device 70, on the one hand, because the sample audio includes simulated audio, and the simulated audio is obtained by superimposing the collected audio of the sample array when the audio is played in multiple areas in the car respectively, and the speaker mechanism plays the audio in each area in the car in turn before the sound source localization model is trained, the collected audio when the audio is played in each area in the car in turn, especially the simulated audio formed by superimposing the collected audio, can simulate the audio data when human-computer interaction occurs in different areas in the real environment as much as possible, and then the model is trained based on this, so that the model can be modeled and distinguished during the training process. Sound representations of different areas can be achieved even in complex scenarios such as multiple people talking at the same time. Accurate sound zone separation can be achieved. On the other hand, there is no need for multiple microphones such as four microphones, but only two microphones are needed to support multiple areas to meet the isolation between different areas in the front and rear rows on the same side. Therefore, under the premise of realizing multiple sound zones, the hardware cost, operation and maintenance difficulty and performance requirements for the vehicle audio processing system can be reduced as much as possible, and the influence of hardware reasons on the effect of multiple sound zone zoning can be reduced as much as possible. Based on this, the target area where the target sound source is located is determined, and the pickup area is adjusted based on beamforming, which can enhance the sound signal of the target sound source as much as possible and suppress the sound signals of other irrelevant sound sources, which helps to improve the audio quality of the target audio. In addition, speech recognition can be performed on this, that is, speech recognition can be performed on the target audio that has already enhanced the target sound source and suppressed other irrelevant sound sources, which can improve the accuracy of speech recognition.

[0080] See also Figure 8 , Figure 81 is a schematic diagram of a framework of an embodiment of a computer-readable storage medium 80 of the present application. The computer-readable storage medium 80 stores program instructions 81 that can be executed by a processor, and the program instructions 81 are used to implement the steps in any of the above-mentioned sound source localization method embodiments, or to implement the steps in any of the above-mentioned sound enhancement method embodiments, or to implement the steps in any of the above-mentioned speech recognition method embodiments.

[0081] In the above scheme, the computer-readable storage medium 80, on the one hand, because the sample audio includes simulated audio, and the simulated audio is obtained by superimposing the collected audio of the sample array when the audio is played in multiple in-car areas respectively, and before the sound source localization model is trained, the speaker mechanism plays the audio in each in-car area in turn, so by collecting the audio when the audio is played in each in-car area in turn, especially the simulated audio formed by superimposing the collected audio, it is possible to simulate the audio data when human-computer interaction occurs in different areas in the real environment as much as possible, and then the model is trained based on this, so that the model can be modeled and distinguished in the training process. Sound representations of different areas can be achieved even in complex scenarios such as multiple people talking at the same time. Accurate sound zone separation can be achieved. On the other hand, there is no need for multiple microphones such as four microphones, but only two microphones are needed to support multiple areas to meet the mutual isolation between different areas of the front and rear rows on the same side. Therefore, under the premise of realizing multiple sound zones, the hardware cost, operation and maintenance difficulty and performance requirements for the vehicle audio processing system can be reduced as much as possible, and the hardware reasons that affect the effect of multiple sound zone zoning can be reduced as much as possible. Based on this, the target area where the target sound source is located is determined, and the pickup area is adjusted based on beamforming, which can enhance the sound signal of the target sound source as much as possible and suppress the sound signals of other irrelevant sound sources, which helps to improve the audio quality of the target audio. In addition, speech recognition can be performed on this, that is, speech recognition can be performed on the target audio that has already enhanced the target sound source and suppressed other irrelevant sound sources, which can improve the accuracy of speech recognition.

[0082] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0083] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.

[0084] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0085] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0086] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0087] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

[0088] If the technical solution of this application involves personal information, the product using the technical solution of this application has clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using the technical solution of this application has obtained the individual's separate consent before processing the sensitive personal information, and at the same time meets the "explicit consent" requirement. For example, on personal information collection devices such as cameras, clear and prominent signs are set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that he or she agrees to the collection of his or her personal information; or on the device that processes personal information, the personal information processing rules are notified by obvious signs / information, and the individual's authorization is obtained through pop-up information or by asking the individual to upload his or her personal information; among them, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.

Claims

1. A sound source localization method, characterized in that: include: Acquire the audio to be processed collected by the sound pickup array including two microphones; The audio to be processed is processed based on a sound source localization model to predict the relative position of the target sound source to the pickup array; wherein the relative position includes a relative direction and a relative distance, and the sound source localization model is obtained based on sample audio training, and the sample audio is marked with an actual area where the sample sound source is located, and the sample audio at least includes simulated audio, and the simulated audio is obtained by superimposing the collected audio of the sample array when the audio is played in multiple in-vehicle areas respectively, and before the sound source localization model is trained, the speaker mechanism plays the audio in each of the in-vehicle areas in turn.

2. The method according to claim 1, characterized in that The step of acquiring the simulated audio comprises: Selecting each of the vehicle interior areas as the current area, and controlling the speaker mechanism to play audio in the current area; Acquire the collected audio of the sample array when the speaker mechanism plays audio; Acquire at least one area combination consisting of a plurality of the vehicle interior areas; The collected audio is superimposed when the speaker mechanism plays audio in sequence in each of the in-vehicle areas in the area combination to obtain the simulated audio.

3. The method according to claim 2, characterized in that The method further comprises: Recording the in-vehicle area where the audio is collected; wherein the in-vehicle area where the audio is collected represents the current area where the speaker mechanism is located when the sample array acquires the collected audio; Based on each of the in-vehicle areas in the area combination, the actual area is labeled for the simulated audio.

4. The method according to claim 3, characterized in that The step of labeling the actual area for the simulated audio based on each of the in-vehicle areas in the area combination includes: The in-vehicle area in the target direction is selected in the area combination and marked as the actual area of ​​the simulated audio.

5. The method according to claim 2, characterized in that: After selecting each of the vehicle interior areas as the current area, and before controlling the speaker mechanism to play audio in the current area, the method further includes: Dividing the current area into a plurality of sub-areas; The controlling the speaker mechanism to play audio in the current area includes: The speaker mechanism is controlled to play audio in the sub-areas in the current area in sequence.

6. The method according to claim 1, characterized in that The training steps of the sound source localization model include: Processing the sample audio based on a sound source localization model to predict a predicted position of the sample sound source relative to the sample array; wherein the predicted position includes a predicted direction and a predicted distance of the sample sound source relative to the sample array; Based on the predicted direction, the predicted distance and the position layout of each of the in-vehicle areas, selecting the in-vehicle area as the predicted area of ​​the sample sound source; Based on the difference between the predicted area and the actual area, a network parameter of the sound source localization model is adjusted.

7. The method according to claim 1, characterized in that The audio to be processed includes the first audio and the second audio respectively collected by the dual microphones. Before processing the audio to be processed based on the sound source localization model to predict the relative position of the target sound source to the pickup array, the method further includes: Obtaining an arrival time difference between the first audio and the second audio, and obtaining a sound intensity difference between the first audio and the second audio; The processing of the audio to be processed based on the sound source localization model to predict the relative position of the target sound source to the sound pickup array includes: The arrival time difference and the sound intensity difference are processed based on the sound source localization model to predict the relative position.

8. The method according to any one of claims 1 to 7, characterized in that: The sound pickup array is arranged in the front row of the vehicle, and the multiple vehicle interior areas at least include the rear row area; And / or, the sample array and the sound pickup array have the same microphone layout.

9. A sound enhancement method, characterized in that: include: Predicting the relative position of the target sound source to the sound pickup array; wherein the relative position is determined based on the sound source localization method according to any one of claims 1 to 7; Based on the relative position of the target sound source to the sound pickup array, obtaining the vehicle interior area where the target sound source is located as the target area; The sound pickup area of ​​the sound pickup array is adjusted based on beam forming to cover the target area, thereby obtaining target audio.

10. A speech recognition method, characterized in that: include: Acquire target audio; wherein the target audio is obtained based on the sound enhancement method according to claim 9; Speech recognition is performed based on the target audio to obtain a recognized text.

11. A sound source localization device, characterized in that: include: An audio acquisition module, used to acquire the audio to be processed collected by a sound pickup array including two microphones; A position prediction module is used to process the audio to be processed based on a sound source localization model, and predict the relative position of the target sound source to the pickup array; wherein the relative position includes a relative direction and a relative distance, and the sound source localization model is obtained based on sample audio training, and the sample audio is marked with the actual area where the sample sound source is located, and the sample audio at least includes simulated audio, and the simulated audio is obtained by superimposing the collected audio of the sample array when the audio is played in multiple in-vehicle areas respectively, and before the sound source localization model is trained, the sound speaker mechanism plays the audio in each of the in-vehicle areas in turn.

12. A sound enhancement device, characterized in that: include: A position prediction module, used to predict the relative position of the target sound source to the sound pickup array; wherein the relative position is determined based on the sound source localization device according to claim 11; An area determination module, used to obtain the area in the vehicle where the target sound source is located as the target area based on the relative position of the target sound source to the sound pickup array; The beamforming module is used to adjust the sound pickup area of ​​the sound pickup array based on beamforming to cover the target area and obtain target audio.

13. A speech recognition device, characterized in that: include: An audio acquisition module, used to acquire target audio; wherein the target audio is obtained based on the sound enhancement device according to claim 12; The audio recognition module is used to perform speech recognition based on the target audio to obtain recognized text.

14. An electronic device, characterized in that: The method comprises at least a memory and a processor coupled to each other, wherein the memory stores at least program instructions, and the processor is used to execute the program instructions to implement the sound source localization method according to any one of claims 1 to 8, or the sound enhancement method according to claim 9, or the speech recognition method according to claim 10.

15. A computer-readable storage medium, characterized in that: Program instructions that can be executed by a processor are stored, and the program instructions are used to implement the sound source localization method described in any one of claims 1 to 8, or the sound enhancement method described in claim 9, or the speech recognition method described in claim 10.

Images