Speech recognition model construction method and device, speech recognition method and device and storage medium

Abstract

The embodiment of the invention provides a speech recognition model construction method, a speech recognition method, a speech recognition device and a storage medium. The speech recognition model construction method comprises the steps of obtaining hardware acoustic parameters of acoustic hardware in at least two pieces of equipment; obtaining target relative position information between a target device and other devices, wherein the at least two devices comprise the target device and the other devices; and based on the hardware acoustic parameters and the target relative position information, constructing a voice recognition model for the target device, the voice recognition model being used for the target device to process sound signals collected by each device so as to recognize whether the sound signals belong to voice signals. Through the technical scheme of the invention, the recognition accuracy of the sound signal can be improved.

Description

Technical Field

[0001] The present application relates to the field of speech recognition technology, and in particular to a speech recognition model construction method, a speech recognition method, a device, a storage medium and an electronic device. Background Art

[0002] Normally, devices with voice recognition functions (such as smart speakers, smart air conditioners, etc.) will use pre-installed voice recognition models to identify whether the collected sound information is human voice information, and then perform further operations based on the recognition results. Generally, the more microphones installed in a device or the higher the quality, the better the recognition effect of the voice recognition model, but increasing the number of microphones or improving the quality of microphones will inevitably increase the configuration cost of the device. Based on this, how to build a voice recognition model that can improve the recognition accuracy of sound signals without increasing the configuration cost of the device is a technical problem that needs to be solved urgently. Summary of the invention

[0003] The embodiments of the present application provide a speech recognition model construction method, a speech recognition method, an apparatus, a storage medium and an electronic device. The technical solution provided in the present application can improve the recognition accuracy of sound signals.

[0004] Other features and advantages of the present application will become apparent from the following detailed description, or may be learned in part by the practice of the present application.

[0005] According to the first aspect of an embodiment of the present application, a method for constructing a speech recognition model is provided, the method comprising: obtaining hardware acoustic parameters of acoustic hardware in at least two devices; obtaining target relative position information between a target device and other devices, the at least two devices including the target device and the other devices; based on the hardware acoustic parameters and the target relative position information, constructing a speech recognition model for the target device, the speech recognition model being used by the target device to process sound signals collected by each device to identify whether the sound signal is a speech signal.

[0006] In some embodiments of the present application, based on the aforementioned scheme, obtaining the hardware acoustic parameters of the acoustic hardware in at least two devices includes: obtaining the device model of each device in the at least two devices; and searching for the hardware acoustic parameters of the acoustic hardware in the at least two devices according to the device model.

[0007] In some embodiments of the present application, based on the aforementioned scheme, the obtaining of the target relative position information between the target device and other devices includes: triggering the target device to play test audio; obtaining audio signals for the test audio collected by each device; and based on the audio signals collected by each device, determining the first relative position information between the target device and the other devices as the target relative position information.

[0008] In some embodiments of the present application, based on the aforementioned scheme, the method also includes: obtaining second relative position information between the target device and the other device estimated by the user; if the error between the first relative position information and the second relative position information meets the preset requirements, then using the first relative position information as the target relative position information.

[0009] In some embodiments of the present application, based on the aforementioned scheme, constructing a speech recognition model for the target device based on the hardware acoustic parameters and the target relative position information includes: comparing the hardware acoustic parameters of each of the other devices with the hardware acoustic parameters of the target device to obtain comparison information, wherein the comparison information is used to characterize the performance difference between the acoustic hardware in each other device and the acoustic hardware in the target device; constructing a speech recognition model for the target device based on the comparison information and the target relative position information.

[0010] In some embodiments of the present application, based on the aforementioned scheme, after building a speech recognition model for the target device, the method also includes: obtaining speech training data, and training the speech recognition model based on the speech training data; sending the trained speech recognition model to the target device to cover the speech recognition model historically installed on the target device.

[0011] According to the second aspect of an embodiment of the present application, a speech recognition method is provided, the method comprising: acquiring sound signals collected by acoustic hardware in each device; processing the sound signals collected by each device through a speech recognition model to identify whether the sound signals belong to speech signals, the speech recognition model being obtained by the speech recognition model construction method as described in any one of the first aspects above.

[0012] According to the third aspect of the embodiment of the present application, a speech recognition model construction device is provided, comprising a first acquisition unit for acquiring hardware acoustic parameters of acoustic hardware in at least two devices; a second acquisition unit for acquiring target relative position information between a target device and other devices, wherein the at least two devices include the target device and the other devices; and a construction unit for constructing a speech recognition model for the target device based on the hardware acoustic parameters and the target relative position information, wherein the speech recognition model is used by the target device to process sound signals collected by each device to identify whether the sound signals belong to speech signals.

[0013] According to the fourth aspect of an embodiment of the present application, a speech recognition device is provided, wherein a third acquisition unit is used to acquire sound signals collected by acoustic hardware in each device; and a recognition unit is used to process the sound signals collected by each device through a speech recognition model to identify whether the sound signals belong to speech signals, and the speech recognition model is obtained by the speech recognition model construction method as described in any one of the first aspects above.

[0014] According to the fifth aspect of an embodiment of the present application, a computer-readable storage medium is provided, characterized in that at least one program code is stored in the computer-readable storage medium, and the at least one program code is loaded and executed by a processor to implement the operations performed by the method described in any one of the first and second aspects above.

[0015] According to the sixth aspect of an embodiment of the present application, an electronic device is provided, comprising one or more processors and one or more memories, wherein at least one program code is stored in the one or more memories, and the at least one program code is loaded and executed by the one or more processors to implement the operations performed by the method described in any one of the first and second aspects above.

[0016] The technical solution of the present application first obtains the hardware acoustic parameters of the acoustic hardware in at least two devices; and obtains the target relative position information between the target device and other devices, and the at least two devices include the target device and the other devices; finally, based on the hardware acoustic parameters and the target relative position information, a speech recognition model for the target device is constructed, and the speech recognition model is used by the target device to process the sound signals collected by each device to identify whether the sound signal belongs to a speech signal. It can be seen that the speech recognition model constructed by the present application, in the process of processing and identifying the sound signal, not only processes and identifies the sound signal collected by the target device, but also processes and identifies the sound signal collected by other devices, thereby improving the recognition accuracy of the sound signal.

[0017] That is to say, based on the technical solution of the present application, the target device can convert the microphones of other devices for its own use, and combine the microphones of other devices and its own microphone into a microphone array connected to the speech recognition model, so that the speech recognition model can not only receive the sound signals collected by the target device itself, but also receive the sound signals collected by other devices in the process of processing and identifying sound signals. Without increasing the number of microphones or improving the quality of microphones, the corresponding speech recognition model of the target device can also improve the recognition accuracy of sound signals, thereby enabling the target device to achieve high-configuration-cost sound signal recognition effects under low-configuration-cost conditions.

[0018] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present application, and together with the specification, are used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0020] Figure 1 A flow chart of a method for constructing a speech recognition model according to an embodiment of the present application is shown;

[0021] Figure 2 A schematic diagram of an application scenario of speech recognition according to an embodiment of the present application is shown;

[0022] Figure 3 A detailed flow chart of obtaining target relative position information between a target device and other devices according to an embodiment of the present application is shown;

[0023] Figure 4 A block diagram of a speech recognition model building device according to an embodiment of the present application is shown;

[0024] Figure 5 A block diagram of a speech recognition device according to an embodiment of the present application is shown;

[0025] Figure 6 A schematic structural diagram of an electronic device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0026] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be more comprehensive and complete and fully convey the concept of the example embodiments to those skilled in the art.

[0027] In addition, described feature, structure or characteristic can be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present application. However, those skilled in the art will appreciate that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, known methods, devices, realizations or operations are not shown or described in detail to avoid blurring the various aspects of the application.

[0028] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0029] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0030] It should be noted that the "multiple" mentioned in this article refers to two or more. "And / or" describes the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.

[0031] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the objects used in this way can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those shown or described.

[0032] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0033] Some embodiments of the present application will be described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments may be combined with each other.

[0034] It should be noted that the devices mentioned in this application refer to smart devices with voice recognition functions, including but not limited to smart speakers, smart refrigerators, smart TVs, smart air conditioners, smart washing machines, etc.

[0035] In the present application, a method for constructing a speech recognition model is provided.

[0036] It should be noted that the executor of the speech recognition model construction method mentioned in this application can be a cloud server, a mobile phone client with a speech recognition model construction function, or a device with a speech recognition model construction function. Specifically, this application does not limit the specific executor of the method.

[0037] See also Figure 1 , shows a flow chart of a method for constructing a speech recognition model according to an embodiment of the present application, which specifically includes the following steps S110 to S130.

[0038] S110: Acquire hardware acoustic parameters of acoustic hardware in at least two devices.

[0039] It should be noted that the acoustic hardware includes but is not limited to a microphone with a sound pickup function, a speaker with a playback function, etc.

[0040] It should also be noted that the hardware acoustic parameters include but are not limited to microphone gain, echo gain, speaker gain, maximum speaker playback volume, speaker playback volume at each level, microphone sensitivity, number of microphones, microphone spacing, etc.

[0041] In some implementations, at least two devices in this embodiment may be devices in the same application environment, such as devices with voice recognition capabilities in the same home environment, or devices with voice recognition capabilities in the same office environment.

[0042] In some implementations, the at least two devices in this embodiment may be devices of the same device type or devices of different device types, and the present application does not limit this. For example, the at least two devices may include a smart air conditioner and a smart refrigerator, or include two smart air conditioners.

[0043] In some implementations, at least two devices in this embodiment may be devices with voice recognition functions that need to be bound together and used as specified by the user, for example, Figure 2 As shown, assuming that the user wants to bind device 1 in the master bedroom, device 2 in the second bedroom, and device 3 in the living room together for use, then the at least two devices in this embodiment include the following Figure 2 Device 1, device 2, and device 3 are shown.

[0044] In some implementations, in the above step S110, the specific implementation of obtaining the hardware acoustic parameters of the acoustic hardware in at least two devices may be the following steps S111 to S112:

[0045] S111, obtaining a device model of each device in the at least two devices.

[0046] It should be noted that each of the two devices may be configured in advance so that each device is connected to the same wireless network, thereby obtaining the device model of each device.

[0047] S112: Searching for hardware acoustic parameters of acoustic hardware in the at least two devices according to the device model.

[0048] Usually, the local cache of the device stores parameters related to the device's hardware. Therefore, after obtaining the device model of each device, the device model can be used as a clue to obtain the corresponding hardware acoustic parameters of each device in the local cache.

[0049] It should be noted that, in the above step S110, the purpose of obtaining the hardware acoustic parameters of the acoustic hardware in each device is to prepare data for the subsequent construction of a speech recognition model for the target device.

[0050] Continue to see Figure 1 , S120, obtaining target relative position information between the target device and other devices, the at least two devices including the target device and the other devices.

[0051] It should be noted that the relative position information between the target device and other devices includes but is not limited to the distance information between the target device and each other device, and the angle information between the target device and each other device. It can be understood that the relative position can reflect the spatial position relationship between the target device and each other device.

[0052] In some implementations, in step S120, the specific implementation of obtaining the target relative position information between the target device and other devices can be as follows: Figure 3 Follow the steps shown.

[0053] See also Figure 3 , shows a detailed flow chart of obtaining target relative position information between a target device and other devices according to an embodiment of the present application, specifically including the following steps S121 to S123:

[0054] S121, triggering the target device to play a test audio.

[0055] It is understandable that if the cloud server is responsible for building the speech recognition model for the target device, then the cloud server can send a play instruction to the target device, so that the target device will start playing the test audio when it receives the play instruction sent by the cloud server.

[0056] It should be noted that, in this embodiment, the specific content of the test audio is not limited, and it can be white noise, music, or audio of other content.

[0057] Continue to see Figure 3 , step S122, obtaining the audio signal for the test audio collected by each device.

[0058] In this embodiment, if the cloud server is responsible for building the speech recognition model, when the microphone of each device collects the audio signal for the test audio, the collected audio signal will be automatically sent to the cloud server.

[0059] Continue to see Figure 3 , step S123, based on the audio signals collected by each device, determine the first relative position information between the target device and the other devices as the target relative position information.

[0060] In some implementations, in step S123, when the audio signals collected by each device are obtained, the first relative position information between the target device and other devices can be determined in at least the following three ways.

[0061] The first method is based on beamforming.

[0062] This method is a controllable beamforming technology based on maximum output power. The audio signals collected by each device are weighted and summed to form a beam. The beam is guided by searching for the possible location of the sound source, and the weight is modified to maximize the output signal power of the microphone array. This method can be used in both the time domain and the frequency domain, and has strong adaptability to the test environment.

[0063] The second method is based on high-resolution spectrum estimation.

[0064] This method is based on high-resolution spectrum estimation, including autoregressive AR model, minimum variance spectrum estimation (MV) and eigenvalue decomposition method (such as Music algorithm). All these methods obtain the signal of microphone array to calculate the correlation matrix of spatial spectrum, which can effectively estimate the direction of sound source.

[0065] The third method is based on the time difference of arrival (TDOA) method.

[0066] The sound source localization technology of time difference of arrival (TDOA) is generally divided into two steps. First, the sound source localization time difference is estimated, and the acoustic delay (TDOA) between the array elements in the microphone array is obtained from it; then the sound source position is further determined by combining the obtained sound source time difference with the known spatial position of the microphone array.

[0067] In order to make those skilled in the art better understand this embodiment, Figure 2 Give an example.

[0068] See also Figure 2 , shows a schematic diagram of an application scenario of speech recognition according to an embodiment of the present application.

[0069] Figure 2 The scenario illustrated in the figure is: device 1 in the master bedroom of a user's home is taken as the target device, device 2 in the second bedroom, and devices 3 and 4 in the living room are taken as other devices. Then, through the above steps S110 to S120, the hardware acoustic parameters of the acoustic hardware in devices 1-4 can be determined, and the first relative position information between device 1 and device 2, device 3, and device 4 can also be obtained.

[0070] In some implementations, after obtaining the first relative position information based on the above steps S121 to S123, the obtained first relative position information can be directly used as the target relative position information to prepare data for the subsequent construction of a speech recognition model for the target device.

[0071] In other implementations, the first relative position information obtained based on the above steps S121 to S123 is not directly used as the target relative position information, but can be verified through the following steps S124 to S125:

[0072] S124: Acquire second relative position information between the target device and the other device estimated by the user.

[0073] In some implementations, the user may input the estimated second relative position information between the target device and other devices through a mobile client, so that the cloud server may obtain the estimated second relative position information input by the user.

[0074] In some embodiments, after obtaining the hardware acoustic parameters of the acoustic hardware in each device, the cloud server can also send all the obtained hardware acoustic parameters to the mobile client, so as to facilitate the user to view. When the user estimates the second relative position information between the target device and the other devices, the user can make an estimate based on the hardware acoustic parameters of each device. For example, the microphone gain of device A among the other devices is larger, which can reduce the distance between device A and the target device.

[0075] S125: If the error between the first relative position information and the second relative position information meets a preset requirement, use the first relative position information as target relative position information.

[0076] In some embodiments, after receiving the second relative position information estimated by the user, the cloud server will compare the second relative position information with the first relative position information obtained in advance through theoretical calculation. If the error generated does not meet the preset requirements, the cloud server can recalculate the first relative position information according to the aforementioned step S123, or notify the user to re-check the input second relative position information.

[0077] It can be understood that, through the above steps S124 to S125, the accuracy of the determined target relative position information can be improved, thereby providing more reliable data preparation for the subsequent construction of a speech recognition model for the target device.

[0078] Continue to see Figure 1 , S130, based on the hardware acoustic parameters and the target relative position information, construct a speech recognition model for the target device, and the speech recognition model is used by the target device to process the sound signals collected by each device to identify whether the sound signal is a speech signal.

[0079] In order to enable those skilled in the art to better understand this embodiment, the background technology for executing step S130 is described below by way of example.

[0080] Assume the target device is Figure 2 The device 1 in the master bedroom shown has a voice recognition function because a corresponding voice recognition model is installed in the device 1. The voice recognition model can be pre-stored in a cloud server, and then the device 1 can download the voice recognition model from the cloud server when it is first installed in the user's home.

[0081] Typically, the speech recognition model installed in device 1 is constructed based on the hardware acoustic parameters of device 1. The speech recognition model can only receive the sound signal collected by device 1, and process and recognize the sound signal collected by device 1 to determine whether the sound signal collected by device 1 is a speech signal emitted by a person.

[0082] Based on this, in this application, if you want to convert the acoustic hardware in device 2, device 3, and device 4 to be used by device 1, you need to obtain the hardware acoustic parameters of device 2-4, and obtain the target relative position information between device 1 and device 2-4, so as to reconstruct the speech recognition model for device 1. Based on the reconstructed speech recognition model for device 1, in the process of processing and recognizing sound signals, not only the sound signals collected by device 1 can be processed, but also the sound signals collected by devices 2-4 can be processed.

[0083] In some implementations, in the above step S130, the specific implementation of constructing the speech recognition model for the target device based on the hardware acoustic parameters and the target relative position information can be performed according to the following steps S131 to S132:

[0084] S131, respectively comparing the hardware acoustic parameters of each of the other devices with the hardware acoustic parameters of the target device to obtain comparison information, where the comparison information is used to characterize the performance difference between the acoustic hardware in each of the other devices and the acoustic hardware in the target device.

[0085] In order to make those skilled in the art better understand this embodiment, Figure 2 Give an example.

[0086] Assumptions Figure 2In the example, device 1 in the master bedroom of a user's home is taken as the target device, device 2 in the second bedroom, and devices 3 and 4 in the living room are taken as other devices. If the microphone gain of device 1 is GainA, and the speaker gain is LineoutA; the microphone gain of device 2 is GainB, and the speaker gain is LineoutB; the microphone gain of device 3 is GainC, and the speaker gain is LineoutC; the microphone gain of device 4 is GainD, and the speaker gain is LineoutC; then the comparison information obtained includes GainA / GainB, GainA / GainC, GainA / GainD, LineoutA / LineoutB, LineoutA / LineoutC, and LineoutA / LineoutD.

[0087] It should be noted that the purpose of step S131 is to enable the constructed speech recognition model for the target device to convert the sound signals collected by other devices into sound signals collected by the target device at the position of other devices during the process of processing the sound signals of other devices. In the conversion process, echo cancellation algorithm, noise cancellation algorithm, etc. need to be considered.

[0088] For example, assuming Figure 2 In the figure, device 1 is the target device. After receiving the sound signal 1 collected by device 2, the speech recognition model of device 1 will convert the sound signal 1 into the sound signal collected by device 1 at the position of device 2. At this time, it is necessary to introduce predetermined comparison information. Assuming that the microphone gain of device 1 is GainA, the microphone gain of device 2 is GainB, the microphone sensitivity of device 1 is SenA, and the microphone sensitivity of device 2 is SenB, then the sound signal 1 is converted into the sound signal collected by device 1 at the position of device 2: X1*GainA / GainB*(SenA-SenB), where X1 is the sound signal 1.

[0089] For example, assuming Figure 2In the figure, device 1 is the target device. After receiving the sound signal 1 collected by device 2, the speech recognition model of device 1 will convert the sound signal 1 into the sound signal collected by device 1 at the position of device 2. At this time, it is necessary to introduce predetermined comparison information. Assuming that the maximum broadcast volume of the speaker of device 1 is VA, the maximum broadcast volume of the speaker of device 2 is VB, the speaker gain of device 1 is LineoutA, the maximum speaker gain of device 2 is LineoutB, the echo gain of device 1 is EA, and the echo gain of device 2 is EB, then the echo cancellation audio considered in converting the sound signal 1 into the sound signal collected by device 1 at the position of device 2 is: M1*(VA-VB)*(LineoutA / LintoutB)*(EA / EB), where M1 is the existing echo cancellation module in device 1.

[0090] S132: construct a speech recognition model for the target device based on the comparison information and the target relative position information.

[0091] In some implementations, the specific implementation of constructing the speech recognition model for the target device based on the comparison information and the target relative position information can be constructed according to actual conditions, and this application is not limited here. Specifically, it can be constructed in the following manner:

[0092] Step 1: Using mathematical modeling of array signals, the spatial phase information brought by the array can be used to expand the time-frequency processing technology to joint time-frequency-space processing.

[0093] Mathematical modeling formulas include the following formulas 1-6:

[0094] X=[exp(-jwτ 1 )exp(-jwτ 2 )...exp(-jwτ N )] T s=d s (Formula 1)

[0095] Z=X+V (Formula 2)

[0096]

[0097]

[0098]

[0099]

[0100] Among them, d represents the steering vector; s represents the target sound signal; V represents the target signal received by the array; Z represents the signal received by the array; W represents the estimated filter; inSNR represents the input signal-to-noise ratio; outSNR represents the output signal-to-noise ratio; distortion represents distortion.

[0101] Step 2: Based on the above criteria, a convex optimization scheme under linear constraints is adopted, which is mathematically expressed as Formula 7. In principle, it guarantees the target speaker's voice while maximally suppressing noise.

[0102]

[0103] It should be noted that after the speech recognition model for the target device is constructed based on the above steps S110 to S130 of the present application, the speech recognition model does not have the actual processing and recognition function of the sound signal, that is, the speech recognition model needs further training before it can process and recognize the sound signal. In some embodiments, after the above step S130, that is, after the speech recognition model for the target device is constructed, the technical solution of the present application further includes the following steps S140 to S150:

[0104] S140, acquiring speech training data, and training the speech recognition model based on the speech training data.

[0105] In some implementations, the sample audio may be played to each device, so that each device may use the collected sound signal for the sample audio as the voice training data.

[0106] For example, Figure 2 Device 1 is used as the target device, and devices 2-4 are used as other devices. The user can issue continuous wake-up commands to device 1, or play music at device 1, so that the wake-up commands or the played music can be collected by devices 1-4 as the voice training data.

[0107] It should be noted that the trained speech recognition model has the function of processing and recognizing sound signals. In other words, the trained speech recognition model can judge the received sound signal, thereby determining whether the sound signal is a speech signal emitted by a human.

[0108] S150, sending the trained speech recognition model to the target device to cover the speech recognition model historically installed on the target device.

[0109] In order to make those skilled in the art better understand this embodiment, Figure 2 Give an example.

[0110] Assumptions Figure 2 In the example, device 1 in the master bedroom of a user's home is taken as the target device, device 2 in the second bedroom, and devices 3 and 4 in the living room are taken as other devices. According to the above steps S110 to S150, the speech recognition model for device 1 is obtained as the target speech recognition model.

[0111] If the user sends a voice signal of "device 1 turned on" in the living room, it is assumed that only devices 3 and 4 in the living room will collect the sound signal of "device 1 turned on", so that devices 3 and 4 will send the collected sound signals to the target speech recognition model, so that the target speech recognition model can process the sound signal of "device 1 turned on" collected by devices 3 and 4, and identify whether the sound signal is a speech signal sent by a person. It can be understood that because the sound signal is the sound signal of "device 1 turned on" sent by the user, the target speech recognition model can accurately identify the sound signal as a speech signal, so that the obtained recognition result can be further sent to the speech recognition system for speech semantic recognition.

[0112] Based on the above examples, it can be understood that the target speech recognition model for device 1 obtained through the technical solution of the present application enables the target speech recognition model for device 1 to accurately determine whether there is a sound signal emitted for device 1 with the help of sound signals collected by other devices, even if device 1 does not collect the sound signal emitted for device 1, thereby facilitating precise voice control of device 1.

[0113] In the present application, based on the same inventive concept as the aforementioned speech recognition model construction method, a speech recognition method is also provided. The method can be executed on a target device, a cloud server, or a mobile client. The method includes the following steps S10 to S20:

[0114] S10, obtaining sound signals collected by acoustic hardware in each device.

[0115] In some implementations, the sound signal may be collected by each microphone installed in each device.

[0116] For example, Figure 2 As shown, device 1 is used as the target device and devices 2-4 are used as other devices. If the user sends a voice signal of "turn on the target device" in the living room, the voice recognition model of device 1 can obtain the sound signal for "turn on the target device" collected by the microphones of devices 1-4.

[0117] S20, processing the sound signals collected by each device through a speech recognition model to identify whether the sound signal belongs to a speech signal, and the speech recognition model is obtained by any implementation of the speech recognition model construction method as described above.

[0118] In some implementations, after the above step S20, the technical solution of the present application may further execute the following steps S30 to S50.

[0119] S30: If the speech recognition model recognizes that the sound signal is a speech signal, the recognition result is sent to the speech recognition system.

[0120] S40, recognizing the speech semantics of the speech signal through the speech recognition system, and determining whether the speech semantics is a voice control instruction.

[0121] S50: If the speech semantics is a speech control instruction, control the target device to execute the speech control instruction.

[0122] It can be understood that through the technical solution of the present application, since the sound signals collected by other devices are utilized, the accuracy of the speech recognition model of the target device in recognizing the sound signals can be improved, thereby improving the processing efficiency of the speech recognition system in processing the speech signals.

[0123] In the technical solutions provided in some embodiments of the present application, the hardware acoustic parameters of the acoustic hardware in at least two devices are first obtained; and the target relative position information between the target device and other devices is obtained, and the at least two devices include the target device and the other devices; finally, based on the hardware acoustic parameters and the target relative position information, a speech recognition model for the target device is constructed, and the speech recognition model is used by the target device to process the sound signals collected by each device to identify whether the sound signal belongs to a speech signal. It can be seen that the speech recognition model constructed by the present application, in the process of processing and identifying the sound signal, not only processes and identifies the sound signal collected by the target device, but also processes and identifies the sound signal collected by other devices, thereby improving the recognition accuracy of the sound signal.

[0124] That is to say, based on the technical solution of the present application, the target device can convert the microphones of other devices for its own use, and combine the microphones of other devices and its own microphone into a microphone array connected to the speech recognition model, so that the speech recognition model can not only receive the sound signals collected by the target device itself, but also receive the sound signals collected by other devices in the process of processing and identifying sound signals. Without increasing the number of microphones or improving the quality of microphones, the corresponding speech recognition model of the target device can also improve the recognition accuracy of sound signals, thereby enabling the target device to achieve high-configuration-cost sound signal recognition effects under low-configuration-cost conditions.

[0125] Based on the same inventive concept, the embodiment of the present application also provides a speech recognition model construction device, which can be used to execute the speech recognition model construction method described above in the present application. For details not disclosed in the embodiment of the present application, please refer to the embodiment of the speech recognition model construction method described above in the present application.

[0126] See also Figure 4 , shows a block diagram of a speech recognition model building device according to an embodiment of the present application.

[0127] like Figure 4 As shown, a speech recognition model construction device 400 according to an embodiment of the present application includes: a first acquisition unit 401, a second acquisition unit 402, and a construction unit 403.

[0128] Among them, the first acquisition unit 401 is used to acquire hardware acoustic parameters of acoustic hardware in at least two devices; the second acquisition unit 402 is used to acquire target relative position information between the target device and other devices, and the at least two devices include the target device and the other devices; the construction unit 403 is used to construct a speech recognition model for the target device based on the hardware acoustic parameters and the target relative position information, and the speech recognition model is used by the target device to process the sound signals collected by each device to identify whether the sound signal belongs to a speech signal.

[0129] In some embodiments of the present application, based on the aforementioned scheme, the first acquisition unit 401 is further used to: obtain a device model of each device in the at least two devices; and search for hardware acoustic parameters of the acoustic hardware in the at least two devices according to the device model.

[0130] In some embodiments of the present application, based on the aforementioned scheme, the second acquisition unit 402 is also used to: trigger the target device to play test audio; obtain audio signals for the test audio collected by each device; and determine the first relative position information between the target device and the other devices based on the audio signals collected by each device as the target relative position information.

[0131] In some embodiments of the present application, based on the aforementioned scheme, the speech recognition model construction device of the present application also includes a verification unit, which is used to: obtain second relative position information between the target device and the other device estimated by the user; if the error between the first relative position information and the second relative position information meets the preset requirements, then use the first relative position information as the target relative position information.

[0132] In some embodiments of the present application, based on the aforementioned scheme, the construction unit 403 is also used to: compare the hardware acoustic parameters of each of the other devices with the hardware acoustic parameters of the target device to obtain comparison information, wherein the comparison information is used to characterize the performance difference between the acoustic hardware in each of the other devices and the acoustic hardware in the target device; based on the comparison information and the target relative position information, construct a speech recognition model for the target device.

[0133] In some embodiments of the present application, based on the aforementioned scheme, the speech recognition model building device of the present application also includes a training unit, which is used to: obtain speech training data, and train the speech recognition model based on the speech training data; send the trained speech recognition model to the target device to cover the speech recognition model historically installed on the target device.

[0134] Based on the same inventive concept, an embodiment of the present application also provides a speech recognition device, which can be used to execute the speech recognition method as described above in the present application.

[0135] See also Figure 5 , shows a block diagram of a speech recognition device according to an embodiment of the present application.

[0136] like Figure 5 As shown, a speech recognition device 500 according to an embodiment of the present application includes: a third acquisition unit 501 and a recognition unit 502.

[0137] Among them, the third acquisition unit 501 is used to obtain the sound signal collected by the acoustic hardware in each device; the recognition unit 502 is used to process the sound signal collected by each device through a speech recognition model to identify whether the sound signal belongs to a speech signal, and the speech recognition model is obtained by any implementation method of the speech recognition model construction method described above.

[0138] Based on the same inventive concept, an embodiment of the present application further provides a computer-readable storage medium, in which at least one computer program instruction is stored. The at least one computer program instruction is loaded and executed by a processor to implement the operations performed by the method described above.

[0139] Based on the same inventive concept, an embodiment of the present application also provides an electronic device.

[0140] See also Figure 6 , shows a schematic diagram of the structure of an electronic device according to an embodiment of the present application, wherein the electronic device includes one or more memories 604, one or more processors 602, and at least one computer program (computer program instruction) stored in the memory 604 and executable on the processor 602, and the processor 602 implements the method described above when executing the computer program.

[0141] Among them, Figure 6 In the embodiment of the present invention, a bus architecture (represented by bus 600) is shown, which may include any number of interconnected buses and bridges, and bus 600 links various circuits including one or more processors represented by processor 602 and memory represented by memory 604. Bus 600 may also link various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. Bus interface 605 provides an interface between bus 600 and receiver 601 and transmitter 603. Receiver 601 and transmitter 603 may be the same element, namely a transceiver, providing a unit for communicating with various other devices over a transmission medium. Processor 602 is responsible for managing bus 600 and general processing, while memory 604 may be used to store data used by processor 602 when performing operations.

[0142] The functions described herein may be implemented in hardware, software executed by a processor, firmware, or any combination thereof. If implemented in software executed by a processor, the functions may be stored as one or more instructions or codes on a computer-readable medium or transmitted via a computer-readable medium. Other examples and implementations are within the scope and spirit of the present application and the appended claims. For example, due to the nature of software, the functions described above may be implemented using software executed by a processor, hardware, firmware, hard wiring, or a combination of any of these. In addition, each functional unit may be integrated into a processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit.

[0143] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0144] The units described as separate components may or may not be physically separated, and the components of the control device may or may not be physical units, that is, they may be located in one place or distributed in multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0145] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly or all or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk, etc., which can store computer program instructions.

[0146] The above description is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present application shall be included in the scope of the claims of the present application.

Claims

1. A method for constructing a speech recognition model, characterized in that: The method comprises: Obtaining hardware acoustic parameters of acoustic hardware in at least two devices; Acquire target relative position information between a target device and other devices, wherein the at least two devices include the target device and the other devices; Based on the hardware acoustic parameters and the target relative position information, a speech recognition model for the target device is constructed, and the speech recognition model is used by the target device to process the sound signals collected by each device to identify whether the sound signals are speech signals.

2. The method according to claim 1, characterized in that The obtaining of hardware acoustic parameters of acoustic hardware in at least two devices includes: Obtaining a device model of each of the at least two devices; The hardware acoustic parameters of the acoustic hardware in the at least two devices are searched according to the device model.

3. The method according to claim 1, characterized in that The step of obtaining target relative position information between the target device and other devices includes: Triggering the target device to play a test audio; Acquire the audio signal for the test audio collected by each device; Based on the audio signals collected by each device, first relative position information between the target device and the other devices is determined as target relative position information.

4. The method according to claim 3, characterized in that The method further comprises: Acquiring second relative position information between the target device and the other device estimated by the user; If the error between the first relative position information and the second relative position information meets a preset requirement, the first relative position information is used as the target relative position information.

5. The method according to claim 1, characterized in that The step of constructing a speech recognition model for the target device based on the hardware acoustic parameters and the target relative position information includes: Comparing the hardware acoustic parameters of each of the other devices with the hardware acoustic parameters of the target device to obtain comparison information, where the comparison information is used to characterize the performance difference between the acoustic hardware in each of the other devices and the acoustic hardware in the target device; Based on the comparison information and the target relative position information, a speech recognition model for the target device is constructed.

6. The method according to claim 1, characterized in that After constructing the speech recognition model for the target device, the method further includes: Acquiring speech training data, and training the speech recognition model based on the speech training data; The trained speech recognition model is sent to the target device to overwrite the speech recognition model historically installed on the target device.

7. A speech recognition method, characterized in that: The method comprises: Acquire the sound signals collected by the acoustic hardware in each device; The sound signals collected by each device are processed by a speech recognition model to identify whether the sound signal belongs to a speech signal, and the speech recognition model is obtained by the speech recognition model construction method according to any one of claims 1 to 6.

8. A speech recognition model construction device, characterized in that: The device comprises: A first acquisition unit, configured to acquire hardware acoustic parameters of acoustic hardware in at least two devices; A second acquisition unit, configured to acquire target relative position information between a target device and other devices, wherein the at least two devices include the target device and the other devices; A construction unit is used to construct a speech recognition model for the target device based on the hardware acoustic parameters and the target relative position information, and the speech recognition model is used by the target device to process the sound signals collected by each device to identify whether the sound signal is a speech signal.

9. A speech recognition device, characterized in that: The device comprises: A third acquisition unit is used to acquire the sound signal collected by the acoustic hardware in each device; A recognition unit is used to process the sound signals collected by each device through a speech recognition model to identify whether the sound signal belongs to a speech signal, and the speech recognition model is obtained by the speech recognition model construction method according to any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one program code, and the at least one program code is loaded and executed by a processor to implement the operations performed by the method according to any one of claims 1 to 7.

11. An electronic device, characterized in that: The method comprises one or more processors and one or more memories, wherein at least one program code is stored in the one or more memories, and the at least one program code is loaded and executed by the one or more processors to implement the method according to any one of claims 1 to 7.

Images