Speech recognition method, terminal device, and computer-readable storage medium

Abstract

The application discloses a speech recognition method, a terminal device and a computer readable storage medium. The method comprises the following steps: obtaining acoustic spectrum feature data of an audio file to be recognized, determining an acoustic spectrum graph corresponding to the audio file to be recognized according to the acoustic spectrum feature data; taking the acoustic spectrum graph as an input of a pre-trained convolutional neural network model, determining an audio spelling sequence corresponding to the acoustic spectrum graph based on the convolutional neural network model; and identifying the voice content corresponding to a pre-stored text spelling sequence matched with the audio spelling sequence as voice content contained in the audio file to be recognized. The effect of identifying telecom fraud in advance before the implementation of telecom fraud is achieved.

Description

Technical Field

[0001] This invention relates to the field of speech recognition, and more particularly to speech recognition methods, terminal devices, and computer-readable storage media. Background Technology

[0002] In related technologies, identifying telecom fraud requires access to user data such as bank card information and ATM withdrawal information on the bank's side, and the identification of fraudulent behavior during the withdrawal process. This means that the method can only identify telecom fraud after the user has already been successfully defrauded.

[0003] It should be noted that the above content is only used to help understand the technical problem solved by the present invention, and does not represent an admission that the above content is prior art. Summary of the Invention

[0004] This application provides a voice recognition method, a terminal device, and a computer-readable storage medium, which solves the technical problem of delayed identification of telecommunications fraud in related technologies, and achieves the effect of identifying telecommunications fraud in advance before it is completed.

[0005] This application provides a speech recognition method, which includes the following steps:

[0006] Obtain the spectrogram feature data of the audio file to be identified, and determine the spectrogram corresponding to the audio file to be identified based on the spectrogram feature data;

[0007] The spectrogram is used as input to a pre-trained convolutional neural network model, and the audio pinyin sequence corresponding to the spectrogram is determined based on the convolutional neural network model.

[0008] The speech content corresponding to the pre-stored text pinyin sequence that matches the audio pinyin sequence is identified as the speech content contained in the audio file to be identified.

[0009] Optionally, before the step of identifying the speech content corresponding to the pre-stored text pinyin sequence that matches the audio pinyin sequence as the speech content contained in the audio file to be identified, the method further includes:

[0010] Get the preset text;

[0011] Generate the pre-stored text pinyin sequence based on the preset text.

[0012] Optionally, after the step of generating the pre-stored text pinyin sequence based on the preset text, the method further includes:

[0013] Generate a pinyin data dictionary corresponding to the pre-stored text pinyin sequence, wherein the key of the pinyin data dictionary is the text pinyin and the value is the text index of the preset text containing the text pinyin;

[0014] Based on the Pinyin data dictionary, query the pre-stored text Pinyin sequence that matches the audio Pinyin sequence.

[0015] Optionally, in the pinyin sequence, each syllable corresponds to at least one audio pinyin. The convolutional neural network model determines the probability that each optional pinyin in the pinyin library is the audio pinyin corresponding to the syllable based on the spectrogram, and selects at least one of the optional pinyin as the audio pinyin corresponding to the syllable according to the probability.

[0016] Optionally, before the step of using the spectrogram as input to a pre-trained convolutional neural network model and determining the audio pinyin sequence corresponding to the spectrogram based on the convolutional neural network model, the method further includes:

[0017] Obtain sample spectrograms for model training, and sample pinyin sequences corresponding to the sample spectrograms;

[0018] The loss function is constructed using the CTC algorithm, and the convolutional neural network model is trained based on the loss function, the sample spectrogram, and the sample pinyin sequence.

[0019] Optionally, the step of obtaining the spectrogram feature data of the audio file to be identified and determining the spectrogram corresponding to the audio file to be identified based on the spectrogram feature data includes:

[0020] Read the audio file to be identified and determine the frequency domain signal corresponding to the audio file to be identified;

[0021] Data framing is performed based on the frequency domain signal;

[0022] Add a plaintext window to each frame of data after framing;

[0023] Based on the Fast Fourier Transform, separate the component frequencies corresponding to each frame of windowed data;

[0024] The spectrogram is generated based on the component frequencies of the classification process and the temporal information corresponding to each frame of data.

[0025] Optionally, the audio file is a call recording, the pre-stored text pinyin sequence is the pinyin sequence corresponding to the fraudulent script text, and the voice content is the fraudulent script.

[0026] In addition, to achieve the above effects, embodiments of the present invention also provide a terminal device, including:

[0027] The acquisition module is used to acquire the spectrogram feature data of the audio file to be identified, and determine the spectrogram corresponding to the audio file to be identified based on the spectrogram feature data.

[0028] The analysis module is used to take the spectrogram as input to a pre-trained convolutional neural network model and determine the audio pinyin sequence corresponding to the spectrogram based on the convolutional neural network model.

[0029] The recognition module is used to recognize the speech content corresponding to the pre-stored text pinyin sequence that matches the audio pinyin sequence as the speech content contained in the audio file to be recognized.

[0030] In addition, to achieve the above effects, embodiments of the present invention also provide a terminal device, including a memory, a processor, and a speech recognition program stored in the memory and executable on the processor. When the processor executes the speech recognition program, it implements the speech recognition method described above.

[0031] In addition, to achieve the above effects, embodiments of the present invention also provide a computer-readable storage medium storing a speech recognition program thereon, which, when executed by a processor, implements the speech recognition method as described above.

[0032] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:

[0033] 1. Because the voice recognition method provided in this implementation can directly identify whether there are fraudulent statements in the call recording, it can identify telecommunications fraud in advance before the fraud is completed.

[0034] 2. Since the identification process can be completed on the terminal device, the call recording does not need to be leaked from the terminal device, thus improving the effectiveness of user privacy protection in the fraud identification process. Attached Figure Description

[0035] Figure 1 This is a flowchart illustrating an embodiment of the speech recognition method of the present invention;

[0036] Figure 2 This is an audio frequency domain signal diagram related to an embodiment of the present invention;

[0037] Figure 3 for Figure 2 The image shown is a single-frame data diagram of the audio frequency domain signal after it has been framed.

[0038] Figure 4 for Figure 3 The image shown is an illustration of the effect of windowing a single frame of data.

[0039] Figure 5 For based on Figure 4The diagram shows the frequency composition of the signal.

[0040] Figure 6 This is a spectrogram related to an embodiment of the present invention;

[0041] Figure 7 This is a schematic diagram of the Pinyin data dictionary involved in an embodiment of the present invention;

[0042] Figure 8 This is a modular schematic diagram of the terminal device involved in an embodiment of the present invention;

[0043] Figure 9 This is a simplified structural diagram of the terminal device involved in an embodiment of the present invention. Detailed Implementation

[0044] In related technologies, telecommunications fraud is generally identified through the following two methods.

[0045] Firstly, user data is accessed on the operator's side for analysis and identification of fraudulent calls. The accessed data includes signaling data, IMEI (International Mobile Equipment Identity) data, and call voice data. Analysis methods include type matching and natural language processing.

[0046] Secondly, user data is accessed on the bank side to identify fraudulent activities during the withdrawal process. The accessed data includes bank card information, ATM (Automated Teller Machine) withdrawal information, etc.

[0047] The aforementioned identification methods require obtaining large amounts of user privacy data from telecom operators and banks, such as call logs, contacts, call content, IMEI, bank card information, and photos of cash withdrawals. This means that implementing telecom fraud identification will result in the leakage of user privacy data.

[0048] Furthermore, fraud detection based on telecom operator data can only alert users via SMS or disconnect their calls after identifying telecom fraud, directly disrupting normal communication. In contrast, fraud detection based on bank data only identifies fraud after the user has suffered financial loss, resulting in a delay in fraud detection.

[0049] To address the shortcomings of existing telecom fraud identification methods, such as privacy breaches and delays, this invention proposes a voice recognition method. This method deploys the fraud identification function within the user's personal terminal, ensuring that the data used for identification does not leak from the user's personal terminal, thus protecting user privacy. Simultaneously, by performing voice recognition on the user's call data, telecom fraud can be quickly identified, allowing for the extraction and identification of fraudulent activities before they are successfully committed, thus alerting the user to take precautions.

[0050] The speech recognition method proposed in this embodiment will be further explained below with reference to the accompanying drawings.

[0051] In one embodiment, the speech recognition method includes the following steps:

[0052] Step S10: Obtain the spectrogram feature data of the audio file to be identified, and determine the spectrogram corresponding to the audio file to be identified based on the spectrogram feature data;

[0053] Step S20: Use the spectrogram as input to a pre-trained convolutional neural network model, and determine the audio pinyin sequence corresponding to the spectrogram based on the convolutional neural network model;

[0054] Step S30: Identify the speech content corresponding to the pre-stored text pinyin sequence that matches the audio pinyin sequence as the speech content contained in the audio file to be identified.

[0055] In this embodiment, a speech recognition method is implemented to identify whether the audio corresponding to an audio file contains speech content corresponding to a preset text library or database.

[0056] For example, in one application scenario of the speech recognition method proposed in this embodiment, the audio file to be recognized may be a call recording. This allows the speech recognition method proposed in this embodiment to identify whether the call recording contains the phonetic equivalent of a preset fraudulent script. Furthermore, if the phonetic equivalent of a fraudulent script is present in the call recording, it is determined that the call recording poses a risk of telecommunications fraud, and the user is then given a fraud prevention alert.

[0057] In another application scenario, the audio file to be identified can also be the audio file corresponding to a video file. For example, during monitoring and investigation, the speech recognition method proposed in this embodiment can be used to identify whether the target speech appears in the monitoring video. The target speech can be a custom setting, for example, set to "xx, are you ready to take action tonight?"

[0058] In another application scenario, the audio file to be identified can be the audio file corresponding to a film or television work, a short video work, or a music file. In this application scenario, the speech recognition method proposed in this embodiment of the invention can be used to identify whether the film or television coordinates, short video works, or music files contain target lines or lyrics.

[0059] The following explanation, in conjunction with a telecommunications fraud early warning scenario, further illustrates the speech recognition method proposed in this invention. It should be understood that the following content is intended to help those skilled in the art understand the scope of the speech recognition method of this invention, and does not limit the invention.

[0060] In this embodiment, the device executing the speech recognition method can be a mobile terminal, such as a mobile phone or tablet computer. The mobile terminal can establish a call connection with other terminals based on a mobile network. For example, it can establish telephone communication based on a calling network, or establish a network voice call based on platforms such as WeChat, QQ, DingTalk, or Lark.

[0061] When the terminal detects that a call has started, it can activate the recording device to record the call audio. This recording is then used as the audio file to be recognized. After the call ends, the speech recognition method is executed based on the audio file to be recognized.

[0062] Once the audio file to be identified is determined, its spectroscopic feature data can be obtained, and the spectrogram corresponding to the audio file to be identified can be determined based on the spectroscopic feature data.

[0063] For example, the terminal can read the audio file to be identified and determine the frequency domain signal corresponding to the audio file. That is, by reading the audio file to be identified, extracting the sampling frequency and sampling data of the audio file, and obtaining the original audio data information. Figure 2 As shown, this is the time-domain representation of speech, where amplitude represents sound intensity, and zero amplitude represents silence. Therefore, these amplitudes cannot represent the content of the speech and need to be converted to a frequency-domain representation.

[0064] After converting the amplitude to a frequency domain representation, i.e., obtaining the frequency domain signal corresponding to the audio file to be identified, data framing is performed based on the frequency domain signal, and then a Minghan window is added to each frame of data after framing. It is understood that humans produce sound through their vocal cords, and different vibration frequencies of the vocal cords produce sounds with different meanings, typically within the range of 10ms-30ms. The vibration frequency of human sound production remains relatively stable; therefore, the original speech data is divided into 20ms segments, with each frame having a length of 20ms. Optionally, to ensure a smooth transition between frames, overlapping framing can be used. For example, one frame overlaps with the next frame by 10ms. A single frame of data is shown below. Figure 3 As shown.

[0065] After framing, because each frame's data is extracted from the original speech data, the data in each frame is not from a single cycle, resulting in spectral leakage. Therefore, a Hamming window needs to be added to the frame data to mitigate spectral leakage.

[0066] Alternatively, in one implementation, the window function w(t) can be:

[0067]

[0068] like Figure 4 As shown, after windowing a frame of data, the frame data approximately appears as periodic data.

[0069] After windowing the frame data, the constituent frequencies of each windowed frame can be separated using the Fast Fourier Transform (FFT). This is analogous to how a sound signal is composed of sound waves of different frequencies; the FFT separates these different frequencies and determines their magnitudes. For example... Figure 5 The figure shows the component frequencies of the sound signal separated by Fourier transform.

[0070] Finally, the spectrogram is generated based on the component frequencies of the classification process and the temporal information corresponding to each frame of data. That is, it is generated based on the component frequencies of the classification process and the temporal information corresponding to each frame of data. Figure 6 The spectrogram shown.

[0071] It should be noted that the above-described method for determining the spectrogram is one optional implementation of the speech recognition method of this invention. In telecommunications fraud early warning scenarios, the terminal can determine the spectrogram of the audio file to be identified based on the above method without the recording file leaving the terminal, thereby protecting user privacy. Of course, in some scenarios where user privacy is not a concern, such as audio recognition of movies or songs, the terminal can call cloud services to determine the spectrogram of the audio file to be identified. Determining the spectrogram through cloud services can effectively reduce the computational overhead of the terminal device.

[0072] Furthermore, after determining the spectrogram, the spectrogram can be used as input to a pre-trained convolutional neural network model, and the audio pinyin sequence corresponding to the spectrogram can be determined based on the convolutional neural network model.

[0073] Understandably, traditional speech recognition methods require converting call recordings into text before text recognition, resulting in slow recognition speeds and high computational demands, making them unsuitable for devices with limited computing power, such as mobile phones and tablets. However, using acoustic models for speech analysis eliminates the speech-to-text process, directly identifying harassing and fraudulent speech features at the pronunciation level. This approach requires less computation, offers faster recognition speeds, and has a smaller model size, allowing for deployment and smooth operation on mobile devices. It also supports the recognition of unfamiliar words and unclear pronunciations, significantly improving the speech recognition rate. Therefore, in this embodiment, a pre-trained convolutional neural network model can be used to determine the corresponding pinyin sequence of the speech file to be recognized based on the spectrogram.

[0074] For example, after obtaining the spectrogram corresponding to the audio file to be recognized, the speech recognition is converted into image recognition. For instance, the network structure of the convolutional neural network can be as follows:

[0075] The first convolutional layer has 32 convolutional kernels, each 3×3 in size, and uses the ReLU activation function.

[0076] The second convolutional layer has 32 kernels, each 3×3 in size, and uses the ReLU activation function.

[0077] The third pooling layer has a 2×2 kernel and uses max pooling.

[0078] The fourth convolutional layer has 64 kernels, each 3×3 in size, and uses the ReLU activation function.

[0079] The fifth convolutional layer has 64 kernels, each 3×3 in size, and uses the ReLU activation function.

[0080] The sixth pooling layer has a 2×2 kernel and uses max pooling.

[0081] The seventh convolutional layer has 128 kernels, each 3×3 in size, and uses the ReLU activation function.

[0082] The eighth convolutional layer has 128 kernels, each 3×3 in size, and uses ReLU as the activation function.

[0083] Ninth pooling layer: 2×2 kernel, max pooling.

[0084] The tenth convolutional layer has 128 kernels, each 3×3 in size, and uses ReLU as the activation function.

[0085] The eleventh convolutional layer has 128 kernels, each 3×3 in size, and uses ReLU as the activation function.

[0086] The twelfth convolutional layer has 128 kernels, each 3×3 in size, and uses the ReLU activation function.

[0087] The thirteenth convolutional layer has 128 kernels, each 3×3 in size, and uses the ReLU activation function.

[0088] The fourteenth fully connected layer contains 256 neurons.

[0089] The fifteenth fully connected layer has the same number of neurons as the number of words in the pinyin dictionary, uses the softmax activation function, and is the final output layer.

[0090] It should be noted that during model training, the CTC (Connectionist Temporal Classification) algorithm can be used to construct a loss function for model training. Then, sample spectrograms and corresponding sample pinyin sequences are obtained for model training. The convolutional neural network model is trained based on the loss function, the sample spectrograms, and the sample pinyin sequences. This allows the trained convolutional neural network model to directly determine the corresponding pinyin sequence based on the input spectrogram. In the pinyin sequence, each syllable corresponds to at least one audio pinyin. The convolutional neural network model, based on the spectrogram, determines the probability that each selectable pinyin in the pinyin library is the audio pinyin corresponding to the syllable, and selects at least one of the selectable pinyin as the audio pinyin corresponding to the syllable based on the probability. For example, the top five pinyin with the highest probability can be used as the audio pinyin corresponding to the syllable. This allows the final recognition result to achieve a fuzzy query effect, that is, to realize the recognition of fuzzy pronunciations.

[0091] Furthermore, after obtaining the audio pinyin sequence, the speech content corresponding to the pre-stored text pinyin sequence that matches the audio pinyin sequence is identified as the speech content contained in the audio file to be identified.

[0092] Optionally, as an implementation scheme, a preset text can be obtained in the terminal first, and a pre-stored text pinyin sequence can be generated based on the preset text. Then, a pinyin data dictionary corresponding to the pre-stored text pinyin sequence can be generated, where the key of the pinyin data dictionary is the text pinyin and the value is the text index of the preset text containing the text pinyin. Based on the pinyin data dictionary, a pre-stored text pinyin sequence that matches the audio pinyin sequence can be queried.

[0093] For example, in a telecommunications fraud identification scenario, the call audio (the audio file to be identified) can be converted into an audio pinyin sequence and then matched with the fraud script in the fraud script (preset text) to calculate the fraud probability of the call and determine whether it is a fraudulent call.

[0094] In this example, each Chinese phrase in the scam script is first converted into its corresponding pinyin sequence for comparison with the pinyin sequence of the recorded conversation. Then, the average length of all phrases in the scam script after conversion to pinyin is calculated using the following formula:

[0095]

[0096] Then, based on the audio pinyin sequence corresponding to the fraudulent script and the average length of the pinyin, a pinyin data dictionary is generated, where the key is the pinyin of the text corresponding to the text, and the value is the index of the script containing that pinyin. An example is shown below. Figure 7 As shown.

[0097] After determining the audio pinyin sequence and pinyin data dictionary, the frequency of the currently processed call pinyin in each dialogue can be calculated based on the pinyin data dictionary and sorted in descending order. Then, based on the sorting result, the top N dialogues can be selected as candidate dialogues for this dialogue matching. The larger the value of N, the more comprehensive the matching; the smaller the value of N, the faster the dialogue matching. The value of N can be flexibly set according to the actual situation. In this application scenario, to maintain the most comprehensive matching efficiency, N is set to the maximum value, i.e., the number of dialogues.

[0098] Furthermore, based on the obtained candidate dialogue set, a loop is performed. Each candidate dialogue is retrieved and compared with the audio pinyin sequence. Since the lengths of the dialogue pinyin sequence (i.e., the pre-stored text pinyin sequence) and the audio pinyin sequence are not fixed and are not equal, a sliding window can be set. The sliding window length is the length of the dialogue pinyin, with a step size of 1. The audio pinyin sequence is sequentially truncated and compared with the dialogue pinyin sequence. The comparison algorithm uses the longest common subsequence algorithm based on dynamic programming. If the matching rate is greater than 50%, the dialogue is matched, the sliding window training is stopped, and the next candidate dialogue is compared.

[0099] After comparing all the alternative scripts, the probability that this call was a scam call is calculated using the following formula:

[0100]

[0101] Optionally, after determining the probability, the identification results of this fraudulent call and the matched script can be fed back to the user through the terminal.

[0102] It is understood that in the speech recognition method provided in this embodiment, since the training process of the waveform does not involve the pre-stored text pinyin sequence to be recognized, there is no need to retrain the convolutional neural network model when updating the speech content to be recognized, i.e., adding a new pre-stored text pinyin sequence. This allows the method to support new word recognition.

[0103] In the technical solution disclosed in this embodiment, the spectrogram feature data of the audio file to be identified is first obtained. Based on the spectrogram feature data, the spectrogram corresponding to the audio file to be identified is determined. Then, the spectrogram is used as input to a pre-trained convolutional neural network model. Based on the convolutional neural network model, the audio pinyin sequence corresponding to the spectrogram is determined. Finally, the speech content corresponding to a pre-stored text pinyin sequence that matches the audio pinyin sequence is identified as the speech content contained in the audio file to be identified. In the scenario of preventing telecommunications fraud, the speech recognition method provided in this embodiment can directly identify whether fraudulent language exists in call recordings, thus identifying telecommunications fraud before it is completed. Furthermore, since the recognition process can be completed on the terminal device, the call recording does not need to leave the terminal device, thereby improving the effectiveness of user privacy protection during fraud identification.

[0104] Furthermore, this embodiment of the invention also proposes a terminal device, which includes: a memory, a processor, and a speech recognition program stored in the memory and executable on the processor. When the speech recognition program is executed by the processor, it implements the steps of the speech recognition method described in the above embodiments.

[0105] Furthermore, embodiments of the present invention also propose a computer-readable storage medium storing a speech recognition program thereon, which, when executed by a processor, implements the steps of the speech recognition method as described in the above embodiments.

[0106] In addition, please refer to Figure 8 The present invention also proposes a terminal device 100, comprising:

[0107] The acquisition module 101 is used to acquire the spectral feature data of the audio file to be identified, and determine the spectral map corresponding to the audio file to be identified based on the spectral feature data.

[0108] Analysis module 102 is used to take the spectrogram as input to a pre-trained convolutional neural network model and determine the audio pinyin sequence corresponding to the spectrogram based on the convolutional neural network model;

[0109] The recognition module 103 is used to recognize the speech content corresponding to the pre-stored text pinyin sequence that matches the audio pinyin sequence as the speech content contained in the audio file to be recognized.

[0110] like Figure 9 As shown, Figure 9 This is a schematic diagram of the terminal structure of the hardware operating environment involved in the embodiments of the present invention.

[0111] like Figure 9As shown, the control terminal may include: a processor 1001, such as a CPU, a network interface 1003, a memory 1004, and a communication bus 1002. The communication bus 1002 is used to enable communication between these components. The network interface 1003 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1004 may be a high-speed RAM or a stable memory. Alternatively, the memory 1004 may be a storage device independent of the aforementioned processor 1001.

[0112] Those skilled in the art will understand that Figure 9 The terminal structure shown does not constitute a limitation on the terminal and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0113] like Figure 9 As shown, the memory 1004, which serves as a computer storage medium, may include an operating system, a network communication module, and a speech recognition program.

[0114] exist Figure 9 In the terminal shown, the processor 1001 can be used to call the speech recognition program stored in the memory 1004 and perform the following operations:

[0115] Obtain the spectrogram feature data of the audio file to be identified, and determine the spectrogram corresponding to the audio file to be identified based on the spectrogram feature data;

[0116] The spectrogram is used as input to a pre-trained convolutional neural network model, and the audio pinyin sequence corresponding to the spectrogram is determined based on the convolutional neural network model.

[0117] The speech content corresponding to the pre-stored text pinyin sequence that matches the audio pinyin sequence is identified as the speech content contained in the audio file to be identified.

[0118] Optionally, the processor 1001 may call the speech recognition program stored in the memory 1004 and further perform the following operations:

[0119] Get the preset text;

[0120] Generate the pre-stored text pinyin sequence based on the preset text.

[0121] Optionally, the processor 1001 may call the speech recognition program stored in the memory 1004 and further perform the following operations:

[0122] Generate a pinyin data dictionary corresponding to the pre-stored text pinyin sequence, wherein the key of the pinyin data dictionary is the text pinyin and the value is the text index of the preset text containing the text pinyin;

[0123] Based on the Pinyin data dictionary, query the pre-stored text Pinyin sequence that matches the audio Pinyin sequence.

[0124] Optionally, the processor 1001 may call the speech recognition program stored in the memory 1004 and further perform the following operations:

[0125] Obtain sample spectrograms for model training, and sample pinyin sequences corresponding to the sample spectrograms;

[0126] The loss function is constructed using the CTC algorithm, and the convolutional neural network model is trained based on the loss function, the sample spectrogram, and the sample pinyin sequence.

[0127] Optionally, the processor 1001 may call the speech recognition program stored in the memory 1004 and further perform the following operations:

[0128] Read the audio file to be identified and determine the frequency domain signal corresponding to the audio file to be identified;

[0129] Data framing is performed based on the frequency domain signal;

[0130] Add a plaintext window to each frame of data after framing;

[0131] Based on the Fast Fourier Transform, separate the component frequencies corresponding to each frame of windowed data;

[0132] The spectrogram is generated based on the component frequencies of the classification process and the temporal information corresponding to each frame of data.

[0133] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0134] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0135] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (such as a mobile phone, tablet computer, etc.) to execute the methods described in the various embodiments of the present invention.

[0136] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A speech recognition method, characterized in that, The speech recognition method includes the following steps: Obtain the spectrogram feature data of the audio file to be identified, and determine the spectrogram corresponding to the audio file to be identified based on the spectrogram feature data; The spectrogram is used as input to a pre-trained convolutional neural network model, and the audio pinyin sequence corresponding to the spectrogram is determined based on the convolutional neural network model. The process of identifying the speech content corresponding to the pre-stored text pinyin sequence that matches the audio pinyin sequence as the speech content contained in the audio file to be identified includes: generating a pinyin data dictionary based on the audio pinyin sequence corresponding to the fraudulent script and the average length of the pinyin in the script; calculating the number of times the currently processed call pinyin appears in each script based on the pinyin data dictionary; sorting the number of times in descending order; selecting the top N scripts as candidate scripts based on the sorting result; comparing each candidate script with the audio pinyin sequence; and after comparing all candidate scripts, calculating the probability that the current call is a fraudulent call and determining whether it is a fraudulent call. The calculation formula is: fraud probability = Σ length of the pinyin in the matched script / length of the pinyin in the current call. The identification results of this fraudulent call and the matched script will be fed back to the user through the terminal.

2. The speech recognition method as described in claim 1, characterized in that, Before the step of identifying the speech content corresponding to the pre-stored text pinyin sequence that matches the audio pinyin sequence as the speech content contained in the audio file to be identified, the method further includes: Get the preset text; Generate the pre-stored text pinyin sequence based on the preset text.

3. The speech recognition method as described in claim 2, characterized in that, After the step of generating the pre-stored text pinyin sequence based on the preset text, the method further includes: Generate a pinyin data dictionary corresponding to the pre-stored text pinyin sequence, wherein the key of the pinyin data dictionary is the text pinyin and the value is the text index of the preset text containing the text pinyin; Based on the Pinyin data dictionary, query the pre-stored text Pinyin sequence that matches the audio Pinyin sequence.

4. The speech recognition method as described in claim 1, characterized in that, In the pinyin sequence, each syllable corresponds to at least one audio pinyin. The convolutional neural network model determines the probability that each optional pinyin in the pinyin library is the audio pinyin corresponding to the syllable based on the spectrogram, and selects at least one of the optional pinyin as the audio pinyin corresponding to the syllable according to the probability.

5. The speech recognition method as described in claim 1, characterized in that, Before the step of using the spectrogram as input to a pre-trained convolutional neural network model and determining the audio pinyin sequence corresponding to the spectrogram based on the convolutional neural network model, the method further includes: Obtain sample spectrograms for model training, and sample pinyin sequences corresponding to the sample spectrograms; The loss function is constructed using the CTC algorithm, and the convolutional neural network model is trained based on the loss function, the sample spectrogram, and the sample pinyin sequence.

6. The speech recognition method as described in claim 1, characterized in that, The step of obtaining the spectrogram feature data of the audio file to be identified, and determining the spectrogram corresponding to the audio file to be identified based on the spectrogram feature data, includes: Read the audio file to be identified and determine the frequency domain signal corresponding to the audio file to be identified; Data framing is performed based on the frequency domain signal; Add a plaintext window to each frame of data after framing; Based on the Fast Fourier Transform, separate the component frequencies corresponding to each frame of windowed data; The spectrogram is generated based on the component frequencies of the classification process and the temporal information corresponding to each frame of data.

7. The speech recognition method as described in claim 1, characterized in that, The audio file is a call recording, the pre-stored text pinyin sequence is the pinyin sequence corresponding to the fraudulent script, and the voice content is the fraudulent script.

8. A terminal device, characterized in that, include: The acquisition module is used to acquire the spectrogram feature data of the audio file to be identified, and determine the spectrogram corresponding to the audio file to be identified based on the spectrogram feature data. The analysis module is used to take the spectrogram as input to a pre-trained convolutional neural network model and determine the audio pinyin sequence corresponding to the spectrogram based on the convolutional neural network model. The recognition module is used to identify the speech content corresponding to the pre-stored text pinyin sequence that matches the audio pinyin sequence as the speech content contained in the audio file to be recognized. This includes: generating a pinyin data dictionary based on the audio pinyin sequence corresponding to the fraudulent script and the average length of the script pinyin; calculating the number of times the currently processed call pinyin appears in each script based on the pinyin data dictionary; sorting the number of times in descending order; selecting the top N scripts as candidate scripts based on the sorting result; comparing each candidate script with the audio pinyin sequence; calculating the probability that the current call is a fraudulent call after all candidate scripts have been compared; determining whether it is a fraudulent call; the calculation formula is: fraud probability = Σ length of the pinyin of the hit script / length of the pinyin of the current call; and feeding back the recognition result of the current fraudulent call and the hit script to the user through the terminal.

9. A terminal device, characterized in that, The method includes a memory, a processor, and a speech recognition program stored in the memory and executable on the processor, wherein the processor, when executing the speech recognition program, implements the method of any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, It stores a speech recognition program, which, when executed by a processor, implements the speech recognition method according to any one of claims 1-7.

Images