Multi-language long recording transcription method, system and device with adaptive noise suppression

Abstract

The present invention provides a method, system and device for transcribing multilingual long recordings with adaptive noise suppression, including: performing audio segmentation on the multilingual long recording to obtain multiple audio segments; performing spectrum analysis on each audio segment to obtain corresponding spectrum features; performing noise detection on each audio segment according to the spectrum features to determine the noise segment; performing adaptive noise suppression on each noise segment in the multilingual long recording to obtain denoised multilingual audio; performing voice activity detection on the denoised multilingual audio to determine voice segments and non-voice segments; performing feature extraction on the voice segments to obtain voice features; inputting the voice features into a pre-trained multilingual transcription model for transcription to obtain corresponding text results. In the present invention, by performing adaptive noise suppression on each noise segment in the multilingual long recording, the interference of noise is effectively reduced and the voice quality is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of speech processing, and in particular to a multi-language long recording transcription method, system and device with adaptive noise suppression. Background Art

[0002] In various international conferences, academic exchanges, cross-border business negotiations, and multilingual media content, multilingual long recordings are becoming more and more common. However, there are currently many challenges in processing multilingual long recordings.

[0003] On the one hand, there are often various noise interferences in the actual environment, such as background human voices, equipment noise, environmental noise, etc. These noises will seriously affect the quality of the audio, reduce the clarity of the speech, and bring great difficulties to the subsequent transcription work.

[0004] On the other hand, different languages ​​have different phonetic characteristics, and their pronunciation rules, intonation, speaking speed, etc. are all different, which increases the difficulty of accurate transcription, resulting in low efficiency and difficulty in meeting actual needs. Summary of the invention

[0005] The main purpose of the present invention is to provide a method, system and device for transcribing multi-language long recordings with adaptive noise suppression, aiming to adaptively remove noise interference and improve the efficiency of transcribing multi-language long recordings.

[0006] To achieve the above object, the present invention provides a multi-language long recording transcription method with adaptive noise suppression, comprising the following steps:

[0007] Perform audio segmentation on multi-language long recordings to obtain multiple audio segments; perform spectrum analysis on each audio segment to obtain the corresponding spectrum features;

[0008] Perform noise detection on each audio segment according to the frequency spectrum characteristics to determine the noise segment;

[0009] Adaptively suppressing noise in each noise segment in the multilingual long recording to obtain denoised multilingual audio;

[0010] Perform voice activity detection on the denoised multi-language audio to determine voice segments and non-voice segments; perform feature extraction on the voice segments to obtain voice features;

[0011] The speech features are input into the pre-trained multilingual transcription model for transcription to obtain the corresponding text results.

[0012] Furthermore, the performing noise detection on each audio segment according to the frequency spectrum feature to determine the noise segment includes:

[0013] Perform energy distribution analysis on the frequency spectrum characteristics of the audio clip to obtain the energy distribution in different frequency intervals;

[0014] The energy distribution is compared with a preset noise energy distribution threshold. If the energy value of any frequency interval is lower than the noise energy distribution threshold and the duration reaches the preset duration, the audio segment corresponding to the frequency interval is marked as a suspected noise segment.

[0015] A periodic detection process is performed on the suspected noise segment, and if there is no periodicity, the suspected noise segment is determined to be a noise segment.

[0016] Furthermore, the step of adaptively suppressing noise on each noise segment in the multi-language long recording to obtain denoised multi-language audio includes:

[0017] Performing noise feature extraction processing on the noise segment to obtain noise feature parameters;

[0018] Construct an adaptive filter based on noise characteristic parameters;

[0019] The multi-language long recording is input into an adaptive filter for filtering to obtain denoised multi-language audio; wherein the adaptive filter adjusts the filter parameters in real time during the filtering process to adapt to different noise changes.

[0020] Furthermore, constructing an adaptive filter according to noise characteristic parameters includes:

[0021] Set the initial parameters of the filter; the initial parameters include the filter order, convergence coefficient and step factor;

[0022] The weight vector of the filter is trained using the noise characteristic parameters, and the weight vector is continuously adjusted through the minimum mean square error algorithm to minimize the error between the filter output and the noise segment. During the training process, the error changes are monitored in real time. When the error stabilizes within the preset range, the training is stopped and the construction of the adaptive filter is completed.

[0023] Furthermore, the step of inputting the speech features into a pre-trained multilingual transcription model for transcription to obtain a corresponding text result includes:

[0024] The speech features are input into the input layer of the multilingual transcription model and passed to the hidden layer after being preprocessed by the input layer;

[0025] Perform deep feature extraction on speech features based on hidden layers to extract semantic and grammatical features of different languages;

[0026] Based on the output layer of the multilingual transcription model, the probability of the extracted semantic features and grammatical features is calculated to determine the most matching character or word at each time step;

[0027] The characters or words obtained at each time step in the output layer are combined in sequence, and the repeated and erroneous parts are removed to obtain the final text result.

[0028] Furthermore, before the multi-language long recording is segmented to obtain multiple audio segments, the following steps are included:

[0029] Negotiate with multiple translation terminals to establish encrypted communication connections; wherein each translation terminal is a terminal held by each user;

[0030] Obtain the recorded audio of each translation terminal;

[0031] The audio recordings of the various translation terminals are combined in chronological order to obtain the multi-language long recording.

[0032] The present invention also provides a multi-language long recording transcription system with adaptive noise suppression, comprising:

[0033] The segmentation module is used to segment the audio of a long multi-language recording to obtain multiple audio segments; perform spectrum analysis on each audio segment to obtain the corresponding spectrum features;

[0034] A detection module, used to perform noise detection on each audio segment according to the spectrum characteristics to determine the noise segment;

[0035] A denoising module, used for adaptively suppressing noise in each noise segment in the multi-language long recording to obtain denoised multi-language audio;

[0036] An extraction module is used to perform voice activity detection on the denoised multi-language audio to determine voice segments and non-voice segments; perform feature extraction on the voice segments to obtain voice features;

[0037] The transcription module is used to input speech features into a pre-trained multilingual transcription model for transcription to obtain the corresponding text results.

[0038] The present invention also provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any one of the above methods when executing the computer program.

[0039] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned methods are implemented.

[0040] The adaptive noise-suppressed multilingual long recording transcription method, system and device provided by the present invention include: performing audio segmentation on the multilingual long recording to obtain multiple audio segments; performing spectrum analysis on each audio segment to obtain corresponding spectrum features; performing noise detection on each audio segment according to the spectrum features to determine the noise segment; performing adaptive noise suppression on each noise segment in the multilingual long recording to obtain denoised multilingual audio; performing voice activity detection on the denoised multilingual audio to determine voice segments and non-voice segments; performing feature extraction on the voice segment to obtain voice features; inputting the voice features into a pre-trained multilingual transcription model for transcription to obtain corresponding text results. In the present invention, by adaptive noise suppression on each noise segment in the multilingual long recording, the interference of noise is effectively reduced and the voice quality is improved; by inputting the voice features into a pre-trained multilingual transcription model for transcription, it is possible to adapt to the characteristics of different languages, accurately convert the voice features into text, and improve the accuracy and efficiency of transcribing multilingual long recordings. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 is a flow chart of a multi-language long audio recording transcription method with adaptive noise suppression in one embodiment of the present invention;

[0042] Figure 2 1 is a schematic diagram of the steps of a multi-language long audio recording transcription method with adaptive noise suppression in one embodiment of the present invention;

[0043] Figure 3 is a structural block diagram of a multi-language long audio recording transcription system with adaptive noise suppression in one embodiment of the present invention;

[0044] Figure 4 It is a schematic block diagram of the structure of a computer device according to an embodiment of the present invention.

[0045] The implementation, functional features and advantages of the present invention will be further described in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0047] Reference Figure 1 as well as Figure 2 In one embodiment of the present invention, a multi-language long recording transcription method with adaptive noise suppression is provided, comprising the following steps:

[0048] Step S1, performing audio segmentation on a multi-language long recording to obtain multiple audio segments; performing spectrum analysis on each audio segment to obtain corresponding spectrum features;

[0049] Step S2, performing noise detection on each audio segment according to the frequency spectrum characteristics to determine the noise segment;

[0050] Step S3, performing adaptive noise suppression on each noise segment in the multi-language long recording to obtain denoised multi-language audio;

[0051] Step S4, performing voice activity detection on the denoised multilingual audio to determine voice segments and non-voice segments; performing feature extraction on the voice segments to obtain voice features;

[0052] Step S5, inputting the speech features into a pre-trained multilingual transcription model for transcription to obtain corresponding text results.

[0053] In this embodiment, as described in step S1 above, first, audio segmentation is performed on a long multilingual recording. This step is to divide the long recording into multiple smaller audio clips. The advantage is that for long recordings, the overall processing will face problems such as high computing resource pressure and long processing time. After being divided into small clips, each clip can be processed separately more efficiently, reducing the complexity of subsequent processing. Next, a spectral analysis is performed on each audio clip. Spectral analysis is a method of revealing the frequency composition of an audio signal by converting it from the time domain to the frequency domain. Through spectral analysis, spectral features such as the energy distribution of each audio clip at different frequencies can be obtained. The above spectral features provide a key data basis for subsequent noise detection, which can help accurately identify the frequency differences between noise and speech signals.

[0054] As described in step S2 above, noise detection is performed on each audio segment based on the obtained spectral features. Specifically, by analyzing the energy distribution, frequency characteristics and other relevant parameters in the spectral features, it can be determined whether a certain audio segment contains noise. For example, if the energy value of a certain frequency interval is lower than the preset noise energy distribution threshold and the duration reaches the preset duration, the portion of the audio segment corresponding to the frequency interval may be marked as a suspected noise segment. Then, the suspected noise segment is periodically detected, and if no obvious periodicity is detected, the suspected noise segment is determined to be a noise segment. In this way, it is possible to accurately determine which parts are noise segments, providing a target for subsequent noise suppression.

[0055] As described in step S3 above, adaptive noise suppression is performed on each noise segment in the multilingual long recording. First, noise features are extracted from the noise segments to obtain noise feature parameters. These parameters include the frequency range, intensity, and variation pattern of the noise. Then, an adaptive filter is constructed based on the noise feature parameters. The adaptive filter can adjust its own parameters in real time according to the changes in the input signal to better adapt to different noise conditions. In the process of inputting the multilingual long recording into the adaptive filter for filtering, the filter parameters will be continuously adjusted according to the actual situation to adapt to the changes in noise. Finally, the filtered audio is post-processed to remove residual noise and distortion introduced by filtering, thereby obtaining denoised multilingual audio.

[0056] As described in step S4 above, voice activity detection is performed on the denoised multilingual audio. The purpose of voice activity detection is to determine which parts of the audio are voice segments and which parts are non-voice segments. By detecting features such as energy changes and periodicity in the audio, the intervals of voice activity can be accurately identified. After the voice segment is determined, feature extraction is performed on the voice segment. The feature extraction process is to extract parameters that can accurately reflect the essential characteristics of the voice from the voice segment, such as Mel-frequency cepstral coefficients. These voice features will be passed as input to the subsequent multilingual transcription model.

[0057] As described in the above step S5: the obtained speech features are input into a pre-trained multilingual transcription model for transcription. The multilingual transcription model is trained with a large amount of multilingual data and can learn the speech characteristics and grammatical rules of different languages. After the speech features are input, the model first performs preprocessing in the input layer, and then performs deep feature extraction in the hidden layer to extract the semantic features and grammatical features of different languages. Finally, the most likely characters or words at each time step are determined by probability calculation in the output layer, and the output characters or words are combined and optimized to remove repeated and erroneous parts to obtain the final text result.

[0058] In this embodiment, by adaptively suppressing noise segments in the multilingual long recording, noise interference is effectively reduced and voice quality is improved; by inputting voice features into a pre-trained multilingual transcription model for transcription, it is possible to adapt to the characteristics of different languages ​​and accurately convert voice features into text, thereby improving the accuracy and efficiency of transcribing multilingual long recordings.

[0059] In one embodiment, performing noise detection on each audio segment according to the frequency spectrum feature to determine the noise segment includes:

[0060] Perform energy distribution analysis on the frequency spectrum characteristics of the audio clip to obtain the energy distribution in different frequency intervals;

[0061] The energy distribution is compared with a preset noise energy distribution threshold. If the energy value of any frequency interval is lower than the noise energy distribution threshold and the duration reaches the preset duration, the audio segment corresponding to the frequency interval is marked as a suspected noise segment.

[0062] A periodic detection process is performed on the suspected noise segment, and if there is no periodicity, the suspected noise segment is determined to be a noise segment.

[0063] In this embodiment, first, the spectrum characteristics of the audio clip are subjected to energy distribution analysis, and this step is intended to determine the energy distribution of the audio signal in different frequency intervals. The spectrum characteristics obtained by spectrum analysis contain the energy information of the audio signal at each frequency. Further processing and analysis of this information can obtain the specific energy distribution of different frequency intervals. This analysis can help understand the intensity changes of the audio signal at different frequencies, and provide a basis for subsequent comparison with the noise energy distribution threshold.

[0064] Then, the energy distribution is compared with the preset noise energy distribution threshold. The above noise energy distribution threshold is determined based on a large number of experiments and experience, and is used to distinguish possible noise and speech signals. If the energy value of any frequency interval is lower than the noise energy distribution threshold and the duration reaches the preset duration, the audio segment corresponding to the frequency interval is marked as a suspected noise segment. Such a judgment standard takes into account the two factors of the energy value and the duration, and can more accurately screen out possible noise segments.

[0065] Finally, the suspected noise segment is subjected to periodicity detection. Speech signals usually have a certain periodicity, while noise often lacks obvious periodicity. By performing periodicity detection on the suspected noise segment, it can be further determined whether the segment is real noise. If the detection result shows that the suspected noise segment has no periodicity, then it can be determined that the suspected noise segment is a noise segment.

[0066] In one embodiment, the step of adaptively suppressing noise on each noise segment in the multi-language long recording to obtain denoised multi-language audio includes:

[0067] Performing noise feature extraction processing on the noise segment to obtain noise feature parameters;

[0068] Construct an adaptive filter based on noise characteristic parameters;

[0069] The multi-language long recording is input into an adaptive filter for filtering to obtain denoised multi-language audio; wherein the adaptive filter adjusts the filter parameters in real time during the filtering process to adapt to different noise changes.

[0070] In this embodiment, first, each noise segment in the multi-language long recording is subjected to noise feature extraction processing. This step is intended to obtain characteristic parameters of the noise, such as the frequency range, intensity, and variation pattern of the noise. By accurately extracting the noise features, an important basis can be provided for the subsequent construction of an adaptive filter. The above-mentioned noise feature extraction can adopt a variety of methods, such as spectrum analysis, statistical analysis, etc., to fully understand the characteristics of the noise.

[0071] Then, an adaptive filter is constructed based on the noise characteristic parameters. An adaptive filter is a filter that can adjust its own parameters in real time according to the changes in the input signal. When constructing an adaptive filter, the extracted noise characteristic parameters are used to determine the initial parameter value and structure of the filter. The above-mentioned adaptive algorithm is selected according to the time-varying characteristics of the noise, such as the least mean square algorithm or the recursive least squares algorithm. The above-mentioned algorithm can continuously adjust the filter parameters during the filtering process to adapt to different noise changes.

[0072] Then, the multilingual long recording is input into the adaptive filter for filtering. During the filtering process, the adaptive filter adjusts the filter parameters in real time according to the noise situation in the input signal to achieve the best denoising effect. The ability to adjust the filter parameters in real time enables this method to adapt to changes in noise of different types and intensities, thereby effectively removing noise while preserving the integrity of the speech signal to the greatest extent.

[0073] In this embodiment, the filter parameters can be adjusted in real time according to the change of noise, adapting to different noise environments, and improving the effect and stability of denoising. By extracting the noise features, the constructed adaptive filter can remove noise more specifically and reduce the impact on the speech signal.

[0074] In one embodiment, constructing an adaptive filter according to noise characteristic parameters includes:

[0075] Set the initial parameters of the filter; the initial parameters include the filter order, convergence coefficient and step factor;

[0076] The weight vector of the filter is trained using the noise characteristic parameters, and the weight vector is continuously adjusted through the minimum mean square error algorithm to minimize the error between the filter output and the noise segment. During the training process, the error changes are monitored in real time. When the error stabilizes within the preset range, the training is stopped and the construction of the adaptive filter is completed.

[0077] In transcribing long multilingual recordings, building an efficient adaptive filter is crucial for removing noise. Traditional filter construction methods cannot adapt well to the complex noise conditions in long multilingual recordings, and a more accurate and adaptive construction method is needed to improve the noise suppression effect.

[0078] In this embodiment, the initial parameters of the filter are first set, including the filter order, convergence coefficient and step factor. The filter order determines the complexity of the filter and the signal processing capability. The convergence coefficient and step factor affect the convergence speed and stability of the filter during the training process. The selection of the above initial parameters can be determined based on empirical values ​​or through some preliminary experiments to ensure that the filter has a good starting point in the subsequent training process.

[0079] The noise characteristic parameters are then used to train the weight vector of the filter. The noise characteristic parameters provide important information about the noise, such as frequency characteristics, intensity, etc. By combining these parameters with the input signal, the weight vector of the filter can be better adjusted so that it can remove noise more accurately. The minimum mean square error algorithm is used to continuously adjust the weight vector. The algorithm calculates the error between the filter output and the noise fragment, and adjusts the weight vector according to the size of the error to minimize the error. Each time the weight vector is adjusted, the algorithm will update the weight vector according to the preset rules based on the current input signal and error situation to gradually approach the optimal denoising effect.

[0080] During the training process, the error changes are monitored in real time. By continuously calculating the error between the filter output and the noise fragment, the performance of the filter and the progress of the training can be understood. If the error gradually decreases and stabilizes within the preset range, it means that the filter has achieved a good denoising effect and the training can be stopped. The above preset range can be set according to the specific application requirements and noise conditions. Generally speaking, when the error is small enough so that the denoised audio quality meets the requirements, the training can be stopped to avoid overfitting problems caused by excessive training.

[0081] In this embodiment, by using noise feature parameters to train the weight vector and continuously adjusting it using the minimum mean square error algorithm, the filter can remove noise more accurately and improve the denoising effect. The above method for constructing an adaptive filter can complete the training in a shorter time, improve the efficiency of filter construction, and provide a faster noise suppression solution for transcribing long multilingual recordings.

[0082] In one embodiment, the step of inputting the speech features into a pre-trained multilingual transcription model for transcription to obtain a corresponding text result includes:

[0083] The speech features are input into the input layer of the multilingual transcription model and passed to the hidden layer after being preprocessed by the input layer;

[0084] Perform deep feature extraction on speech features based on hidden layers to extract semantic and grammatical features of different languages;

[0085] Based on the output layer of the multilingual transcription model, the probability of the extracted semantic features and grammatical features is calculated to determine the most matching character or word at each time step;

[0086] The characters or words obtained at each time step in the output layer are combined in sequence, and the repeated and erroneous parts are removed to obtain the final text result.

[0087] In transcribing long multilingual recordings, accurately converting speech features into text results is the ultimate goal. Since different languages ​​have different semantic and grammatical characteristics, traditional transcription methods may not work well when dealing with multiple languages. Therefore, an efficient transcription model that can adapt to multilingual environments is needed.

[0088] In this embodiment, the speech features are first input into the input layer of the multilingual transcription model. The speech features are parameters that can reflect the essential features of the speech obtained through the previous steps. The input layer preprocesses the speech features. The above preprocessing includes operations such as data normalization and format conversion to ensure that the input data meets the requirements of the model and provides a better basis for subsequent processing.

[0089] The preprocessed speech features are passed to the hidden layer. The hidden layer performs deep feature extraction on the speech features, aiming to extract the semantic and grammatical features of different languages. Deep feature extraction can be achieved through the multi-layer structure of the neural network, using different neurons and connection methods to capture the deep features of the language. This can better understand the meaning and structure of different languages ​​and provide a basis for accurate transcription.

[0090] Then, based on the output layer of the multilingual transcription model, the probability of the extracted semantic features and grammatical features is calculated. The output layer determines the most matching character or word at each time step by evaluating the probability of different characters or words. The probability calculation can be based on the trained model parameters and statistical methods, taking into account factors such as language context information and semantic relevance to improve the accuracy of transcription.

[0091] Finally, the characters or words obtained at each time step in the output layer are combined in sequence. The results of consecutive time steps are combined to form a preliminary text result. Then the repeated and erroneous parts are removed. Through some post-processing methods, such as checking repeated characters or words, correcting common errors, etc., the preliminary text results are optimized to obtain the final text results.

[0092] In this embodiment, it is possible to process the transcription of long recordings in multiple languages, adapt to the semantic and grammatical characteristics of different languages ​​through deep feature extraction and probability calculation, and improve the accuracy and versatility of transcription. By using the deep feature extraction of the hidden layer and the probability calculation of the output layer, the characteristics and meaning of the language can be captured more accurately, and transcription errors can be reduced. The quality and readability of the text results are improved by combining and removing the post-processing of repeated erroneous parts.

[0093] In one embodiment, before the audio segmentation of the multi-language long recording to obtain multiple audio segments includes:

[0094] Negotiate with multiple translation terminals to establish encrypted communication connections; wherein each translation terminal is a terminal held by each user;

[0095] Obtain the recorded audio of each translation terminal;

[0096] The audio recordings of the various translation terminals are combined in chronological order to obtain the multi-language long recording.

[0097] In this embodiment, in a multilingual environment, especially in a scenario involving multiple users and different translation terminals, it is of great significance to obtain high-quality multilingual long recordings and perform accurate transcription.

[0098] First, negotiate with multiple translation terminals to establish an encrypted communication connection. In a multi-language communication scenario, each translation terminal is a terminal held by each user and can be distributed in different geographical locations. By establishing an encrypted communication connection, the security and confidentiality of audio transmission can be ensured to prevent information leakage. The above encrypted communication connection can use encryption algorithms and protocols to ensure that audio data is not stolen or tampered with during transmission.

[0099] After the connection is established, the recorded audio of each translation terminal is obtained. Each translation terminal records voice content in different languages, and these audios contain important information for multilingual communication. The recorded audio data can be received from each translation terminal through specific communication protocols and interfaces, and appropriately stored and managed.

[0100] Finally, the audio recordings of each translation terminal are combined in chronological order to obtain a multi-language long recording. Since different translation terminals may record at different time points, they need to be combined in chronological order to ensure the coherence and integrity of the audio. The above combination process can be aligned by means of timestamps, etc., to ensure the correct temporal order of each audio clip.

[0101] In this embodiment, by establishing an encrypted communication connection, the security of the audio acquisition process is guaranteed, and user privacy and data security are protected. The audio recordings are obtained from multiple translation terminals, which enriches the audio sources and improves the comprehensiveness and diversity of multi-language long recordings. The combination in chronological order ensures the coherence and accuracy of multi-language long recordings, providing a good foundation for subsequent audio segmentation and transcription.

[0102] In one embodiment, the step of negotiating with a plurality of translation terminals to establish an encrypted communication connection includes:

[0103] Obtaining the language type of the user on each translation terminal, and expressing the language type as a standardized language code;

[0104] Acquire identification information of each translation terminal, and combine the identification information of each translation terminal with its corresponding language code to obtain a terminal information combination;

[0105] According to the sequence of the serial numbers of the translation terminals, the terminal information combinations are sequentially combined to obtain an identification combination; wherein each translation terminal is preset with a corresponding serial number;

[0106] Creating a preset centrosymmetric binary tree structure, and adding the characters in the identification combination to the binary tree structure one by one in sequence to obtain an identification binary tree;

[0107] De-duplication of language codes of various translation terminals is performed to obtain de-duplication codes;

[0108] Searching for the characters in the deduplication code in the identification binary tree in sequence, and using the positions of the characters as the target positions;

[0109] Connect each target position in sequence to obtain multiple lines;

[0110] In the identification binary tree, characters that satisfy a preset position relationship with the multiple lines are obtained as target characters, and the target characters are combined to obtain a target character combination; the target character combination is used as a communication key negotiated by multiple translation terminals.

[0111] In this embodiment, it is crucial to establish a secure encrypted communication connection with multiple translation terminals during the transcription of long multilingual recordings. This not only ensures the confidentiality and integrity of audio data during transmission, but also prevents unauthorized access and data leakage. Traditional encryption methods have the problem of insufficient security or not being suitable for complex multi-terminal environments.

[0112] In this embodiment, first, the language type selected by the user on the terminal or detected by the system is obtained by communicating with each translation terminal. For example, the language type information can be included in a specific request message sent by the terminal, or the language setting status can be obtained using the system setting interface of the terminal. Once the language type is obtained, it is converted into a standardized language code. For example, if it is detected that the user uses Chinese, it is converted into the "zh" code in the ISO 639 standard; if it is English, it is converted into the "en" code, etc. This ensures that the language types of different terminals have a unified identification in subsequent processing.

[0113] Next, obtain the identification information of each translation terminal. The identification information can be the hardware serial number of the terminal, the unique device identifier, or a specific number assigned by the system. Then, combine the identification information of each translation terminal with the language code corresponding to the terminal. For example, if the identification information of a terminal is "Terminal001" and its corresponding language code is "fr" (indicating French), the combined terminal information is "Terminal001fr".

[0114] Then, according to the sequence of numbers preset by each translation terminal, the terminal information combinations are combined in sequence. Assuming there are three terminals, the sequence numbers are 1, 2, and 3, and the corresponding terminal information combinations are "Terminal001fr", "Terminal002en", and "Terminal003es", then the identification combination after sequential combination is "Terminal001frTerminal002enTerminal003es".

[0115] Then, create a preset centrally symmetric binary tree structure. The above binary tree structure has a specific symmetry in design, such as the number of nodes and structural symmetry of the left and right subtrees. It can be a predefined standard template of a data structure, which is instantiated as needed at runtime. Add the characters in the logo combination to the binary tree structure one by one. For example, first add "T" to the root node of the binary tree, and then add the subsequent characters to the corresponding child nodes according to the order, and gradually build a complete logo binary tree.

[0116] In this embodiment, the language codes of each translation terminal also need to be deduplicated. For example, if multiple terminals use the same language, only one corresponding language code is retained. For example, if two terminals both use English, only one "en" code is retained after deduplication. In the constructed identification binary tree, each character in the deduplicated language code is searched in turn, and its position in the binary tree is determined, and this position is used as the target position. For example, if the language code after deduplication is "en", first search the binary tree for the position of the character "e", and then search for the position of the character "n".

[0117] Then, each target position is connected in sequence to form a plurality of lines. For example, if the position of the character "e" in the binary tree is node A, and the position of the character "n" is node B, then A and B are connected to form a line. In the identification binary tree, the characters related to these lines are determined as target characters according to the preset positional relationship. For example, the characters adjacent to the lines can be defined as target characters, or the target characters can be determined according to a preset distance rule. The determined target characters are combined to obtain a target character combination. This target character combination is used as a communication key negotiated by multiple translation terminals for subsequent encrypted communication processes. For example, in one embodiment, the characters through which the above-mentioned multiple lines pass in the identification binary tree can be used as the above-mentioned target characters; or the closest distance between each character in the identification binary tree and the multiple lines is calculated. If the closest distance is less than a preset value, the character is used as the target character. It can be understood that the characters at each endpoint of the above-mentioned multiple lines are not within the calculation range of the positional relationship.

[0118] In this embodiment, a central symmetric binary tree structure and a specific character combination method are used to generate communication keys, which has higher security and innovation compared with traditional encryption methods. Combining the language type with the encryption process makes the encrypted communication connection closely associated with the multilingual environment, improving the practicality of the solution in multilingual scenarios. The above communication key is generated through dynamic information negotiation of multiple terminals, is not easy to be cracked, and can be changed according to different terminal combinations and language types, enhancing the security of the system.

[0119] Reference Figure 3 In another embodiment of the present invention, a multi-language long recording transcription system with adaptive noise suppression is provided, comprising:

[0120] The segmentation module is used to segment the audio of a long multi-language recording to obtain multiple audio segments; perform spectrum analysis on each audio segment to obtain the corresponding spectrum features;

[0121] A detection module, used to perform noise detection on each audio segment according to the spectrum characteristics to determine the noise segment;

[0122] A denoising module, used for adaptively suppressing noise in each noise segment in the multi-language long recording to obtain denoised multi-language audio;

[0123] An extraction module is used to perform voice activity detection on the denoised multi-language audio to determine voice segments and non-voice segments; perform feature extraction on the voice segments to obtain voice features;

[0124] The transcription module is used to input speech features into a pre-trained multilingual transcription model for transcription to obtain the corresponding text results.

[0125] In this embodiment, for the specific implementation of each module in the above system embodiment, please refer to the above method embodiment, which will not be repeated here.

[0126] Reference Figure 4 In an embodiment of the present invention, a computer device is also provided. The computer device may be a server, and its internal structure may be as follows: Figure 4 As shown. The computer device includes a processor, a memory, a display screen, an input device, a network interface and a database connected through a system bus. Among them, the processor designed by the computer is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store the corresponding data in this embodiment. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the above method is implemented.

[0127] Those skilled in the art will understand that Figure 4 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present invention, and does not constitute a limitation on the computer device to which the solution of the present invention is applied.

[0128] An embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the above method is implemented. It can be understood that the computer-readable storage medium in this embodiment can be a volatile readable storage medium or a non-volatile readable storage medium.

[0129] In summary, the multi-language long recording transcription method, system and device with adaptive noise suppression provided in the embodiments of the present invention include: performing audio segmentation on the multi-language long recording to obtain multiple audio segments; performing spectrum analysis on each audio segment to obtain corresponding spectrum features; performing noise detection on each audio segment according to the spectrum features to determine the noise segment; performing adaptive noise suppression on each noise segment in the multi-language long recording to obtain denoised multi-language audio; performing voice activity detection on the denoised multi-language audio to determine voice segments and non-voice segments; performing feature extraction on the voice segment to obtain voice features; inputting the voice features into a pre-trained multi-language transcription model for transcription to obtain the corresponding text results. In the present invention, by performing adaptive noise suppression on each noise segment in the multi-language long recording, the interference of noise is effectively reduced and the voice quality is improved; by inputting the voice features into a pre-trained multi-language transcription model for transcription, it is possible to adapt to the characteristics of different languages, accurately convert the voice features into text, and improve the accuracy and efficiency of transcribing multi-language long recordings.

[0130] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media provided by the present invention and used in the embodiments may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double-speed data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM.

[0131] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the presence of other identical elements in the process, device, article or method including the element.

[0132] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A multi-language long recording transcription method with adaptive noise suppression, characterized in that: The following steps are involved: Perform audio segmentation on multi-language long recordings to obtain multiple audio segments; perform spectrum analysis on each audio segment to obtain the corresponding spectrum features; Perform noise detection on each audio segment according to the frequency spectrum characteristics to determine the noise segment; Adaptively suppressing noise in each noise segment in the multilingual long recording to obtain denoised multilingual audio; Perform voice activity detection on the denoised multi-language audio to determine voice segments and non-voice segments; perform feature extraction on the voice segments to obtain voice features; Input the speech features into the pre-trained multilingual transcription model for transcription to obtain the corresponding text results; Before the multi-language long audio recording is segmented to obtain multiple audio segments, the method includes: Negotiate with multiple translation terminals to establish encrypted communication connections; wherein each translation terminal is a terminal held by each user; Obtain the recorded audio of each translation terminal; Combining the audio recordings of the various translation terminals in chronological order to obtain the multi-language long recording; The step of negotiating with a plurality of translation terminals to establish an encrypted communication connection includes: Obtaining the language type of the user on each translation terminal, and expressing the language type as a standardized language code; Acquire identification information of each translation terminal, and combine the identification information of each translation terminal with its corresponding language code to obtain a terminal information combination; According to the sequence of the serial numbers of the translation terminals, the terminal information combinations are sequentially combined to obtain an identification combination; wherein each translation terminal is preset with a corresponding serial number; Creating a preset centrosymmetric binary tree structure, and adding the characters in the identification combination to the binary tree structure one by one in sequence to obtain an identification binary tree; De-duplication of language codes of various translation terminals is performed to obtain de-duplication codes; Searching for the characters in the deduplication code in the identification binary tree in sequence, and using the positions of the characters as the target positions; Connect each target position in sequence to obtain multiple lines; In the identification binary tree, characters that satisfy a preset position relationship with the multiple lines are obtained as target characters, the target characters are combined to obtain a target character combination; and the target character combination is used as a communication key negotiated by multiple translation terminals.

2. The multi-language long recording transcription method with adaptive noise suppression according to claim 1, characterized in that: The step of performing noise detection on each audio segment according to the frequency spectrum feature to determine the noise segment includes: Perform energy distribution analysis on the frequency spectrum characteristics of the audio clip to obtain the energy distribution in different frequency intervals; The energy distribution is compared with a preset noise energy distribution threshold. If the energy value of any frequency interval is lower than the noise energy distribution threshold and the duration reaches the preset duration, the audio segment corresponding to the frequency interval is marked as a suspected noise segment. A periodic detection process is performed on the suspected noise segment, and if there is no periodicity, the suspected noise segment is determined to be a noise segment.

3. The multi-language long recording transcription method with adaptive noise suppression according to claim 1, characterized in that: The step of adaptively suppressing noise on each noise segment in the multi-language long recording to obtain denoised multi-language audio includes: Performing noise feature extraction processing on the noise segment to obtain noise feature parameters; Construct an adaptive filter based on noise characteristic parameters; The multi-language long recording is input into an adaptive filter for filtering to obtain denoised multi-language audio; wherein the adaptive filter adjusts the filter parameters in real time during the filtering process to adapt to different noise changes.

4. The multi-language long recording transcription method with adaptive noise suppression according to claim 3, characterized in that: The step of constructing an adaptive filter according to noise characteristic parameters comprises: Set the initial parameters of the filter; the initial parameters include the filter order, convergence coefficient and step factor; The weight vector of the filter is trained using the noise characteristic parameters, and the weight vector is continuously adjusted through the minimum mean square error algorithm to minimize the error between the filter output and the noise segment. During the training process, the error changes are monitored in real time. When the error stabilizes within the preset range, the training is stopped and the construction of the adaptive filter is completed.

5. The multi-language long recording transcription method with adaptive noise suppression according to claim 1, characterized in that: The speech features are input into a pre-trained multilingual transcription model for transcription to obtain corresponding text results, including: The speech features are input into the input layer of the multilingual transcription model and passed to the hidden layer after being preprocessed by the input layer; Perform deep feature extraction on speech features based on hidden layers to extract semantic and grammatical features of different languages; Based on the output layer of the multilingual transcription model, the probability of the extracted semantic features and grammatical features is calculated to determine the most matching character or word at each time step; The characters or words obtained at each time step in the output layer are combined in sequence, and the repeated and erroneous parts are removed to obtain the final text result.

6. A multi-language long recording transcription system with adaptive noise suppression, characterized in that: include: A segmentation module is used to segment long multilingual recordings into multiple audio segments; Perform spectrum analysis on each audio clip to obtain the corresponding spectrum features; A detection module, used to perform noise detection on each audio segment according to the spectrum characteristics to determine the noise segment; A denoising module, used for adaptively suppressing noise in each noise segment in the multi-language long recording to obtain denoised multi-language audio; An extraction module is used to perform voice activity detection on the denoised multi-language audio to determine voice segments and non-voice segments; perform feature extraction on the voice segments to obtain voice features; The transcription module is used to input the speech features into a pre-trained multilingual transcription model for transcription to obtain the corresponding text results; Before the multi-language long audio recording is segmented to obtain multiple audio segments, the method includes: Negotiate with multiple translation terminals to establish encrypted communication connections; wherein each translation terminal is a terminal held by each user; Obtain the recorded audio of each translation terminal; Combining the audio recordings of the various translation terminals in chronological order to obtain the multi-language long recording; The step of negotiating with a plurality of translation terminals to establish an encrypted communication connection includes: Obtaining the language type of the user on each translation terminal, and expressing the language type as a standardized language code; Acquire identification information of each translation terminal, and combine the identification information of each translation terminal with its corresponding language code to obtain a terminal information combination; According to the sequence of the serial numbers of the translation terminals, the terminal information combinations are sequentially combined to obtain an identification combination; wherein each translation terminal is preset with a corresponding serial number; Creating a preset centrosymmetric binary tree structure, and adding the characters in the identification combination to the binary tree structure one by one in sequence to obtain an identification binary tree; De-duplication of language codes of various translation terminals is performed to obtain de-duplication codes; Searching for the characters in the deduplication code in the identification binary tree in sequence, and using the positions of the characters as the target positions; Connect each target position in sequence to obtain multiple lines; In the identification binary tree, characters that satisfy a preset position relationship with the multiple lines are obtained as target characters, and the target characters are combined to obtain a target character combination; the target character combination is used as a communication key negotiated by multiple translation terminals.

7. A computer device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images