Video communication voice transcription method and device adopting artificial intelligence, and electronic equipment

Abstract

The invention discloses a video communication voice transliteration method and device adopting artificial intelligence and electronic equipment, and relates to the technical field of voice transliteration, and the method comprises the following steps: obtaining a voice signal in a video communication process, and segmenting the voice signal according to a preset frame length to obtain a plurality of voice segments; respectively carrying out feature extraction on the plurality of voice segments, constructing a fuzzy feature vector and identifying a fuzzy pronunciation section; constructing a voice membership curve by adopting a triangular membership function, and evaluating the fuzzy feature vector change before and after the inflection point of the voice membership curve; correcting the speech membership curve slope corresponding to the fuzzy pronunciation section based on the evaluation result to obtain a corrected fuzzy interval; according to the method, the dynamically generated phoneme candidate paths are screened, and the character transliteration result is generated based on the screened optimal phoneme path, so that the problem that fuzzy sounds are forcibly distributed to non-main semantic paths due to overlarge curve inflection point offset when the semantic variation is excessively amplified is solved.

Description

Technical Field

[0001] This invention relates to the field of speech transcription technology, and more specifically, to a video communication speech transcription method, apparatus, and electronic device employing artificial intelligence. Background Technology

[0002] In applications such as water conservancy that require field operations and real-time consultations, highly integrated portable devices have emerged to meet the emergency command needs of different locations and conditions. These devices integrate display modules, 4G / 5G network access modules, and ultra-long-lasting power supply modules. These devices not only provide comprehensive audio / video and network expansion interfaces, but also optimize the enclosure material and internal module layout. While ensuring over 4 hours of battery life for outdoor operations, they also include a real-time enclosure temperature display module, improving safety and reliability, and expanding the number of interfaces to support more audio / video and network devices, thereby enhancing the system's interface expansion capabilities.

[0003] Furthermore, these devices typically possess powerful audio processing capabilities in modern video conferencing systems, employing advanced audio encoding and decoding technologies to effectively suppress echo and background noise, ensuring clear and smooth voice transmission and providing a superior auditory experience for remote participants. Simultaneously, the system supports sharing of various data formats, including files, images, and videos, making data transfer and discussion during meetings more efficient and convenient.

[0004] However, despite employing advanced audio coding and noise suppression technologies to ensure clear speech transmission, these devices still face the challenge of recognizing ambiguous pronunciation segments during actual speech-to-text transcription. Especially when semantic variations are excessively amplified, the inflection point of the speech membership curve may shift significantly, causing the system to incorrectly assign ambiguous sounds to non-primary semantic paths, thus affecting the accuracy and semantic coherence of the transcription results. This shift not only leads to inaccurate localization of ambiguous pronunciation segments but may also cause errors in speech recognition path selection, ultimately impacting the final text transcription quality.

[0005] To address the above problems, this invention proposes a solution. Summary of the Invention

[0006] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide a video communication speech transcription method, apparatus, and electronic device employing artificial intelligence. By correcting the speech membership curve and correctly assigning ambiguous speech to the main semantic path, the present invention addresses the problem that when the semantic change is excessively amplified, the curve inflection point shifts too much, causing ambiguous speech to be forcibly assigned to a non-main semantic path.

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] A video communication speech-to-text method using artificial intelligence includes the following steps: acquiring speech signals during video communication and segmenting them according to a preset frame length to obtain several speech segments; extracting features from each speech segment, constructing fuzzy feature vectors, and identifying fuzzy pronunciation regions; constructing a speech membership curve using a triangular membership function based on the fuzzy feature vectors, and evaluating the changes in fuzzy feature vectors before and after the inflection point of the speech membership curve; correcting the slope of the speech membership curve corresponding to the fuzzy pronunciation region based on the evaluation results to obtain a corrected fuzzy interval; filtering dynamically generated phoneme candidate paths in the corrected fuzzy interval, and generating a text transcription result based on the selected optimal phoneme path.

[0009] In a preferred embodiment, the step of acquiring the audio signal during video communication and segmenting it according to a preset frame length to obtain several audio segments specifically involves: acquiring the audio signal during video communication, using an adaptive noise suppression algorithm to reduce the noise of the audio signal to obtain a denoised audio signal; performing endpoint detection on the denoised audio signal to identify valid audio; and segmenting the valid audio into equal-length segments based on the preset frame length to obtain several audio segments.

[0010] In a preferred embodiment, the step of extracting features from several speech segments, constructing fuzzy feature vectors, and identifying fuzzy pronunciation segments specifically involves: extracting the Mel-frequency cepstral coefficients and first-order differences of each speech segment to construct a first feature vector; calculating the harmonic noise ratio and formant bandwidth deviation within each speech segment to construct a second feature vector; tensor concatenating the first and second feature vectors to obtain a fuzzy feature vector; inputting the fuzzy feature vector into a preset bidirectional long short-term memory network to output a fuzzy probability value; and marking consecutive speech segments with fuzzy probability values ​​exceeding a preset threshold as fuzzy pronunciation segments.

[0011] In a preferred embodiment, the step of constructing a speech membership curve based on a fuzzy feature vector and using a triangular membership function specifically involves: constructing a triangular membership function with the starting point of the fuzzy pronunciation segment as the vertex; inputting the fuzzy feature vector into the triangular membership function according to a preset time sequence, calculating the membership value at each time point; and generating a speech membership curve based on the membership values.

[0012] In a preferred embodiment, the evaluation of the changes in fuzzy feature vectors before and after the inflection point of the speech membership curve specifically involves: detecting the zero point of the second derivative of the speech membership curve to obtain the inflection point of the speech membership curve; extracting the fuzzy feature vectors within a preset time window before and after the inflection point, and calculating the difference in the trace of the Mahalanobis distance covariance matrix of the fuzzy feature vectors within the preset time window before and after the inflection point to obtain the confidence weight of the inflection point.

[0013] In a preferred embodiment, the step of filtering the dynamically generated phoneme candidate paths in the corrected fuzzy interval specifically involves: constructing a phoneme transition probability matrix based on the slope of the corrected speech membership curve in the corrected fuzzy interval; constructing a hidden Markov model state transition graph; inputting the phoneme transition probability matrix into the Viterbi algorithm to obtain the cumulative probability of each phoneme candidate path; and filtering the phoneme candidate paths based on the cumulative probability.

[0014] In a preferred embodiment, generating the text transcription result based on the selected optimal phoneme path specifically involves: inputting the selected optimal phoneme path into a preset converter to generate a candidate word set; inputting the candidate word set into a bidirectional long short-term memory network to model the grammatical structure and reconstruct sentences that conform to preset grammatical rules; inserting punctuation marks into the sentences and formatting the output to obtain the text transcription result.

[0015] The technical effects and advantages of the video communication speech transcription method, device, and electronic device using artificial intelligence of the present invention are as follows:

[0016] 1. This invention constructs a speech membership curve using a triangular membership function and evaluates the credibility of the inflection point by combining the Mahalanobis distance change of fuzzy feature vectors before and after the inflection point. Based on the credibility weights, slope correction and smoothing are performed, significantly improving the accuracy of fuzzy sound attribution. Simultaneously, by combining a phoneme transition probability matrix and a Hidden Markov Model, the optimal phoneme path is effectively screened. A bidirectional Long Short-Term Memory (LSTM) network is used to model the grammatical structure of the candidate word set and reconstruct sentences, achieving high-precision, real-time, and semantically coherent text transcription. This technology not only improves the accuracy and robustness of speech transcription in multi-point video communication scenarios and reduces recognition errors caused by noise and speech fuzziness, but also enhances the system's adaptability to complex speech environments, providing users with a more natural, fluent, and reliable speech transcription service, significantly optimizing the communication efficiency and user experience of video conferencing. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating a video communication speech transcription method using artificial intelligence according to the present invention.

[0018] Figure 2 This is a schematic diagram of the device structure for a video communication speech transcription method using artificial intelligence according to the present invention.

[0019] Figure 3 This is a schematic diagram of an electronic device structure for a video communication speech transcription method using artificial intelligence according to the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0021] Example 1, Figure 1 This invention presents a video communication speech transcription method using artificial intelligence, comprising the following steps:

[0022] S1, acquire the audio signal during video communication and segment it according to the preset frame length to obtain several audio segments;

[0023] In this example, the audio signal during video communication is acquired and segmented according to a preset frame length to obtain several audio segments, specifically:

[0024] The audio signal during video communication is acquired, and an adaptive noise suppression algorithm is used to reduce the noise of the audio signal to obtain the noise-reduced audio signal.

[0025] Endpoint detection is performed on the denoised speech signal to identify valid speech;

[0026] Based on the preset frame length, the effective speech is divided into segments of equal length, resulting in several speech segments.

[0027] It should be noted that when acquiring multi-channel audio streams during video communication using a microphone array, an adaptive noise suppression algorithm is introduced because audio signals are often interfered with by background noise during transmission, such as environmental noise, human conversations, or electromagnetic interference from equipment. This algorithm can automatically adjust noise suppression parameters according to the background environment of the audio signal, dynamically filter noise components, and retain effective audio information, thereby improving the accuracy and stability of subsequent processing.

[0028] After noise suppression, endpoint detection is performed on the denoised speech signal. This operation aims to distinguish speech from silence and identify the speech region containing valid pronunciation. The endpoint detection module employs a multi-dimensional judgment strategy, such as energy threshold and zero-crossing rate, to comprehensively determine the start and end points of the speech, ensuring that subsequent processing focuses on the actual pronunciation region and avoiding redundant data from interfering with the recognition results.

[0029] After recognizing valid speech, the valid speech signal is segmented into multiple consecutive speech segments according to a preset frame length. These speech segments serve as the basic processing units for subsequent feature extraction and fuzzy analysis, and will then enter the fuzzy feature vector construction stage.

[0030] S2, extract features from several speech segments, construct fuzzy feature vectors, and identify fuzzy pronunciation segments;

[0031] In this example, features are extracted from several speech segments to construct fuzzy feature vectors and identify fuzzy pronunciation segments, specifically as follows:

[0032] Extract the Mel frequency cepstral coefficients and first-order differences of each speech segment to construct the first feature vector;

[0033] Calculate the harmonic noise ratio and formant bandwidth deviation within each speech segment to construct a second feature vector;

[0034] The first feature vector and the second feature vector are concatenated using tensors to obtain a fuzzy feature vector.

[0035] The fuzzy feature vector is input into a pre-defined bidirectional long short-term memory network, and the fuzzy probability value is output.

[0036] Continuous speech segments with fuzziness probability values ​​exceeding a preset threshold are marked as fuzzy pronunciation segments.

[0037] It should be noted that Mel-frequency cepstral coefficients (MFCC) is a feature extraction method commonly used in speech recognition. Based on the Mel scale perceived by human hearing, it transforms the spectral information of the speech signal into a set of coefficients representing the essential features of speech, used to characterize the timbre and pitch characteristics of speech, and is a compact expression describing the frequency structure of speech. The first-order difference, on the other hand, is the derivative of the Mel-frequency cepstral coefficient features in the time dimension, used to represent the changing trend of speech features over time, enhancing the sensitivity to changes in speech rate and pitch.

[0038] Furthermore, Harmonic Noise Ratio (HNR) is an indicator that measures the ratio of periodic to non-periodic components in speech. It reflects the degree of speech intelligibility and stability. A high HNR indicates clear and continuous pronunciation, while a low HNR may mean unclear, ambiguous, or speech interfered with by background noise. Formant bandwidth deviation refers to the abnormal expansion or compression of the bandwidth of formants (i.e., regions of concentrated energy in the spectrum) in speech. It is usually used to identify whether pronunciation is abnormal, ambiguous, or interfered with. By extracting the HNR and formant bandwidth deviation of each speech segment, a judgment on speech intelligibility and structural stability can be formed. These parameters together constitute the second feature vector. The first and second feature vectors are concatenated using a tensor. Specifically, the two vectors are connected in the last dimension of the tensor. For example, if the first feature dimension is 20 and the second feature dimension is 4, the resulting fuzzy feature vector will have a dimension of 24.

[0039] Furthermore, the fuzzy feature vector is a probabilistic feature representation used to express whether there is ambiguity in the pronunciation of a speech segment after combining multi-dimensional speech features (such as MFCC, HNR, bandwidth deviation, etc.). It is not a simple physical or spectral value, but a high-level semantic vector that reflects the clarity, pronunciation stability and recognizability of speech after fusing multi-dimensional features. The purpose of this vector is to serve as input data so that intelligent models can determine whether there are ambiguous pronunciation segments in the speech, thereby improving the accuracy and robustness of the speech transcription system under complex conditions such as noise, ambiguity, and accent.

[0040] Finally, inputting the fuzzy feature vector into the preset bidirectional long short-term memory network means inputting the fuzzy feature sequence of each frame as time series data into the neural network model. This model can understand the speech context information from both forward and backward time directions. Therefore, when analyzing whether a speech segment is fuzzy, it can combine the preceding and following contexts to determine whether there are fuzzy pronunciation phenomena such as abnormal speech rate, connected speech, and omissions. The model finally outputs a probability value at each time step to represent the probability that the current speech segment belongs to fuzzy pronunciation. Probabilities exceeding a set threshold are marked as fuzzy segments. These probability values ​​constitute a fuzzy probability sequence for subsequent membership function processing and path selection.

[0041] S3, based on fuzzy feature vectors, constructs speech membership curves using triangular membership functions, and evaluates the changes in fuzzy feature vectors before and after the inflection point of the speech membership curves;

[0042] In this example, based on fuzzy feature vectors, a triangular membership function is used to construct a speech membership curve, and the changes in fuzzy feature vectors before and after the inflection point of the speech membership curve are evaluated, specifically as follows:

[0043] Construct a triangular membership function with the starting point of the ambiguous pronunciation segment as the vertex;

[0044] The fuzzy feature vector is input into the triangular membership function according to the preset time series, and the membership value at each time point is calculated.

[0045] Generate speech membership curves based on membership values.

[0046] It should be noted that the triangular membership function is a membership calculation method used for fuzzy logic modeling. It measures the degree of fuzziness of speech segments in the time dimension. Its construction process takes the starting point of the fuzzy pronunciation segment as the vertex of the function, that is, the time point where the membership degree is 1. Then, a descending boundary point is set forward and backward. The membership degree at the time positions corresponding to these two boundary points is 0, thus forming a symmetrical or asymmetrical triangular function curve. The shape of the entire function is a sharp-topped gradient change structure, indicating that starting from the central vertex, as the time distance increases, the membership degree of fuzziness gradually weakens until it reaches zero.

[0047] After constructing the triangular membership function, the fuzzy feature vectors are input into the function sequentially according to the time order of the speech segments. Each speech segment corresponds to a time point. The triangular function calculates the fuzzy membership value of that time point based on the distance between that point and the vertex. Throughout the process, the membership values ​​of all speech segments are arranged sequentially to generate a continuous speech membership curve. This curve reflects the degree of fuzziness at each moment in the entire speech stream. The peak appears at the center of the fuzzy segment, and the edges gradually decay, forming a continuous and differentiable fuzzy trend line.

[0048] In this example, the changes in fuzzy feature vectors before and after the inflection point of the speech membership curve are evaluated, specifically as follows:

[0049] The inflection point of the speech membership curve is obtained by detecting the zero point of the second derivative of the speech membership curve.

[0050] Extract the fuzzy feature vectors within the preset time window before and after the inflection point, and calculate the difference of the Mahalanobis distance covariance matrix trace of the fuzzy feature vectors within the preset time window before and after the inflection point to obtain the inflection point confidence weight.

[0051] It should be noted that, in order to detect the inflection points of the speech membership curve, the second derivative zero-point detection method is adopted. The specific process is as follows: first, the generated speech membership curve is regarded as a continuous function that changes with time. Then, the second derivative of the curve is approximated at each time point using numerical methods. The second derivative represents the change in the curvature of the curve. When its value changes from positive to negative or from negative to positive, it indicates that the curve has a local extremum or a change in shape at that point. Such points are regarded as inflection points of the curve. By detecting these points where the second derivative is zero, the position where the change in speech ambiguity is most obvious can be located. These inflection points often correspond to the transition boundary between speech pronunciation ambiguity and clarity, and are an important basis for subsequent correction of the ambiguity range.

[0052] Furthermore, after detecting the inflection point of the speech membership curve, it is necessary to further evaluate whether the inflection point truly reflects the fuzziness change. To do this, it is necessary to analyze the change characteristics of the fuzzy feature vectors in the two time windows before and after the inflection point. Specifically, the fuzzy feature vector sets in the two windows are extracted, and their covariance matrices are calculated separately. Then, the trace value of their respective Mahalanobis distance covariance matrices is calculated. This trace value can be regarded as the overall dispersion of the feature change distribution. If the difference between the trace values ​​before and after the inflection point is large, it indicates that the fuzziness features have changed significantly, and the inflection point is highly reliable. Conversely, if the difference is small, it indicates that the inflection point may just be a natural fluctuation rather than an actual fuzzy boundary.

[0053] S4. Based on the evaluation results, the slope of the speech membership curve corresponding to the fuzzy pronunciation segment is corrected to obtain the corrected fuzzy interval.

[0054] In this example, the slope of the speech membership curve corresponding to the ambiguous pronunciation segment is corrected based on the evaluation results to obtain the corrected ambiguous interval, specifically:

[0055] When the confidence weight of the inflection point exceeds the preset threshold, the principal component analysis projection direction of the feature vector at the inflection point is extracted.

[0056] The speech membership curve is piecewise linearly transformed along the projection direction to obtain the transformed speech membership curve.

[0057] The transformed speech membership curve is smoothed by cubic spline interpolation to obtain a smoothed curve.

[0058] The boundaries of intervals with membership degrees lower than a preset first threshold are redefined based on the smooth curve to obtain the corrected fuzzy intervals.

[0059] It should be noted that after performing piecewise linear transformation and cubic spline interpolation smoothing on the speech membership curve, a more continuous, smooth, and structurally stable fuzzy membership curve is obtained, which reflects the changing trend of speech fuzziness. Next, to more accurately define which speech segments belong to fuzzy pronunciation regions, the membership values ​​at each time point on this smoothed curve are traversed, and continuous intervals with membership values ​​below a preset first threshold are identified as new fuzzy interval boundaries. This threshold-based re-division strategy can eliminate misclassified segments caused by noise or erroneous detection in the original fuzzy regions, while expanding adjacent regions that show increased fuzziness after smoothing. This makes the finally identified fuzzy pronunciation regions more consistent with real speech characteristics and auditory perception patterns, achieving a correction effect with precise boundaries and reasonable range.

[0060] S5 filters the dynamically generated phoneme candidate paths in the corrected fuzzy interval and generates the text transcription result based on the filtered optimal phoneme path.

[0061] In this example, the candidate phoneme paths dynamically generated within the corrected fuzzy interval are filtered, specifically as follows:

[0062] In the corrected fuzzy interval, a phoneme transition probability matrix is ​​constructed based on the slope of the corrected speech membership curve;

[0063] Construct the state transition graph of the Hidden Markov Model, input the phoneme transition probability matrix into the Viterbi algorithm, and obtain the cumulative probability of each phoneme candidate path;

[0064] Phoneme candidate paths are filtered based on cumulative probability.

[0065] It should be noted that after correcting the ambiguous segments, a membership curve that more accurately reflects the changes in speech ambiguity is obtained. By analyzing the slope changes of this curve in each time segment, the dynamic characteristics of the transition between phonemes in the speech signal can be captured. Specifically, the steeper the slope, the more drastic the speech change, which usually indicates a high probability area of ​​phoneme boundary or phoneme transition, while a gentle slope indicates that the current speech state may continue. Therefore, based on the magnitude and direction of the slope in different time periods, the probability weights of state transitions between phonemes are adjusted to construct a phoneme transition probability matrix with time dynamic characteristics.

[0066] Furthermore, the phoneme transition probability matrix is ​​a statistical structure that describes the probability of transitions between various phonemes in speech. It constructs a two-dimensional matrix with all candidate phonemes as the state set, where each matrix element represents the probability of transitioning from one phoneme state to another. This matrix not only includes transitions between common phoneme pairs (such as transitions from vowels to consonants and from consonants to vowels), but also dynamically assigns weights by combining the temporal information and fuzzy slope information of speech.

[0067] Finally, after constructing the phoneme transition probability matrix, it is used as input to the Viterbi algorithm. Combined with the Hidden Markov Model structure, the global probability of all possible phoneme paths is calculated. At each step, the Viterbi algorithm selects the current optimal transition path and accumulates the probability value of the entire path, thus obtaining the most probable candidate phoneme path in the current speech segment. After the calculation is completed, all candidate paths are sorted from high to low according to the accumulated probability, and the one or several paths with the highest accumulated probability are selected as the final optimal phoneme path. This can effectively eliminate erroneous branches caused by ambiguous pronunciation or noise interference, ensuring that the final phoneme sequence used for text transcription has the highest reliability and linguistic rationality.

[0068] In this example, the text transcription result is generated based on the selected optimal phoneme path, specifically as follows:

[0069] The selected optimal phoneme path is input into a preset converter to generate a candidate word set;

[0070] The candidate word set is input into a bidirectional long short-term memory network to model the grammatical structure and reconstruct sentences that conform to the preset grammatical rules;

[0071] Insert punctuation marks into the sentence and format the output to obtain the text transcription result.

[0072] It should be noted that the preset converter usually refers to a mapping module that converts phoneme sequences into candidate words or syllables. It can be a phoneme-word lookup table based on dictionary rules, or a trained neural network model, such as a sequence-to-sequence (Seq2Seq) converter based on an attention mechanism, or a mapping tool based on a CTC (Connected Temporal Classification) decoder. Its main function is to synthesize, spell, and map these phonemes based on the known optimal phoneme paths, combined with language models or pronunciation rules, thereby generating multiple possible corresponding text candidate words. It is a core component connecting the speech recognition front-end and the language understanding back-end.

[0073] After determining the phoneme path, the path is passed as input to a pre-defined converter. The converter will search for multiple word expressions that may correspond to the phoneme sequence in the language model or phoneme-word lookup table based on the structure and pronunciation rules of the phoneme sequence. At the same time, in order to enhance the ability to handle homophones, polyphonic characters, or ambiguous sounds, the converter is usually designed to output a set of candidate words. That is, for a set of phoneme sequences, it not only provides a definite word, but also gives multiple possible word combinations. Each candidate word may also have a confidence score, so that the subsequent model can make context judgments and sort and select.

[0074] Furthermore, after obtaining the candidate word set, these word sequences are input as time steps into a trained bidirectional long short-term memory network. This network can simultaneously understand the sentence's grammatical structure, word order relationships, and contextual semantics from both the preceding and following directions, thereby selecting the word combination that best conforms to language rules and actual context from multiple candidate words. The network captures long-term dependencies through a memory mechanism and generates hidden state sequences to predict the most likely structure of the sentence. Finally, it outputs a complete sentence structure after grammatical modeling. Subsequently, according to preset language formatting rules, such as sentence segmentation, punctuation insertion, and uniform capitalization, the sentence is formatted to generate the final transcribed text for display or storage, ensuring that it is not only accurate in content but also conforms to the expression habits of natural language.

[0075] like Figure 2The diagram shows a device structure for a video communication speech transcription method using artificial intelligence. Depending on the functions implemented, the device may include a speech acquisition module, a fuzzy analysis module, a fuzzy evaluation module, a fuzzy correction module, and a speech transcription module. The module in this invention can also be called a unit, which refers to a series of computer program segments that can be executed by the processor of the personnel performance data processing device and can perform a fixed function. These segments are stored in the memory of the personnel performance data processing device.

[0076] In this embodiment, the functions of each module are as follows:

[0077] The voice acquisition module is used to acquire voice signals during video communication and segment them according to a preset frame length to obtain several voice segments;

[0078] The fuzzy analysis module is used to extract features from several speech segments, construct fuzzy feature vectors, and identify fuzzy pronunciation segments.

[0079] The fuzzy evaluation module is used to construct a speech membership curve based on fuzzy feature vectors and a triangular membership function, and to evaluate the changes in fuzzy feature vectors before and after the inflection point of the speech membership curve.

[0080] The fuzzy correction module is used to correct the slope of the speech membership curve corresponding to the fuzzy pronunciation segment based on the evaluation results, so as to obtain the corrected fuzzy interval.

[0081] The speech-to-text module is used to filter the dynamically generated phoneme candidate paths in the corrected fuzzy range and generate the text transcription result based on the optimal phoneme path after filtering.

[0082] In detail, the modules in the video communication speech transcription device using artificial intelligence described in this embodiment of the invention employ the same technical means as the video communication speech transcription method using artificial intelligence described in the accompanying drawings, and can produce the same technical effects, which will not be repeated here.

[0083] It should be noted that an AI-powered video communication speech-to-text device is particularly suitable for mobile, portable, all-domain emergency video conferencing and command systems with extremely high requirements for real-time performance, accuracy, and environmental adaptability. In such emergency command scenarios, the device is often deployed in complex and ever-changing on-site environments, facing interference from background noise and network fluctuations. Furthermore, the command personnel may speak rapidly, with potentially unclear or hurried pronunciation. By accurately identifying unclear pronunciation segments, effectively evaluating and correcting the inflection point shift of the speech membership curve, dynamically selecting the optimal phoneme path, and ultimately generating a grammatically and semantically coherent text transcription result, the device can significantly improve the accuracy and semantic coherence of AI functions integrated into such systems, such as real-time speech-to-text, automatic subtitle generation, and intelligent meeting minutes generation. It also considers efficiency, operating stably even in environments with limited computing resources on mobile portable devices. This provides more reliable and clearer language information support for emergency command decisions, effectively improving conferencing efficiency and the user experience of the command system.

[0084] like Figure 3 The diagram shown is a schematic representation of an electronic device structure for a video communication speech transcription method using artificial intelligence according to the present invention.

[0085] The electronic device may include a processor, a memory, a communication bus, and a communication interface, and may also include a computer program stored in the memory and capable of running on the processor, such as a video communication speech transcription program employing artificial intelligence.

[0086] The processor is the control unit of the electronic device. It connects to various components of the electronic device through various interfaces and lines. It performs various functions and processes data by running or executing programs or modules stored in the memory and calling data stored in the memory.

[0087] The memory includes at least one type of readable storage medium. In some embodiments, the memory may be an internal storage unit of an electronic device, such as a portable hard drive. The memory can be used to store not only application software installed on the electronic device but also various types of data.

[0088] The communication bus is configured to enable communication between the memory and at least one processor.

[0089] The communication interface is used for communication between the aforementioned electronic device and other devices, including a network interface and a user interface.

[0090] The figure only shows an electronic device with components. Those skilled in the art will understand that the structure shown in the figure does not constitute a limitation on the electronic device and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0091] It should be understood that the embodiments described are for illustrative purposes only and are not limited to this structure in the scope of the patent application.

[0092] The memory in the electronic device stores a video communication speech transcription program using artificial intelligence, which is a combination of multiple instructions. When run in the processor, it can implement the steps in the aforementioned video communication speech transcription method using artificial intelligence.

[0093] Specifically, the processor's specific implementation system of the above instructions can be found in the description of the relevant steps in the corresponding embodiments of the accompanying drawings, which will not be repeated here.

[0094] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0095] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, in the form of a computer program product.

[0096] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0097] In addition, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0098] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0099] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A video communication speech transcription method using artificial intelligence, characterized in that, Includes the following steps: The audio signal during video communication is acquired and segmented according to a preset frame length to obtain several audio segments; Feature extraction is performed on several speech segments to construct fuzzy feature vectors and identify fuzzy pronunciation segments; Based on fuzzy feature vectors, a triangular membership function is used to construct a speech membership curve, and the changes in fuzzy feature vectors before and after the inflection point of the speech membership curve are evaluated. Based on the evaluation results, the slope of the speech membership curve corresponding to the ambiguous pronunciation segment is corrected to obtain the corrected ambiguous interval. The candidate phoneme paths dynamically generated in the corrected fuzzy interval are filtered, and the text transcription result is generated based on the optimal phoneme path after filtering.

2. The video communication speech transcription method using artificial intelligence according to claim 1, characterized in that, The process of acquiring audio signals during video communication and segmenting them according to a preset frame length to obtain several audio segments is as follows: The audio signal during video communication is acquired, and an adaptive noise suppression algorithm is used to reduce the noise of the audio signal to obtain the noise-reduced audio signal. Endpoint detection is performed on the denoised speech signal to identify valid speech; Based on the preset frame length, the effective speech is divided into segments of equal length, resulting in several speech segments.

3. The video communication speech transcription method using artificial intelligence according to claim 2, characterized in that, The step of extracting features from several speech segments, constructing fuzzy feature vectors, and identifying fuzzy pronunciation segments specifically involves: Extract the Mel frequency cepstral coefficients and first-order differences of each speech segment to construct the first feature vector; Calculate the harmonic noise ratio and formant bandwidth deviation within each speech segment to construct a second feature vector; The first feature vector and the second feature vector are concatenated using tensors to obtain a fuzzy feature vector. The fuzzy feature vector is input into a pre-defined bidirectional long short-term memory network, and the fuzzy probability value is output. Continuous speech segments with fuzziness probability values ​​exceeding a preset threshold are marked as fuzzy pronunciation segments.

4. The video communication speech transcription method using artificial intelligence according to claim 3, characterized in that, The method for constructing speech membership curves based on fuzzy feature vectors and using triangular membership functions is as follows: Construct a triangular membership function with the starting point of the ambiguous pronunciation segment as the vertex; The fuzzy feature vector is input into the triangular membership function according to the preset time series, and the membership value at each time point is calculated. Generate speech membership curves based on membership values.

5. The video communication speech transcription method using artificial intelligence according to claim 4, characterized in that, The evaluation of the changes in fuzzy feature vectors before and after the inflection point of the speech membership curve is specifically as follows: The inflection point of the speech membership curve is obtained by detecting the zero point of the second derivative of the speech membership curve. Extract the fuzzy feature vectors within the preset time window before and after the inflection point, and calculate the difference of the Mahalanobis distance covariance matrix trace of the fuzzy feature vectors within the preset time window before and after the inflection point to obtain the inflection point confidence weight.

6. The video communication speech transcription method using artificial intelligence according to claim 5, characterized in that, The process of filtering dynamically generated phoneme candidate paths within the corrected fuzzy intervals specifically involves: In the corrected fuzzy interval, a phoneme transition probability matrix is ​​constructed based on the slope of the corrected speech membership curve; Construct the state transition graph of the Hidden Markov Model, input the phoneme transition probability matrix into the Viterbi algorithm, and obtain the cumulative probability of each phoneme candidate path; Phoneme candidate paths are filtered based on cumulative probability.

7. The video communication speech transcription method using artificial intelligence according to claim 6, characterized in that, The process of generating text transcription results based on the selected optimal phoneme paths is as follows: The selected optimal phoneme path is input into a preset converter to generate a candidate word set; The candidate word set is input into a bidirectional long short-term memory network to model the grammatical structure and reconstruct sentences that conform to the preset grammatical rules; Insert punctuation marks into the sentence and format the output to obtain the text transcription result.

8. An apparatus for using a video communication speech transcription method employing artificial intelligence as described in any one of claims 1-7, characterized in that, It includes a voice acquisition module, a fuzzy analysis module, a fuzzy evaluation module, a fuzzy correction module, and a speech-to-text module: The voice acquisition module is used to acquire voice signals during video communication and segment them according to a preset frame length to obtain several voice segments; The fuzzy analysis module is used to extract features from several speech segments, construct fuzzy feature vectors, and identify fuzzy pronunciation segments. The fuzzy evaluation module is used to construct a speech membership curve based on fuzzy feature vectors and a triangular membership function, and to evaluate the changes in fuzzy feature vectors before and after the inflection point of the speech membership curve. The fuzzy correction module is used to correct the slope of the speech membership curve corresponding to the fuzzy pronunciation segment based on the evaluation results, so as to obtain the corrected fuzzy interval. The speech-to-text module is used to filter the dynamically generated phoneme candidate paths in the corrected fuzzy range and generate the text transcription result based on the optimal phoneme path after filtering.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores computer programs that can be executed by the at least one processor. The computer program is executed by the at least one processor. So that the at least one processor can execute a video communication speech transcription method using artificial intelligence as described in any one of claims 1 to 7.

Images