Multimedia-based enterprise teleconference method

Abstract

The invention discloses a multimedia-based enterprise teleconference method, which comprises the following steps of: initializing a multimedia conference system, and establishing audio, video and data transmission channels; acquiring environment audio data in real time by using a microphone array, and identifying and classifying background noise; filtering background noise in real time according to a classification result, and transmitting high-quality audio data; transcribing the audio data into a text through a voice recognition technology, and performing multi-language translation to generate real-time display subtitles; and optimization and improvement are carried out according to user feedback. According to the invention, by introducing a noise identification and classification mechanism, the system can identify external noise more accurately, and by classifying the noise levels, different noise elimination strategies can be adopted according to different noise levels, so that the definition and integrity of original voice signals can be reserved to the maximum extent, and the communication effect can be ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of enterprise communications, and in particular to a multimedia-based enterprise remote conferencing method. Background Art

[0002] With the acceleration of globalization and the rapid development of information technology, enterprises are increasingly relying on teleconferencing to achieve cross-regional collaboration and communication in their daily operations. As an important enterprise management tool, teleconferencing systems connect participants in different geographical locations through the network to achieve real-time information sharing, business communication, and decision-making. However, in actual teleconferencing, background noise interference is always one of the main issues affecting the quality of meetings.

[0003] Traditional noise suppression technology usually uses a unified filtering or noise reduction algorithm. Although this method can reduce noise interference to a certain extent, it often cannot distinguish between different types or intensities of noise, which may cause distortion of voice signals or residual noise. This single processing method is powerless in the face of complex and changing noise environments and is difficult to meet the needs of high-quality remote conferences. Summary of the invention

[0004] The purpose of this section is to summarize some aspects of embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the specification abstract and the invention title of this application to avoid blurring the purpose of this section, the specification abstract and the invention title, and such simplifications or omissions cannot be used to limit the scope of the present invention.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0006] A multimedia-based enterprise teleconference method comprises the following steps:

[0007] S1. Initialize the multimedia conference system and establish audio, video and data transmission channels;

[0008] S2, using microphone array to collect environmental audio data in real time, identify and classify background noise;

[0009] S3, filtering the background noise in real time according to the classification results and transmitting high-quality audio data;

[0010] S4, transcribe the audio data into text through speech recognition technology, perform multi-language translation, and generate real-time display subtitles;

[0011] S5. Optimize and improve based on user feedback.

[0012] As a preferred solution of the multimedia-based enterprise teleconference method of the present invention, the specific steps of identifying background noise include:

[0013] Standardized spectral features of audio signals: Perform Fourier transform on audio signals to extract spectral features and perform standardized processing;

[0014] Nonlinear changes: nonlinear changes are made to the standardized spectral characteristics and time derivatives;

[0015] Feature matrix similarity: The similarity between the spectral characteristics of the noise and the time-frequency feature matrix is ​​expressed by an exponential function;

[0016] Time integration: perform time integration on the above nonlinear transformation results and similarity;

[0017] Multi-microphone weighted summation: The results of all microphones are weighted summed to obtain a comprehensive score.

[0018] As a preferred solution of the multimedia-based enterprise teleconference method described in the present invention, the calculation formula of the standardized spectrum feature of the audio signal is as follows:

[0019]

[0020] Where: X(t) is the time series audio signal collected by the microphone array; σ(t) is the standard deviation of the noise, indicating the intensity change of the noise; μ(t) is the mean of the noise, indicating the average level of the noise; FFT(X(t)) is the Fourier transform in the audio signal, which extracts the spectral features.

[0021] As a preferred solution of the multimedia-based enterprise teleconferencing method described in the present invention, the nonlinear transformation calculation formula is as follows:

[0022] ReLUFreq(X i (t)) = ReLU(NormFreq(X i (t)))

[0023]

[0024] Where: ReLU(t) is the rectified linear unit function; tanh(t) is the hyperbolic tangent function.

[0025] As a preferred solution of the multimedia-based enterprise teleconference method described in the present invention, the characteristic matrix similarity calculation formula is as follows:

[0026]

[0027] Where: λ is the regularization parameter; φ(t) is the spectral characteristics of the noise; ψ(t) is the time-frequency characteristic matrix of the noise.

[0028] As a preferred solution of the multimedia-based enterprise teleconference method described in the present invention, the time integral calculation formula is as follows:

[0029] IntegralFeature

[0030] Among them: α, β, γ are weight coefficients, which are used to adjust the importance of different features.

[0031] As a preferred solution of the multimedia-based enterprise teleconference method described in the present invention, the multi-microphone weighted sum calculation formula is as follows:

[0032]

[0033] Where: W i are the weights of the microphones i=1, 2, ..., n.

[0034] As a preferred solution of the multimedia-based enterprise teleconference method of the present invention, the classification of the background noise includes:

[0035] S<0.2, quiet environment: The noise level in this range is extremely low, and the system can recognize that the environment is very quiet;

[0036] 0.2≤S<0.5, mild noise: within this range, the noise level is more obvious, there is a certain amount of interference noise in the environment, and certain noise suppression and filtering are required;

[0037] 0.5≤S<1.0, moderate noise: within this range, the noise level is more obvious, there is a certain amount of interference noise in the environment, and certain noise suppression and filtering are required;

[0038] S≥1.0, severe noise: Within this range, the noise level is very high and there is serious noise interference in the environment, requiring strong noise suppression and filtering technology.

[0039] As a preferred solution of the multimedia-based enterprise teleconference method of the present invention, the noise elimination method specifically includes:

[0040] Quiet environment: continue to observe;

[0041] Mild noise: Slightly filter the audio signal to remove background noise;

[0042] Moderate noise: Use a bandpass filter to filter the audio signal more strongly;

[0043] Severe noise: Adaptive filtering is used to suppress and eliminate noise.

[0044] As a preferred solution of the multimedia-based enterprise teleconferencing method described in the present invention, the multi-language translation adopts a machine translation model to translate and optimize between different languages.

[0045] Beneficial effects of the present invention:

[0046] By introducing a noise recognition and classification mechanism, the system can more accurately identify external noise. By classifying noise levels, the invention can adopt different noise reduction strategies according to different noise levels. This targeted noise reduction method not only improves the efficiency of noise reduction, but also avoids voice distortion caused by excessive noise reduction. By accurately identifying and classifying noise and taking appropriate noise reduction measures according to the noise level, the clarity and integrity of the original voice signal can be retained to the greatest extent, ensuring communication effectiveness. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. Among them:

[0048] Figure 1 A method flow chart of a multimedia-based enterprise teleconferencing method proposed by the present invention;

[0049] Figure 2 This is a framework diagram of a noise classification and processing method for a multimedia-based enterprise remote conferencing method proposed by the present invention. DETAILED DESCRIPTION

[0050] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0051] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0052] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0053] Secondly, the present invention is described in detail with reference to the schematic diagram. When describing the embodiments of the present invention in detail, for the sake of convenience, the cross-sectional diagrams showing the device structure will not be partially enlarged according to the general scale, and the schematic diagrams are only examples, which should not limit the scope of protection of the present invention. In addition, in actual production, the three-dimensional dimensions of length, width and depth should be included.

[0054] Reference Figure 1-2 The present invention provides a multimedia-based enterprise teleconference method, comprising the following steps:

[0055] S1. Initialize the multimedia conference system and establish audio, video and data transmission channels;

[0056] S2, using microphone array to collect environmental audio data in real time, identify and classify background noise;

[0057] S3, filtering the background noise in real time according to the classification results and transmitting high-quality audio data;

[0058] S4, transcribe the audio data into text through speech recognition technology, perform multi-language translation, and generate real-time display subtitles;

[0059] S5. Optimize and improve based on user feedback.

[0060] Among them, the specific steps of identifying background noise include:

[0061] Standardized spectral features of audio signals: Perform Fourier transform on audio signals to extract spectral features and perform standardized processing;

[0062] Nonlinear changes: nonlinear changes are made to the standardized spectral characteristics and time derivatives;

[0063] Feature matrix similarity: The similarity between the spectral characteristics of the noise and the time-frequency feature matrix is ​​expressed by an exponential function;

[0064] Time integration: perform time integration on the above nonlinear transformation results and similarity;

[0065] Multi-microphone weighted summation: The results of all microphones are weighted and summed to obtain a comprehensive score. By combining the spectral features, temporal features, and video features of the audio signal, accurate identification and classification of background noise can be achieved.

[0066] Furthermore, the calculation formula of the standardized spectrum characteristics of the audio signal is as follows:

[0067]

[0068] Where: X(t) is the time series audio signal collected by the microphone array; σ(t) is the standard deviation of the noise, indicating the intensity change of the noise; μ(t) is the mean of the noise, indicating the average level of the noise; FFT(X(t)) is the Fourier transform of the audio signal, which extracts the spectral features, and the spectral features are extracted and standardized through Fourier transform.

[0069] Furthermore, the nonlinear transformation calculation formula is as follows:

[0070] ReLUFreq(X i (t)) = ReLU(NormFreq(X i (t)))

[0071]

[0072] Where: ReLU(x) is the rectified linear unit function; tanh(x) is the hyperbolic tangent function, which performs a nonlinear transformation on the time derivative.

[0073] Furthermore, the feature matrix similarity calculation formula is as follows:

[0074]

[0075] Where: λ is the regularization parameter; φ(t) is the spectral characteristics of the noise; Ψ(t) is the time-frequency characteristic matrix of the noise. The similarity between the spectral characteristics of the noise and the time-frequency characteristic matrix is ​​expressed by an exponential function, and the regularization parameter λ is used to control the influence of the similarity.

[0076] Furthermore, the time integral calculation formula is as follows:

[0077] IntegralFeature

[0078] Among them: α, β, γ are weight coefficients, which are used to adjust the importance of different features, integrate time, and accumulate time features.

[0079] Furthermore, the multi-microphone weighted summation calculation formula is as follows:

[0080]

[0081] Where: W iThe weights of the microphones are i=1, 2, ..., n, and the signals collected by all microphones are weighted and summed to integrate the characteristics of multiple microphones.

[0082] Further, the classification of background noise includes:

[0083] S<0.2, quiet environment: The noise level in this range is extremely low, and the system can recognize that the environment is very quiet;

[0084] 0.2≤S<0.5, mild noise: within this range, the noise level is more obvious, there is a certain amount of interference noise in the environment, and certain noise suppression and filtering are required;

[0085] 0.5≤S<1.0, moderate noise: within this range, the noise level is more obvious, there is a certain amount of interference noise in the environment, and certain noise suppression and filtering are required;

[0086] S≥1.0, severe noise: Within this range, the noise level is very high and there is serious noise interference in the environment, requiring strong noise suppression and filtering technology. By classifying the severity of the noise, the system can better process and identify background noise in different environments.

[0087] Furthermore, the noise elimination method specifically includes:

[0088] Quiet environment: continue to observe;

[0089] Mild noise: Slightly filter the audio signal to remove background noise;

[0090] Moderate noise: Use a bandpass filter to filter the audio signal more strongly;

[0091] Severe noise: Adaptive filtering is used to suppress and eliminate noise. Different noise elimination methods are used for different levels of noise to achieve the best noise suppression effect.

[0092] Furthermore, multilingual translation uses machine translation models to translate and optimize between different languages.

[0093] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A multimedia-based enterprise teleconference method, characterized by: The following steps are involved: S1. Initialize the multimedia conference system and establish audio, video and data transmission channels; S2, using microphone array to collect environmental audio data in real time, identify and classify background noise; S3, filtering the background noise in real time according to the classification results and transmitting high-quality audio data; S4, transcribe the audio data into text through speech recognition technology, perform multi-language translation, and generate real-time display subtitles; S5. Optimize and improve based on user feedback.

2. A multimedia-based enterprise teleconferencing method according to claim 1, characterized in that: The specific steps of identifying background noise include: Standardized spectral features of audio signals: Perform Fourier transform on audio signals to extract spectral features and perform standardized processing; Nonlinear changes: nonlinear changes are made to the standardized spectral characteristics and time derivatives; Feature matrix similarity: The similarity between the spectral characteristics of the noise and the time-frequency feature matrix is ​​expressed by an exponential function; Time integration: perform time integration on the above nonlinear transformation results and similarity; Multi-microphone weighted summation: The results of all microphones are weighted summed to obtain a comprehensive score.

3. A multimedia-based enterprise teleconferencing method according to claim 2, characterized in that: The calculation formula of the audio signal standardized spectrum feature is as follows: Where: X(t) is the time series audio signal collected by the microphone array; σ(t) is the standard deviation of the noise, indicating the intensity change of the noise; μ(t) is the mean of the noise, indicating the average level of the noise; FFT(X(t)) is the Fourier transform in the audio signal, which extracts the spectral features.

4. A multimedia-based enterprise teleconference method according to claim 3, characterized in that: The nonlinear transformation calculation formula is as follows: ReLUFreq(X i (t))=ReLU(NormFreq(X i (t))) Where: ReLU(x) is the rectified linear unit function; tanh(x) is the hyperbolic tangent function.

5. A multimedia-based enterprise teleconference method according to claim 4, characterized in that: The feature matrix similarity calculation formula is as follows: Where: λ is the regularization parameter; φ(t) is the spectral characteristics of the noise; Ψ(t) is the time-frequency characteristic matrix of the noise.

6. A multimedia-based enterprise teleconference method according to claim 5, characterized in that: The time integral calculation formula is as follows: IntegralFeature Among them: α, β, γ are weight coefficients, which are used to adjust the importance of different features.

7. A multimedia-based enterprise teleconference method according to claim 6, characterized in that: The noise score calculation formula obtained by weighted summation of multiple microphones is as follows: Where: W i are the weights of the microphones i=1, 2, ..., n.

8. The multimedia-based enterprise teleconference method according to claim 7, characterized in that: The classification of background noise includes: S<0.2, quiet environment: The noise level in this range is extremely low, and the system can recognize that the environment is very quiet; 0.2≤S<0.5, mild noise: within this range, the noise level is more obvious, there is a certain amount of interference noise in the environment, and certain noise suppression and filtering are required; 0.5≤S<1.0, moderate noise: within this range, the noise level is more obvious, there is a certain amount of interference noise in the environment, and certain noise suppression and filtering are required; S≥1.0, severe noise: Within this range, the noise level is very high and there is serious noise interference in the environment, requiring strong noise suppression and filtering technology.

9. The multimedia-based enterprise teleconference method according to claim 8, characterized in that: The noise elimination method specifically comprises: Quiet environment: continue to observe; Mild noise: Slightly filter the audio signal to remove background noise; Moderate noise: Use a bandpass filter to filter the audio signal more strongly; Severe noise: Adaptive filtering is used to suppress and eliminate noise.

10. The multimedia-based enterprise teleconference method according to claim 9, characterized in that: The multilingual translation adopts a machine translation model to translate and optimize between different languages.

Images