126 results about "Speech spectrum" patented technology

The invention relates to a user psychological state monitoring method and system based on voice and semantic recognition, and the method comprises the steps: firstly obtaining voice data and corresponding text data of a user, and respectively extracting a voice spectrum feature vector and a text feature vector; then, inputting the two features into a cross-modal joint coding network model, and aligning a speech spectrum with a text embedding space by using a multi-head attention mechanism to obtain a joint feature; thirdly, confidence coefficients of the joint features are extracted through a Bayesian network model, weighted features are obtained, a Gaussian mixture model is used for fitting an emotion fluctuation trend, and an emotion distribution probability is calculated; the confidence coefficient weight is dynamically adjusted according to the emotion distribution probability, new weighted features are obtained, and after time sequence alignment processing, local information is extracted by using a convolutional neural network and the features are fused; and finally, calculating a classification probability through a softmax function, and determining a psychological state classification result of the user.

The invention relates to the technical field of classroom behavior analysis, and discloses a classroom behavior analysis system based on an adaptive algorithm, which comprises a detection module, a parameter extraction module, a feature integration module and a decision generation module. The detection module collects classroom multi-modal behavior data and identifies active student terminals; the parameter extraction module obtains a real-time posture vector, a voice spectrum feature, an interaction response time delay and a teaching content semantic tag of a target student; the feature integration module performs cross-modal fusion on the data to generate attitude, voice, interactive fusion features and semantic association features; and the decision generation module performs hierarchical association modeling and dynamic priority ranking through a multi-head attention mechanism, generates a behavior matching degree and determines a target analysis object. The system realizes multi-modal data fusion by means of algorithms such as space-time decomposition and dynamic density clustering, improves the accuracy and real-time performance of classroom behavior analysis, and provides support for teaching optimization.

The invention relates to the technical field of wireless communication, and discloses a low-power-consumption communication method and device for wireless audio equipment, an SoC chip and a storage medium. The method comprises the following steps: sampling acoustic characteristics of heterogeneous audio data of a plurality of target audio devices, calculating a voice spectrum corresponding to device characteristics, and extracting multi-device acoustic characteristics, performing multi-frequency sub-band speech enhancement, heterogeneous segmentation and grouping of multi-target audio equipment, verification value calculation and differential coding on each extracted acoustic feature sequence to obtain a multi-source coding speech segment; performing transmission unit division and cooperative signal modulation, cooperative resource allocation and data stream transmission of each transmission unit on the multi-source coding voice segment to obtain a transmission voice data stream; and obtaining a transmission voice data stream transmitted with low power consumption and performing audio recovery on the obtained data to obtain a target audio data stream transmitted by the multi-target audio equipment. The communication power consumption of the multiple target audio devices is reduced, and the audio transmission quality is ensured.

The invention relates to the technical field of artificial intelligence, and particularly provides an artificial intelligence-based speech recognition method, which comprises the following steps of: acquiring multiple paths of speech signals through a microphone array; performing noise reduction processing on each path of voice signal, removing background noise, enhancing the voice signal by adopting an adaptive beam forming algorithm, and extracting a target voice signal; performing short-time Fourier transform on the preprocessed voice signals, extracting voice spectrum features, and extracting high-level semantic features of the voice through a deep learning model; inputting the extracted speech features into a speech recognition model based on an attention mechanism, generating text transcription of target speech, dynamically adjusting model parameters through an adaptive learning module, and optimizing a recognition result; according to the method, the recognition result is corrected according to the real-time feedback of the user, the corrected data is used for online updating of the model, the robustness and adaptability of the system are improved, and the method has the effects that training and evaluation are conducted through high-quality data, and delay is reduced in real-time processing.

The invention relates to the technical field of financial science and technology, and discloses an intelligent financial customer service interaction optimization method and system, and the method comprises the steps: collecting a real-time session data flow and a session path when a target customer carries out the session interaction with an intelligent customer service system, calculating a satisfaction degree attenuation value of the intelligent customer service system, and evaluating an interaction quality index of the session path; extracting speech spectrum features and text semantic vectors in the real-time session data stream, drawing an emotion fluctuation curve of the target customer in the session process, extracting a wave crest semantic fragment corresponding to an emotion wave crest in the emotion fluctuation curve, and calculating an emotion entropy value of the target customer; calculating the service integrating degree of the intelligent customer service system in the interaction channel; generating a session strategy optimization scheme of the target customer; and executing interaction processing about the target client, calculating a client journey conversion rate corresponding to the session strategy optimization scheme, and performing optimization adjustment on the session strategy optimization scheme to obtain a final session strategy. According to the invention, the accuracy of financial customer service interaction optimization can be improved.

The invention discloses a key information extraction method and system based on a multi-modal large model, and belongs to the technical field of information processing.The key information extraction method and system based on the multi-modal large model comprise the following specific steps that firstly, input heterogeneous modal data are subjected to standardized preprocessing, and the preprocessed data are subjected to data preprocessing; comprising text vectorization, image feature coding and speech spectrum analysis; 2, establishing an inter-modal incidence matrix through a cross-modal attention mechanism, and dynamically adjusting the feature weight of each modal; and 3, sequentially executing context semantic understanding, entity relationship modeling and core information positioning by adopting a hierarchical feature extraction architecture. According to the method, intelligent selection of inter-modal features is realized through dynamic weight distribution, the key information recall rate is greatly improved in a noise interference scene, and after a feature distillation loss function is introduced and redundant features are eliminated, the error propagation rate is effectively reduced in a noisy voice and low-resolution image coexistence scene.

The invention relates to the technical field of voice signal processing and synthesis, and particularly discloses an information processing method and system for voice conversion, and the method comprises the steps: extracting a language feature vector of each word from an input text; calculating posterior probabilities of the words in different languages by combining a Bayesian reasoning mechanism, and generating language attribution confidence coefficient characteristic values; a Monte Carlo sampling method is adopted to carry out multiple times of context sensitive simulation, and pronunciation path selection probability distribution characteristic values are generated; further fusing the feature values into a multi-language pronunciation decision vector, inputting the multi-language pronunciation decision vector into a multi-language end-to-end speech synthesis model, calling a phoneme mapping rule of a corresponding language and an acoustic parameter prediction module, and generating a high-quality target speech spectrogram; and finally, dynamically adjusting Bayesian prior distribution and a language recognition threshold according to the output speech spectrogram.

The invention discloses an ultra-lightweight speech enhancement neural network and method based on network architecture search. The neural network comprises an encoder, an enhancer and a decoder, and the encoder comprises a frequency spectrum compression module used for compressing a noisy speech frequency spectrum in a frequency dimension; the down-sampling convolution module is used for performing feature extraction and frequency dimension down-sampling on the feature map; the grouping depth separation convolution module is used for performing feature extraction and feature enhancement on the feature map; the intensifier comprises a two-way packet recurrent neural network which is used for carrying out time-frequency domain modeling on a feature map; the decoder comprises a grouping depth separation convolution module which is used for performing feature extraction and feature enhancement on the feature map; the up-sampling convolution module is used for performing feature reconstruction and frequency dimension up-sampling on the feature map; and the frequency spectrum expansion module is used for expanding the frequency dimension of the feature map. According to the invention, high-performance speech enhancement can be realized with relatively low calculation overhead.

The invention relates to the technical field of voice communication, and provides an online conference transcription method capable of realizing intelligent conference summary, which comprises the following steps: collecting audio data of each speaker in a conference process in real time through a microphone array, and carrying out primary processing on the collected audio signals; performing sound source positioning on the audio signal and determining a source direction of the audio signal according to a sound source positioning result; performing voiceprint recognition on the audio signal to extract voice spectrum features of the speaker, and marking the voiceprint of the speaker according to a voiceprint recognition result; and the weighted audio signal is used for character transcription, and the speaker is positioned to generate a complete conference summary. The problems that in a traditional audio transfer system, multiple speakers cannot be accurately recognized, speech content cannot be distinguished, and manual proofreading work cannot be reduced are effectively solved, and the accuracy and efficiency of conference recording are improved.

The invention discloses an ultra-lightweight speech enhancement neural network system and method based on adaptive convolution. The system comprises an encoder; a frequency spectrum compression module, which is used for carrying out dynamic range and frequency dimension compression on a noisy speech frequency spectrum; the self-adaptive convolution attention module is used for carrying out frequency dimension downsampling and feature extraction analysis on the compressed feature map through a self-adaptive convolution layer and time channel attention, and then sending the feature map into an intensifier; the intensifier is a dual-path packet recurrent neural network and is used for modeling sub-band time features and intra-frame frequency features of the voice; the decoder comprises a self-adaptive transpose convolution attention module which is used for performing frequency dimension upsampling and feature reconstruction on the feature map output by the enhancer; and the frequency spectrum expansion module is used for expanding the frequency dimension of the feature map output by the self-adaptive transpose convolution attention module. According to the invention, high-performance speech enhancement can be realized with extremely low calculation complexity.

The invention discloses a multi-modal data transmission method and system based on dynamic priority, and the method comprises the steps: respectively extracting image, voice and text feature data from image, voice and text data in collected multi-modal data of a service, respectively adding corresponding modal embedding vectors, and then carrying out the splicing; inputting the spliced feature data into a cross attention mechanism, and adding the attention scores of the feature data of each modal to obtain the attention score of the multi-modal fusion feature data of the service; performing joint evaluation on the attention score of the multi-modal fusion feature data of the service and a priority preset for the service to obtain a dynamic priority of the service; and preferentially transmitting the data of the service with high dynamic priority. According to the invention, transmission of data with high dynamic priority can be guaranteed preferentially, and key and important data can be guaranteed to be transmitted preferentially.

The invention discloses an electric power emerging business risk keyword identification and early warning method, system and device based on an NLP technology and a medium, and belongs to the technical field of electric power risk, and the method comprises the steps: obtaining multi-source customer service text data, and extracting risk keywords and corresponding voice spectrum data from the multi-source customer service text data; inputting the risk keyword into a power service knowledge graph, and generating a first risk research and judgment result through knowledge reasoning, the first risk research and judgment result comprising a risk traceability path; performing acoustic feature analysis on the speech spectrum data, extracting acoustic feature parameters, calculating an emotional excitation index, and generating a second risk identification result; performing weighted fusion on the first risk research and judgment result and the second risk identification result to generate a comprehensive risk score; and when the comprehensive risk score reaches a preset threshold value, generating a structured early warning work order, and carrying out data binding on the structured early warning work order, the original customer service work order and the call record. According to the invention, the efficiency of risk management and control is improved.

The present invention discloses a continuous speech formant extraction method based on peak selection, comprising: performing a preprocessing operation on a single frame of input speech; using a linear prediction method to preliminarily estimate the peak value in the spectral envelope of the speech frame; establishing a reference point and a formant trough, and then using a peak selection method to establish a mapping relationship between the peak value and the reference point; using the mapping relationship between the peak value and the reference point and the formant trough to determine the formant of the speech frame; and performing formant estimation on the continuous speech: dividing the continuous speech into frames according to different frame numbers, using the above algorithm to loop 100 times to obtain the formant parameters under different frame number tests, averaging the results after 100 loops, and obtaining the final result after smoothing. The method of the present invention can eliminate the influence of merged peaks and false peaks, and has a fast convergence speed and strong robustness.

The invention relates to the technical field of artificial intelligence, can be applied to the fields of financial science and technology and medical science and technology, and discloses a voice coding and decoding method, device, equipment and medium. A continuous vector is obtained by encoding the data through an encoder trained by a first-stage mirror image architecture; segmenting a continuous vector into sub-vectors, matching the sub-vectors with corresponding sub-coding dictionaries through a quantizer to obtain sub-indexes, and combining the sub-indexes to generate an overall index; analyzing the overall index to obtain sub-indexes, calling sub-discrete vectors and splicing the sub-discrete vectors into a discrete vector; and reconstructing the discrete vector through a decoder trained by a second-stage non-mirror-image architecture to obtain a target speech spectrum feature, and converting the target speech spectrum feature into a target speech signal. According to the method, double-stage training is adopted, the first-stage mirror image architecture guarantees the coding stability, the second-stage non-mirror image architecture improves the decoding flexibility, and the product quantization technology is combined, so that the calculation efficiency and the storage overhead are balanced, and meanwhile, the voice reconstruction quality is improved.

The invention discloses a speech enhancement method based on AI echo cancellation, and relates to the technical field of speech processing. According to the voice enhancement method based on AI echo cancellation, voice data sets in different echo environments are obtained through a recording device, time-frequency conversion is carried out on voice signals, and voice spectrum features are extracted. Time-frequency features are extracted through a convolutional neural network, and an echo cancellation model is constructed by combining a time sequence dependency relationship of a long and short term memory network capture signal. And according to the echo type label and the echo cancellation model, target echo signal characteristics are extracted, and echo-removed signal estimation is generated in combination with an adaptive filter. According to the method, the echo cancellation effect index is calculated by comprehensively analyzing the signal quality and echo residual characteristics after the echo is removed, the filter parameters are dynamically adjusted, voice enhancement is further carried out on the signal after the echo is removed through the deep convolution auto-encoder, and dual improvement of echo cancellation and voice enhancement is achieved.

The invention discloses an audio feature extraction method based on a cranial nerve model, and the method comprises the steps: obtaining voice signal data, carrying out the preprocessing of a voice signal, and segmenting the voice signal into N voice segments; extracting acoustic features representing emotional conditions from each voice segment, normalizing the acoustic features, and synthesizing the acoustic features into a multi-channel voice spectrogram; and inputting the multi-channel speech spectrogram into the RBA-FE model, outputting a BDI score, carrying out depression degree prediction classification according to the BDI score, adjusting parameters of each network layer in the RBA-FE model according to the difference between prediction classification and actual classification, and obtaining an optimal RBA-FE model to carry out depression degree detection. According to the method, the cell selectivity in the auditory cortex of the human brain is simulated, the problems of overfitting and non-noise resistance of the LSTM are solved by adopting a self-adaptive activation mechanism, and precise recognition of the depression voice under different noise interferences is realized.

The invention provides a voice recognition method, system and device based on artificial intelligence and a medium. The method comprises the steps of obtaining a to-be-recognized voice signal; performing keyword extraction on an initial voice text corresponding to the voice signal based on a pitch period determined by an amplitude deviation between adjacent voice amplitudes in the voice signal and a voice spectrogram of the voice signal to obtain a plurality of voice keywords; determining a homonym group of each voice keyword, and performing recognition verification on each voice keyword based on context semantic information of each voice keyword in the initial voice text and semantic features of homonyms in each homonym group to obtain recognition ambiguity of each voice keyword; and recognizing text probability distribution of the voice signal according to the recognition ambiguity and the homonym group corresponding to each voice keyword, and analyzing the text probability distribution to obtain a text recognition result of the voice signal. According to the scheme, based on the text probability distribution, the speech recognition accuracy can be improved under homonym confusion.

The invention relates to a voice signal processing method and device, electronic equipment and a readable storage medium. Under the condition that a part of sound signals of a user object are collected, corresponding intermediate feature representation is extracted from the part of sound signals, supplementary sound features corresponding to the user object are determined, synthesis processing is performed on the intermediate feature representation and the supplementary sound features, and a complete speech spectrum corresponding to the part of sound signals is obtained. And carrying out acoustic code conversion on the complete speech spectrum to obtain a converted audio signal. Since the intermediate feature representation is text content extracted from a part of sound signals, and the supplementary sound features describe the sound characteristics and styles of the user object, the intermediate feature representation of the part of sound signals of the user object and the supplementary sound features of the user object are synthesized, so that the sound characteristics and styles of the user object are obtained. Therefore, the finally obtained audio signal is not interfered by environmental noise, and the quality of the audio signal can be ensured.

The invention discloses a speech spectrum reconstruction method and system combining time-domain half-wave rectification and a weighted Gaussian mixture model decoder, a terminal and a medium, and relates to the technical field of speech signal processing, the method comprises the following steps: acquiring a low-resolution audio, executing time-domain half-wave rectification, and executing weighted Gaussian mixture model decoding; obtaining a mixed amplitude spectrum based on the amplitude spectrum of the low-resolution audio and the amplitude spectrum of the rectification signal; outputting GRU features based on an encoder, inputting the GRU features into a weighted Gaussian mixture model decoder, and generating a plurality of groups of Gaussian component parameters through three parallel linear layers and applying three constraint designs; and calculating a frame-level frequency distribution weight, combining the frame-level frequency distribution weight with the mixed amplitude spectrum to obtain a model output amplitude spectrum, inputting the model output amplitude spectrum and the amplitude spectrum of the low-resolution audio into a frequency band guide masking module to obtain a final amplitude spectrum, and reconstructing an audio time domain waveform based on the final amplitude spectrum and the phase of the low-resolution audio. According to the method, high-fidelity and high-frequency reconstruction can be realized while the complexity of the model is greatly reduced.

The invention discloses an artificial intelligence psychological assessment method and equipment based on multiple modes, particularly relates to the technical field of psychological assessment analysis, and aims to solve the problem of high detection error rate caused by biological characteristic distortion due to privacy desensitization in existing psychological assessment. The method comprises the following steps of: dynamically generating an evaluation instruction matched with a user behavior, collecting a facial micro-expression video, segmenting and quantifying a privacy interference level based on region pixels, and fuzzifying a non-sensitive region while keeping key micro-expression region characteristics; synchronously acquiring a voice spectrum waveform and reconstructing voiceprint tone modification and fundamental frequency waveform; extracting text sentiment keywords to construct a cross-stage sentiment association chain to repair semantic fracture caused by local desensitization; and according to the privacy interference level, dynamically adjusting the fusion weight of the video and the voice to generate a stage evaluation parameter and a time sequence label, and finally constructing a psychological state evolution graph and outputting an evaluation report, so that the emotional feature fidelity and the psychological state analysis precision are remarkably improved.

The invention discloses a Bluetooth earphone call intelligent noise reduction method based on cloud cooperation, and the method comprises the following steps: S1, obtaining a call voice signal and an environment state parameter, carrying out the preprocessing, and constructing a voice spectrum sequence and an environment feature sequence; s2, performing noise estimation and suppression processing on the speech spectrum sequence by using a DCCRN model to obtain a spectrum feature sequence; s3, constructing a cooperative processing sequence based on the environment feature sequence and the spectrum feature sequence; s4, using a Conv-TasNet model to execute speech enhancement on the co-processing sequence, and generating a target spectrum sequence; s5, performing phase reconstruction and inverse frequency spectrum transformation based on the target frequency spectrum sequence to generate a target voice signal; and S6, constructing a loss function according to the target voice signal, and updating the Conv-TasNet model. According to the method, the DCCRN model and the like are fused, and the method has the advantages of high noise reduction precision, high environment adaptability and sustainable optimization.

The invention discloses a radio analog telephone communication encryption method, device and equipment and a medium, and the method comprises the steps: generating a frequency spectrum segment number and a pseudo-random sorting sequence through a preset encryption algorithm; and carrying out controllable segmentation on the total spectrum of the analog voice according to a continuous sequence from low frequency to high frequency, carrying out sequence recombination on each sub-spectrum according to a pseudo-random sequence, finally combining and outputting encrypted voice, and recovering the original voice by utilizing the same pseudo-random sequence reverse recombination at a receiving end. Therefore, the time-frequency continuity and statistical characteristics of the voice spectrum in the traditional radio analog communication are fundamentally changed. According to the method, frequency spectrums are recombined as a whole according to a pseudo-random rule, so that an eavesdropper can only obtain disordered voice signals after the frequency spectrums are rearranged even if the eavesdropper intercepts the signals, and the difficulty of passive cracking means such as frequency spectrum analysis, filtering reduction and voice recognition is remarkably increased; the problem that in the prior art, radio analog telephone communication is insufficient in encryption and easy to crack is solved.

This invention discloses an online speech separation method and apparatus based on the sparsity constraints of the speech spectrum domain, belonging to the field of information processing. The method includes the following steps: S1: Perform a short-time Fourier transform on the time-domain signal of each microphone to obtain a time-frequency domain representation, and construct the original frequency domain vector; S2: Construct an initialization separation matrix of dimension 2×M according to the direction of the target speech relative to the microphone array; S3: Construct an optimization function and calculate the separation matrix used to separate the target speech from the remaining signal; S4: Obtain the frequency domain signal of the target speech based on the calculated separation matrix, and then obtain the time domain signal of the target speech. This invention, based on the sparsity characteristics of speech signals in the spectrum domain, designs a new optimization function that does not require pre-whitening processing, ensuring the real-time performance of conference communication.

The invention discloses a rhythm-based voice conversion method and device, equipment and a medium. The method comprises the following steps: receiving a source voice input by a user and a selected target voice, and respectively extracting frame-level rhythm features from the source voice and the target voice through a preset feature extraction method; performing feature processing on the source voice, the target voice and the corresponding rhythm features through an encoder and a converter to obtain to-be-input features; inputting the to-be-input features adjusted by the user into a preset decoder model to obtain a speech spectrogram; and synthesizing the voice spectrogram into a target voice waveform through a vocoder, and obtaining converted source voice according to the target voice waveform. By implementing the method provided by the invention, the problem that the frame-level rhythm conversion cannot be effectively performed on the voice in the prior art can be solved, and the flexibility and the efficiency of the financial industry, especially the insurance customer service, are improved.

The present invention provides an emotion-controllable joint encoding VITS speech synthesis method and related devices, which relate to the field of speech synthesis technology. The method extracts emotion features from input speech samples and generates an emotion feature vector; constructs a set of emotion category relative ranking functions to generate a relative attribute vector; concatenates the emotion feature vector with the relative attribute vector to obtain a fused emotion feature representation; performs weighted concatenation of the text feature vector and the fused emotion feature representation to generate a joint feature vector; performs joint encoding on the joint feature vector; converts the jointly encoded features into a speech spectrum, and dynamically controls the emotional expression of the synthesized speech by adjusting the weight ratio of each emotion category in the relative attribute vector, thereby achieving a technical effect of effectively, conveniently, and flexibly controlling the emotion of the synthesized speech.

A low-power communication method and apparatus for a wireless audio device, an SoC chip, and a storage medium. The low-power communication method for a wireless audio device comprises: sampling acoustic features of heterogeneous audio data from multiple target audio devices, calculating speech spectra corresponding to device characteristics, extracting multi-device acoustic features, and performing multi-frequency sub-band speech enhancement, heterogeneous segmentation and grouping of the multiple target audio devices, check value calculation, and differentiated encoding on each extracted acoustic feature sequence to obtain a multi-source encoded speech segment; dividing the multi-source encoded speech segment into transmission units, and performing cooperative signal modulation, cooperative resource allocation, and data stream transmission on each transmission unit to obtain a transmitted speech data stream; and acquiring the transmitted speech data stream transmitted with low power, and performing audio recovery on the transmitted speech data stream to obtain a target audio data stream transmitted by the multiple target audio devices, thereby reducing the communication power consumption of the multiple target audio devices and ensuring the audio transmission quality.

The present invention discloses an audio feature extraction method based on a brain neural model, comprising the following steps: obtaining speech signal data, preprocessing the speech signal, and dividing it into N speech segments; extracting acoustic features representing the emotional state from each speech segment, normalizing the acoustic features and synthesizing a multi-channel speech spectrogram; inputting the multi-channel speech spectrogram into an RBA-FE model, outputting a BDI score, predicting and classifying the degree of depression based on the BDI score, adjusting the parameters of each network layer in the RBA-FE model based on the difference between the predicted classification and the actual classification, and obtaining an optimal RBA-FE model for depression detection. The present invention simulates the cell selectivity in the auditory cortex of the human brain and adopts an adaptive activation mechanism to solve the overfitting and noise resistance of LSTM, thereby achieving accurate recognition of depressive speech under different noise interferences.

The application relates to a speech synthesis model training method, a speech synthesis method, equipment and products. The speech synthesis model training method comprises the following steps: obtaining training speech spectrum information corresponding to a training speech sample of a training speech object; inputting the training speech spectrum information into a first coding module in a speech synthesis model to be trained, obtaining a first coding vector corresponding to the training speech spectrum information through the first coding module, and determining training vector distribution parameters corresponding to the first coding vector; obtaining a training object timbre feature corresponding to the training speech object based on the training vector distribution parameters, obtaining first synthesized speech according to the training object timbre feature and training text information corresponding to the training speech sample; obtaining a first model loss value according to the difference between the first synthesized speech and the training speech sample; and adjusting model parameters according to the first model loss value to obtain a pre-trained speech synthesis model, which can effectively improve the similarity between the timbre of the synthesized speech and the timbre of the speaker.

The invention discloses a voice enhancement method and system based on cross-modal knowledge migration. The speech enhancement method adopted by the invention comprises the following steps: inputting a text corresponding to an audio into a large language model to obtain context-related language embedding features; performing cross-modal knowledge migration, and fusing the language embedding features into the voice embedding features to obtain cross-modal embedding features; training a speech enhancement model; calculating loss between an enhanced speech spectrum and a target speech spectrum by using a mean absolute error loss function; using a cosine similarity loss function to calculate the loss between the cross-modal embedded feature and the output of the large language model; integrating the two losses, and optimizing parameters of the speech enhancement model through a gradient descent algorithm; and in the reasoning stage, voice enhancement processing is carried out only by using the trained voice enhancement model. According to the method, the enhancement effect of the voice in the noisy environment can be effectively improved, and text data or a language model does not need to participate in the reasoning stage.

The invention relates to the technical field of information security, and discloses a multi-point recording interference method and system, and the method comprises the steps: obtaining environment information, and determining a basic interference volume according to the environment information; acquiring a sound source-node distance vector and sound field uniformity, and converting the sound source-node distance vector and the sound field uniformity into a spatial distribution coefficient; acquiring a voice initial instantaneous slope and an interference-voice cross-correlation function peak value, and converting the slope and the peak value into a dynamic response coefficient; receiving a target voice, obtaining the spectral characteristics and the voice spectrum centroid of the target voice, and converting the spectral characteristics and the voice spectrum centroid into a spectral efficiency coefficient; reading the basic interference volume, the spatial distribution coefficient, the dynamic response coefficient and the spectral efficiency coefficient, and generating an adjustment instruction; according to the invention, through collaborative analysis of the environment reference coefficient, the spatial distribution coefficient, the dynamic response coefficient and the spectrum efficiency coefficient, multi-dimensional adaptive matching of the interference volume and the environment state, the spatial layout, the voice dynamic characteristic and the spectrum characteristic is realized, and the flexibility is extremely high.