Knowledge question-answering method and system based on artificial intelligence technology and electronic equipment

Abstract

The invention relates to the field of knowledge questioning and answering, and particularly discloses a knowledge questioning and answering method and system based on the artificial intelligence technology and electronic device.Firstly, to-be-recognized audio collected by a sound sensor and an oscillogram extracted from the to-be-recognized audio collected by the sound sensor are obtained; and then feature extraction and correlation analysis are performed on the two, and finally a classification result is obtained through a classifier to judge whether the to-be-identified audio is the human voice audio needing knowledge questioning and answering, so that it is ensured that the system only processes the human voice audio needing knowledge questioning and answering, and the performance and user experience of the knowledge questioning and answering system are improved.

Description

Technical Field

[0001] The present application relates to the field of knowledge question answering, and more specifically, to a knowledge question answering method, system and electronic device based on artificial intelligence technology. Background Art

[0002] Knowledge quizzes can use technologies such as machine learning and speech recognition to enable computers to understand and answer questions asked through voice. By training models, computers can convert voice input into text, and use natural language processing technology to extract information from a large amount of knowledge base and data, match and reason with questions, and finally generate accurate answers. This technology has a wide range of applications and can be used in smart speakers, voice assistants, smart customer service and other fields, providing users with a convenient and natural voice interaction method, realizing the intelligence of voice input and output, and improving the convenience and efficiency of users in acquiring knowledge and solving problems.

[0003] However, in current knowledge question-answering technology, when there is noise in the voice input, the voice recognition system may be inaccurate. The noise may include environmental noise, background music, other people's voices, etc. These noises may make it difficult for the voice recognition system to correctly recognize and transcribe the user's voice input, thereby affecting the understanding of the question and the generation of the answer.

[0004] Therefore, a knowledge question answering method, system and electronic device based on artificial intelligence technology are desired. Summary of the invention

[0005] In order to solve the above technical problems, this application is proposed. The embodiments of this application provide a knowledge question answering method, system and electronic device based on artificial intelligence technology, which first obtains the audio to be identified collected by the sound sensor and the waveform extracted from the audio to be identified collected by the sound sensor, and then performs feature extraction and correlation analysis on the two, and finally obtains the classification result through the classifier to determine whether the audio to be identified is human voice audio that needs to be answered for knowledge, thereby ensuring that the system only processes human voice audio that needs to be answered for knowledge, thereby improving the performance and user experience of the knowledge question answering system.

[0006] According to one aspect of the present application, a knowledge question answering method based on artificial intelligence technology is provided, which includes:

[0007] Acquire the audio to be recognized collected by the sound sensor and a waveform extracted from the audio to be recognized collected by the sound sensor;

[0008] Extracting a knowledge question and answer audio-related feature vector and an audio time domain-audio frequency domain-related feature vector from the audio to be recognized collected by the sound sensor and the waveform graph extracted from the audio to be recognized collected by the sound sensor;

[0009] Performing hidden perturbation optimization based on data reconstruction mapping on the knowledge question and answer audio associated feature vector and the audio time domain-audio frequency domain associated feature vector to obtain a knowledge question and answer audio classification feature vector;

[0010] The knowledge question and answer audio classification feature vector is passed through a classifier to obtain a classification result, and the classification result is used to determine whether the audio to be identified is human voice audio that requires knowledge question and answer.

[0011] According to another aspect of the present application, a knowledge question answering system based on artificial intelligence technology is provided, which includes:

[0012] A knowledge question and answer data acquisition module, used to acquire the audio to be recognized collected by the sound sensor and a waveform extracted from the audio to be recognized collected by the sound sensor;

[0013] A knowledge question and answer data feature extraction module, used to extract a knowledge question and answer audio-related feature vector and an audio time domain-audio frequency domain-related feature vector from the audio to be recognized collected by the sound sensor and the waveform extracted from the audio to be recognized collected by the sound sensor;

[0014] A target domain joint correlation migration module is used to perform hidden perturbation optimization based on data reconstruction mapping on the knowledge question and answer audio correlation feature vector and the audio time domain-audio frequency domain correlation feature vector to obtain a knowledge question and answer audio classification feature vector;

[0015] The human voice audio judgment generation module is used to pass the knowledge question and answer audio classification feature vector through a classifier to obtain a classification result, and the classification result is used to determine whether the audio to be identified is human voice audio that requires knowledge question and answer.

[0016] According to another aspect of the present application, an electronic device is provided, comprising: a processor; and a memory, wherein computer program instructions are stored in the memory, and when the computer program instructions are executed by the processor, the processor executes the knowledge question and answer method based on artificial intelligence technology as described above.

[0017] Compared with the prior art, the present application provides a knowledge question and answer method, system and electronic device based on artificial intelligence technology, which first obtains the audio to be identified collected by a sound sensor and a waveform graph extracted from the audio to be identified collected by the sound sensor, then performs feature extraction and correlation analysis on the two, and finally obtains the classification result through a classifier to determine whether the audio to be identified is human voice audio that requires knowledge question and answer, thereby ensuring that the system only processes human voice audio that requires knowledge question and answer, thereby improving the performance and user experience of the knowledge question and answer system. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0019] Figure 1 The present invention is a flowchart of a knowledge question answering method based on artificial intelligence technology according to an embodiment of the present application.

[0020] Figure 2 The present invention is a flowchart of extracting text features from the audio to be recognized collected by the sound sensor in the knowledge question answering method based on artificial intelligence technology according to an embodiment of the present application.

[0021] Figure 3 The present invention is a flowchart of extracting spectral features of the audio to be recognized collected by the sound sensor in the knowledge question answering method based on artificial intelligence technology according to an embodiment of the present application.

[0022] Figure 4 The present invention is a flowchart of data preprocessing of the waveform graph extracted from the audio to be recognized collected by the sound sensor in the knowledge question answering method based on artificial intelligence technology according to an embodiment of the present application.

[0023] Figure 5 It is a block diagram of a knowledge question answering system based on artificial intelligence technology according to an embodiment of the present application.

[0024] Figure 6 is a block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0025] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0026] Figure 1 Flow chart of the knowledge question answering method based on artificial intelligence technology according to an embodiment of the present application. Figure 1As shown, according to the knowledge question and answer method based on artificial intelligence technology in an embodiment of the present application, it includes: S110, obtaining the audio to be recognized collected by the sound sensor and the waveform extracted from the audio to be recognized collected by the sound sensor; S120, extracting the knowledge question and answer audio related feature vector and the audio time domain-audio frequency domain related feature vector from the audio to be recognized collected by the sound sensor and the waveform extracted from the audio to be recognized collected by the sound sensor; S130, performing hidden perturbation optimization based on data reconstruction mapping on the knowledge question and answer audio related feature vector and the audio time domain-audio frequency domain related feature vector to obtain a knowledge question and answer audio classification feature vector; S140, passing the knowledge question and answer audio classification feature vector through a classifier to obtain a classification result, and the classification result is used to determine whether the audio to be recognized is human voice audio that requires knowledge question and answer.

[0027] In the above-mentioned knowledge question and answer method based on artificial intelligence technology, the step S110 obtains the audio to be recognized collected by the sound sensor and the waveform extracted from the audio to be recognized collected by the sound sensor. It should be understood that the audio to be recognized collected by the sound sensor is the original sound signal, which contains the voice content to be recognized. These audios can be used to determine whether the audio to be recognized is a human voice audio containing knowledge questions and answers. The waveform extracted from the audio to be recognized is a visual representation of the audio signal, which shows the amplitude of the sound signal changing over time. The waveform can provide the time domain characteristics of the audio signal, including information such as the intensity, frequency and duration of the sound. By analyzing and processing these data, the voice content can be recognized and understood, so as to perform corresponding questions and answers or interactive operations.

[0028] In the above-mentioned knowledge question and answer method based on artificial intelligence technology, the step S120 extracts the knowledge question and answer audio-related feature vector and the audio time domain-audio frequency domain-related feature vector from the audio to be identified collected by the sound sensor and the waveform extracted from the audio to be identified collected by the sound sensor. It should be understood that the knowledge question and answer audio-related feature vector is a feature vector extracted from the audio to be identified, which includes text features and spectral features. The text feature can convert the audio into text data through a speech processing encoder. The spectral feature is based on the spectral information of the audio, such as Mel spectrum, cochlear spectrogram and constant Q transform spectrum, so as to represent the frequency domain features of the audio. The audio time domain-audio frequency domain-related feature vector is a feature vector obtained by correlating and extracting the time domain features and frequency domain features of the audio. The time domain features include information such as the amplitude, energy and time domain waveform of the audio signal, and the frequency domain features represent the distribution of the audio signal in the frequency domain. By correlating and extracting the time domain features and the frequency domain features, the feature information of the audio can be described more comprehensively. Extracting knowledge question-answering audio-related feature vectors and audio time domain-audio frequency domain-related feature vectors can help the system better understand and analyze audio content, thereby improving the accuracy and effectiveness of knowledge question-answering.

[0029] In a specific embodiment of the present application, the step S120 includes: extracting text features from the audio to be recognized collected by the sound sensor to obtain a knowledge question and answer associated feature vector; extracting spectral features from the audio to be recognized collected by the sound sensor to obtain a knowledge question and answer spectral feature vector; associating the knowledge question and answer associated feature vector with the knowledge question and answer spectral feature vector to obtain the knowledge question and answer audio associated feature vector; extracting features from the waveform graph extracted from the audio to be recognized collected by the sound sensor to obtain the audio time domain-audio frequency domain associated feature vector. It should be understood that the purpose of extracting text features from the audio to be recognized collected by the sound sensor is to convert the speech content in the audio into a processable text form. Knowledge question and answer tasks usually need to be matched and queried with information in the knowledge base. After converting the audio into text, it can be more convenient to match it with the text data in the knowledge base, thereby achieving more accurate question and answer results. In addition, by converting the audio into text, natural language processing technology can be applied to semantically understand and reason about the text. This helps to understand the speech content more deeply, extract the intent and context information of the question, and thus provide more accurate and intelligent answers to knowledge questions. Among them, the speech signal in the audio is converted into corresponding text data by using text feature extraction, which can adopt methods such as speech recognition technology or speech processing encoder.

[0030] Furthermore, in order to describe the characteristics of the audio from the perspective of the frequency domain, the spectral features of the audio to be identified collected by the sound sensor are extracted. It should be understood that the spectral features may include features such as Mel-spectrogram, cochlear spectrogram, constant Q transform spectrogram, etc. Spectral feature extraction is the process of converting the audio signal from the time domain to the frequency domain by performing Fourier transform or other spectral analysis methods on the audio signal to obtain the spectral information of the audio. In this way, the frequency domain characteristics of the audio can be captured, thereby obtaining information on the speech content, pitch, timbre, etc. in the audio. Then, by combining the spectral features and other features, the system can more accurately understand and analyze the audio content, and provide help to improve the accuracy and effectiveness of knowledge questions and answers.

[0031] Furthermore, in order to obtain a more comprehensive and accurate knowledge question and answer audio-related feature vector, the knowledge question and answer related feature vector and the knowledge question and answer spectral feature vector are associated. The knowledge question and answer related feature vector is obtained by text feature extraction, and it contains the semantic information of the text content converted in the audio. The knowledge question and answer spectral feature vector is obtained by spectral feature extraction, and it contains the feature information of the audio in the frequency domain. By associating the knowledge question and answer related feature vector and the knowledge question and answer spectral feature vector, the different information they provide can be comprehensively utilized. For example, the text features and spectral features can be fused to improve the question and answer system's ability to understand and express the audio content. At the same time, the associated feature vector can provide more feature dimensions and diversity, thereby enhancing the accuracy and robustness of the question and answer system.

[0032] In particular, by performing feature extraction on the waveform graph extracted from the audio to be identified collected by the sound sensor, an audio time domain-audio frequency domain correlation feature vector can be obtained. It should be understood that the audio waveform graph describes the change of the audio signal over time, which reflects the amplitude and waveform shape of the audio. By performing feature extraction on the waveform graph, the time domain features and frequency domain features of the audio can be extracted to analyze and understand the audio at different levels and provide more audio information.

[0033] Figure 2 The flowchart is a method for extracting text features from the audio to be recognized collected by the sound sensor in the knowledge question answering method based on artificial intelligence technology according to an embodiment of the present application. Figure 2As shown, in a specific embodiment of the present application, the audio to be recognized collected by the sound sensor is subjected to text feature extraction to obtain a knowledge question and answer associated feature vector, including: S210, passing the audio to be recognized collected by the sound sensor through a speech processing encoder to obtain knowledge question and answer text data; S220, performing word segmentation processing on the knowledge question and answer text data to obtain the multiple knowledge question and answer text input data; S230, passing the multiple knowledge question and answer text input data through a knowledge question and answer context encoder based on the Clip model to obtain multiple knowledge question and answer feature vectors; S240, cascading the multiple knowledge question and answer feature vectors to obtain the knowledge question and answer associated feature vector. It should be understood that the speech processing encoder is a technology for converting audio signals into text data. It can convert the speech content in the audio into a corresponding text representation, so that the content of the audio can be processed and analyzed in text form. By converting the audio to be recognized into knowledge question and answer text data, multi-language processing and multi-modal processing can be more easily realized, that is, the text data can be fused with other modal data (such as images and videos) while performing operations such as language translation and cross-language retrieval. This helps the knowledge question answering system to better understand and answer questions in the audio. Specifically, first, features are extracted from the audio to be identified collected by the sound sensor using short-time Fourier transform or Mel spectrum coefficients, and these features represent the distribution of the audio in time and frequency. Then, these feature representations are converted into text data, that is, the text data is converted into a fixed-length vector sequence using an embedding layer, that is, the knowledge question answering text data.

[0034] Furthermore, word segmentation processing can be performed on the knowledge question and answer text data to split the text into multiple words or phrases, thereby obtaining multiple knowledge question and answer text input data. Word segmentation is a basic task in natural language processing, which divides a continuous text sequence into discrete words or phrases. By performing word segmentation processing on the knowledge question and answer text data, the text can be converted into smaller semantic units, and features such as word frequency, part of speech and other information can be extracted for each word or phrase to understand the meaning and semantic structure of the text, thereby providing a more fine-grained text representation, thereby improving the understanding and expression of the text. Specifically, a word segmenter (such as NLTK or spaCy) is used to split the text data into a sequence of words or subwords to obtain multiple knowledge question and answer text input data.

[0035] Furthermore, multiple knowledge question and answer text input data are passed through a knowledge question and answer context encoder based on the CLIP model to capture the semantic and contextual information of the text. The CLIP model can map text and images to the same feature space to achieve cross-modal semantic understanding. Specifically, by inputting multiple knowledge question and answer text input data into the knowledge question and answer context encoder of the CLIP model, the text and image are mapped to the same feature space, thereby achieving cross-modal semantic understanding, so as to obtain a feature vector representation of each text input. These feature vectors can capture the semantic information and contextual relationships of the text, and reflect the similarities and differences between texts, that is, similar text inputs are closer in the feature space, and dissimilar text inputs are farther away. This allows feature vectors to be used for tasks such as text classification and similarity matching. More specifically, the embedding layer of the Clip model-based knowledge question and answer context encoder is used to map each of the multiple knowledge question and answer text input data into a knowledge question and answer text embedding vector to obtain a sequence of knowledge question and answer text embedding vectors; and the Clip model of the Clip model-based knowledge question and answer context encoder is used to perform global context semantic encoding on the sequence of knowledge question and answer text embedding vectors to obtain multiple knowledge question and answer feature vectors.

[0036] In particular, cascading multiple feature vectors can increase the dimension and expressiveness of the features, thereby better representing and distinguishing different knowledge question and answer texts. Specifically, the knowledge question and answer feature vector is a vector representation obtained by encoding and representing the knowledge question and answer text. Each knowledge question and answer text may contain different information and semantics. By cascading multiple knowledge question and answer feature vectors, these different information and semantics can be integrated and fused.

[0037] Figure 3 FIG. 1 is a flow chart of extracting the spectral features of the audio to be recognized collected by the sound sensor in the knowledge question answering method based on artificial intelligence technology according to an embodiment of the present application. Figure 3As shown, in a specific embodiment of the present application, the audio to be recognized collected by the sound sensor is subjected to spectral feature extraction to obtain a knowledge question and answer spectral feature vector, including: S310, extracting multiple spectrograms including a logarithmic Mel spectrogram, a cochlear spectrogram, and a constant Q transform spectrogram from the audio to be recognized collected by the sound sensor; S320, arranging the multiple spectrograms including the logarithmic Mel spectrogram, the cochlear spectrogram, and the constant Q transform spectrogram to obtain a knowledge question and answer audio multi-channel input spectrogram; S330, passing the knowledge question and answer audio multi-channel input spectrogram through a knowledge question and answer spectral feature extractor using a channel attention mechanism to obtain the knowledge question and answer spectral feature vector. It should be understood that extracting multiple spectrograms including a logarithmic Mel spectrogram, a cochlear spectrogram, and a constant Q transform spectrogram from the audio to be recognized collected by the sound sensor can extract different feature representations of the audio so as to better capture the voice information of the audio in audio recognition and processing tasks. Specifically, the log-mel spectrogram is a commonly used spectrogram representation method, which converts the audio signal into a spectrogram and performs a logarithmic transformation on the spectrogram. The log-mel spectrogram is widely used in speech recognition tasks because it can capture the frequency and time information of the audio and is more consistent with human auditory perception. The cochlear spectrogram is a spectrogram representation method based on the human ear auditory perception model. It simulates the cochlear structure in the human ear and converts the audio signal into a spectrogram. The cochlear spectrogram can better capture the auditory characteristics of the audio and has certain advantages for some audio processing tasks (such as speech recognition and speech enhancement). The constant Q transform spectrogram is a spectrogram representation method based on the constant Q transform. The constant Q transform is a method of converting an audio signal into a spectrogram with a fixed Q value in frequency. The constant Q transform spectrogram has better adaptability to different frequency resolutions in the audio and can better represent the sound features in different frequency ranges. Extracting these different spectrogram features can provide multiple audio representations, covering different aspects and characteristics of the audio, thereby providing more information and enhancing the model's understanding and differentiation of the audio.

[0038] Furthermore, each spectrogram channel represents different feature information, such as a different frequency range or a different sound feature. By arranging these feature information, they can be combined into a multi-channel input, thereby providing a richer and more comprehensive audio feature representation, thereby better representing and distinguishing different audio signals.

[0039] Furthermore, by inputting the multi-channel spectrogram of the knowledge question and answer audio into the knowledge question and answer spectrogram feature extractor using the channel attention mechanism, a more informative and important knowledge question and answer spectrogram feature vector can be obtained. The channel attention mechanism is a variant of the attention mechanism, which is used to adaptively learn the importance weight of each channel in multi-channel input. By using the channel attention mechanism in the knowledge question and answer spectrogram feature extractor, the weight of the channel can be automatically adjusted according to the contribution of each channel, so that important feature channels get larger weights and unimportant feature channels get smaller weights. By using the channel attention mechanism, the weight of each channel can be adaptively learned, thereby highlighting important feature channels so that they get more attention and weight in the feature extraction process. This helps to improve the distinguishing ability and expressiveness of the features, making the final knowledge question and answer spectrogram feature vector more representative and informative. Specifically, each layer of the knowledge question and answer spectrogram feature extractor using the channel attention mechanism performs the following on the knowledge question and answer audio multi-channel input spectrogram in the forward pass of the layer: convolution processing is performed on the knowledge question and answer audio multi-channel input spectrogram to generate a spectrogram convolution feature map; pooling processing is performed on the spectrogram convolution feature map to generate a spectrogram pooling feature map; activation processing is performed on the spectrogram pooling feature map to generate a spectrogram activation feature map; the quotient of the eigenvalue mean of the feature matrix corresponding to each channel in the spectrogram activation feature map and the sum of the eigenvalue mean of the feature matrix corresponding to all channels is calculated as the weighting coefficient of the feature matrix corresponding to each channel; and the feature matrix of each channel is weighted by the weighting coefficient of each channel in the spectrogram activation feature map to generate a spectrogram channel attention feature map; wherein the output of the last layer of the knowledge question and answer spectrogram feature extractor using the channel attention mechanism is the knowledge question and answer spectrogram feature vector.

[0040] In a specific embodiment of the present application, the waveform extracted from the audio to be identified collected by the sound sensor is subjected to feature extraction to obtain the audio time domain-audio frequency domain correlation feature vector, including: performing data preprocessing on the waveform extracted from the audio to be identified collected by the sound sensor to obtain an audio time domain feature input vector and an audio frequency domain feature input vector; calculating the association matrix of the audio time domain feature input vector and the audio frequency domain feature input vector to obtain an audio time domain-audio frequency domain association matrix; passing the audio time domain-audio frequency domain association matrix through an audio time domain-audio frequency domain feature extractor based on a convolutional neural network to obtain the audio time domain-audio frequency domain correlation feature vector. It should be understood that the time domain feature of audio refers to the feature of analyzing and representing the audio signal in the time domain. Common audio time domain features include audio waveforms, energy, zero crossing rate, etc. By performing data preprocessing on the waveform of audio, the amplitude and change information of the waveform can be extracted, thereby obtaining the time domain feature input vector of audio. The time domain feature can reflect the time change and waveform shape of the audio signal. The frequency domain features of audio refer to the features of analyzing and representing audio signals in the frequency domain. Common audio frequency domain features include audio spectrograms, power spectra, etc. By preprocessing the frequency domain representation of audio, the energy distribution and spectral characteristics of audio at different frequencies can be extracted to obtain the frequency domain feature input vector of audio. Frequency domain features can reflect the frequency components and spectral characteristics of audio signals. By extracting the time domain features and frequency domain features of audio and converting them into input vector form, different aspects and characteristics of audio can be better represented and captured.

[0041] Furthermore, audio time domain features and audio frequency domain features usually come from the same audio signal, but they represent different feature representations. By calculating the correlation matrix between them, we can explore their mutual relationships and correlations, such as determining which time domain features are highly correlated with frequency domain features, so as to select the most informative features for subsequent audio processing or recognition tasks. Specifically, time domain features and frequency domain features can be associated using linear weighting or nonlinear fusion.

[0042] Furthermore, the audio time domain-audio frequency domain correlation matrix can be passed through an audio time domain-audio frequency domain feature extractor based on a convolutional neural network to obtain an audio time domain-audio frequency domain correlation feature vector. The feature extractor based on the convolutional neural network can be constructed by components such as a convolution layer, a pooling layer, and a fully connected layer to extract audio time domain-audio frequency domain correlation features. These features can be abstract high-level features that can better represent the structure, semantics, and correlation of the audio. Specifically, each layer of the audio time domain-audio frequency domain feature extractor of the convolutional neural network performs convolution processing, pooling processing, and nonlinear activation processing on the input data in the forward pass of the layer to output the audio time domain-audio frequency domain correlation feature vector by the last layer of the audio time domain-audio frequency domain feature extractor of the convolutional neural network, wherein the input of the audio time domain-audio frequency domain feature extractor of the convolutional neural network is the audio time domain-audio frequency domain correlation matrix.

[0043] Figure 4 The flowchart is a data preprocessing method for the knowledge question answering method based on artificial intelligence technology according to an embodiment of the present application for extracting the waveform graph from the audio to be recognized collected by the sound sensor. Figure 4 As shown, in a specific embodiment of the present application, the waveform extracted from the audio to be identified collected by the sound sensor is subjected to data preprocessing to obtain an audio time domain feature input vector and an audio frequency domain feature input vector, including: S410, the waveform extracted from the audio to be identified collected by the sound sensor is intercepted along a time sequence into multiple sampling windows to obtain multiple audio waveform sampling windows to be identified; S420, time domain features and frequency domain features are extracted from the multiple audio waveform sampling windows to be identified to obtain multiple audio time domain features to be identified and multiple audio frequency domain features to be identified; S430, the multiple audio time domain features to be identified and the multiple audio frequency domain features to be identified are arranged to obtain the audio time domain feature input vector and the audio frequency domain feature input vector. It should be understood that an audio signal is usually a continuous time series, and in practical applications, the audio usually needs to be segmented or framed. By dividing the audio into multiple sampling windows, the audio signal can be decomposed into a series of shorter time segments, each window representing a local area of ​​the audio. In this way, the dynamic changes of audio signals can be modeled in time to better understand the evolution process and contextual information of the audio and preserve the temporal relationship of the audio signal. Specifically, an appropriate sampling window size and overlap rate can be selected to balance the requirements of time resolution and frequency resolution, so that a series of audio sampling windows with temporal continuity can be obtained to obtain local features of the audio and capture the dynamic changes of the audio signal in time.

[0044] Furthermore, obtaining audio feature representations in the time domain and frequency domain dimensions can more comprehensively describe the characteristics of audio signals, improve the accuracy and robustness of audio recognition and processing tasks, adapt to the needs of different tasks, and provide richer audio feature representations. Furthermore, by arranging the time domain features and frequency domain features of multiple audios to be recognized, the time domain feature input vector and the frequency domain feature input vector of the audio can be obtained to integrate the features of the audio signal into one vector, thereby retaining the time domain information and frequency domain information of each audio.

[0045] In the above-mentioned knowledge question and answer method based on artificial intelligence technology, the step S130 includes: performing hidden perturbation optimization based on data reconstruction mapping on the knowledge question and answer audio related feature vector and the audio time domain-audio frequency domain related feature vector to obtain the knowledge question and answer audio classification feature vector.

[0046] In particular, considering that the knowledge question and answer audio association feature vector and the audio time domain-audio frequency domain association feature vector may be distributed in different feature spaces, the performance degradation when the source domain model is directly applied to the target domain is caused by the fact that the two feature vectors originate from different types of data processing paths and represent different levels of information abstraction. The knowledge question and answer audio association feature vector is obtained by converting the audio to be recognized collected by the sound sensor into text data, and then undergoing natural language processing steps such as word segmentation and context encoding. This process mainly captures the semantic information of the audio content, that is, the meaning of the specific question or statement conveyed by the voice. This feature vector emphasizes the understanding of language structure and content, and is suitable for processing text-related tasks such as knowledge question and answer. On the other hand, the audio time domain-audio frequency domain association feature vector is a time domain and frequency domain feature calculated from waveforms and spectrograms (such as logarithmic Mel spectrograms, cochlear spectrograms, and constant Q transform spectrograms) extracted from the same audio segment. These features reflect more of the physical properties of the sound signal, including pitch, loudness, timbre, etc., and their changing patterns over time. The features further extracted by the convolutional neural network are more focused on capturing patterns and structures in the audio signal, and are suitable for identifying non-semantic characteristics such as identity features and emotional states of speech. Since the above two feature vectors focus on information at the semantic level and acoustic level respectively, even if they are derived from the same original audio data, they will be different in numerical representation, and more importantly, they represent different information dimensions. Therefore, when trying to directly transfer knowledge from one field (for example, a model trained only for semantic information) to another field (that is, a new environment that integrates semantic and acoustic information), it is difficult for the model to directly and effectively transfer its learned knowledge. Based on this, in the technical solution of the present application, the knowledge question and answer audio association feature vector and the audio time domain-audio frequency domain association feature vector are subjected to hidden perturbation optimization based on data reconstruction mapping to obtain the knowledge question and answer audio classification feature vector.

[0047] Specifically, in an embodiment of the present application, the knowledge question and answer audio associated feature vector and the audio time domain-audio frequency domain associated feature vector are subjected to hidden perturbation optimization based on data reconstruction mapping to obtain a knowledge question and answer audio classification feature vector, including: mapping the knowledge question and answer audio associated feature vector and the audio time domain-audio frequency domain associated feature vector to an eigendecomposition space to obtain a knowledge question and answer audio associated eigenfeature vector and an audio time domain-audio frequency domain associated eigenfeature vector; domain-analyzing the knowledge question and answer audio associated eigenfeature vector and the audio time domain-audio frequency domain associated eigenfeature vector The method comprises the following steps: calibrating the features between the two components to obtain the knowledge question and answer audio-related modulated intrinsic feature vector and the audio time domain-audio frequency domain related modulated intrinsic feature vector; constructing an audio time domain-audio frequency domain related fine-grained semantic information field between the knowledge question and answer audio-related modulated intrinsic feature vector and the audio time domain-audio frequency domain related modulated intrinsic feature vector; and based on the audio time domain-audio frequency domain related fine-grained semantic information field, fusing the knowledge question and answer audio-related modulated intrinsic feature vector and the audio time domain-audio frequency domain related modulated intrinsic feature vector to obtain the knowledge question and answer audio classification feature vector.

[0048] More specifically, in an embodiment of the present application, an audio time domain-audio frequency domain associated fine-grained semantic information field is constructed between the audio-associated intrinsic feature vector of the knowledge question and answer and the audio time domain-audio frequency domain associated intrinsic feature vector, including: multiplying the transposed vector of the audio-associated intrinsic feature vector and the audio time domain-audio frequency domain associated intrinsic feature vector, and then dividing by the scale of the audio time domain-audio frequency domain associated intrinsic feature vector to obtain an audio time domain-audio frequency domain associated fine-grained interaction matrix; and performing convolution encoding on the audio time domain-audio frequency domain associated fine-grained matrix to obtain the audio time domain-audio frequency domain associated fine-grained semantic information field.

[0049] More specifically, in an embodiment of the present application, the audio-related knowledge question and answer modulated intrinsic feature vector and the audio-related audio-frequency domain modulated intrinsic feature vector are fused based on the audio-time-frequency domain associated fine-grained semantic information field to obtain the audio-time-frequency domain associated feature vector, including: multiplying the audio-time-frequency domain associated fine-grained semantic information field by the audio-related knowledge question and answer modulated intrinsic feature vector to obtain the knowledge question and answer audio-related feature vector after information field mapping; multiplying the audio-time-frequency domain associated fine-grained semantic information field by the audio-time-frequency domain associated modulated intrinsic feature vector to obtain the audio-time-frequency domain associated feature vector after information field mapping; weighted fusion of the knowledge question and answer audio-related feature vector after information field mapping and the audio-time-frequency domain associated feature vector after information field mapping to obtain the knowledge question and answer audio classification feature vector.

[0050] In an embodiment of the present application, specifically, performing hidden perturbation optimization based on data reconstruction mapping on the knowledge question and answer audio association feature vector and the audio time domain-audio frequency domain association feature vector to obtain the knowledge question and answer audio classification feature vector includes: processing the knowledge question and answer audio association feature vector and the audio time domain-audio frequency domain association feature vector according to the following formula to obtain the knowledge question and answer audio classification feature vector; wherein the formula is:

[0051]

[0052]

[0053] v c =αv kt +βv st

[0054] Among them, V1 represents the knowledge question and answer audio correlation feature vector, V2 represents the audio time domain-audio frequency domain correlation feature vector, PCA(·) represents principal component analysis, U1 represents the sequence of knowledge question and answer audio correlation eigendecomposition vectors, Λ1 represents the knowledge question and answer audio correlation diagonal matrix, λ 11 and λ 1m denote the eigenvalues ​​of the first and mth positions of the knowledge question and answer audio correlation diagonal matrix, T denotes the transpose of the vector, concat{·;·} denotes vector concatenation, and v 11 、v 12 、v 1m represent the first, second, and mth eigenvectors of the sequence of the knowledge question and answer audio-related eigendecomposition vectors, V krepresents the knowledge question and answer audio correlation eigenvector, U2 represents the sequence of audio time domain-audio frequency domain correlation eigendecomposition vectors, Λ2 represents the knowledge question and answer audio correlation diagonal matrix, λ 21 and λ 2m They represent the eigenvalues ​​of the first and mth positions of the audio time domain-audio frequency domain correlation diagonal matrix, respectively. 21 、v 22 、v 2m Respectively represent the first, second and mth eigenvectors of the sequence of audio time domain-audio frequency domain correlation eigendecomposition vectors, V s represents the audio time domain-audio frequency domain correlation eigenvector, v i represents the i-th eigenvalue of the knowledge question and answer audio associated eigenvector, v j represents the jth eigenvalue of the audio time domain-audio frequency domain correlation eigenvector, represents the square of the vector norm, exp represents the natural exponential function, V' k represents the intrinsic feature vector after the knowledge question and answer audio is modulated, V' s Represents the intrinsic eigenvector after audio time domain-audio frequency domain correlation modulation, represents matrix multiplication, S represents the scale of the vector, M x Represents the audio time domain-audio frequency domain correlation fine-grained interaction matrix, Conv 3×3 (M x ) indicates that M x A convolutional coding operation based on a convolution kernel of 3×3 is performed, Ω represents the audio time domain-audio frequency domain associated fine-grained semantic information field, v kt represents the knowledge question and answer audio related feature vector after information field mapping, v st represents the audio time domain-audio frequency domain correlation feature vector after information field mapping, α and β represent weighted hyperparameters, and v c Represents the knowledge question and answer audio classification feature vector.

[0055] That is, in the technical solution of the present application, the knowledge question and answer audio-related feature vector and the audio time-domain-audio frequency domain-related feature vector are subjected to hidden perturbation optimization based on data reconstruction mapping. The process first maps the knowledge question and answer audio-related feature vector and the audio time-domain-audio frequency domain-related feature vector to the intrinsic decomposition space to obtain the knowledge question and answer audio-related intrinsic feature vector and the audio time-domain-audio frequency domain-related intrinsic feature vector. Those of ordinary skill in the art should know that the process of mapping to the intrinsic decomposition space is a combination of dimensionality reduction and feature extraction. In a specific example, the eigenvalue decomposition technique of principal component analysis (PCA) is used to find the intrinsic decomposition space, and the intrinsic decomposition space mapping is performed. The knowledge question and answer audio-related intrinsic feature vector and the audio time-domain-audio frequency domain-related intrinsic feature vector after information field mapping not only represent the projection of the original feature vector in the new coordinate system, but also reflect the main direction of change of the data, which helps to avoid the "dimensionality disaster", that is, the problem that the sample density drops sharply as the number of features increases.

[0056] Next, after mapping the feature vector to the eigendecomposition space, it is necessary to perform inter-domain feature calibration on the knowledge question and answer audio-related eigenfeature vector and the audio time-domain-audio frequency-domain-related eigenfeature vector to adjust the source domain feature vector to better match the data distribution of the target domain. This operation aims to capture and compensate for the systematic deviation between the two domains, so as to reduce the hidden disturbance of the class expression of the local feature distribution of the knowledge question and answer audio-related eigenfeature vector and the audio time-domain-audio frequency-domain-related eigenfeature vector to the overall class expression. After modulation, the knowledge question and answer audio-related modulated eigenfeature vector and the audio time-domain-audio frequency-domain-related modulated eigenfeature vector can retain the source domain characteristics while being more in line with the actual needs of the target domain, thereby enhancing the model's understanding and application of the target domain data.

[0057] After completing the scale modulation of the feature vector, an audio time domain-audio frequency domain associated fine-grained semantic information field is constructed between the knowledge question and answer audio associated modulated intrinsic feature vector and the audio time domain-audio frequency domain associated modulated intrinsic feature vector. The audio time domain-audio frequency domain associated fine-grained semantic information field is used to describe a more detailed relationship between feature vectors. The introduction of the audio time domain-audio frequency domain associated fine-grained semantic information field enables the model to go beyond simple point-to-point comparisons and consider more complex patterns and contextual information, thereby enhancing the model's sensitivity to subtle differences to improve the quality of feature representation. Finally, the knowledge question and answer audio associated modulated intrinsic feature vector and the audio time domain-audio frequency domain associated modulated intrinsic feature vector are fused based on the audio time domain-audio frequency domain associated fine-grained semantic information field to obtain the knowledge question and answer audio classification feature vector.

[0058] In the above-mentioned knowledge question and answer method based on artificial intelligence technology, the step S140 includes: passing the knowledge question and answer audio classification feature vector through a classifier to obtain a classification result, and the classification result is used to determine whether the audio to be identified is a human voice audio that requires knowledge question and answer. It should be understood that the classification result can be obtained by classifying the knowledge question and answer audio classification feature vector through a classifier, and this result is used to determine whether the audio to be identified is a human voice audio that requires knowledge question and answer. The classifier can learn the patterns and features in the knowledge question and answer audio classification feature vector, so as to classify the audio to be identified, that is, to divide it into the category of knowledge question and answer audio or non-knowledge question and answer audio. By judging whether the audio to be identified is a human voice audio that requires knowledge question and answer, automatic classification and filtering of the audio can be achieved. This is very important for the knowledge question and answer system because it can help the system automatically select audio suitable for question and answer processing, thereby improving the efficiency and accuracy of the system.

[0059] In summary, the embodiment of the present application first obtains the audio to be identified collected by the sound sensor and the waveform extracted from the audio to be identified collected by the sound sensor, then performs feature extraction and correlation analysis on the two, and finally obtains the classification result through the classifier to determine whether the audio to be identified is human voice audio that requires knowledge question answering, thereby ensuring that the system only processes human voice audio that requires knowledge question answering, thereby improving the performance and user experience of the knowledge question answering system.

[0060] Figure 5 FIG. 1 is a block diagram of a knowledge question answering system based on artificial intelligence technology according to an embodiment of the present application. Figure 5 As shown, according to the embodiment of the present application, the knowledge question and answer system 100 based on artificial intelligence technology includes: a knowledge question and answer data acquisition module 110, which is used to acquire the audio to be recognized collected by the sound sensor and the waveform extracted from the audio to be recognized collected by the sound sensor; a knowledge question and answer data feature extraction module 120, which is used to extract the knowledge question and answer audio related feature vector and the audio time domain-audio frequency domain related feature vector from the audio to be recognized collected by the sound sensor and the waveform extracted from the audio to be recognized collected by the sound sensor; a target domain joint correlation migration module 130, which is used to perform hidden perturbation optimization based on data reconstruction mapping on the knowledge question and answer audio related feature vector and the audio time domain-audio frequency domain related feature vector to obtain a knowledge question and answer audio classification feature vector; a human voice audio judgment generation module 140, which is used to pass the knowledge question and answer audio classification feature vector through a classifier to obtain a classification result, and the classification result is used to judge whether the audio to be recognized is human voice audio that needs knowledge question and answer.

[0061] As described above, the knowledge question and answer system 100 based on artificial intelligence technology according to the embodiment of the present application can be implemented in various terminal devices, such as a server deployed with a knowledge question and answer control algorithm based on artificial intelligence technology. In one example, the knowledge question and answer system 100 based on artificial intelligence technology can be integrated into the terminal device as a software module and / or a hardware module. For example, the knowledge question and answer system 100 based on artificial intelligence technology can be a software module in the operating system of the terminal device, or can be an application developed for the terminal device; of course, the knowledge question and answer system 100 based on artificial intelligence technology can also be one of the many hardware modules of the terminal device.

[0062] Alternatively, in another example, the artificial intelligence technology-based knowledge question and answer system 100 and the terminal device may also be separate devices, and the artificial intelligence technology-based knowledge question and answer system 100 may be connected to the terminal device via a wired and / or wireless network and transmit interactive information in accordance with an agreed data format.

[0063] Here, those skilled in the art can understand that the specific operations of each step in the above-mentioned knowledge question answering system based on artificial intelligence technology have been referred to above. Figures 1 to 4 The invention has been introduced in detail in the description of the knowledge question answering method based on artificial intelligence technology, and therefore, its repeated description will be omitted.

[0064] In addition, an embodiment of the present invention further provides another electronic device, comprising: one or more processors, a memory, a communication component and a communication bus, wherein the processor, the memory and the communication component communicate with each other via the communication bus;

[0065] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute an operation corresponding to the data processing method of any of the above embodiments of the present invention.

[0066] Figure 6 is a block diagram of an electronic device according to an embodiment of the present application. Figure 6 , which shows a schematic diagram of the structure of an electronic device suitable for implementing a terminal device or a server in an embodiment of the present application. Figure 6As shown, the electronic device includes one or more processors, a communication unit, etc. The one or more processors are, for example: one or more central processing units (CPUs), and / or one or more graphics processing units (GPUs), etc. The processor can perform various appropriate actions and processes according to executable instructions stored in a read-only memory (ROM) or executable instructions loaded from a storage part to a random access memory (RAM). The communication unit may include but is not limited to a network card, and the network card may include but is not limited to an IB (Infiniband) network card. The processor may communicate with a read-only memory and / or a random access memory to execute executable instructions, and is connected to the communication unit through a bus and communicates with other target devices through the communication unit, thereby completing the operation corresponding to any method provided in the embodiments of the present application, for example, the primary shard manager shards the features in the feature queue; for any shard, the primary shard manager stores the any shard in the corresponding shard feature library, and records the mapping relationship between the any shard and the corresponding shard feature library in a mapping table; in response to a failure of the primary shard manager, a shard manager is reselected from at least one slave shard manager as the new primary shard manager; the new primary shard manager obtains shard information of the shard currently being written from all shard feature libraries, and recreates the mapping relationship of the shard currently being written based on the shard information of the shard currently being written in all shard feature libraries.

[0067] In addition, various programs and data required for the operation of the device can also be stored in the RAM. The CPU, ROM and RAM are connected to each other via a bus. In the case of RAM, the ROM is an optional module. The RAM stores executable instructions, or writes executable instructions into the ROM at runtime, and the executable instructions enable the processor to perform operations corresponding to any of the above methods of the present invention. The input / output (I / O) interface is also connected to the bus. The communication unit can be integrated or can be configured to have multiple submodules (e.g., multiple IB network cards) and be linked on the bus.

[0068] The following components are connected to the I / O interface: an input section including a keyboard, a mouse, etc.; an output section including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section including a hard disk, etc.; and a communication section including a network interface card such as a LAN card, a modem, etc. The communication section performs communication processing via a network such as the Internet. A drive is also connected to the I / O interface as needed. Removable media, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., are installed on the drive as needed so that a computer program read therefrom is installed into the storage section as needed.

[0069] If necessary, Figure 6The architecture shown is only an optional implementation method. In the specific practice, the above can be modified according to actual needs. Figure 6 The number and type of components can be selected, deleted, added or replaced; different functional components can be set up by separate settings or integrated settings, for example, the GPU and CPU can be set up separately or the GPU can be integrated on the CPU, the communication unit can be set up separately or integrated on the CPU or GPU, etc. These replaceable implementations all fall within the protection scope disclosed in the present invention.

[0070] In addition, an embodiment of the present invention further provides a computer program, including computer-readable code. When the computer-readable code runs on a device, a processor in the device executes instructions for implementing each step in the data processing method of any of the above embodiments of the present invention.

[0071] The method and apparatus of the present invention may be implemented in many ways. For example, the method and apparatus of the present invention may be implemented by software, hardware, firmware, or any combination of software, hardware, firmware. The above order of steps for the method is only for illustration, and the steps of the method of the present invention are not limited to the order specifically described above, unless otherwise specifically stated. In addition, in some embodiments, the present invention may also be implemented as a program recorded in a recording medium, which includes machine-readable instructions for implementing the method according to the present invention. Thus, the present invention also covers a recording medium storing a program for executing the method according to the present invention.

[0072] The description of the present invention is given for the purpose of illustration and description, and is not intended to be exhaustive or to limit the invention to the disclosed form. Many modifications and variations will be apparent to those of ordinary skill in the art. The embodiments are selected and described in order to better illustrate the principles and practical applications of the present invention and to enable those of ordinary skill in the art to understand the present invention and thereby design various embodiments with various modifications suitable for particular uses.

Claims

1. A knowledge question answering method based on artificial intelligence technology, characterized in that: include: Acquire the audio to be recognized collected by the sound sensor and a waveform extracted from the audio to be recognized collected by the sound sensor; Extracting a knowledge question and answer audio-related feature vector and an audio time domain-audio frequency domain-related feature vector from the audio to be recognized collected by the sound sensor and the waveform graph extracted from the audio to be recognized collected by the sound sensor; Performing hidden perturbation optimization based on data reconstruction mapping on the knowledge question and answer audio associated feature vector and the audio time domain-audio frequency domain associated feature vector to obtain a knowledge question and answer audio classification feature vector; The knowledge question and answer audio classification feature vector is passed through a classifier to obtain a classification result, and the classification result is used to determine whether the audio to be identified is human voice audio that requires knowledge question and answer.

2. The knowledge question answering method based on artificial intelligence technology according to claim 1 is characterized in that: Extracting a knowledge question and answer audio-related feature vector and an audio time domain-audio frequency domain-related feature vector from the audio to be recognized collected by the sound sensor and the waveform graph extracted from the audio to be recognized collected by the sound sensor, including: Performing text feature extraction on the audio to be recognized collected by the sound sensor to obtain a knowledge question and answer related feature vector; Extracting the spectral features of the audio to be recognized collected by the sound sensor to obtain a spectral feature vector of the knowledge question and answer; Associating the knowledge question and answer associated feature vector with the knowledge question and answer spectrum feature vector to obtain the knowledge question and answer audio associated feature vector; Feature extraction is performed on the waveform graph extracted from the audio to be recognized collected by the sound sensor to obtain the audio time domain-audio frequency domain correlation feature vector.

3. The knowledge question answering method based on artificial intelligence technology according to claim 2 is characterized in that: Performing text feature extraction on the audio to be recognized collected by the sound sensor to obtain a knowledge question and answer related feature vector, including: The audio to be recognized collected by the sound sensor is passed through the audio to be recognized speech processing encoder to obtain knowledge question and answer text data; Performing word segmentation processing on the knowledge question and answer text data to obtain the plurality of knowledge question and answer text input data; Passing the plurality of knowledge question and answer text input data through a knowledge question and answer context encoder based on the Clip model to obtain a plurality of knowledge question and answer feature vectors; The multiple knowledge question and answer feature vectors are cascaded to obtain the knowledge question and answer associated feature vector.

4. The knowledge question answering method based on artificial intelligence technology according to claim 3 is characterized in that: Extracting the spectral features of the audio to be recognized collected by the sound sensor to obtain a knowledge question and answer spectral feature vector, including: Extracting a plurality of spectrograms including a logarithmic Mel spectrogram, a cochlear spectrogram and a constant Q transform spectrogram from the audio to be recognized collected by the sound sensor; Arranging the plurality of spectrograms including the logarithmic Mel spectrogram, the cochlear spectrogram and the constant Q transform spectrogram to obtain a knowledge question answering audio multi-channel input spectrogram; The multi-channel input spectrogram of the knowledge question and answer audio is passed through a knowledge question and answer spectrogram feature extractor using a channel attention mechanism to obtain the knowledge question and answer spectrogram feature vector.

5. The knowledge question answering method based on artificial intelligence technology according to claim 4 is characterized in that: The method of performing feature extraction on the waveform graph extracted from the audio to be recognized collected by the sound sensor to obtain the audio time domain-audio frequency domain correlation feature vector comprises: Performing data preprocessing on the waveform graph extracted from the audio to be recognized collected by the sound sensor to obtain an audio time domain feature input vector and an audio frequency domain feature input vector; Calculating the association matrix of the audio time domain feature input vector and the audio frequency domain feature input vector to obtain an audio time domain-audio frequency domain association matrix; The audio time domain-audio frequency domain correlation matrix is ​​passed through an audio time domain-audio frequency domain feature extractor based on a convolutional neural network to obtain the audio time domain-audio frequency domain correlation feature vector.

6. The knowledge question answering method based on artificial intelligence technology according to claim 5 is characterized in that: The method of preprocessing the waveform image extracted from the audio to be recognized collected by the sound sensor to obtain an audio time domain feature input vector and an audio frequency domain feature input vector comprises: Cutting a plurality of sampling windows of the waveform extracted from the audio to be identified collected by the sound sensor along the time sequence to obtain a plurality of sampling windows of the audio waveform to be identified; Extracting time domain features and frequency domain features from the plurality of audio waveform sampling windows to be identified to obtain a plurality of audio time domain features to be identified and a plurality of audio frequency domain features to be identified; The multiple audio time domain features to be identified and the multiple audio frequency domain features to be identified are arranged to obtain the audio time domain feature input vector and the audio frequency domain feature input vector.

7. The knowledge question answering method based on artificial intelligence technology according to claim 6 is characterized in that: The knowledge question and answer audio associated feature vector and the audio time domain-audio frequency domain associated feature vector are subjected to hidden perturbation optimization based on data reconstruction mapping to obtain a knowledge question and answer audio classification feature vector, including: Mapping the knowledge question and answer audio-related feature vector and the audio time domain-audio frequency domain-related feature vector to an eigendecomposition space to obtain a knowledge question and answer audio-related eigenfeature vector and an audio time domain-audio frequency domain-related eigenfeature vector; Performing inter-domain feature calibration on the knowledge question and answer audio-related intrinsic feature vector and the audio time domain-audio frequency domain-related intrinsic feature vector to obtain the knowledge question and answer audio-related modulated intrinsic feature vector and the audio time domain-audio frequency domain-related modulated intrinsic feature vector; Constructing an audio time domain-audio frequency domain associated fine-grained semantic information field between the knowledge question and answer audio associated modulated intrinsic feature vector and the audio time domain-audio frequency domain associated modulated intrinsic feature vector; Based on the audio time domain-audio frequency domain associated fine-grained semantic information field, the knowledge question and answer audio associated modulated intrinsic feature vector and the audio time domain-audio frequency domain associated modulated intrinsic feature vector are fused to obtain the knowledge question and answer audio classification feature vector.

8. The knowledge question answering method based on artificial intelligence technology according to claim 7 is characterized in that: Constructing an audio time domain-audio frequency domain associated fine-grained semantic information field between the knowledge question and answer audio associated modulated intrinsic feature vector and the audio time domain-audio frequency domain associated modulated intrinsic feature vector, including: Multiply the eigenvector after audio correlation modulation of the knowledge question and answer and the transposed vector of the eigenvector after audio time domain-audio frequency domain correlation modulation, and then divide by the scale of the eigenvector after audio time domain-audio frequency domain correlation modulation to obtain an audio time domain-audio frequency domain correlation fine-grained interaction matrix; The audio time domain-audio frequency domain associated fine-grained matrix is ​​convolutionally encoded to obtain the audio time domain-audio frequency domain associated fine-grained semantic information field.

9. A knowledge question answering system based on artificial intelligence technology, characterized in that: include: A knowledge question and answer data acquisition module, used to acquire the audio to be recognized collected by the sound sensor and a waveform extracted from the audio to be recognized collected by the sound sensor; A knowledge question and answer data feature extraction module, used to extract a knowledge question and answer audio-related feature vector and an audio time domain-audio frequency domain-related feature vector from the audio to be recognized collected by the sound sensor and the waveform extracted from the audio to be recognized collected by the sound sensor; A target domain joint correlation migration module is used to perform hidden perturbation optimization based on data reconstruction mapping on the knowledge question and answer audio correlation feature vector and the audio time domain-audio frequency domain correlation feature vector to obtain a knowledge question and answer audio classification feature vector; The human voice audio judgment generation module is used to pass the knowledge question and answer audio classification feature vector through a classifier to obtain a classification result, and the classification result is used to determine whether the audio to be identified is human voice audio that requires knowledge question and answer.

10. An electronic device, characterized in that: include: at least one processor; And, a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the knowledge question and answer method based on artificial intelligence technology as described in any one of claims 1-8.

Images