Voice state detection system and method based on voice interaction

Abstract

The invention relates to the technical field of voice interaction, in particular to a voice state detection system and method based on voice interaction, and the method comprises the steps: obtaining a first audio signal, storing the first audio signal in a cache region, analyzing and recognizing the first audio signal, and judging whether the first audio signal meets a preset playing condition 1 or not; dividing the first audio signal into multiple segments of audio signals according to a preset time length, and playing the multiple segments of audio signals segment by segment; analyzing the playing state of each segment of audio signal, and adjusting the parameters of the output audio signal; acquiring a second audio signal, identifying the second audio signal, and classifying the second audio signal according to an identification result; and analyzing the classified second audio signal, and judging whether the second audio signal meets a playing condition II or not. According to the invention, in the voice interaction process, the output audio signal and the input audio signal are collected, the two kinds of signals are analyzed, the sound state including the intensity of the audio signals and the like is recognized, and in addition, the content of the output audio can be monitored.

Description

Technical Field

[0001] The present invention relates to the field of voice interaction technology, and in particular to a sound state detection system and method based on voice interaction. Background Art

[0002] Voice interaction is a technology that uses natural language to communicate between humans and machines. It covers the entire process from user voice input to machine voice output. Voice interaction technology combines key technologies such as speech recognition, natural language processing, and speech synthesis to provide users with a more natural and efficient way of interaction. In the financial, insurance, and judicial industries, voice interaction technology is widely used in multiple scenarios such as intelligent question and answer, intelligent quality inspection, and intelligent court records, effectively improving work efficiency and service quality.

[0003] During the voice interaction process, the output audio signal needs to be detected to ensure the audio output effect. In practical applications, it still faces some challenges, such as poor noise suppression and inaccurate volume adjustment. Summary of the invention

[0004] The present invention not only analyzes the basic properties (such as intensity) of the output audio signal and makes real-time adjustments to improve the audio output effect; it also identifies and classifies the content of the audio signal, thereby being able to achieve the effect of audio signal detection from multiple dimensions.

[0005] The technical solution proposed by the present invention is: a method for detecting sound status in voice interaction, the method comprising: Acquire a first audio signal, store the first audio signal in a buffer area, analyze and identify the first audio signal, and determine whether the first audio signal meets a preset playback condition 1; Divide the first audio signal into multiple audio signals according to a preset time length, and play them one by one; analyze the playing state of each audio signal, and adjust the parameters of the output audio signal; Acquire a second audio signal, identify the second audio signal, and classify the second audio signal according to the identification result; Analyze the classified second audio signal to determine whether the second audio signal meets the second playback condition; The obtaining of the first audio signal and storing the first audio signal in the buffer area includes: obtaining the first audio signal from a sound input terminal or from a sound collection device terminal; Divide the local storage into multiple cache areas and assign corresponding addresses; The acquired first audio signals are stored in corresponding buffer areas, and a collection timestamp is added to each first audio signal.

[0006] Preferably, the analyzing and identifying the first audio signal comprises: Extracting a first audio signal from the buffer area and preprocessing the first audio signal to reduce noise and improve signal quality; Reading a locally stored voiceprint feature information database, wherein the voiceprint feature information database includes voiceprint MFCC feature data and identity information of a person corresponding to the voiceprint MFCC feature data; Extract characteristic voiceprint signals from preprocessed audio signals, including: The preprocessed audio signal is converted into a frequency domain signal, a plurality of Mel frequency cepstral coefficient MFCC features are extracted from the corresponding frequency domain signal, and the extracted plurality of MFCC features constitute a frequency domain feature set 1; Import the pre-trained speech recognition model, and input the elements in the frequency domain feature set 1 as input into the speech recognition model; The MFCC features are classified by the speech recognition model; specifically: Using an SVM classifier to classify the MFCC features in the frequency domain feature set 1, the frequency domain feature set 1 is divided into a plurality of frequency domain feature subsets, and a parameter N is generated, wherein the parameter N is used to represent the number of frequency domain feature subsets; Each frequency domain feature subset corresponds to a voiceprint, thereby identifying the type of the voiceprint included in the first audio signal; Match the MFCC features in the frequency domain feature subset with the voiceprint MFCC voiceprint feature data in the voiceprint feature information library; Identify the identity information and special sound information of the speaker corresponding to different voiceprint MFCC features in the first audio signal according to the matching result; After the special sound information is identified, a warning message is generated; the special sound information includes one or more of rain sound, wind sound, and cough sound.

[0007] Preferably, the determining whether the first audio signal satisfies a preset playing condition 1 includes: Get parameter N. When N=1, perform the following steps: Get multiple personnel information from the database, and extract personnel names, personnel roles and personnel permissions from the personnel information; Generate a personnel information table according to the obtained multiple personnel information; According to the identified personnel's identity information, query the personnel information table to determine whether there is matching personnel information; If yes, the first audio signal is played; otherwise, the first audio signal is not played.

[0008] Preferably, the determining whether the first audio signal satisfies a preset playing condition 1 includes: Get parameter N. When N>1, perform the following steps: Get multiple personnel information from the database, and extract personnel names, personnel roles and personnel permissions from the personnel information; Generate a personnel information table according to the obtained multiple personnel information; According to the identified personnel's identity information, query the personnel information table to determine whether there is matching personnel information; If not, the first audio signal is not played; if yes, the identity information of all matching persons is obtained; Query the corresponding personnel authority from the personnel information table according to the personnel's identity information; Arrange the identity information of personnel according to their authority; According to the arrangement order of the identity information of the persons, multiple voiceprint signals included in the first audio signal are played.

[0009] Preferably, the determining whether the first audio signal satisfies a preset playing condition 1 further includes: Preprocessing the first audio signal, converting the preprocessed first audio signal into text one through a natural speech processing algorithm, and storing the text one locally; Use the term frequency-inverse document frequency TF-IDF algorithm to extract keywords from the text information; Importing a preset keyword library to determine whether the first audio signal includes the keyword; If not, a reminder signal is sent and the first audio signal is acquired again; if yes, a sensitive word recognition step is performed; The step of identifying sensitive words includes: Load the first audio signal using the audio processing Librosa library, and convert the first audio signal to a target sampling frequency; Then convert the first audio signal processed in the previous step into a wav file, use the DeepSpeech model to convert the first audio signal into text 2, and then use regular expressions to search for sensitive words in the text; storing the second text locally; Record the identified sensitive words and the timestamps of their occurrence; Import the pre-trained LSTM model, count the number of times each sensitive word appears, and output the statistical results; Construct a sensitive word replacement table and map sensitive words to replacement words in the sensitive word replacement table; The sensitive words in the second text are replaced by replacement words to form the third text; the third text is converted into audio data by using the text-to-speech (TTS) technology to replace the original first audio signal and output.

[0010] Preferably, the step of dividing the first audio signal into multiple audio signals according to a preset time length and playing them one by one; analyzing the playing state of each audio signal and adjusting the parameters of the output audio signal comprises: Dividing the first audio signal into a plurality of audio segments of the same time length T; Playing an audio segment, collecting an audio signal of the played audio segment, identifying the strength of the audio signal, comparing it with a preset strength threshold, determining whether the output audio signal meets the strength requirement, and if so, continuing to play the next audio segment; If not, increase the volume of the next audio segment and play it.

[0011] Preferably, the identifying the intensity of the audio signal and comparing it with a preset intensity threshold to determine whether the output audio signal meets the intensity requirement includes: Performing noise removal and normalization processing on the audio signal of the collected audio segment; Get the root mean square value RMS of each audio segment; specifically including: Set the sampling frequency and obtain discrete sample values ​​of multiple audio signals; After squaring each discrete sample value, calculate the average of all square values; Take the square root of the average value to obtain the RMS value; The obtained RMS value is compared with a preset intensity threshold. If the RMS value is greater than the intensity threshold, it is determined that the sound intensity of the audio segment meets the requirement; otherwise, it is determined that the sound intensity of the audio segment does not meet the requirement.

[0012] Preferably, the acquiring the second audio signal, identifying the second audio signal, and classifying the second audio signal according to the identification result comprises the following steps: Determine whether the first audio signal has been played; if not, continue to play the first audio signal; if yes, proceed to the next step; Collecting a second audio signal through a sound collecting device; After preprocessing the second audio signal, perform voiceprint recognition to obtain the voiceprint MFCC features; Matching the obtained voiceprint MFCC features with the voiceprint MFCC feature data in the voiceprint feature information library to obtain the personal information of the speaker corresponding to the second audio signal; According to the personal information query person information table obtained in the previous step, judging whether to perform the operation corresponding to the second audio signal according to the query result; specifically including: querying whether the person information table includes personal information of the speaker corresponding to the second audio signal; If yes, parse the second audio signal, and identify keywords in the second audio signal through the speech recognition model to perform corresponding operations; Otherwise, a warning message is issued, and the second audio signal is re-collected, and the number of times M of re-collecting the second audio signal is recorded; If M reaches the set maximum collection limit M max , the second audio signal is no longer collected.

[0013] The present invention also provides a system for detecting a sound state in voice interaction, wherein the system is used to execute the method for detecting a sound state in voice interaction.

[0014] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method for detecting sound status in voice interaction.

[0015] Beneficial effects of the present invention: 1. In the process of voice interaction, the present invention segments the first audio signal (output audio signal), monitors the playing status (volume, etc.) of the segmented audio, and adjusts the playing volume of the next audio segment as needed to ensure that the volume of the played audio can overcome the influence of environmental noise. Due to the segmented approach, the volume can be adjusted in real time to adapt to changes in the environment.

[0016] 2. The present invention can identify the content of the first audio signal and the identity information of the speaker in the audio, identify the keywords and sensitive words in the first audio signal, ensure that the output audio includes necessary content and does not leak corresponding sensitive information, identify the identity information of the speaker corresponding to the first audio signal, avoid outputting audio information of unauthorized persons, and adapt to voice interaction scenarios with strict restrictions on the identities of the interacting parties. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 The present invention is a flow chart of a method for detecting sound status in voice interaction. DETAILED DESCRIPTION

[0018] The following description is used to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are only examples, and those skilled in the art can think of other obvious variations. The basic principles of the present invention defined in the following description can be applied to other embodiments, variations, improvements, equivalents, and other technical solutions that do not deviate from the spirit and scope of the present invention.

[0019] It is to be understood that the term "one" should be understood as "at least one" or "one or more", that is, in one embodiment, the number of an element may be one, while in another embodiment, the number of the element may be multiple, and the term "one" should not be understood as a limitation on the quantity.

[0020] Embodiment 1: refer to Figure 1 The technical solution provided by the present invention is: a method for detecting sound state in voice interaction, comprising the following steps: 1. Obtain a first audio signal and store the first audio signal in a buffer area, specifically: obtain the first audio signal from a sound input terminal or from a sound collection device terminal, divide a plurality of buffer areas in a local storage, assign corresponding addresses, store the obtained first audio signal in the corresponding buffer area, and add a collection timestamp to each first audio signal.

[0021] Then, the first audio signal is analyzed and identified. Specifically, the steps include:

[0022] 1.1. Extract the first audio signal from the buffer area and pre-process the first audio signal, including noise reduction and normalization. In this embodiment, the first audio signal is subjected to noise reduction processing by spectral subtraction. The so-called normalization processing refers to normalizing the amplitude of the audio signal to a uniform range to eliminate the influence of different sound collection devices.

[0023] 1.2. Read the locally stored voiceprint feature information database, which includes voiceprint MFCC feature data and identity information of the person corresponding to the voiceprint MFCC feature data; wherein the voiceprint MFCC feature data is used to represent the voice characteristics of an individual's identity, such as timbre, pitch and pronunciation speed, etc., through which functions such as speaker recognition and emotion analysis can be realized.

[0024] 1.3. Extract characteristic voiceprint signals from pre-processed audio signals, including: The preprocessed audio signal is converted into a frequency domain signal, a plurality of Mel frequency cepstral coefficient MFCC features are extracted from the corresponding frequency domain signal, and the extracted plurality of MFCC features constitute a frequency domain feature set 1; Import the pre-trained speech recognition model, and input the elements in the frequency domain feature set 1 as input into the speech recognition model; 1.4. Classify MFCC features through the speech recognition model, specifically: Using an SVM classifier to classify the MFCC features in the frequency domain feature set 1, the frequency domain feature set 1 is divided into a plurality of frequency domain feature subsets, and a parameter N is generated, wherein the parameter N is used to represent the number of frequency domain feature subsets; Each frequency domain feature subset corresponds to a voiceprint, thereby identifying the type of the voiceprint included in the first audio signal; Match the MFCC features in the frequency domain feature subset with the voiceprint MFCC voiceprint feature data in the voiceprint feature information library; The identity information of the speaker corresponding to different voiceprint MFCC features in the first audio signal is identified according to the matching result.

[0025] In this embodiment, the adopted speech recognition model is constructed based on a support vector machine (SVM) model, wherein the SVM model is trained with historical audio data before use to meet the performance requirements of model deployment.

[0026] After the analysis process is completed, it is necessary to determine whether the first audio signal meets the preset playback condition 1. The specific steps are as follows: Get parameter N. When N=1, perform the following steps: Obtain multiple personnel information from the database, extract personnel names, personnel roles and personnel permissions from the personnel information; generate a personnel information table based on the obtained multiple personnel information; query the personnel information table based on the identified personnel identity information to determine whether there is matching personnel information; If yes, the first audio signal is played; otherwise, the first audio signal is not played.

[0027] In some embodiments, the first audio signal is not input through a specific recording device, but is collected on-site. During the collection process, audio signals of other speakers may be collected at the same time. This requires identifying the information of each speaker and judging the authority of all speakers, so as to exclude voiceprint signals that do not need to be processed. The specific steps are as follows: Get parameter N. When N>1, perform the following steps: Get multiple personnel information from the database, and extract personnel names, personnel roles and personnel permissions from the personnel information; Generate a personnel information table based on the obtained multiple personnel information; query the personnel information table based on the identified personnel identity information to determine whether there is matching personnel information; If not, the first audio signal is not played; if yes, the identity information of all matching persons is obtained; According to the identity information of the personnel, the corresponding personnel authority is queried from the personnel information table; the identity information of the personnel is arranged according to the level of the personnel authority; according to the arrangement order of the identity information of the personnel, the multiple voiceprint signals included in the first audio signal are played.

[0028] 2. dividing the first audio signal into multiple audio signals according to a preset time length, and playing them one by one; analyzing the playing status of each audio signal, and adjusting the parameters of the output audio signal; The first audio signal is divided into multiple audio segments of the same time length T; the audio segments are played, and the audio signals of the played audio segments are collected at the same time, and the intensity of the audio signals is identified, and compared with the preset intensity threshold to determine whether the output audio signal meets the intensity requirement, and if so, the next audio segment is played; otherwise, the volume of the next audio segment is increased before playing. The specific increased volume amplitude can be calculated by collecting the ambient noise data and calculating the intensity of the ambient noise, and the increased volume amplitude is not less than 1.5 times the intensity of the ambient noise.

[0029] After identifying the special sound information, a warning message is generated; the special sound information includes rain, wind, cough, knocking, background music, etc. These special sounds can be used to indicate the environment where the interlocutor is located or the danger warning signal issued by the interlocutor. The warning message can be sent secretly through the background. When the first audio signal is played, the special sound information will not be played.

[0030] The process of identifying the audio signal strength and judging whether it meets the requirements is as follows: The audio signal of the collected audio segment is subjected to noise removal and normalization processing; the strength of the audio signal can be calculated in a variety of ways, common methods include: root mean square value (RMS) and peak detection (finding the maximum amplitude value in the audio signal). In this embodiment, the root mean square value (RMS) method is used.

[0031] The specific steps of obtaining the root mean square value RMS of each audio segment include: setting a sampling frequency to obtain discrete sample values ​​of multiple audio signals; performing a square operation on each discrete sample value and then calculating the average value of all square values; and performing a square root operation on the average value to obtain an RMS value.

[0032] The obtained RMS value is compared with a preset intensity threshold. If the RMS value is greater than the intensity threshold, it is determined that the sound intensity of the audio segment meets the requirement; otherwise, it is determined that the sound intensity of the audio segment does not meet the requirement.

[0033] The audio data processing in this embodiment uses the librosa library. The RMS value is obtained by defining the calculate_rms function. This function includes multiple functions (square calculation function, average function and square root function, etc.), and after receiving an audio signal array as input, returns the calculated RMS value.

[0034] 3. Acquire the second audio signal, identify the second audio signal, and classify the second audio signal according to the identification result. Specifically, the steps include: Determine whether the first audio signal has been played; if not, continue to play the first audio signal; if yes, proceed to the next step; Collecting a second audio signal through a sound collecting device; After preprocessing the second audio signal, perform voiceprint recognition to obtain the voiceprint MFCC features; The obtained voiceprint MFCC features are matched with the voiceprint MFCC feature data in the voiceprint feature information library to obtain the personal information of the speaker corresponding to the second audio signal.

[0035] 4. Analyze the classified second audio signal to determine whether the second audio signal meets the second playback condition; specifically including: querying whether the person information table includes personal information of the speaker corresponding to the second audio signal; If yes, parse the second audio signal, and identify keywords in the second audio signal through the speech recognition model to perform corresponding operations; Otherwise, a warning message is issued to remind the speaker that he does not have the authority for voice interaction, and the second audio signal is re-collected, and the number of times the second audio signal is re-collected is recorded; if M reaches the set maximum collection limit M max , the second audio signal is no longer collected.

[0036] Embodiment 2: The difference between this embodiment and the first embodiment is that the step of determining whether the first audio signal satisfies the first preset playing condition further includes: Preprocessing the first audio signal, converting the preprocessed first audio signal into text one through a natural speech processing algorithm, and storing the text one locally; Use the term frequency-inverse document frequency TF-IDF algorithm to extract keywords from the text information; Importing a preset keyword library to determine whether the first audio signal includes the keyword; If not, a reminder signal is sent and the first audio signal is acquired again; if yes, a sensitive word recognition step is performed.

[0037] The step of identifying sensitive words includes: Use the Librosa library to load the audio file and convert it to the target sampling rate (16kHz) to reduce the difference caused by different sampling rates. Save the preprocessed audio as a WAV file and then convert it to text using the DeepSpeech model. Use regular expressions to find sensitive words in the recognized text;

[0038] The second text is stored locally for later query.

[0039] Record the identified sensitive words and the timestamps of their occurrence; import the pre-trained LSTM model, count the number of times each sensitive word appears, and output the statistical results; Construct a sensitive word replacement table and map sensitive words to replacement words in the sensitive word replacement table; The sensitive words in the second text are replaced by replacement words to form the third text; the third text is converted into audio data by using the text-to-speech (TTS) technology to replace the original first audio signal and output.

[0040] In this embodiment, keyword and sensitive word restrictions are set in the first playback condition. Keyword recognition is helpful for determining whether the first audio signal includes necessary information; and sensitive word recognition is helpful for avoiding leakage of sensitive information.

[0041] In the first and second embodiments, the first audio signal may be considered as feedback to the second audio information (ie, an output audio signal), or a response to a wake-up voice or an active voice.

[0042] For example, in a smart court scenario, the first audio signal may be the collected audio data of the judge, and the second audio signal may be the reply audio of the person being tried; For example, in an artificial intelligence robot device, the first audio signal may be a reply audio output by the intelligent robot after receiving a wake-up command; or it may be audio information output after executing the second audio signal.

[0043] The present invention also provides a system for detecting a sound state in voice interaction, wherein the system is used to execute the method for detecting a sound state in voice interaction.

[0044] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method for detecting sound status in voice interaction.

[0045] The embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. The embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part, and / or installed from a removable medium. When the computer program is executed by the central processing unit (CPU), the above-mentioned functions defined in the method of the present application are executed. It should be noted that the above-mentioned computer-readable medium of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, but not limited to, a system, device or device of an electrical, magnetic, optical, electromagnetic, infrared segment, or semiconductor, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection with one or more wire segments, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, electrical wire, optical cable, RF, etc., or any suitable combination of the foregoing.

[0046] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0047] Those skilled in the art should understand that the embodiments of the present invention described above and shown in the accompanying drawings are only examples and do not limit the present invention. The objectives of the present invention have been fully and effectively achieved. The functional and structural principles of the present invention have been demonstrated and explained in the embodiments. Without departing from the principles, the implementation methods of the present invention may be deformed or modified in any way.

Claims

1. A method for detecting sound status in voice interaction, characterized in that: The method comprises: Acquire a first audio signal, store the first audio signal in a buffer area, analyze and identify the first audio signal, and determine whether the first audio signal meets a preset playback condition 1; Divide the first audio signal into multiple audio signals according to a preset time length, and play them one by one; analyze the playing state of each audio signal, and adjust the parameters of the output audio signal; Acquire a second audio signal, identify the second audio signal, and classify the second audio signal according to the identification result; Analyze the classified second audio signal to determine whether the second audio signal meets the second playback condition; The obtaining of the first audio signal and storing the first audio signal in the buffer area includes: obtaining the first audio signal from a sound input terminal or from a sound collection device terminal; Divide the local storage into multiple cache areas and assign corresponding addresses; The acquired first audio signals are stored in corresponding buffer areas, and a collection timestamp is added to each first audio signal.

2. The method for detecting sound status in voice interaction according to claim 1, characterized in that: The analyzing and identifying a first audio signal comprises: Extracting a first audio signal from the buffer area and preprocessing the first audio signal to reduce noise and improve signal quality; Reading a locally stored voiceprint feature information database, wherein the voiceprint feature information database includes voiceprint MFCC feature data and identity information of a person corresponding to the voiceprint MFCC feature data; Extract characteristic voiceprint signals from preprocessed audio signals, including: The preprocessed audio signal is converted into a frequency domain signal, a plurality of Mel frequency cepstral coefficient MFCC features are extracted from the corresponding frequency domain signal, and the extracted plurality of MFCC features constitute a frequency domain feature set 1; Import the pre-trained speech recognition model, and input the elements in the frequency domain feature set 1 as input into the speech recognition model; The MFCC features are classified by the speech recognition model; specifically: Using an SVM classifier to classify the MFCC features in the frequency domain feature set 1, the frequency domain feature set 1 is divided into a plurality of frequency domain feature subsets, and a parameter N is generated, wherein the parameter N is used to represent the number of frequency domain feature subsets; Each frequency domain feature subset corresponds to a voiceprint, thereby identifying the type of the voiceprint included in the first audio signal; Match the MFCC features in the frequency domain feature subset with the voiceprint MFCC voiceprint feature data in the voiceprint feature information library; Identify the identity information and special sound information of the speaker corresponding to different voiceprint MFCC features in the first audio signal according to the matching result; After the special sound information is identified, a warning message is generated; the special sound information includes one or more of rain sound, wind sound, and cough sound.

3. The method for detecting sound state in voice interaction according to claim 2, characterized in that: The determining whether the first audio signal satisfies a preset playing condition 1 includes: Get parameter N. When N=1, perform the following steps: Get multiple personnel information from the database, and extract personnel names, personnel roles and personnel permissions from the personnel information; Generate a personnel information table according to the obtained multiple personnel information; According to the identified personnel's identity information, query the personnel information table to determine whether there is matching personnel information; If yes, the first audio signal is played; otherwise, the first audio signal is not played.

4. The method for detecting sound status in voice interaction according to claim 3, characterized in that: The determining whether the first audio signal satisfies a preset playing condition 1 includes: Get parameter N. When N>1, perform the following steps: Get multiple personnel information from the database, and extract personnel names, personnel roles and personnel permissions from the personnel information; Generate a personnel information table according to the obtained multiple personnel information; According to the identified personnel's identity information, query the personnel information table to determine whether there is matching personnel information; If not, the first audio signal is not played; if yes, the identity information of all matching persons is obtained; Query the corresponding personnel authority from the personnel information table according to the personnel's identity information; Arrange the identity information of personnel according to their authority; According to the arrangement order of the identity information of the persons, multiple voiceprint signals included in the first audio signal are played.

5. The method for detecting sound status in voice interaction according to claim 4, characterized in that: The step of determining whether the first audio signal satisfies a preset playback condition 1 further includes: Preprocessing the first audio signal, converting the preprocessed first audio signal into text one through a natural speech processing algorithm, and storing the text one locally; Use the term frequency-inverse document frequency TF-IDF algorithm to extract keywords from the text information; Importing a preset keyword library to determine whether the first audio signal includes the keyword; If not, a reminder signal is sent and the first audio signal is acquired again; if yes, a sensitive word recognition step is performed; The step of identifying sensitive words includes: Load the first audio signal using the audio processing Librosa library, and convert the first audio signal to a target sampling frequency; Then convert the first audio signal processed in the previous step into a wav file, use the DeepSpeech model to convert the first audio signal into text 2, and then use regular expressions to search for sensitive words in the text; storing the second text locally; Record the identified sensitive words and the timestamps of their occurrence; Import the pre-trained LSTM model, count the number of times each sensitive word appears, and output the statistical results; Construct a sensitive word replacement table and map sensitive words to replacement words in the sensitive word replacement table; The sensitive words in the second text are replaced by replacement words to form the third text; the third text is converted into audio data by using the text-to-speech (TTS) technology to replace the original first audio signal and output.

6. The method for detecting sound status in voice interaction according to claim 5, characterized in that: The first audio signal is divided into a plurality of audio signals according to a preset time length, and played one by one; Analyze the playback status of each audio signal and adjust the parameters of the output audio signal, including: Dividing the first audio signal into a plurality of audio segments of the same time length T; Playing an audio segment, collecting an audio signal of the played audio segment, identifying the strength of the audio signal, comparing it with a preset strength threshold, determining whether the output audio signal meets the strength requirement, and if so, continuing to play the next audio segment; If not, increase the volume of the next audio segment and play it.

7. The method for detecting sound status in voice interaction according to claim 6, characterized in that: The identifying the intensity of the audio signal, comparing it with a preset intensity threshold, and determining whether the output audio signal meets the intensity requirement includes: Performing noise removal and normalization processing on the audio signal of the collected audio segment; Get the root mean square value RMS of each audio segment; specifically including: Set the sampling frequency and obtain discrete sample values ​​of multiple audio signals; After squaring each discrete sample value, calculate the average of all square values; Take the square root of the average value to obtain the RMS value; The obtained RMS value is compared with a preset intensity threshold. If the RMS value is greater than the intensity threshold, it is determined that the sound intensity of the audio segment meets the requirement; otherwise, it is determined that the sound intensity of the audio segment does not meet the requirement.

8. The method for detecting sound status in voice interaction according to claim 7, characterized in that: The acquiring the second audio signal, identifying the second audio signal, and classifying the second audio signal according to the identification result comprises the following steps: Determine whether the first audio signal has been played; if not, continue to play the first audio signal; if yes, proceed to the next step; Collecting a second audio signal through a sound collecting device; After preprocessing the second audio signal, perform voiceprint recognition to obtain the voiceprint MFCC features; Matching the obtained voiceprint MFCC features with the voiceprint MFCC feature data in the voiceprint feature information library to obtain the personal information of the speaker corresponding to the second audio signal; According to the personal information query person information table obtained in the previous step, judging whether to perform the operation corresponding to the second audio signal according to the query result; specifically including: querying whether the person information table includes personal information of the speaker corresponding to the second audio signal; If yes, parse the second audio signal, and identify keywords in the second audio signal through the speech recognition model to perform corresponding operations; Otherwise, a warning message is issued, and the second audio signal is re-collected, and the number of times M of re-collecting the second audio signal is recorded; If M reaches the set maximum collection limit M max , the second audio signal is no longer collected.

9. A sound state detection system based on voice interaction, characterized in that: The system is used to execute a method for detecting sound status in voice interaction according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement a method for detecting sound status in voice interaction according to any one of claims 1 to 8.

Images