Real-time intercom intervention and alarm platform based on AI technology

Abstract

The present invention provides a real-time intercom intervention and alarm platform based on AI technology, including a communication server, a media resource control server and an AI voice training and recognition platform. The communication server is used to provide communication services, and convert the communication content into an audio media stream in real time, and send it to the media resource control server; the media resource control server is used to convert the audio media stream into text content and send it to the AI ​​voice training and recognition platform; the AI ​​voice training and recognition platform is used to identify sensitive information involved in the text content, and identify the audio involved in the text content for classification, send alarm information to the communication server, and start the intervention module in the communication server. The present invention is used to identify risk information such as sensitive words, violent threats, cries for help, abnormal sounds, etc. that appear in multiple business scenarios, and start corresponding intervention actions to achieve the purpose of purifying the conversation environment and handling unexpected events in a timely manner.

Description

Technical Field

[0001] The present invention belongs to the technical field of intercom intervention and alarm, and in particular, relates to a real-time intercom intervention and alarm platform based on AI technology. Background Art

[0002] In daily life, communication is everywhere and there are many ways to communicate. In certain scenarios, even if an administrator is monitoring the call, it is difficult to respond to sensitive information in the call in real time and intervene. Summary of the invention

[0003] In order to solve the above-mentioned deficiencies in the prior art, the purpose of the present invention is to provide a real-time intercom intervention and alarm platform based on AI technology to overcome the defects in the prior art.

[0004] In order to achieve the above-mentioned purpose, the present invention provides a real-time intercom intervention and alarm platform based on AI technology, including a communication server, a media resource control server and an AI voice training and recognition platform, wherein the communication server is electrically connected and connected by signal to the media resource control server, and the communication server includes an MRCP client, a user agent, a session communication component and an intervention module; wherein the user agent is used to access a number of user terminals, the session communication component is used to obtain the communication content of the user terminal in real time and convert it into an audio media stream, and the MRCP client is used to pull the audio media stream in real time and send it to the media resource control server; the media resource control server is electrically connected and connected by signal to the AI ​​voice training and recognition platform, and ... The control server is used to convert the audio media stream into text content and send it to the AI ​​voice training and recognition platform; the AI ​​voice training and recognition platform is electrically and signal-connected to the communication server, and the AI ​​voice training and recognition platform includes a voice recognition engine, a training module and an alarm module; wherein the voice recognition engine is used to receive the text content of the media resource control server, the training module is used to identify the sensitive information involved in the text content through the voice recognition model, and to identify the audio involved in the text content through the sound classification model for classification, and the alarm module is used to generate an alarm message with a message code of the corresponding category and send it to the communication server; the intervention module in the communication server initiates the corresponding intervention action according to the alarm message with the message code.

[0005] Through the above technical solution, when two user terminals are conversing, the conversation content is converted into an audio stream in real time and sensitive words are identified. When sensitive information is detected, it can be identified what kind of sound the current audio is, or what state or scene the sound is in, and send corresponding category alarm information to the communication server for timely intervention. It can be used to identify risk information such as sensitive words, cries for help, abnormal sounds, etc. that appear in multiple business scenarios, and initiate corresponding intervention actions to achieve the purpose of purifying the conversation environment and handling unexpected events in a timely manner.

[0006] As a further explanation of the real-time intercom intervention and alarm platform based on AI technology described in the present invention, preferably, the media resource control server includes a main server and multiple slave servers, the MRCP client communicates with the main server, and the main server communicates with the multiple slave servers, so that the MRCP client sends the IP address and port number of the user terminal to the main server, and the main server controls the idle slave servers to establish a communication connection with the MRCP client.

[0007] As a further explanation of the real-time intercom intervention and alarm platform based on AI technology described in the present invention, preferably, the speech recognition engine includes a word segmentation module and a semantic analysis module; the word segmentation module is used to divide the text content into a word vector set according to the word segmentation set and transmit it to the semantic analysis module, and the semantic analysis module is used to perform semantic analysis on the word vector set, preliminarily determine the classification category corresponding to the word vector set, and transmit it to the training module.

[0008] As a further explanation of the real-time intercom intervention and alarm platform based on AI technology described in the present invention, preferably, the alarm module includes an encoder, an alarm information generation module and an alarm information transmission module; the training module is connected to the encoder, the encoder is connected to the alarm information generation module, the alarm information generation module is connected to the alarm information transmission module, and the alarm information transmission module is connected to the intervention module; wherein, the encoder is used to receive the sensitive information and audio classification results of the training module and generate a corresponding message code and send it to the alarm information generation module, the alarm information generation module is used to generate an alarm message with a message code after receiving the message code, and the alarm information transmission module is used to send the alarm message with the message code to the intervention module.

[0009] As a further explanation of the real-time intercom intervention and alarm platform based on AI technology described in the present invention, preferably, the intervention module includes an alarm information receiving module, a decoder, an interruption intervention module, a reminder intervention module and a keyword silencing module; the alarm information transmitting module is connected to the alarm information receiving module, the alarm information receiving module is connected to the decoder, and the decoder is respectively connected to the interruption intervention module, the reminder intervention module and the keyword silencing module; wherein, the alarm information receiving module is used to receive the alarm information with message coding from the alarm information transmitting module and send it to the decoder, the decoder is used to parse the message coding and start the interruption intervention module, the reminder intervention module or the keyword silencing module according to the message coding, the interruption intervention module is used to cut off the call of the user terminal, the reminder intervention module is used to send text warnings or insert voice to the user terminal, and the keyword silencing module is used to silence sensitive words in the communication content of the user terminal.

[0010] As a further explanation of the real-time intercom intervention and alarm platform based on AI technology described in the present invention, preferably, the AI ​​voice training and recognition platform includes a database module, which is used to store sensitive word data sets and audio classification data sets, and provide model training data of training sets and test sets for the training module.

[0011] As a further illustration of the real-time intercom intervention and alarm platform based on AI technology described in the present invention, preferably, the communication server, the media resource control server and the AI ​​voice training and recognition platform are connected via real-time media streaming transmission.

[0012] As a further illustration of the real-time intercom intervention and alarm platform based on AI technology described in the present invention, preferably, the communication server communicates with the media resource control server via the SIP protocol.

[0013] As a further illustration of the real-time intercom intervention and alarm platform based on AI technology described in the present invention, preferably, the speech recognition model and the sound classification model are deployed on a private CPU / GPU server.

[0014] Through the above technical solutions, the model can be used in an intranet or offline environment to ensure data privacy.

[0015] As a further illustration of the real-time intercom intervention and alarm platform based on AI technology described in the present invention, preferably, the speech recognition model and the sound classification model are the speech recognition system DeepASR based on PaddlePaddle Fluid and Kaldi.

[0016] Through the above technical solutions, DeepASR uses the Fluid framework to complete the configuration and training of the acoustic model in speech recognition, and integrates Kaldi's decoder to achieve fast and large-scale training of the acoustic model, and uses Kaldi to complete complex speech data preprocessing and the final decoding process.

[0017] The beneficial effects of the present invention are as follows: the present invention provides an intervention and alarm platform that supports real-time intercom. Multiple different user terminals can be connected through a communication server. When two user terminals are conversing, the communication connection established between the communication server and the media resource control server and the AI ​​voice training and recognition platform can realize real-time conversion of the conversation content into an audio stream and perform sensitive word recognition. When sensitive information is detected, it can identify what kind of sound the current audio is, or what state or scene the sound is in, send corresponding category of alarm information to the communication server, and intervene in time. It can be used to identify risk information such as sensitive words, cries for help, abnormal sounds, etc. that appear in multiple business scenarios, and initiate corresponding intervention actions to achieve the purpose of purifying the conversation environment and handling accidents in time. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a structural schematic diagram of the real-time intercom intervention and alarm platform based on AI technology of the present invention.

[0019] Figure 2 It is a structural diagram of the media resource control server of the present invention.

[0020] Figure 3 It is a structural schematic diagram of the speech recognition engine of the present invention.

[0021] Figure 4 It is a structural schematic diagram of the alarm module and the intervention module of the present invention. DETAILED DESCRIPTION

[0022] In order to further understand the structure, features and other purposes of the present invention, the preferred embodiments are described in detail with reference to the accompanying drawings as follows. The embodiments described in the drawings are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention.

[0023] First, please refer to Figure 1 , Figure 1 The structure diagram of the real-time intercom intervention and alarm platform based on AI technology of the present invention is shown in FIG. The real-time intercom intervention and alarm platform based on AI technology includes a communication server 1, a media resource control server 2 and an AI voice training and recognition platform 3.

[0024] The communication server 1 is electrically and signally connected to the media resource control server 2. The communication server is used to provide communication services. The communication server 1 includes an MRCP client 11, a user agent 12, a session communication component 13 and an intervention module 14; wherein the user agent 12 is used to access a number of user terminals, the session communication component 13 is connected to the user agent 12, the session communication component 13 is used to obtain the communication content of the user terminal in real time and convert it into an audio media stream, the MRCP client 11 is connected to the session communication component 13, the MRCP client 11 is connected to the media resource control server 2, and the MRCP client 11 is used to pull the audio media stream in real time and send it to the media resource control server 2.

[0025] The media resource control server 2 is electrically and signal-connected to the AI ​​voice training and recognition platform 3, and the media resource control server 2 is used to convert the audio media stream into text content and send it to the AI ​​voice training and recognition platform 3. Figure 2As shown, the media resource control server 2 includes a main server 21 and multiple slave servers 22. The MRCP client 11 communicates with the main server 21, and the main server 21 communicates with the multiple slave servers 22, so that the MRCP client 11 sends the IP address and port number of the user terminal to the main server 21, and the main server 21 controls the idle slave servers 22 to establish a communication connection with the MRCP client 11.

[0026] The AI ​​voice training and recognition platform 3 is electrically and signal-connected to the communication server 1. The AI ​​voice training and recognition platform 3 includes a voice recognition engine 31, a training module 31, and an alarm module 33. The media resource control server 2 is connected to the voice recognition engine 31. The voice recognition engine 31 is used to receive the text content of the media resource control server 2. The voice recognition engine 31 is connected to the training module 32. Figure 3 As shown, the speech recognition engine 31 includes a word segmentation module 311 and a semantic analysis module 312; the word segmentation module 311 is used to divide the text content into word vector sets according to the word segmentation set and transmit it to the semantic analysis module 312; the semantic analysis module 312 is used to perform semantic analysis on the word vector set, preliminarily determine the classification category corresponding to the word vector set, and transmit it to the training module 32. The training module 32 includes a speech recognition model and a sound classification model. The training module 32 is used to identify sensitive information involved in the text content through the speech recognition model, and to identify and classify the audio involved in the text content through the sound classification model. The training module 32 is connected to the alarm module 33, and the alarm module 33 is connected to the intervention module 14. The alarm module 33 is used to generate an alarm message with a message code of the corresponding category and send it to the communication server 1 when the training module 32 detects sensitive information; the intervention module 14 in the communication server 1 initiates the corresponding intervention action according to the alarm message with the message code. Among them, as Figure 4As shown, the alarm module 33 includes an encoder 331, an alarm information generating module 332 and an alarm information transmitting module 333; the training module 32 is connected to the encoder 331, the encoder 331 is connected to the alarm information generating module 332, the alarm information generating module 332 is connected to the alarm information transmitting module 333, and the alarm information transmitting module 333 is connected to the intervention module 14; wherein, the encoder 331 is used to receive the sensitive information and audio classification results of the training module 32 and generate a corresponding message code and send it to the alarm information generating module 332, the alarm information generating module 332 is used to generate an alarm message with a message code after receiving the message code, and the alarm information transmitting module 333 is used to send the alarm message with a message code to the intervention module 14. The intervention module 14 includes an alarm information receiving module 141, a decoder 142, an interruption intervention module 143, a reminder intervention module 144 and a keyword silencing module 145; the alarm information transmitting module 333 is connected to the alarm information receiving module 141, the alarm information receiving module 141 is connected to the decoder 142, and the decoder 142 is respectively connected to the interruption intervention module 143, the reminder intervention module 144 and the keyword silencing module 145; wherein, the alarm information receiving module 141 is used to receive the alarm information with message coding from the alarm information transmitting module 333 and send it to the decoder 142, the decoder 142 is used to parse the message coding and start the interruption intervention module 143, the reminder intervention module 144 or the keyword silencing module 145 according to the message coding, the interruption intervention module 143 is used to cut off the call of the user terminal, the reminder intervention module 144 is used to send a text warning or insert a voice to the user terminal, and the keyword silencing module 145 is used to silence sensitive words in the communication content of the user terminal. Therefore, the intervention actions initiated by the intervention module 14 include cutting off the call, issuing warnings, interrupting, and silencing sensitive words on the communication content of the user terminal. The encoder 331 in the alarm module 33 and the decoder 142 in the intervention module 14, as well as the alarm information transmitting module 333 in the alarm module 33 and the alarm information receiving module 141 in the intervention module 14 are matched with each other to ensure the correct transmission of the alarm information and the accuracy of the decoding and encoding, and also improve security. When the AI ​​voice training and recognition platform detects sensitive information, it can identify what kind of sound the current audio is, or what state or scene the sound is, send the corresponding category of alarm information to the communication server, and intervene in time. It can be used to identify risk information such as sensitive words, cries for help, abnormal sounds, etc. that appear in multiple business scenarios, and initiate corresponding intervention actions to achieve the purpose of purifying the conversation environment and handling accidents in a timely manner.

[0027] Preferably, the AI ​​voice training and recognition platform 3 also includes a database module 34, which is connected to the training module 32. The database module 34 is used to store sensitive word data sets and audio classification data sets, and provide model training data of training sets and test sets for the training module 32. The AI ​​voice training and recognition platform 3 has a real-time voice transcription interface using a websocket protocol connection method, which can achieve the acquisition of recognition results while uploading audio, and recognize the audio stream as text in real time. The speech recognition model and the sound classification model are DeepASR, a speech recognition system based on PaddlePaddle Fluid and Kaldi. DeepASR uses the Fluid framework to complete the configuration and training of the acoustic model in speech recognition, and integrates Kaldi's decoder to achieve fast and large-scale training of the acoustic model, and uses Kaldi to complete complex speech data preprocessing and the final decoding process. The trained speech recognition model and the sound classification model are deployed on a private CPU / GPU server, and the model is used in an intranet or off-network environment to ensure data privacy. The model can also be published as an API and used by calling the model.

[0028] Preferably, the communication server 1, the media resource control server 2 and the AI ​​speech training and recognition platform 3 are connected through real-time media streaming. The communication server 1 communicates with the media resource control server 2 through the SIP protocol. The MRCP client 11 includes a SIP protocol stack and an MRCP protocol stack, wherein the MRCP protocol stack of the MRCP client 11 is used to call the API interface of the media resource control server 2, and the API interface creates a SIP dialog through the SIP protocol stack of the MRCP client 11 and carries the information of the media resource control server 2; the SIP protocol stack of the MRCP client 11 is used to initialize a media session for the media resource control server 2 through RTP, and create a control session for the media resource control server 2 through the MRCP protocol stack of the MRCP client 11. The media resource control server 2 also includes the MRCP protocol stack and the SIP protocol stack, and the media resource control server 2 includes various media resources, such as speech recognition, speech synthesis, speech recording, speaker verification, and voiceprint matching.

[0029] The above-mentioned real-time intercom intervention and alarm platform based on AI technology can be applied to live audio. The live broadcast device is connected to the user agent 12 of the communication server 1. The communication server 1 sends the live broadcast room audio to the media resource control server 2. The media resource control server 2 processes the audio into text and sends it to the AI ​​voice training and recognition platform 3. The AI ​​voice training and recognition platform 3 detects whether there are sensitive words in the text of the live broadcast room audio. The sensitive words can be muted, and an alarm message can be sent to the communication server 1 for silencing, or the live broadcast can be cut off and an alarm message can be sent to the live broadcast room, which saves manpower supervision costs, ensures the safety of the live broadcast room content, and purifies the network environment; it can also be applied to identify the content of the conversation and intervene in the occurrence of accidents in a timely manner; it can also be applied to public places such as schools and banks, and the AI ​​voice training and recognition platform 3 can identify the audio content and handle accidents in a timely manner.

[0030] It should be noted that the above invention content and specific implementation methods are intended to demonstrate the practical application of the technical solution provided by the present invention and should not be interpreted as limiting the scope of protection of the present invention. Those skilled in the art can make various modifications, equivalent substitutions or improvements within the spirit and principles of the present invention. The scope of protection of the present invention shall be subject to the attached claims.

Claims

1. A real-time intercom intervention and alarm platform based on AI technology, characterized in that: It includes a communication server (1), a media resource control server (2) and an AI voice training and recognition platform (3), wherein: The communication server (1) is electrically and signal-connected to the media resource control server (2), and the communication server (1) comprises an MRCP client (11), a user agent (12), a session communication component (13) and an intervention module (14); wherein the user agent (12) is used to access a plurality of user terminals, the session communication component (13) is used to obtain the communication content of the user terminals in real time and convert it into an audio media stream, and the MRCP client (11) is used to pull the audio media stream in real time and send it to the media resource control server (2); The media resource control server (2) is electrically and signal-connected to the AI ​​voice training and recognition platform (3), and the media resource control server (2) is used to convert the audio media stream into text content and send it to the AI ​​voice training and recognition platform (3); The AI ​​voice training and recognition platform (3) is electrically and signal-connected to the communication server (1), and the AI ​​voice training and recognition platform (3) comprises a voice recognition engine (31), a training module (32) and an alarm module (33); wherein the voice recognition engine (31) is used to receive text content from the media resource control server (2), the training module (32) is used to identify sensitive information involved in the text content through a voice recognition model, and to classify audio involved in the text content through a sound classification model, and the alarm module (33) is used to generate an alarm message with a message code of a corresponding category and send it to the communication server (1); the intervention module (14) in the communication server (1) initiates a corresponding intervention action according to the alarm message with the message code; The alarm module (33) comprises an encoder (331), an alarm information generation module (332) and an alarm information transmission module (333); the training module (32) is connected to the encoder (331), the encoder (331) is connected to the alarm information generation module (332), the alarm information generation module (332) is connected to the alarm information transmission module (333), and the alarm information transmission module (333) is connected to the intervention module (14); wherein the encoder (331) is used to receive the sensitive information and audio classification results of the training module (32) and generate a corresponding message code and send it to the alarm information generation module (332), the alarm information generation module (332) is used to generate an alarm message with a message code after receiving the message code, and the alarm information transmission module (333) is used to send the alarm message with the message code to the intervention module (14); The intervention module (14) comprises an alarm information receiving module (141), a decoder (142), an interruption intervention module (143), a reminder intervention module (144) and a keyword silencing module (145); the alarm information transmitting module (333) is connected to the alarm information receiving module (141), the alarm information receiving module (141) is connected to the decoder (142), and the decoder (142) is respectively connected to the interruption intervention module (143), the reminder intervention module (144) and the keyword silencing module (145); wherein the alarm information receiving module (141) is connected to the alarm information receiving module (141) and the keyword silencing module (145); The invention relates to a method for receiving an alarm message with a message code from an alarm message transmitting module (333) and sending the alarm message to a decoder (142). The decoder (142) is used to parse the message code and start an interruption intervention module (143), a reminder intervention module (144) or a keyword silencing module (145) according to the message code. The interruption intervention module (143) is used to cut off the call of the user terminal. The reminder intervention module (144) is used to send a text warning or insert a voice to the user terminal. The keyword silencing module (145) is used to silence sensitive words in the communication content of the user terminal.

2. The real-time intercom intervention and alarm platform based on AI technology as claimed in claim 1, characterized in that: The media resource control server (2) comprises a main server (21) and a plurality of slave servers (22). The MRCP client (11) communicates with the main server (21), and the main server (21) communicates with the plurality of slave servers (22), so that the MRCP client (11) sends the IP address and port number of the user terminal to the main server (21), and the main server (21) controls the idle slave servers (22) to establish a communication connection with the MRCP client (11).

3. The real-time intercom intervention and alarm platform based on AI technology as claimed in claim 1, characterized in that: The speech recognition engine (31) comprises a word segmentation module (311) and a semantic analysis module (312); the word segmentation module (311) is used to divide the text content into word vector sets according to the word segmentation set and transmit the word vector sets to the semantic analysis module (312); the semantic analysis module (312) is used to perform semantic analysis on the word vector sets, preliminarily determine the classification category corresponding to the word vector sets, and transmit the word vector sets to the training module (32).

4. The real-time intercom intervention and alarm platform based on AI technology as claimed in claim 1, characterized in that: The AI ​​speech training and recognition platform (3) includes a database module (34), which is used to store sensitive word data sets and audio classification data sets, and provide model training data of training sets and test sets for the training module (32).

5. The real-time intercom intervention and alarm platform based on AI technology as claimed in claim 1, characterized in that: The communication server (1), the media resource control server (2) and the AI ​​voice training and recognition platform (3) are connected via real-time media streaming.

6. The real-time intercom intervention and alarm platform based on AI technology as claimed in claim 1, characterized in that: The communication server (1) communicates with the media resource control server (2) via the SIP protocol.

7. The real-time intercom intervention and alarm platform based on AI technology as claimed in claim 1, characterized in that: The speech recognition model and the sound classification model are deployed on a private CPU / GPU server.

8. The real-time intercom intervention and alarm platform based on AI technology as claimed in claim 1, characterized in that: The speech recognition model and the sound classification model are the speech recognition system DeepASR based on PaddlePaddle Fluid and Kaldi.

Images