Video conference security access method oriented to IMS exchange network multi-scene

By building an intelligent communication service platform and combining AI-powered intelligent threat detection, IoT security management, and encryption technology, the secure access and data protection issues of the IMS video conferencing system in multiple scenarios have been resolved, achieving an efficient and secure video conferencing environment.

CN118921355AActive Publication Date: 2024-11-08STATE GRID GRID GANSU ELECTRIC POWER CO QINGYANG POWER SUPPLY CO

Patent Information

Application Number
CN202411301220.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-18
Publication Date
2024-11-08
Estimated Expiration
2044-09-18

AI Technical Summary

Technical Problem

IMS-based video conferencing systems face security threats such as unauthorized access, data breaches, and denial-of-service attacks. In particular, in multi-scenario applications supported by 5G and IoT technologies, traditional security measures are difficult to fully cover, increasing the complexity and difficulty of system security management.

Method used

Artificial intelligence is used for intelligent threat detection, IoT device security management, and encryption technology to protect data transmission. An intelligent communication service platform is built, which combines audio security access mechanisms such as speech recognition STT and text-to-speech TTS, as well as image recognition mechanisms based on deep learning, to achieve identity authentication and data encryption.

Benefits of technology

It improves the security and reliability of video conferencing systems, provides secure and flexible communication services, enhances user experience and system security capabilities, and ensures the confidentiality and integrity of data transmission.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118921355A_ABST
    Figure CN118921355A_ABST
Patent Text Reader

Abstract

The invention provides an IMS (IP Multimedia Subsystem) exchange network-oriented multi-scene video conference security access method, which comprises the following steps of: 1, constructing an intelligent communication service platform based on an ICT (Information Communication Technology) and IoT (Internet of Things) fusion architecture of an IP multimedia subsystem IMS; step 2, establishing an IP multimedia subsystem IMS network audio security access mechanism based on speech recognition STT and text-to-speech TTS in the intelligent communication service platform; and step 3, establishing an IP multimedia subsystem IMS network image recognition mechanism based on deep learning in the platform. According to the invention, the accessibility, the security and the user experience of a video conference and voice communication system are effectively improved, and a security identification function for audios is realized. According to the IP multimedia subsystem IMS network image recognition mechanism based on deep learning, the accessed image can be effectively and safely recognized.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the fields of information and communication technology, Internet of Things technology and artificial intelligence, and in particular to a video conference secure access method in multiple scenarios of an IMS switching network. Background Art

[0002] In recent years, with the rapid development and widespread application of the new generation of information and communication technology (ICT), Internet of Things (IoT) and artificial intelligence (AI), the potential of IMS networks has been further released and enhanced. As a typical communication application scenario supported by IMS networks, video conferencing systems have played an important role in many fields such as enterprise collaborative work, distance education, online medical care and personal social networking.

[0003] The study found that the video conferencing system based on IP Multimedia Subsystem (IMS) has strong network adaptability, excellent audio and video transmission quality, and flexible interactive functions, meeting the user's efficient and convenient communication needs. However, with the continuous expansion of application scenarios and the sharp increase in the number of users, the video conferencing system faces security challenges from multiple levels of network, software and hardware. Unauthorized access, data leakage, denial of service attack (DDoS), side-channel attack and other security threats are frequent, which brings huge risks to user privacy protection and data security. In addition, with the deep integration of video conferencing systems and Internet of Things (IoT) devices, the security boundary of video conferencing systems has been greatly expanded. Each IoT device connected to the system may become a potential entry point for security attacks, bringing new challenges to security management. The diversity of IoT devices and the increase in system access points make it difficult for traditional security protection measures to fully cover, increasing the complexity and difficulty of system security management. Therefore, in the context of the rapid development of information technology today, effectively ensuring the secure access and data protection of IMS-based video conferencing systems is not only a need for technological development, but also an urgent need of the market and users. In the future, with the continuous advancement of technology and the increasing complexity of security threats, the security protection measures of video conferencing systems will also need to be continuously updated and optimized to meet new security challenges.

[0004] The integration of artificial intelligence technology has brought revolutionary improvements to IMS-based video conferencing systems. Through AI technology, video conferencing systems can achieve automated conference content management, intelligent identification of participants, real-time monitoring and optimization of conference quality, and automatic detection and response to security threats, significantly improving the efficiency, experience and security of video conferencing. In addition, the popularization of 5G technology provides a more stable high-speed network connection for video conferencing, making high-definition, ultra-high-definition and even 360-degree panoramic video conferencing possible, greatly improving the immersion and interactivity of the conference. At the same time, IoT technology enables video conferencing systems to seamlessly connect with more smart devices and sensors, realizing new functions such as environmental control, intelligent assistance and virtual interaction, further broadening the application field of video conferencing. IP multimedia subsystem is a new generation of communication exchange technology standard for telecommunications systems that promotes the integration of mobile and fixed networks, and provides an architectural framework for implementing IP multimedia and voice services. The IMS network can integrate a variety of communication services, including voice, video, SMS, etc., to ensure quality of service (QoS) and meet a variety of application scenario requirements. Based on IMS, these technologies will be integrated and deepened with video conferencing services. Video conferencing will present more diverse application scenarios and more intelligent and efficient service models, bringing more convenience and innovative experience to people's work and life.

[0005] Based on the above background, in order to effectively improve the security protection capability of the video conferencing system and maintain the normal meeting order and confidentiality of conference information, it is necessary to propose a video conferencing security access method for multiple scenarios in the IMS switching network. Summary of the invention

[0006] Purpose of the invention: With the rapid development of information and communication technology, the Internet of Things and artificial intelligence, video conferencing systems based on IP Multimedia Subsystem (IMS) have been widely used. However, this widespread application also makes it face challenges brought about by frequent security threats such as unauthorized access, data leakage and denial of service attacks (DDoS). The present invention aims to solve the security access and data protection problems faced by video conferencing services for multi-scenario applications of IMS networks with the support of 5G and IoT technologies. In order to improve the security and service efficiency of video conferencing and ensure the safe, stable and efficient operation of video conferencing systems, the present invention proposes a secure access method for video conferencing in multiple scenarios of IMS switching networks, including strategies such as using AI for intelligent threat detection, IoT to strengthen terminal security management, and encryption technology to protect data transmission.

[0007] The present invention specifically provides a video conference security access method for multiple scenarios in an IMS switching network. By comprehensively using multiple advanced technologies such as artificial intelligence, data encryption, identity authentication, and secure access to IoT devices, the security of the video conference system is effectively improved, providing users with a safe and reliable conference environment.

[0008] The method of the present invention comprises the following steps:

[0009] Step 1: Build an intelligent communication service platform based on the ICT (Information and Communication Technology, ICT) and IoT (Internet of Things, IoT) converged architecture of the IP Multimedia Subsystem IMS;

[0010] Step 2, establishing an IP multimedia subsystem IMS network audio security access mechanism based on speech recognition STT (Speech-to-Text, STT) and text-to-speech TTS (Text-to-Speech, TTS) in the intelligent communication service platform;

[0011] Step 3: Establish an IP Multimedia Subsystem IMS network image recognition mechanism based on deep learning in the platform.

[0012] Step 1 includes: the intelligent communication service platform includes an application layer, a network layer, a device layer and a business capability layer;

[0013] The application layer is responsible for comprehensively processing the interaction, business logic and service interface of various applications of users, including IMS applications, ICT applications, Tas applications, etc.

[0014] The network layer integrates multiple network protocols, connects multiple networks such as IMS, PSTN, PLMN and IP, realizes heterogeneous and multi-network data transmission and network communication, and ensures smooth and reliable data transmission between different layers and different protocols; the network layer uses IMS technology to process call control, session management, network connection and data transmission, and supports and integrates multiple network protocols and communication standards;

[0015] The device layer is responsible for sensing the environment and collecting data, providing the required information to the application layer;

[0016] The business capability layer integrates ICT and IoT technologies, and provides core business logic and service capabilities, including data processing, storage, analysis, and execution of business rules;

[0017] The intelligent communication service platform is used to provide speech recognition STT and text-to-speech TTS functions. Users can use voice input devices to control IoT devices through the speech recognition STT function, and the application layer notifies users of IoT sensor information through the text-to-speech TTS function.

[0018] In step 1, the smart communication service platform is connected to the CHT IoT smart platform, which provides a set of open application programming interfaces.

[0019] Step 2 includes the following steps:

[0020] Step 2-1, converting the user's voice input into text information through the voice recognition STT function;

[0021] Step 2-2, converting the text information back into speech and outputting it to the user through the text-to-speech (TTS) function;

[0022] Step 2-3, perform security authentication: Combined with biometric technology, the intelligent communication service platform can identify and verify the user's voiceprint to ensure that only authorized users can access the system and prevent malicious information systems from accessing.

[0023] Step 2-1 includes:

[0024] Step 2-1-1, preprocess the input speech signal, including removing noise and unnecessary information, and performing standardization processing; the unnecessary information refers to the part of the speech signal that interferes with speech recognition or other processing processes. For example: background noise (such as wind, keyboard tapping), echo and reverberation, silent part (part without speech content) and non-language sound (such as cough, laughter), etc. Standardization processing refers to a series of transformations of the speech signal for subsequent processing. The standardization processing includes: amplitude normalization, adjusting the amplitude of the speech signal so that it fluctuates within a certain range to avoid recognition difficulties due to excessive or insufficient signal strength, so as to facilitate subsequent digital processing; spectrum standardization, converting the speech signal into a spectrum representation through Mel frequency cepstral coefficients, and standardizing the spectrum to reduce the recognition difficulties caused by differences in timbre or speaking speed of different speakers. The technologies used in the preprocessing step are all in the existing field of speech processing.

[0025] Step 2-1-2, extracting features: extracting Mel frequency cepstral coefficients and Mel filter bank features from the preprocessed speech signal, and normalizing the features;

[0026] The step of extracting Mel frequency cepstral coefficients and Mel filter bank features from the preprocessed speech signal specifically includes:

[0027] Extract Mel-frequency cepstral coefficients:

[0028] Framing and windowing: the preprocessed speech signal is divided into small segments, each frame is 20 to 40 milliseconds, and a Hamming window is applied to each frame to reduce spectrum leakage;

[0029] Fast Fourier Transform (FFT), perform fast Fourier transform on each frame signal y(t) to obtain the frequency domain signal X(t,w), the formula is:

[0030]

[0031] Where w(t) is the window function at time t, which is used to localize the audio signal in each small period of time; e is a natural constant, w is the frequency; dτ represents the integral, and j is the imaginary unit;

[0032] The power spectrum P(t,w) of the frame signal y(t) at time t is calculated using the following formula:

[0033] P(t,w)=|X(t,w) 2 (2)

[0034] The obtained fast Fourier transform spectrum is input into a set of Mel filter banks M(m,w), and the output is the Mel frequency energy spectrum

[0035] Perform logarithmic operation on the Mel frequency energy spectrum and record the result as S(t,m):

[0036]

[0037] Using discrete cosine transform (DCT), the logarithmic Mel energy spectrum is subjected to discrete cosine transform, and the first 13 coefficients are extracted to obtain a set of Mel frequency cepstrum coefficients;

[0038] Extract Mel filter bank features: directly convert the Mel frequency energy spectrum as Mel filter bank features;

[0039] The standardization of features includes:

[0040] Mean normalization: Calculate the mean of the Mel filter bank feature data set and subtract the mean from each feature to make the mean of the feature data zero;

[0041] Standard deviation normalization: Calculate the standard deviation of the feature data set and divide each feature by the standard deviation to make the standard deviation of the feature data equal to one;

[0042] Step 2-1-3, establish a convolutional neural network acoustic model, and map the convolutional neural network acoustic model to the sound unit of the language;

[0043] The convolutional neural network acoustic model includes an input layer, a convolution layer, a pooling layer, a fully connected layer and an output layer;

[0044] The input layer receives the features extracted in step 2-1-2, and the features are input into the convolutional neural network acoustic model in the form of a two-dimensional matrix, where one dimension represents the time step and the other dimension represents the frequency feature;

[0045] The convolution layer performs a convolution operation on the input two-dimensional matrix through a convolution kernel, extracts local time-frequency domain features, and generates a feature map, where different convolution kernels can capture features in different time and frequency ranges; the operation of the lth convolution layer is expressed as:

[0046]

[0047] in, is the kth feature map of the lth layer, It is the convolution kernel from the i-th feature map to the k-th feature map; is the bias term, f is the activation function;

[0048] The pooling layer downsamples the feature map generated by the convolution layer through a maximum pooling operation, reduces the spatial dimension of the feature, and retains the most important information to generate a two-dimensional feature map;

[0049] The fully connected layer flattens the two-dimensional feature map output by the pooling layer into a one-dimensional vector, and further abstracts the features, and finally maps the features to higher-level language features. k , the formula is:

[0050]

[0051] in, is the jth element of the flattened feature vector output by the last convolutional layer; and They represent the weight and bias of the fully connected layer respectively, and g is the activation function;

[0052] The output layer uses the Softmax function to convert the final output of the convolutional neural network acoustic model into a probability distribution. According to the probability distribution, the sound unit with the highest probability is selected as the final prediction result. The formula is:

[0053]

[0054] Step 2-1-4, establish a neural network language model, and learn the co-occurrence probability of words based on text data; a language model is a tool for predicting the probability of word occurrence in a sentence or text, and is responsible for understanding the vocabulary and grammatical structure in speech. A neural network language model specifically refers to a model built using a neural network. A neural network is an algorithmic structure that mimics the way the human brain works and can learn complex patterns and relationships from large amounts of data. The co-occurrence probability refers to the probability that two or more words appear together in a given set of text. The present invention uses an N-gram model to learn the co-occurrence probability of words based on a large amount of text data;

[0055] Step 2-1-5, the intelligent communication service platform also includes a decoder and a semantic analysis system. The decoder uses Bayesian decision theory to optimize search efficiency, combines the output of the convolutional neural network acoustic model and the neural network language model, and generates the final text output; the semantic analysis system performs semantic analysis on the text data through the neural network language model to check whether the text language is legal and safe.

[0056] Step 2-3 includes: when caller A calls callee B, if the call status is responding, the intelligent communication service platform converts the voice into text information for analysis, the semantic analysis system will check the context of the conversation, and if the result of the conversation is illegal, the semantic analysis system will notify the callee B; otherwise, the conversation ends.

[0057] In step 2-1 and step 2-2, data in the speech recognition STT and text-to-speech TTS processing process is encrypted, including:

[0058] Data encryption transmission: When data is transmitted from user devices to the intelligent communication service platform, the transmission link is encrypted through the Transport Layer Security (TLS) protocol to prevent data from being eavesdropped and intercepted during transmission;

[0059] Data encryption storage, using Advanced Encryption Standard (AES) to encrypt and store users' voice and text data to ensure data confidentiality;

[0060] End-to-end encryption uses end-to-end encryption (E2EE) technology throughout the data transmission process to ensure that the entire process from user input to final output is carried out in an encrypted environment and only the communicating parties can decrypt it;

[0061] Data integrity verification, using Hash-based Message Authentication Code (HMAC) to verify the integrity of the data to ensure that the data has not been tampered with during transmission;

[0062] Together, these encryption technologies provide comprehensive security for the IP Multimedia Subsystem IMS system based on speech recognition STT and text-to-speech TTS.

[0063] Step 3 includes:

[0064] Step 3-1, capturing images: capturing image data containing legal and illegal information in multimedia communications, building a local training data set, and performing preliminary formatting on the data set;

[0065] Step 3-2, data preprocessing: preprocess the image data captured in step 3-1, including resizing, format conversion, and normalization to meet the input requirements of the YOLOv5 model. For resizing, resize the image to 640x640 pixels, and keep the aspect ratio of the image by padding the edges during adjustment; for format conversion, convert the formats of all images to a three-channel format in RGB format; for normalization, convert the image data to a floating-point type and divide it by 255 so that the image pixel values ​​input to the YOLOv5 model are scaled to the range of [0,1]; for real-time video stream data, frame extraction and buffer management are also required; for frame extraction, extract image frames from the video stream at a fixed frame rate, where the frame rate is set to 5fps; for buffer management, use a circular buffer to manage frames to ensure that new frames do not overwrite frames that have not been processed during processing;

[0066] Step 3-3, YOLOv5 detection: The preprocessed image data is fed into the trained YOLOv5 model for object recognition. The YOLOv5 model predicts the objects and bounding boxes in the image and classifies each object in a single forward pass.

[0067] Step 3-4: Process and analyze the results output by the YOLOv5 model.

[0068] Step 3-3 includes:

[0069] The input of the YOLOv5 model uses Mosaic data enhancement and adaptive anchor box calculation, and the optimal anchor box value is adaptively calculated according to different training sets during each training. The different training sets include public data sets and constructed local training data sets. Different training sets mean that different data sets or different divisions of data sets (such as the division of training sets and validation sets) can be used during each training to improve the generalization ability of the YOLOv5 model. The local data set refers to a data set constructed locally that is labeled for legal or illegal data, that is, the data set for training the YOLOv5 model constructed by capturing image data in step 3-1;

[0070] The focus module of the backbone network of the YOLOv5 model performs the following operations:

[0071] Take a value from every other pixel in an image to get four independent feature layers;

[0072] Four independent feature layers are stacked to convert width and height information into channel information, increasing the number of input channels by 4 times, from the original red, green and blue three-channel mode to 12 channels;

[0073] Perform a convolution operation on the obtained new image to obtain a two-fold downsampled feature map without information loss;

[0074] The Conv module in the YOLOv5 model includes the convolutional layer Conv2d, the normalization Batch Normalization and the activation function SiLU;

[0075] The C3 module in the YOLOv5 model is a residual structure that enables the entire network to achieve richer gradient combinations, while reducing the number of parameters and calculations, and improving the training speed of the YOLOv5 model;

[0076] The SPPF (Spatial Pyramid Pooling Fast) module in the YOLOv5 model can expand the receptive field, allowing the YOLOv5 model to obtain global feature information;

[0077] The Neck in the YOLOv5 model uses a path aggregation network to fuse the deep semantic information extracted by the backbone network with the shallow detail information, and performs upsampling through the UpSample operation to convert the W×H image into 2W×2H, where W and H represent the width and height of the image respectively; then the Concat operation is performed, which means splicing feature maps of the same size;

[0078] The output Prediction of the YOLOv5 model has three branches, which are used for the detection output of large, medium and small targets respectively.

[0079] Steps 3-4 include:

[0080] Step 3-4-1, data collection: The image capture module integrated in each node of the IP multimedia subsystem IMS network is responsible for collecting the incoming multimedia content. The multimedia content undergoes preliminary screening (using a combination of manual and automated screening. The preliminary screening includes content type filtering, image quality detection, deduplication processing, and time window screening. Content type filtering, through content recognition algorithm, automatically filters out non-target content; image quality detection, using edge detection algorithm to automatically detect and filter out content that does not meet the image quality standards; deduplication processing, automatically identifying and deleting duplicate image frames; time window screening, automatically filtering data based on timestamps, and only processing images within a specific time period), and the relevant image data is marked and transmitted to the central processing server in the intelligent communication service platform; the central processing server is responsible for centralized processing and management of data flows in the IMS network;

[0081] Step 3-4-2, data transmission: the image data is transmitted to the data processing center in the intelligent communication service platform through the secure channel of the IP multimedia subsystem IMS network, and encryption and integrity verification are performed during the transmission process;

[0082] Step 3-4-3, real-time processing: In the data processing center, the YOLOv5 model processes and analyzes the image data in real time or near real time. "Near real time" usually refers to a relatively short delay in data processing and response, but it is not exactly the concept of "real time". "Near real-time" processing usually involves a time delay from data acquisition to processing and output of results, which can be quantified by the delay time. In the intelligent communication service platform, if the YOLOv5 model can complete the processing and return the results within 100ms to 1 second after the image data is transmitted to the data processing center, such a processing speed can usually be called "near real time".

[0083] Beneficial effects: The secure access method for video conferencing in multiple scenarios of the IMS switching network of the present invention effectively improves the security of the video conferencing system by adopting AI for intelligent threat detection, IoT to strengthen terminal security management, and encryption technology to protect data transmission and other strategies, providing users with a safe and reliable conference environment. The present invention builds an intelligent communication service platform based on the ICT and IoT fusion architecture of the IP multimedia subsystem IMS. The intelligent communication service platform integrates the communication capabilities of IMS as well as ICT and IoT technologies to provide users with more flexible and intelligent communication services that can meet the needs of various application scenarios. The IP multimedia subsystem IMS network audio security access mechanism based on speech recognition STT and text to speech TTS of the present invention effectively improves the accessibility, security and user experience of video conferencing and voice communication systems, and realizes the security recognition function of audio. The IP multimedia subsystem IMS network image recognition mechanism based on deep learning of the present invention can effectively and securely identify accessed images. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments, and the above and / or other advantages of the present invention will become more clear.

[0085] Figure 1 This is a system architecture diagram of the intelligent communication service platform of the present invention.

[0086] Figure 2 The present invention is an access flow chart of the IMS network audio security access mechanism based on STT and TTS.

[0087] Figure 3 This is a flow chart of the present invention for preventing malicious information system access.

[0088] Figure 4 This is a flowchart of YOLOv5 detection of the present invention.

[0089] Figure 5 This is a structural diagram of the C3 module of the present invention.

[0090] Figure 6 It is a structural diagram of the SPPF module of the present invention. DETAILED DESCRIPTION

[0091] In an embodiment of the present invention, a method for secure access to video conferences in multiple scenarios of an IMS switching network is specifically provided, including an IMS-based converged network architecture, an IMS network audio security access mechanism based on STT and TTS, and an IMS network image recognition mechanism based on deep learning.

[0092] 1. ICT and IoT converged architecture based on IP multimedia subsystem IMS

[0093] Based on the ICT and IoT convergence architecture of IMS, the present invention builds an intelligent communication service platform. The platform integrates the communication capabilities of IMS and ICT and IoT technologies, providing users with more flexible and intelligent communication services and supporting the needs of various application scenarios.

[0094] Reference Figure 1 The intelligent communication service platform provided by the present invention includes an application layer, a business capability layer, a network layer and a device layer. Figure 1 In the figure, CS stands for conferencing service, STT / TTS stands for speech-to-text / text-to-speech service, IP stands for Internet, MS stands for media service, IMS stands for IP-based multimedia subsystem, PLMN stands for public land mobile network, TAS stands for telephone AP service, PSTN stands for public switched telephone network, and CRS stands for call recording service.

[0095] The application layer is responsible for processing user interaction and business logic, providing application programs and service interfaces, and realizing specific intelligent services and application functions. Among them, IMS APs are application program interfaces for realizing and managing multimedia services based on IMS networks, ICT APs are application program interfaces for managing and integrating information and communication technology services, and IoT APs are application program interfaces for connecting and managing IoT devices.

[0096] The business capability layer integrates ICT and IoT technologies, provides core business logic and service capabilities, and provides the required services by processing application layer requests.

[0097] The network layer is responsible for data transmission and network communication, ensuring smooth and reliable data transmission between different layers; the network layer uses IMS technology to handle communication tasks such as call control, session management, network connection and data transmission, and supports multiple network protocols and communication standards; PSTN is the public switched telephone network, a circuit switching network for voice communication, and PLMN refers to the public land mobile network, which provides mobile communication support and allows voice and data communication through cellular networks;

[0098] The device layer includes hardware devices that interact with the intelligent communication service platform, which is responsible for data collection and transmission and provides the required information to the application layer;

[0099] In order to support users to develop IMS-based IoT services, the intelligent communication service platform is connected to the CHT IoT intelligent platform. The CHT IoT intelligent platform provides a set of open application programming interfaces (APIs), called Tas APIs. Using TasAPI, users can independently develop customized applications to meet their specific needs. Through the intelligent communication service platform, business functions such as network audio security access can be realized. Its diversified open interfaces can be customized to add a variety of business needs.

[0100] 2. IMS network audio security access mechanism based on STT and TTS Establishing an IMS network audio security access mechanism based on STT and TTS on the intelligent communication service platform can effectively improve the accessibility, security and user experience of video conferencing and voice communication systems. Figure 2 The access flow chart of the IMS network audio security access mechanism based on STT and TTS is shown. The following are the specific implementation steps of the mechanism:

[0101] Step 1: Speech recognition (STT), converting the user's voice input into text information. Through deep learning and neural network technology, the STT system can accurately understand and transcribe the user's voice instructions or queries. Among them, the speech recognition STT process includes the following steps:

[0102] Preprocess the input speech signal, including removing noise and unnecessary information, and performing standardization.

[0103] Feature extraction, according to an embodiment of the present invention, extracting Mel frequency cepstral coefficients and Mel filter bank features from the preprocessed speech signal and normalizing the features;

[0104] Specifically, let y(t) be the speech signal, and the short-time Fourier transform (STFT) be:

[0105]

[0106] Where w(t) is a window function used to localize the audio signal in each small time segment.

[0107] The power spectrum P is calculated from the square of the modulus of the STFT:

[0108] P(t,w)=|X(t,w) 2 (2)

[0109] The power spectrum P is then converted to a Mel spectrum S, which is obtained by applying a Mel filter bank to the power spectrum and taking its logarithm:

[0110]

[0111] Where M(m,w) is a Mel filter bank, which is used to convert the frequency to the Mel scale to make the spectrum more consistent with human auditory characteristics.

[0112] The standardization of the extracted features includes: mean normalization, calculating the mean of the feature data set, and subtracting the mean from each feature to make the mean of the feature data zero; standard deviation normalization, calculating the standard deviation of the feature data set, and dividing each feature by the standard deviation to make the standard deviation of the feature data one.

[0113] Establishing a convolutional neural network acoustic model. The present invention adopts the Faster-RCNN algorithm to establish the acoustic model. The acoustic model is used to understand the extracted audio features and map them to the sound units of the language;

[0114] For a specific convolutional layer, its operation can be expressed by the following formula:

[0115]

[0116] in, is the k-th feature map of layer l, It is the convolution kernel from the i-th feature map to the k-th feature map. is the bias term, and f is the activation function, such as ReLU;

[0117] The feature map is further processed through the fully connected layer to obtain the final classification result:

[0118]

[0119] in, is the jth element of the flattened feature vector output by the last convolutional layer. and They represent the weight and bias of the fully connected layer respectively, and g is the activation function, which is usually a softmax function in the output layer for multi-class classification:

[0120]

[0121] Build a neural network language model and use the N-gram model to learn the co-occurrence probability of words based on a large amount of text data;

[0122] The decoder uses Bayesian decision theory to optimize search efficiency, combines the outputs of the acoustic model and language model, and generates the final text output.

[0123] Step 2: Text to Speech (TTS), converts text information back into speech output for the user. This allows the system to interact with the user in the form of speech, provide feedback or execute commands.

[0124] Step 3: Security authentication, combined with biometric technology, such as voiceprint recognition, the system can identify and verify the user's voiceprint to realize the user's identity and ensure that only authorized users can access the system. Combining multiple authentication methods such as passwords, biometric technology, and electronic tokens can further enhance the security of the IMS system.

[0125] Reference Figure 3 ,The specific process of preventing malicious information systems from accessing is:

[0126] When caller A calls callee B. If the call status is responding, the STT server converts the voice into text information for analysis;

[0127] The semantic analysis system will check the context of the conversation. If the result of the conversation is illegal, the semantic analysis system will notify the callee B; otherwise, the conversation ends.

[0128] Step 4: Encrypted communication: To ensure the security of data transmission during STT and TTS processing, encrypt the data to prevent it from being intercepted or tampered with during transmission.

[0129] 3. IMS network image recognition mechanism based on deep learning

[0130] The IMS network image recognition system based on YOLO includes the following steps:

[0131] Step 1: Capture the Image

[0132] Capture image data containing legal and illegal information in the IMS network, build a local training data set, and perform preliminary formatting on the data set to facilitate subsequent image recognition analysis.

[0133] Step 2: Data Preprocessing

[0134] The captured image data is preprocessed, including resizing, format conversion, normalization, etc., to meet the input requirements of the YOLO model. For real-time video streaming data, frame extraction and buffer management are also required.

[0135] For resizing, resize the image to 640x640 pixels, and keep the aspect ratio of the image by padding the edges;

[0136] For format conversion, all images are converted to a three-channel RGB format.

[0137] For normalization, the image data is converted to floating point type and divided by 255 so that the image pixel values ​​input to the YOLOv5 model are scaled to the range of [0,1];

[0138] For frame extraction, image frames are extracted from the video stream at a fixed frame rate, where the frame rate is set to 5fps;

[0139] For buffer management, a circular buffer is used to manage frames to ensure that new frames will not overwrite unprocessed frames during processing;

[0140] Step 3: YOLOv5 Detection

[0141] The preprocessed image data is fed into the trained YOLOv5 model for object recognition. Figure 4 , the YOLOv5 detection process includes:

[0142] The input uses Mosaic data enhancement and adaptive anchor box calculation. The optimal anchor box value is adaptively calculated according to different training sets during each training.

[0143] The focus module of the backbone network operates as follows:

[0144] Get a value from every other pixel in an image to get four independent feature layers; stack these four independent feature layers to convert width and height information into channel information, so that the number of input channels increases by 4 times, that is, from the original RGB three-channel mode to 12 channels; perform convolution operation on the obtained new image to obtain a two-fold downsampled feature map without information loss. The Conv module consists of three parts: convolution layer (Conv2d), normalization (Batch Normalization, BN) and activation function (SiLU). Figure 5 The structure diagram of the C3 module is shown. The C3 module is a residual structure that enables the entire network to achieve richer gradient combinations, while reducing the number of parameters and calculations and improving the model training speed. Figure 6 The structure diagram of the SPPF module is shown. The SPPF module can expand the receptive field so that the model obtains global feature information.

[0145] The neck uses a path aggregation network to fuse the deep semantic information extracted by the backbone network with the shallow detail information. The UpSample operation is used for upsampling, converting the W×H image into 2W×2H. The Concat operation is concatenation, which concatenates feature maps of the same size.

[0146] The three branches at the output end (Prediction) are used for the detection output of large, medium and small targets respectively.

[0147] 4. Result processing

[0148] The output of the YOLO model needs to be further analyzed and processed. For example, triggering alarms, logging, or adjusting network configurations based on the recognition results. In addition, the system needs to visualize the results so that network administrators can directly observe and evaluate the image recognition results.

[0149] According to an embodiment of the present invention, in an IMS network, processing of an image data stream includes the following key steps:

[0150] 1) Data Collection

[0151] The image capture module integrated in each node of the IMS network is responsible for collecting incoming multimedia content. After preliminary screening, the relevant image data is marked and transmitted to the central processing server.

[0152] 2) Data transmission

[0153] Image data is transmitted to the data processing center through the secure channel of the IMS network. It is necessary to ensure the security and integrity of the data during transmission, which may involve encryption and integrity verification mechanisms.

[0154] 3) Real-time processing

[0155] In the data processing center, the YOLO model analyzes image data in real time or near real time. The system needs to be optimized to support high concurrency processing and ensure efficient operation even during peak network traffic.

[0156] By deploying a YOLO-based image recognition mechanism in the IMS network, the image content in the network traffic can be monitored in real time to identify potential security threats, illegal content or abnormal behavior, thereby strengthening ISM network security protection and management control and ensuring the legality and security of communication content.

[0157] The present invention provides a method for secure access to video conferences in multiple scenarios of an IMS switching network. There are many methods and ways to implement the technical solution. The above is only a preferred implementation of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the protection scope of the present invention. All components not specified in this embodiment can be implemented by existing technologies.

Claims

1. A video conference secure access method for multiple scenarios in an IMS switching network. It is characterized in that The following steps are involved: Step 1: Build an intelligent communication service platform based on the ICT and IoT convergence architecture of the IP Multimedia Subsystem IMS; Step 2, establishing an IP multimedia subsystem IMS network audio security access mechanism based on speech recognition STT and text-to-speech TTS in the intelligent communication service platform; Step 3: Establish an IP Multimedia Subsystem IMS network image recognition mechanism based on deep learning in the platform.

2. The method according to claim 1, It is characterized in that Step 1 includes: the intelligent communication service platform includes an application layer, a network layer, a device layer and a business capability layer; The application layer is responsible for comprehensively processing the interaction, business logic and service interface of various applications of users; The network layer integrates multiple network protocols, connects IMS, PSTN, PLMN and IP networks, realizes heterogeneous and multi-network data transmission and network communication, and ensures data transmission between different layers and different protocols; the network layer uses IMS technology to process call control, session management, network connection and data transmission, and supports and integrates network protocols and communication standards; The device layer is responsible for sensing the environment and collecting data, providing information to the application layer; The business capability layer integrates ICT and IoT technologies, and provides business logic and service capabilities, including data processing, storage, analysis, and execution of business rules; The intelligent communication service platform is used to provide speech recognition STT and text-to-speech TTS functions. Users can use voice input devices to control IoT devices through the speech recognition STT function, and the application layer notifies users of IoT sensor information through the text-to-speech TTS function.

3. The method according to claim 2, It is characterized in that In step 1, the intelligent communication service platform is connected to the CHT IoT intelligent platform, which provides a set of open application programming interfaces.

4. The method according to claim 3, It is characterized in that Step 2 includes the following steps: Step 2-1, converting the user's voice input into text information through the voice recognition STT function; Step 2-2, converting the text information back into speech and outputting it to the user through the text-to-speech (TTS) function; Step 2-3, perform security authentication: Combined with biometric technology, the intelligent communication service platform can identify and verify the user's voiceprint to ensure that only authorized users can access the system.

5. The method according to claim 4, It is characterized in that Step 2-1 includes: Step 2-1-1, preprocessing the input speech signal, including removing noise and unnecessary information, and performing standardization; Step 2-1-2, extracting features: extracting Mel frequency cepstral coefficients and Mel filter bank features from the preprocessed speech signal, and normalizing the features; The step of extracting Mel frequency cepstral coefficients and Mel filter bank features from the preprocessed speech signal specifically includes: Extract Mel-frequency cepstral coefficients: Framing and windowing: the preprocessed speech signal is divided into small segments and a Hamming window is applied on each frame to reduce spectral leakage; Perform fast Fourier transform on each frame signal y(t) to obtain the frequency domain signal X(t,w), the formula is: Where w(t) is the window function at time t, which is used to localize the audio signal in each small period of time; e is a natural constant, w is the frequency; dτ represents the integral, and j is the imaginary unit; The power spectrum P(t,w) of the frame signal y(t) at time t is calculated using the following formula: P(t,w)=|X(t,w) 2 (2) The obtained fast Fourier transform spectrum is input into a set of Mel filter banks M(m,w), and the output is the Mel frequency energy spectrum Perform logarithmic operation on the Mel frequency energy spectrum and record the result as S(t,m): Using discrete cosine transform, the logarithmic Mel energy spectrum is subjected to discrete cosine transform to obtain a set of Mel frequency cepstrum coefficients; Extract Mel filter bank features: directly convert the Mel frequency energy spectrum as Mel filter bank features; The standardization of features includes: Mean normalization: Calculate the mean of the Mel filter bank feature data set and subtract the mean from each feature to make the mean of the feature data zero; Standard deviation normalization: Calculate the standard deviation of the feature data set and divide each feature by the standard deviation to make the standard deviation of the feature data equal to one; Step 2-1-3, establish a convolutional neural network acoustic model, and map the convolutional neural network acoustic model to the sound unit of the language; The convolutional neural network acoustic model includes an input layer, a convolution layer, a pooling layer, a fully connected layer and an output layer; The input layer receives the features extracted in step 2-1-2, and the features are input into the convolutional neural network acoustic model in the form of a two-dimensional matrix, where one dimension represents the time step and the other dimension represents the frequency feature; The convolution layer performs a convolution operation on the input two-dimensional matrix through a convolution kernel, extracts local time-frequency domain features, and generates a feature map, where different convolution kernels can capture features in different time and frequency ranges; the operation of the lth convolution layer is expressed as: in, is the kth feature map of the lth layer, It is the convolution kernel from the i-th feature map to the k-th feature map; is the bias term, f is the activation function; The pooling layer downsamples the feature map generated by the convolution layer through a maximum pooling operation, reduces the spatial dimension of the feature, and retains the most important information to generate a two-dimensional feature map; The fully connected layer flattens the two-dimensional feature map output by the pooling layer into a one-dimensional vector, and further abstracts the features, and finally maps the features to higher-level language features. k , the formula is: in, is the jth element of the flattened feature vector output by the last convolutional layer; and They represent the weight and bias of the fully connected layer respectively, and g is the activation function; The output layer uses the Softmax function to convert the final output of the convolutional neural network acoustic model into a probability distribution. According to the probability distribution, the sound unit with the highest probability is selected as the final prediction result. The formula is: Step 2-1-4, establish a neural network language model to learn the co-occurrence probability of words based on text data; Step 2-1-5, the intelligent communication service platform also includes a decoder and a semantic analysis system. The decoder uses Bayesian decision theory to optimize search efficiency, combines the output of the convolutional neural network acoustic model and the neural network language model, and generates the final text output; the semantic analysis system performs semantic analysis on the text data through the neural network language model to check whether the text language is legal and safe.

6. The method according to claim 5, It is characterized in that Step 2-3 includes: when caller A calls callee B, if the call status is responding, the intelligent communication service platform converts the voice into text information for analysis, the semantic analysis system will check the context of the conversation, and if the result of the conversation is illegal, the semantic analysis system will notify the callee B; otherwise, the conversation ends.

7. The method according to claim 6, It is characterized in that In step 2-1 and step 2-2, data in the speech recognition STT and text-to-speech TTS processing process is encrypted, including: Data encryption transmission: When data is transmitted from user equipment to the intelligent communication service platform, the transmission link is encrypted through the transport layer security protocol; Data encryption storage, using advanced encryption standards to encrypt and store users' voice and text data; End-to-end encryption uses end-to-end confidentiality technology throughout the entire data transmission process to ensure that the entire process from user input to final output is carried out in an encrypted environment and only the communicating parties can decrypt it; Data integrity verification uses hash message authentication codes to verify the integrity of data to ensure that the data has not been tampered with during transmission.

8. The method according to claim 7, It is characterized in that Step 3 includes: Step 3-1, capturing images: capturing image data containing legal and illegal information in multimedia communications, building a local training data set, and performing preliminary formatting on the data set; Step 3-2, data preprocessing: preprocess the image data captured in step 3-1, including resizing, format conversion, and normalization to meet the input requirements of the YOLOv5 model. For resizing, resize the image to 640x640 pixels, and keep the aspect ratio of the image by padding the edges during adjustment; for format conversion, convert the formats of all images to a three-channel format in RGB format; for normalization, convert the image data to a floating-point type and divide it by 255 so that the image pixel values ​​input to the YOLOv5 model are scaled to the range of [0,1]; for real-time video stream data, frame extraction and buffer management are also required; for frame extraction, extract image frames from the video stream at a fixed frame rate, where the frame rate is set to 5fps; for buffer management, use a circular buffer to manage frames to ensure that new frames do not overwrite frames that have not been processed during processing; Step 3-3, YOLOv5 detection: The preprocessed image data is fed into the trained YOLOv5 model for object recognition. The YOLOv5 model predicts the objects and bounding boxes in the image and classifies each object in a single forward pass. Step 3-4: Process and analyze the results output by the YOLOv5 model.

9. The method according to claim 8, It is characterized in that Step 3-3 includes: The input of the YOLOv5 model uses Mosaic data enhancement and adaptive anchor box calculation. During each training, the optimal anchor box value is adaptively calculated according to different training sets; the different training sets include public data sets and constructed local training data sets; The focus module of the backbone network of the YOLOv5 model performs the following operations: Take a value from every other pixel in an image to get four independent feature layers; Four independent feature layers are stacked to convert width and height information into channel information, increasing the number of input channels by 4 times, from the original red, green and blue three-channel mode to 12 channels; Perform a convolution operation on the obtained new image to obtain a two-fold downsampled feature map without information loss; The Conv module in the YOLOv5 model includes the convolutional layer Conv2d, the normalization Batch Normalization and the activation function SiLU; The C3 module in the YOLOv5 model is a residual structure; The SPPF module in the YOLOv5 model can expand the receptive field, allowing the YOLOv5 model to obtain global feature information; The Neck in the YOLOv5 model uses a path aggregation network to fuse the deep semantic information extracted by the backbone network with the shallow detail information, and performs upsampling through the UpSample operation to convert the W×H image into 2W×2H, where W and H represent the width and height of the image respectively; then the Concat operation is performed, which means splicing feature maps of the same size; The output Prediction of the YOLOv5 model has three branches, which are used for the detection output of large, medium and small targets respectively.

10. The method according to claim 9, It is characterized in that Steps 3-4 include: Step 3-4-1, data collection: The image capture module integrated in each node of the IP Multimedia Subsystem IMS network is responsible for collecting the incoming multimedia content. The multimedia content is initially screened, and the relevant image data is marked and transmitted to the central processing server in the intelligent communication service platform; the central processing server is responsible for centralized processing and management of data flows in the IMS network; Step 3-4-2, data transmission: the image data is transmitted to the data processing center in the intelligent communication service platform through the secure channel of the IP multimedia subsystem IMS network, and encryption and integrity verification are performed during the transmission process; Step 3-4-3, real-time processing: In the data processing center, the YOLOv5 model processes and analyzes real-time or near real-time image data.

Citation Information

Patent Citations

  • Intelligent voice transferring method and system based on deep learning

    CN109741748A

  • Intelligent quality inspection method and system for voice communication

    CN111128241A

  • Smart Internet security platform based on big data technology

    CN114896305A

  • Live conference intelligent subtitle display method and system based on cloud edge collaboration

    CN116527840A

  • Intelligent conference system based on Internet of Things

    CN116545790A

Cited By

  • Multidimensional association-based multimedia data security measurement method

    CN120951176A

  • A multimedia data security measurement method based on multi-dimension correlation

    CN120951176B

  • Method for realizing real-time interaction function in 5G new call agent

    CN121122282A

  • Protocol non-inductive dynamic adaptation method and system for government and enterprise data channels

    CN121486076A