Private area monitoring method and system based on non-semantic feature recognition

Abstract

The invention discloses a privacy area monitoring method and system based on non-semantic feature recognition, and relates to the technical field of voice signal processing. The method comprises the following steps: continuously collecting environment audio, and carrying out audio detection and preprocessing to obtain abnormal audio data; performing non-semantic feature extraction on the abnormal audio data to obtain abnormal sound data; performing keyword detection on the abnormal sound data, and matching the abnormal sound data with a preset sound template; and when the sound templates are successfully matched, triggering an alarm and destroying all the collected and generated data. According to the invention, the problem that the privacy area cannot be monitored is fundamentally solved, the original sound is not recorded and transmitted, and privacy leakage is completely eradicated from the technical source.

Description

Technical Field

[0001] This invention relates to the field of speech signal processing technology, and in particular to a method and system for monitoring privacy regions based on non-semantic feature recognition. Background Technology

[0002] Currently, in private areas of public places such as schools and nursing homes, the installation of image-capturing devices such as cameras is generally prohibited out of respect for personal privacy. This makes these areas "blind spots" for security monitoring. While traditional audio monitoring solutions can capture sound, they also record the entire conversation, severely infringing on personal privacy and thus proving impractical. Based on the above background, existing technologies face the following irreconcilable technical contradictions and specific shortcomings when addressing the security monitoring problem in private areas: The unavailability of surveillance methods: Due to respect for personal privacy, it is not feasible to deploy cameras and other image acquisition devices in private areas such as toilets, bathrooms, and dormitories. This makes these areas "blind spots" for security monitoring, making it impossible to effectively monitor and warn of abnormal events such as bullying and falls.

[0003] Privacy risks associated with existing audio surveillance: While audio surveillance devices can partially overcome blind spots, traditional solutions require continuous recording and transmission of raw audio data for analysis or archiving. This practice fully records sensitive information such as conversations, posing a significant risk of privacy breaches and making compliant deployment in private areas difficult.

[0004] The fundamental contradiction between security and privacy: How to achieve reliable, real-time perception and early warning of specific abnormal security events (such as violent behavior, calls for help, falls, etc.) within a privacy area without infringing on personal privacy (i.e., without recording, parsing, or transmitting any information that can identify an individual or reproduce voice content). Summary of the Invention

[0005] The purpose of this invention is to provide a privacy region monitoring method and system based on non-semantic feature recognition, aiming to solve or improve at least one of the above-mentioned technical problems.

[0006] To achieve the above objectives, the present invention provides the following solution: A privacy region monitoring method based on non-semantic feature recognition includes: Continuously collect environmental audio, perform audio detection and preprocessing to obtain abnormal audio data; Non-semantic feature extraction is performed on abnormal audio data to obtain abnormal sound data; Perform keyword detection on abnormal sound data and match it with preset sound templates; When the sound template is successfully matched, an alarm is triggered and all collected and generated data is destroyed.

[0007] Further, audio detection is performed, including: Energy, main frequency components, and short-time spectral entropy data in ambient audio are acquired to form three-dimensional features. Abnormal audio is detected based on the three-dimensional features. The K-means clustering algorithm is used to adaptively select features and calculate the threshold used for speech detection.

[0008] Furthermore, the preprocessing involves denoising the audio using a joint adaptive threshold active speech detection (VAD) algorithm and a real-time denoising algorithm with minimum mean square error log-spectral amplitude estimation (MMSE-LSA).

[0009] Furthermore, non-semantic feature extraction is performed on the abnormal audio data to obtain abnormal sound data, including: An environmental sound recognition algorithm using Mel-frequency cepstral coefficients (MFCC) and convolutional neural networks (CNNs) is used to collect and identify abnormal sounds in abnormal audio data, thus obtaining abnormal sound data.

[0010] Furthermore, no text is generated during the sound template matching process.

[0011] Furthermore, the specific voice templates include: cries for help, threatening language with the intent to cause physical harm, insulting language, commanding language with the nature of forcing obedience, and dangerous language indicating self-destructive behavior.

[0012] Furthermore, when a sound template matches successfully, an alarm is triggered and all collected and generated data is destroyed, including: When the sound template is successfully matched, an encrypted alarm data packet is generated and uploaded to the early warning platform; Destroy the collected audio data and the generated abnormal audio data.

[0013] Furthermore, the encrypted alarm data packet includes the device ID, timestamp, event type, confidence level, and feature category.

[0014] A privacy zone monitoring system based on non-semantic feature recognition includes: The sound acquisition module is used to continuously acquire ambient audio. The edge computing module is used to process ambient audio and perform sound template matching; when an abnormal event occurs, it triggers an alarm and destroys all collected and generated data. The backend management platform receives alarm information and processes and displays it.

[0015] Furthermore, the edge computing module includes: The preprocessing unit performs audio detection and preprocessing to obtain abnormal audio data. The feature extraction unit performs non-semantic feature extraction on the abnormal audio data to obtain abnormal sound data. The template matching unit performs keyword detection on abnormal sound data and matches it with preset sound templates; The alarm unit triggers an alarm and destroys all collected and generated data when the sound template is successfully matched.

[0016] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects: This invention discloses a method and system for monitoring privacy areas based on non-semantic feature recognition. The method fundamentally solves the problem of privacy areas being unmonitorable, ensuring that the original sound is not recorded or transmitted, thus preventing privacy leaks from the source.

[0017] By turning a passive approach into a proactive one, it can promptly detect malicious incidents such as bullying and harassment hidden in private areas, as well as unexpected situations such as elderly people falling, thus gaining valuable time for human intervention and rescue. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic flowchart of the method of the present invention; Figure 2 This is a schematic diagram of the system structure in this embodiment; Figure 3 This is a schematic diagram of the overall audio data processing flow in this embodiment. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] The purpose of this invention is to provide a privacy region monitoring method and system based on non-semantic feature recognition, aiming to solve or improve at least one of the above-mentioned technical problems.

[0022] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0023] like Figure 1 As shown, the present invention provides a privacy region monitoring method based on non-semantic feature recognition.

[0024] The specific steps include: Step 1: Continuously collect ambient audio, perform audio detection and preprocessing to obtain abnormal audio data, including: Ambient audio is continuously collected, and energy, dominant frequency components, and short-time spectral entropy data are obtained to form three-dimensional features. Anomaly detection is performed based on these three-dimensional features. A K-means clustering algorithm is used to adaptively select features and calculate the threshold used for speech detection. Once the set threshold is reached, the audio segment is marked as a suspicious segment. A joint adaptive threshold active speech detection (VAD) algorithm and a real-time denoising algorithm based on minimum mean square error log-spectral amplitude estimation (MMSE-LSA) are used to denoise the suspicious audio segments, thus obtaining the anomalous audio data.

[0025] In the above steps, energy, dominant frequency components, and short-time spectral entropy exhibit strong complementarity in various types of noise. This complementarity greatly improves the robustness and accuracy of the system in complex noise environments. This is mainly reflected in three aspects: (1) Strong anti-interference capability. The combination of the three can effectively distinguish between sudden sounds (dependent on energy), specific sound sources (such as human voices, dependent on dominant frequency), and complex spectral sounds (dependent on spectral entropy), avoiding false alarms or missed alarms due to single noise interference; (2) More comprehensive coverage: ensuring that different types of abnormal sounds (sudden, frequency-specific, and complex) can be reliably captured, significantly reducing the risk of missed detection; (3) More reliable decision-making: multidimensional features provide a more stable and information-rich basis for subsequent adaptive algorithms (such as K-means clustering), enabling the system to set thresholds and identify anomalies more accurately.

[0026] The algorithm for calculating the three-dimensional feature vector in step 1 is as follows: The expression for the three-dimensional feature vector is: ; In the formula, Let be the three-dimensional feature vector of the m-th audio frame; The energy of the m-th frame is used to reflect the signal strength. The dominant frequency component is the frequency point with the largest spectral amplitude. Short-time spectral entropy is used to measure the complexity or disorder of a spectral distribution. The energy expression is: ; In the formula, N is the number of sampling points in the m-th audio frame; m is the frame number; and n is the intra-frame index. This represents the nth sample point in the time domain of the m-th audio frame. The expression for the dominant frequency component is: ; In the formula, The Discrete Fourier Transform (DFT) result of the m-th audio frame represents the complex value at the k-th frequency point in the frequency domain; k is the frequency index of the DFT; N is the number of points in the DFT, equal to the frame length; j is the imaginary unit. The expression for short-time spectral entropy is: ; In the formula, Let be the normalized probability distribution of the m-th frame at the k-th frequency point.

[0027] The K-means clustering adaptive threshold calculation algorithm in step 1 is as follows: Historical data extracts a three-dimensional feature set from audio data of normal historical environments, expressed as follows: We then perform K-means clustering to divide the features into several clusters. Let the cluster representing the "normal state" be... Its cluster center is The maximum distance from all points in the cluster to the center is used as the threshold for anomaly detection. When the feature vector exceeds the threshold If it does, it is considered abnormal.

[0028] Set the minimum objective function The expression is: ; In the formula, Given the total within-class squared error of K-means clustering, the algorithm attempts to find the optimal cluster centers such that... Minimum; K is the preset number of clusters. In anomaly detection scenarios, it is usually desirable that at least one cluster corresponds to the "normal" state. For the i-th cluster The cluster center is a three-dimensional vector; For the i-th cluster, that is, the set of all eigenvectors assigned to the i-th center; Let be the three-dimensional feature vector of the i-th cluster.

[0029] threshold The expression is: ; In the formula, It is a set of clusters in the normal state; for The cluster center of a set is the mean vector of all features.

[0030] Abnormal audio detection in step 1 includes: Calculate the distance between the real-time audio and the normal cluster; if it exceeds a threshold... Then it is marked as an exception, and the expression is: ; In the formula, This is the distance metric between the feature in the m-th frame and the center of the normal cluster. The three-dimensional feature vector extracted from the m-th audio frame; The real-time noise reduction algorithm in step 1 includes: For abnormal audio, a speech activity detection algorithm (VAD) and a noise reduction algorithm (MMSE-LSA, Minimum Mean Square ErrorLog-Spectral Amplitude) are used.

[0031] Voice Activity Detection (VAD), the expression is: ; In the formula, This is the voice activity flag for the m-th frame. 1 indicates an active voice frame (containing valid voice), and 0 indicates a non-voice frame (silence or noise). The energy of the m-th frame; This is the energy threshold, used to determine if there is a sufficiently strong signal; Let be the short-time spectral entropy of the m-th frame; It is the spectral entropy threshold, used to distinguish structured signals from random noise.

[0032] The MMSE-LSA gain function expression is: ; In the formula, As a gain function, if the signal-to-noise ratio is high at a certain frequency... It has almost no attenuation; if the signal-to-noise ratio is low at a certain frequency... Strongly inhibited; The prior signal-to-noise ratio; For auxiliary variables; k is the frequency index; ; In the formula, The complex spectrum of the m-th frame after noise reduction; The spectrum of noisy speech; This represents the posterior signal-to-noise ratio.

[0033] Step 2: Extract non-semantic features from the abnormal audio data to obtain abnormal sound data, including: An environmental sound recognition algorithm using Mel-frequency cepstral coefficients (MFCC) and convolutional neural networks (CNNs) is used to collect and identify abnormal sounds in abnormal audio data, thus obtaining abnormal sound data.

[0034] The unusual sounds include screams and impacts.

[0035] Step 2, the Mel frequency cepstral coefficients (MFCC) algorithm, includes: The Mel spectrum is calculated using the following expression: ; In the formula, For the m-th frame in the... The output energy on each Mel filter channel constitutes the Mel spectrum; For the first The response values ​​of a Mel filter at frequency bink; is the complex spectrum of the m-th frame after noise reduction; L is the total number of channels in the Mel filter bank; k is the frequency index of the Discrete Fourier Transform (DFT); The MFCC coefficients are calculated using the following expression: ; In the formula, Let be the i-th MFCC coefficient in the m-th frame; M is the final number of MFCC dimensions retained. Step 2, feature extraction using a convolutional neural network (CNN), includes: The MFCC sequence is input into a convolutional neural network (CNN), which outputs a feature vector. The expression is: ; ; In the formula, For the mapping function of the CNN network; For the first The convolutional kernel weight tensor of the layer; For the first The output feature map of the layer; This is a convolution operation; For the first Layer bias vector; The activation function is nonlinear; C is the MFCC sequence; T is the length of the MFCC sequence.

[0036] Step 3: Perform keyword detection on the abnormal sound data and match it with the preset sound template; no text is generated during the matching process. The specific voice templates include: cries for help, threatening language with the intention of causing physical harm, insulting language, commanding language with the nature of forcing obedience, and dangerous language of self-destructive behavior.

[0037] In the above steps, a low-resource speech keyword detection method based on audio feature space trajectory information is used to achieve keyword detection.

[0038] Calculate the cosine similarity between the feature vector z and the preset template t. If the similarity exceeds a threshold, the match is successful. ; In the formula, s is the cosine similarity; The similarity threshold is calculated as follows: ; ; ; ; ; In the formula, This represents the total number of positive samples. The number of negative samples; Let be the similarity score of the i-th positive sample; The similarity score for the j-th negative sample; The mean of the matched samples; The mean of the non-matching samples; The variance of the matching samples; The variance of the non-matching samples.

[0039] Alarm triggering and data destruction

[0040] Step 4: When the sound template matches successfully, trigger an alarm and destroy all collected and generated data, including: When the sound template is successfully matched, an encrypted alarm data packet is generated and uploaded to the early warning platform; Destroy the collected audio data and the generated abnormal audio data.

[0041] In the above steps, the encrypted alarm data packet includes device ID, timestamp, event type, confidence level, and feature category.

[0042] like Figure 3 As shown, a privacy area monitoring system based on non-semantic feature recognition includes: The sound acquisition module is used to continuously acquire ambient audio. The edge computing module is used to process ambient audio and perform sound template matching; when an abnormal event occurs, it triggers an alarm and destroys all collected and generated data. Specifically, it includes: The preprocessing unit performs audio detection and preprocessing to obtain abnormal audio data. The feature extraction unit performs non-semantic feature extraction on the abnormal audio data to obtain abnormal sound data. The template matching unit performs keyword detection on abnormal sound data and matches it with preset sound templates; The alarm unit triggers an alarm and destroys all collected and generated data when the sound template is successfully matched. The backend management platform receives alarm information and processes and displays it.

[0043] like Figure 3 As shown, the overall process of audio data processing specifically includes: The system collects audio information and performs real-time audio stream activity detection to determine if the audio is currently silent. If silent, audio information collection continues; otherwise, 3D feature extraction is performed. If abnormal audio is detected, noise reduction, speech feature extraction, and edge computing local AI model analysis are then performed to determine if an abnormal event has occurred. Upon confirmation of an abnormal event, an encrypted alarm data packet is generated and uploaded to the early warning platform, where manual intervention is initiated based on the alarm information.

[0044] This invention fundamentally solves the problem of unmonitored private areas, ensuring that the original sound is not recorded or transmitted, thus preventing privacy leaks from the source.

[0045] By turning a passive approach into a proactive one, it can promptly detect malicious incidents such as bullying and harassment hidden in private areas, as well as unexpected situations such as elderly people falling, thus gaining valuable time for human intervention and rescue.

[0046] Edge computing distributes the computing load to the front end, reducing the pressure on the back end. It only transmits minimal alarm data, saving network bandwidth.

[0047] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0048] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A privacy region monitoring method based on non-semantic feature recognition, characterized in that, include: Continuously collect environmental audio, perform audio detection and preprocessing to obtain abnormal audio data; Non-semantic feature extraction is performed on abnormal audio data to obtain abnormal sound data; Perform keyword detection on abnormal sound data and match it with preset sound templates; When the sound template is successfully matched, an alarm is triggered and all collected and generated data is destroyed.

2. The privacy region monitoring method based on non-semantic feature recognition according to claim 1, characterized in that, The audio detection includes: Energy, main frequency components, and short-time spectral entropy data in ambient audio are acquired to form three-dimensional features. Abnormal audio is detected based on the three-dimensional features. The K-means clustering algorithm is used to adaptively select features and calculate the threshold used for speech detection.

3. The privacy region monitoring method based on non-semantic feature recognition according to claim 1, characterized in that, The preprocessing involves denoising the audio using a joint adaptive threshold active speech detection (VAD) algorithm and a real-time noise reduction algorithm with minimum mean square error log-spectral amplitude estimation (MMSE-LSA).

4. The privacy region monitoring method based on non-semantic feature recognition according to claim 1, characterized in that, The process of extracting non-semantic features from abnormal audio data to obtain abnormal sound data includes: An environmental sound recognition algorithm using Mel-frequency cepstral coefficients (MFCC) and convolutional neural networks (CNNs) is used to collect and identify abnormal sounds in abnormal audio data, thus obtaining abnormal sound data.

5. A privacy region monitoring method based on non-semantic feature recognition according to claim 1, characterized in that, No text is generated during the sound template matching process.

6. The privacy region monitoring method based on non-semantic feature recognition according to claim 1, characterized in that, The specific voice templates include: cries for help, threatening language with the intent to cause physical harm, insulting language, commanding language with the nature of forcing obedience, and dangerous language of self-destructive behavior.

7. A privacy region monitoring method based on non-semantic feature recognition according to claim 1, characterized in that, When a sound template is successfully matched, an alarm is triggered and all collected and generated data is destroyed, including: When the sound template is successfully matched, an encrypted alarm data packet is generated and uploaded to the early warning platform; Destroy the collected audio data and the generated abnormal audio data.

8. A privacy region monitoring method based on non-semantic feature recognition according to claim 1, characterized in that, The encrypted alarm data packet includes device ID, timestamp, event type, confidence level, and feature category.

9. A privacy area monitoring system based on non-semantic feature recognition, characterized in that, include: The sound acquisition module is used to continuously acquire ambient audio. The edge computing module is used to process ambient audio, extract 3D audio features, perform noise reduction, and match sound templates; when an abnormal event occurs, it triggers an alarm and destroys all collected and generated data. The backend management platform receives alarm information and processes and displays it.

10. A privacy area monitoring system based on non-semantic feature recognition according to claim 9, characterized in that, The edge computing module includes: The preprocessing unit performs audio detection and preprocessing to obtain abnormal audio data. The feature extraction unit performs non-semantic feature extraction on the abnormal audio data to obtain abnormal sound data. The template matching unit performs keyword detection on abnormal sound data and matches it with preset sound templates; The alarm unit triggers an alarm and destroys all collected and generated data when the sound template is successfully matched.

Images