42 results about "Sound event detection" patented technology

A method for real-time sound event detection on an embedded device includes pretraining a contrastive language-audio pretraining model as an audio foundation model and preparing offline multimodal query prototypes for sound events of interest. The pretrained model and query prototypes are deployed on an embedded device. The device receives an input audio stream and extracts audio embeddings using the pretrained model. Similarity scores are calculated between the extracted audio embeddings and the prepared query prototypes. The presence of a sound event is determined based on the calculated similarity scores, and a real-time sound event detection result is output. The system includes a memory storing the pretrained model and query prototypes, an audio input interface, and a processor configured to perform the extraction, calculation, determination, and output operations. A non-transitory computer-readable medium stores instructions that, when executed, cause a processor to perform the method.

In general, the disclosure describes a computing system to automatically identify and classify audio input, including non-speech audio signals. The computing system may also add new classes based on only a limited number of examples of the new classes, to identify classes of sounds for which the system had not been trained.

The invention discloses a semantic prompt-based sound event detection data synthesis and sound event detection model training method. The sound event detection task is converted into the semantic description information, and the structured semantic prompt instruction which accurately reflects the target sound event characteristics is generated in combination with the semantic constraint rule, so that automatic mapping from semantic description to instruction generation is realized, and the manual intervention cost is reduced. And inputting the structured semantic prompt instruction into the audio generation model, and synthesizing the audio data in a large scale, thereby reducing the data acquisition cost and improving the sample diversity and expandability. The structured semantic prompt instruction can guide the model to synthesize audios of various sound event types in batches, and is automatically generated by a large language model to ensure that the synthesized audios are strictly aligned with instruction semantics. When the label is generated, the sample event type can be obtained without manual labeling, and an efficient and reliable data source is provided for sound event detection model training.

The application discloses a sound event detection method, device and equipment and a storage medium. The method comprises the following steps: acquiring to-be-detected audio data, and extracting acoustic characteristics of the to-be-detected audio data; determining a sound event detection result of the to-be-detected audio data according to the acoustic characteristics and a pre-determined sound event relationship characteristic; wherein the sound event relationship characteristic is determined based on a pre-constructed sound event relationship graph; and the sound event relationship graph is determined according to a statistical result of sound events in an audio data set. The technical scheme solves the problems of limited application range and low accuracy of sound event detection, and can effectively expand the application range of the detection scene while improving the accuracy of sound event detection.

The invention discloses a human voice data collection method and device, equipment and a storage medium. The method comprises the following steps: inputting received audio data into a sound event detection model; a feature extraction module of the model performs feature extraction on the audio data to obtain audio features; respectively inputting the audio features into a strong label learning module, a multi-instance learning module and a CTC module of the model, correspondingly outputting a frame-level event probability, a frame-level boundary probability, an audio-level existence probability, a frame-level attention weight and a sound event prediction sequence, and fusing the output results to obtain a target detection result; and when the target detection result represents that the voice event exists in the audio data, extracting target voice data from the audio data based on the target detection result. According to the method, the audio data with different granularity labels are optimized through the multi-task learning framework of the model, different task output results are fused, the time sequence detection problem of multi-sound event overlapping is solved, and the accuracy of the target human voice data is guaranteed.

In one possible example aspect, the present disclosure provides a context-aware noise compensation system for a wearable device playing back media content to a user in a noisy environment, wherein the system is configured to: obtain a sound signal from the noisy environment; determine context information associated with the sound signal, wherein the context information comprises at least one of: vocal sound detection information indicative of presence of one or more vocal sounds of the user in the sound signal, or sound event detection information indicative of presence of one or more environmental sound events in the sound signal; and optimize, based on the context information, the media content for playback by the wearable device.

The invention discloses a small sample sound event detection method based on multi-scale feature aggregation, and relates to the technical field of audio signal processing and machine learning. The method comprises the following steps: firstly, preprocessing an input audio signal and extracting Mel spectrogram features; then, the features are input into a specially designed multi-scale feature aggregation network, the network extracts a time dimension, a frequency dimension and a local time-frequency joint feature through three parallel convolution paths, and deep fusion is carried out on multiple paths of features; thirdly, training the network by adopting a model-independent meta-learning framework, and enabling the network to learn the capability of quickly adapting to new tasks by simulating a large number of small sample classification tasks; and finally, using the trained model to quickly identify a new sound event category only containing a small number of samples. According to the method, through effective combination of multi-scale feature extraction and a meta-learning strategy, the problems of incomplete feature extraction and poor model generalization ability of a traditional method in a data scarcity scene are solved.

The application discloses a bird sound event detection method and system for monitoring state-protected birds, and comprises the following steps: collecting bird sound data and environmental noise data of state-protected birds, and constructing a bird sound spectrogram data set and a noisy bird sound data set; constructing a bird sound noise reduction model and a target detection model; training and jointly optimizing the bird sound noise reduction model and the target detection model according to the bird sound spectrogram data set and the noisy bird sound data set; constructing an image-audio mapping module for converting the image detection result of the target detection model into an audio sound event detection result; acquiring bird sound data to be detected, searching and sound source positioning the target bird sound according to the bird sound noise reduction model, the target detection model and the image-audio mapping module, and determining the species, time-frequency and direction information of the target bird sound. The application can improve the detection effect of field bird sound, and further improve the classification accuracy, and can be widely applied to the technical field of acoustic information processing.

The invention discloses a sound signal periodic feature extraction method, a network model training method, a storage medium and equipment, and belongs to the technical field of sound event detection. The objective of the invention is to solve the problems of high sensing difficulty and poor decoupling effect of overlapped acoustic events in the current acoustic detection process. The method comprises the following steps: for a sound signal i, mapping the sound signal i to a low-dimensional space through two different linear layers to obtain p and g, and respectively carrying out expansion convolution operation on p and g to obtain pconv and gconv; for p and g, feature coding is carried out based on a Fourier basis function and a gating mechanism to obtain Fourier features, for pconv and gconv, Fourier features are obtained in the same mode, and Hadamard product is carried out on the pconv and the gconv to obtain representation of periodic features. And in the training process of the corresponding model, performing reconstruction error on the sum and the original signal i, respectively calculating two norms of the sum, and adding the two obtained two norms to obtain a Fourier series regular term for training the model.

The invention relates to a sound event detection model training method and device, equipment and a storage medium. The method comprises the following steps: inputting a sample audio into a pre-trained sound event positioning model to obtain a prediction label of the sample audio; the sample audio is labeled with an initial label, and the initial label is used for representing whether the sample audio contains a sound event; the prediction label comprises probability distribution information that each audio frame in the sample audio comprises a sound event; cutting out an audio clip conforming to the target audio length from the sample audio, and marking a clip tag corresponding to the audio clip according to the initial tag and the prediction tag; the fragment label is used for representing whether the audio fragment contains a sound event; and according to the audio clip and the clip tag corresponding to the audio clip, training a to-be-trained sound event detection model to obtain a pre-trained sound event detection model. By adopting the method, the training efficiency of the sound event detection model can be improved.

The application relates to the technical field of sound recognition, and provides a low-resource sound event detection method, device and equipment and a storage medium, wherein the method comprises the following steps: acquiring sound to be detected; inputting the sound to be detected into a trained sound group category neural network model to obtain sound group category information; inputting the sound to be detected into an encoder of a pre-trained sound event category judgment model to obtain fine-grained feature information, and splicing and fusing the group category information and the fine-grained feature information to obtain fusion feature information; inputting the fusion feature information into a decoder of the sound event category judgment model, and decoding the fusion feature information based on an attention mechanism and in combination with a pre-obtained group category representation matrix to obtain a sound event judgment result. The application forms large-class group sound group category information containing rich information by using the sound to be detected, and realizes auxiliary discrimination based on large-class sound event results through a self-attention mechanism.

This disclosure relates to a model generation method, a sound event detection method, an apparatus, a medium, and a device. The method includes: extracting audio features from a first sample audio file and inputting the audio features into a sound event detection model to obtain a first detection result; determining the target loss of the model based on the first annotation result of the first sample audio file and the first detection result, the target loss including cross-entropy loss and continuity error penalty loss; and updating the model parameters based on the target loss. Since continuity detection errors are the main cause of misidentification, the continuity error penalty term is included as part of the loss function during the sound event detection model training process. The model trained using this method can maintain high accuracy while keeping the sound event recall rate unchanged. That is, it can significantly reduce misidentifications that affect user experience without introducing more missed detections, thereby improving the accuracy of sound event detection and user experience.

This invention discloses a sound event detection method based on the fusion of multi-scale spatial features and coordinate attention, belonging to the field of sound event detection technology. It aims to address the problems in existing sound event detection methods, such as the limited receptive field of the network, which makes it difficult to capture multi-scale temporal span features, and the inability to accurately focus on target sounds and suppress noise in complex time-frequency spaces. This method innovatively introduces a multi-scale coordinate attention module into the sound event feature extraction network. This module first uses parallel dilated convolutions with an increasing time dilation rate to obtain multi-scale temporal features and performs fusion and dimensionality reduction. Then, it employs a dual-axis time-frequency decoupling mechanism, performing adaptive pooling aggregation along both the time and frequency dimensions to generate directional time attention weights and frequency attention weights. Finally, the dual-axis weights are used to recalibrate the multi-scale features. This method significantly improves the accuracy and robustness of sound event detection.

A method for real-time sound event detection on an embedded device includes pre-training a contrast language-audio pre-training model as an audio base model and preparing an offline multi-modal query prototype for a sound event of interest. The pre-trained model and the query prototype are deployed on an embedded device. The device receives an input audio stream and extracts an audio embedding using a pre-trained model. A similarity score between the extracted audio embedding and the prepared query prototype is calculated. The presence of the sound event is determined based on the calculated similarity score, and the real-time sound event detection result is output. The system includes a memory storing a pre-trained model and a query prototype, an audio input interface, and a processor configured to perform extraction, calculation, determination, and output operations. A non-transitory computer readable medium stores instructions that, when executed, cause a processor to implement the method.

The invention relates to a sound event detection system (1) comprising a data processing unit (2) and a sound event recording unit (11). The data processing unit (2) is suitable for generating a user interface (5) that can be displayed on a data display device (4) that is data-connected to the data processing unit (2). Several user data entries (8) can be generated via the user interface (5) using a data input device that is data-connected to the data processing unit (2). These user data entries can be stored in a database (10) that is connected to the data processing unit (2) via at least one database interface. Functional data (9) can be generated in the data processing unit (2) by comparing and / or data-connected to the system data (7) stored in the data processing unit (2).The sound event recording device (11) is arranged in a transport vehicle (12), in particular a small van, which is connected to the database (10) via data processing. The sound event recording device (11) comprises at least one digital audio interface (15) arranged in a mobile and / or portable frame (14) that can be attached to the transport vehicle (12), in particular a small van, and having at least several microphone inputs for several microphones that can be stowed in the transport vehicle (12), in particular a small van. The transport vehicle (12), in particular a small van, includes a navigation and communication device (16) which is connected to the data processing device (2) via data processing.The transport vehicle (12), in particular a small van, includes a computing device (17), in particular a portable device, which is connected to the audio interface (15) and to the data processing device (2). When navigating the transport vehicle (12), in particular a small van, and / or when communicating with the communication device (16), and / or when exchanging data with the computing device (17), at least some of the functional data (9) can be queried electronically.

This invention relates to the field of artificial intelligence technology, and in particular to a method and apparatus for detecting and locating multiple sound events based on a neural network model. The method includes: innovatively designing time-frequency multi-scale residual convolutional blocks, which, together with a Conformer module and a cross-stitch unit module, form a network model to extract features at multiple scales, enhance long sequence modeling, and promote task-based collaborative optimization, thereby improving performance and accuracy; in terms of data processing, pre-emphasis and frame-by-frame windowing improve feature quality, audio channel swapping and spectrum enhancement increase data diversity and reduce overfitting, and SALSA-Lite features are used to enhance feature representation; in terms of training strategy, a multivariate loss function is used to accelerate convergence while considering task requirements, and hyperparameters are flexibly adjusted using a validation set. This makes the method highly efficient in training, has excellent practical performance, strong generalization ability on unknown data, and can accurately cope with complex and ever-changing real-world scenarios, effectively overcoming the shortcomings of traditional methods.

The invention discloses a sound event detection method based on recursive gating convolution and a self-attention mechanism, and the method comprises the steps: collecting a to-be-detected audio signal, and constructing a sound event detection model; inputting the audio signal into a sound event detection model, and extracting a time-frequency feature through a preprocessing module to obtain a logarithmic Mel spectrum feature; inputting the logarithmic Mel spectrum features into a convolution module, and performing spatial feature fusion through recursive gating convolution to obtain spatial fusion features; inputting the spatial fusion features into a time domain modeling module, and performing context information interaction through a self-attention mechanism to obtain global information features; and inputting the global information features into a KANLinear classifier, and carrying out frame-by-frame classification to obtain each sound event category in the audio and the occurrence time period thereof. According to the method, the accuracy of sound event detection can be remarkably improved on the premise that the number of parameters is small and the number of floating point operation times is low.

The invention discloses a whale overlapped sound event detection method and system based on adaptive multi-scale synthetic attention, and belongs to the technical field of ocean engineering and ocean signals, and the method comprises the steps: collecting historical whale sound signals, and carrying out the preprocessing and ACT transformation of the historical whale sound signals, and obtaining a transformed data set; constructing a time-frequency perception cross-offset feature extraction network, and training the time-frequency perception cross-offset feature extraction network by using the transformed data to obtain an adaptive multi-scale synthesis attention network; constructing an adaptive window size prediction network, and predicting window parameters required for constructing a multi-scale attention fusion network according to a feature map output by the adaptive multi-scale synthesis attention network; constructing a multi-scale attention fusion network based on the window parameters, and carrying out iteration to obtain a whale overlapped sound event detection model; and acquiring a real-time whale sound signal, and detecting the real-time whale sound signal by using the detection model to obtain a detection result.

The invention relates to the technical field of sound detection, and discloses a sound event detection method and device and a computer readable storage medium. The method comprises the steps that spectrum features of audio to be detected are acquired, and the dimensions of the spectrum features comprise the number of frequency points and the number of continuous frames; inputting the spectrum feature into an audio feature extraction model to obtain a general audio feature which is a feature vector used for indicating the audio information; inputting the general audio features into a multi-sound event classification model to obtain a first prediction probability of each sound event in preset multi-class sound events; and judging the existence condition of each sound event in the to-be-detected audio according to the first prediction probability. According to the invention, the number of types of sound event detection is effectively expanded while the bearable computing power requirement of the end side equipment is maintained.

The application discloses a smart audio non-inductive monitoring method and system for a home environment, and belongs to the technical field of smart home and health monitoring, which comprises the following steps: collecting audio data in a room and extracting features to obtain an input feature tensor; based on the input feature tensor, sound event detection and sound source positioning are performed through a SELD model based on a deep neural network to obtain local observation data, which is converted to a global coordinate system; the position of a target object in the global coordinate system is continuously tracked and state estimation is performed based on a particle filter to obtain a global motion trajectory; based on the local observation data and the global motion trajectory, combined with trigger information of infrared and radar sensors, decision-level fusion is performed by using Dempster-Shafer evidence theory to output a final activity state and an environmental abnormal event, thereby solving the problems that in the current monitoring scheme, normal life of a user is disturbed, privacy and self-esteem are highly invasive, and monitoring data is not comprehensive.

This application discloses a user profile generation method, apparatus, electronic device, and computer-readable medium. An embodiment of the method includes: segmenting a user's call audio into an audio segment without human voice and an audio segment containing human voice; extracting a first audio feature from the audio segment without human voice, inputting the first audio feature into a pre-trained first ambient sound event detection model to obtain a first detection result; extracting a second audio feature from the audio segment containing human voice, inputting the second audio feature into a pre-trained second ambient sound event detection model to obtain a second detection result; and generating a user profile based on the first and second detection results. This implementation enriches the methods for generating user profiles. User profiles generated in this way can provide users with services related to their environment, thereby improving service quality.

The application provides a sound event detection method, belonging to the technical field of sound event detection, and comprising the following steps: obtaining an audio signal to be detected; pre-processing the audio signal to be detected; performing data enhancement on the pre-processed audio signal; performing feature extraction on the data-enhanced audio signal; extracting context information from the extracted features to obtain first branch features; inputting the data-enhanced audio signal into a pre-trained audio teacher-student model based on Transfomer to obtain second branch features; and inputting the first branch features and the second branch features into a classifier after splicing to obtain a sound event detection result of the audio signal to be detected. The application can obtain sufficient background information corresponding to different sound events, and improves the multi-scale feature capturing capability of the model.

The application discloses a multi-sound source positioning and detection method based on global-local feature recalibration, which comprises the following steps: calculating the short-time Fourier transform of a multi-channel spatial audio signal in a first-order stereo format, obtaining a log-linear spectrum and a normalized sound intensity vector as input features, and then performing data augmentation on the features of a training set; splicing the augmented spectrum and sound intensity vector as the input of a neural network model, training the neural network model, obtaining optimal network model parameters and saving them; preprocessing a test sample and inputting it into the trained model, outputting the predicted sound event category and position information, drawing a sound event detection graph, a direction angle and an azimuth angle trajectory curve graph according to the prediction result, and comparing them with the visualized image of the test sample real label to analyze the performance of the model. The application can achieve high sound source positioning and detection performance, and the model shows good generalization on real and synthetic data sets.

An apparatus is described that comprises a microphone array and a plurality of transducers. The microphone array and the plurality of transducers are communicatively coupled with at least one processor. The apparatus includes the microphone array for receiving at least one signal. Each transducer of the plurality of transducers is configured to deliver a correction signal along a transducer beam spread axis, wherein the plurality of transducers is positioned on the apparatus for providing a combined transducer beam spread coverage in the horizontal plane. One or more applications running on the at least one processor use information of the at least one signal to detect a sound event. The detecting the sound event includes selecting transducers from the plurality of transducers and instructing the selected transducers to deliver a correction signal.

The present application relates to the technical field of digital medical treatment, and in particular to a sound event detection method, device and equipment based on artificial intelligence and a medium. The method separates mixed sound into independent sound, inputs the independent sound into an encoder to obtain sound features, inputs the sound features into a recurrent layer to obtain timing features, uses a label prediction model to process and predict pseudo-event labels according to the sound features and the timing features, queries the pseudo-event labels as reference labels, uses the reference labels and the independent sound to form training samples, trains an event detection model, and then obtains an event detection result, extracts timing information of the sound features, enriches input of event prediction, improves accuracy of event prediction, queries and filters the pseudo-event labels to determine the reference labels, so that the event detection model better adapts to a scene, improves accuracy of sound event detection, and can assist medical staff in timely discovering abnormal sound events of patients in a medical environment, so as to respond in time.

The invention provides a training method, a detection method and a device of an animal sound event detection model, and relates to the technical field of intelligent audio signal processing. The training method comprises the steps that a sample support set and a multi-class training set are obtained, the sample support set comprises first positive and negative samples and a query sample, and the multi-class training set comprises the first positive sample; extracting sample frame-level acoustic features, wherein the sample frame-level acoustic features comprise a sample Log-Mel feature and a sample PCEN feature; performing multi-task joint fine tuning on the pre-training detection model through the sample frame-level acoustic features to obtain an intermediate detection model; performing animal sound event detection on the query sample through the intermediate detection model to obtain a first detection result so as to determine a second positive sample, and updating the frame-level acoustic features of the sample; and iteratively executing multi-task joint fine tuning and query sample detection to finally obtain an animal sound event detection model. According to the invention, high-precision and high-robustness animal sound event detection can be realized under the condition of small samples.

A method of executing an open audio tracking system is disclosed. A feedback loop between an audio foundation model (AFM) and a large language model (LLM) enables both low-level sound event detection and high-level acoustic scene detection in real-time, which are then used to generate additional text-based event descriptions that are applied to subsequent iteration cycles of the system. The AFM can be similar to a contrastive language-audio pre-training (CLAP) model configured to detect sound events, while the LLM receives the detected given sound events and categorizes these events into acoustic sound classes that can interpret the sound events in the context of the environment.

Methods and systems for predicting aggressive behavior associated with a surveillance scene. Images are generated from a camera of a surveillance scene, and audio is also generated. A local computing system executes an object classification model on the images to predict one or more classes of objects in the scene. The local computing system also executes a sound-event detection model on the audio to predict one or more classes of events occurring in the scene. Metadata is generated associated with the image-based classes and audio-based classes. The metadata is transferred to a remote computing system that executes a knowledge graph on the metadata to implement knowledge graph-based reasoning to predict aggressive behavior occurring in the surveillance scene based on the metadata. The metadata associated with the predicted aggressive behavior is labeled as such, and control commands are output accordingly.

In one possible example aspect, the present disclosure provides a context-aware noise compensation system for a wearable device playing back media content to a user in a noisy environment, wherein the system is configured to: obtain a sound signal from the noisy environment; determine context information associated with the sound signal, wherein the context information comprises at least one of: vocal sound detection information indicative of presence of one or more vocal sounds of the user in the sound signal, or sound event detection information indicative of presence of one or more environmental sound events in the sound signal; and optimize, based on the context information, the media content for playback by the wearable device.

The invention discloses a live pig abnormal sound monitoring and early warning method, system and device based on audio and video fusion. The method comprises the steps of obtaining audio and video stream data; extracting audio frequency spectrum features and inputting the audio frequency spectrum features into a sound event detection and positioning model to obtain a prediction result; analyzing the video to obtain the real-time position and activity state of the live pig; performing multi-modal fusion judgment on the audio prediction result and the live pig state to generate a high-confidence identification result; and generating a structured log and carrying out early warning pushing. According to the method, audio classification, positioning and video analysis technologies are combined, the recognition accuracy is improved through multi-modal comparison, the problem that detection and positioning are difficult in a strong noise environment is effectively solved, and efficient and accurate monitoring of the abnormal state of the live pig is achieved.