383results about "Audio data clustering/classification" patented technology

A method of automated generation of contextually-relevant images for a music segment includes receiving at least one of basic metadata information and lyric information for the music segment, generating a first prompt for a computer-implemented machine-learning language model based on the at least one of the basic metadata information and the lyric information, receiving context information from the computer-implemented machine-learning language model in response to the first prompt, generating a second prompt for the computer-implemented machine-learning language model based on the context information, generating a third prompt by providing the second prompt as an input to the computer-implemented machine-learning language model, and generating an image descriptive of the music segment by providing the third prompt as an input to a computer-implemented machine-learning image generation model.

The invention discloses a digital intelligent teaching recording and broadcasting system and method, and relates to the technical field of data encryption, and the method comprises the steps: collecting teaching videos and audios of a teacher through a high-definition camera and an audio device, carrying out the preprocessing, obtaining the course content, recognizing the key information of the course content through a natural language processing NLP and computer vision technology, and carrying out the recording and broadcasting. The method comprises the following steps: segmenting a video into single knowledge point fragments, performing classified storage, forming a learning resource library, collecting basic information and learning habit data of students, constructing a user portrait, matching learning resources, extracting the learning habit data in the user portrait, performing feature extraction, and obtaining a comprehensive feature vector; constructing a learning state evaluation model by using a convolutional neural network to comprehensively analyze the comprehensive feature vector to obtain a learning state comprehensive score, formulating a playing strategy according to the learning state comprehensive score and matched learning resources, regularly testing knowledge points of students, evaluating the mastering degree of the students, and updating a user portrait according to a result.

Methods and systems for generating training data for training a contrastive language-audio machine-learning model. A plurality of audio segments are retrieved from a speech emotion recognition (SER) database along with metadata associated with the audio segments. The metadata of each audio segment includes an emotion class. Words or terms associated with emotions are retrieved from a lexicon. A large language model (LLM) is executed on (i) the classes of emotion associated with the audio segments and (ii) the words or terms from the lexicon. This generates a plurality of text captions associated with emotion, which are stored in a caption pool. For each audio segment retrieved from the SER database, that audio segment is paired with one or more of the text captions from the caption pool that were generated based on the emotion class associated with that audio segment. This yields audio-text pairs for training a contrastive learning model.

A method and an apparatus for generating a motion of a virtual character, and a method and an apparatus for constructing a motion library of a virtual character are provided, and belong to the field of computer technologies. The method for generating a motion of a virtual character includes: obtaining audio and text of a virtual character, the text indicating semantic information of the audio (201); determining a semantic tag of the text based on the text (202); retrieving a motion category matching the semantic tag and motion data belonging to the motion category from a preset motion library (203); and generating a motion sequence of the virtual character based on the motion data (204).

The invention discloses an image label understanding method and device, electronic equipment and a storage medium, and relates to the technical field of image processing, and the method comprises the steps: marking a target label for a target image; the target label comprises a scene label and an in-scene entity category label; identifying text information in the target image; constructing a knowledge graph of the scene and entities in the scene, and converting the knowledge graph into a knowledge base; inputting the target image, the target label and the text information as question information into a knowledge base for retrieval to obtain related knowledge information; and fusing the related knowledge information and the question information into cue words, and inputting the cue words into the large language model to obtain label understanding information of the target image. The knowledge graph of the scene and the entity in the scene is constructed, the knowledge graph is converted into the knowledge base, and the knowledge base is mounted by using the large language model, so that the large language model can understand the mutual relationship between the scene tag and the text in the target image, and the understanding of the target image tag is improved.

The invention discloses a graph neural network music recommendation method for independent division and aggregation of large and small paths, and belongs to the field of deep learning, and the method comprises the following steps: 1, constructing a heterogeneous graph neural network classification and aggregation model which comprises a path discriminator, a large-path neighbor node selection module and an intra-path aggregation module; step 2, constructing a loss function optimization training heterogeneous graph neural network classification aggregation model; and step 3, acquiring music data of a current user, inputting the trained heterogeneous graph neural network classification aggregation model, and generating a personalized music recommendation list. The invention aims to create more personalized and accurate music recommendation experience for the user and help the user to explore richness and unique charm of the music more deeply, so that a more wonderful music world is presented for music lovers.

The invention relates to the technical field of multimedia, and provides a song recommendation method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring song-based real-time user interaction data and historical user use data; a target model is created based on the real-time user interaction data and the historical user usage data, and the target model is a model representing user preferences; a target music feature and a candidate song set similar to the currently played song are obtained, and the target music feature is a feature obtained after multi-modal music features are fused; and creating a dynamic queue of the candidate song set based on the target model and the target music features to obtain a to-be-played song set. According to the method, diversification of data analysis factors of song recommendation is enhanced, intelligent dynamic adjustment of the music playing queue is realized, highly personalized and dynamically responsive music playing experience is provided for the user, the accuracy of the played songs is improved, and thus the personalized song listening experience of the user is improved.

Methods, systems, and apparatuses, including computer programs encoded on computer storage media, for receiving a query relating to a data item that includes multiple data item samples and processing the query and the data item to generate a response to the query. In particular, the described techniques include adaptively selecting a subset of the data item samples using a selection neural network conditioned on features of the data item samples and the query. Then processing the subset and query using a downstream task neural network to generate a response to the query. By adaptively selecting the subset of data item samples according to the query, the described techniques generate responses to queries that are more accurate and require less computation resources than would be the case using other techniques.

A method and apparatus for classifying generated speech are disclosed. The method for classifying generated speech includes: applying a one-dimensional convolution operation to raw speech data to embed the raw speech data into a feature space and extract a feature vector; quantizing the feature vector by applying it to a residual vector quantizer; and applying the quantized result to a classifier model including a natural language processing model to output a classification label.

Knowledge-based audio-text modeling via automatic multimodal graph construction is performed. An audio dataset is received, the audio dataset including clips of audio data, wherein each of the clips of the audio data is paired with corresponding metadata descriptive of the audio contents of the respective clip of the audio data. Graph nodes of interest are identified from a sematic network, the graph nodes being descriptive of semantics of the knowledge domain of the contents of the audio dataset. A large language model (LLM) is utilized for categorizing the metadata into the graph nodes and for inferring supplemental data for the graph nodes for which there is no metadata, producing an extracted knowledge graph. The extracted knowledge graph is validated utilizing the LLM to perform relation verification of edges between the graph nodes of the extracted knowledge graph, thereby mitigating hallucination effects in the categorizing and inferring of the supplemental data.

The invention provides an enterprise call-out method, device and equipment and a storage medium, and the method comprises the steps: obtaining the enterprise information of a to-be-called customer, converting the enterprise information into a label, and carrying out the matching of verbal skill contents from a historical verbal skill database according to the label, and obtaining a personalized verbal skill list; real-time voice recognition is carried out on an exhaustion call record collected on an exhaustion call site to obtain a real-time text, keyword retrieval is carried out on the real-time text, the real-time text is compared with the personalized verbal skill list, violation items and omission items are obtained respectively, and the violation items and the omission items are used for generating reminding information and sending the reminding information to exhaustion call personnel; after the exhaustion call process is finished, performing voice recognition on the exhaustion call record to obtain a full text, extracting key information from the full text and filling the key information into the report template to obtain an exhaustion call report; and acquiring the position information of the call-out site and the voiceprint characteristics of the call-out record, verifying the position information and the voiceprint characteristics, and marking the call-out report according to the verification result. The problem that in the prior art, the complete dispatch authenticity verification capability is weak is solved.

The invention relates to the technical field of music information retrieval, in particular to an automatic classification method and system for music works, and the method comprises the following steps: based on the classification standard of music genre, analyzing the hierarchical node relationship of the music genre, extracting the corresponding hierarchical characteristics of the large class and the small class of the genre, identifying the association proportion between each hierarchy, and analyzing the hierarchical connection strength. And generating a music genre level weight table. According to the method, through deep analysis of a music genre hierarchical structure, finer and dynamic classification of music works is realized, the classification accuracy and flexibility are significantly improved, through extraction of hierarchical features and revelation of association between hierarchies, music classification not only breaks through limitation of simple labeling, but also can reflect hierarchical relationships between genre, and the classification efficiency is improved. The chord change rule and amplitude are extracted, the dynamic capture of music features is enhanced, the association of chord change and genre level features is matched, the classification weight is dynamically adjusted, genre features are accurately matched, and the diversity and complexity of music creation are better adapted.

The invention relates to the technical field of data processing, and discloses a recommended song menu customization method, device and equipment, a storage medium and a computer program product, and the method comprises the steps: obtaining user identity information of a current user, obtaining a music playing record and a user portrait of the current user based on the user identity information, and obtaining a recommended song menu based on the music playing record; and obtaining a user preference song list of the current user, and screening songs in the user preference song list according to the user portrait to obtain a recommended song list. Therefore, the music playing record and the user portrait of the corresponding current user can be obtained according to the obtained user identity information, the user preference song list of the current user is obtained based on the music playing record of the user, and the songs in the user preference song list are screened according to the user portrait. Due to the fact that the music playing record of the current user and the user portrait are combined, the effect of more accurately matching the song listening requirement of the user is achieved.

The invention belongs to the technical field of audio signal processing, and discloses a music emotion recognition method and system based on parallel local attention and a converter, and the method comprises the steps: inputting an audio feature vector into a parallel local attention network and a converter network, and respectively extracting a local music feature and a global music feature; carrying out weighted fusion to obtain a fused feature vector; a music encoder is used to encode music features; using an emotion encoder to serialize the emotion labels into vectors, and outputting emotion features; using canonical correlation analysis to maximize the correlation between the music features and the emotion features; the similarity between music and emotion distribution is optimized through KL divergence loss; and by calculating the similarity, bidirectional retrieval of music recognition emotion and emotion recognition music is realized. The method can effectively improve the performance of the music emotion recognition task, has a wide application prospect, and particularly has an important application value in the fields of music recommendation, emotion calculation, human-computer interaction and the like.

A method of training a sound classification model for a premises monitoring system may include receiving audio data corresponding to a sound detected at a premises. The audio data may be provided to an active instance of a sound classification model, which may generate classification data indicating the sound is unrecognized. The audio data may be provided to a user device, and updated classification data indicating an identity of the sound may be received from the user device. The audio data and the updated classification data may be stored in a data bucket corresponding to the identity of the sound. When the data bucket contains a threshold quantity of user-classified audio data, a new instance of the sound classification model may be generated and retrained using the user-classified audio data. The active instance of the classification model may be replaced with the new instance of the sound classification model.

A support interaction is guided in real time by generating from audio content featurized audio data that includes audio segments and audio features; generating in real time classification scores associated with certain audio segments; and displaying in real time the classifications scores and information associated with the corresponding audio segments.

The invention provides an interrogation record and audio and video recording linkage method and system, and belongs to the technical field of information processing, and the method comprises the steps: obtaining interrogation record text data and segmented video recording data, and respectively labeling category labels; performing semantic extraction on the record text data to obtain semantic feature information, and converting the feature information into text semantic feature vectors; frame extraction is carried out on the video data, image features of each frame of the video data are extracted, similarity screening is carried out, and the image features are converted into key frame semantic feature vectors; constructing a cross-modal embedding space model, respectively encoding the text semantic feature vector and the key frame semantic feature vector of the category label through two encoders, then splicing and fusing to obtain a fused feature vector, converting and mapping the fused feature vector into a sharing layer, completing feature association of the record text data and the segmented video recording data, and obtaining the segmented video recording data. And subsequently, cross-modal search is carried out. According to the method, cross-modal information association is realized through association of the semantic feature vectors of the record and video recording information, and a basis is provided for subsequent rapid calling, research and judgment.

The invention relates to a pet language translation method and system based on audio learning, and belongs to the technical field of animal training. The method comprises the following steps: constructing a pet standardized sound library; when the preset scene is triggered, playing the target sound signal; the target sound signal is a sound signal related to a preset scene in a pet standardized sound library; when it is detected that the first sound signal sent by the pet is matched with the sound signal in the pet standardized sound library, triggering a feedback operation corresponding to the matched sound signal; collecting pet sound in the current environment in real time, responding to the matching of the pet sound and the sound signal in the pet standardized sound library, and outputting a pet demand of the matched sound signal; the pet demand corresponds to a semantic translation result of the pet sound. By means of the mode, the training logic which can be stably recognized and can be copied and executed can be provided, accurate and reasonable pet language translation can be achieved, and the probability of mistranslation or wrong translation is reduced.

The embodiment of the invention discloses an audio content pushing method and system. The method comprises the steps that electroencephalogram signals of a user are collected; recognizing the health state of the user based on the electroencephalogram signal; according to the health state, matching with the audio content in the audio content set, and calculating a first matching degree of each audio content and the health state; and pushing the audio content to the user according to the first matching degree, and repeating the steps until the health state reaches a preset value. According to the technical scheme provided by the invention, the personalized audio content can be pushed according to the health states of different users, and the technical problem that the audio is difficult to adaptively push to each user in the prior art is relieved; in addition, the audio content does not need to be manually researched and tested, and the technical problems that an existing music healing system is long in development period and high in cost are solved.

An apparatus for speech recognition includes a user prompt playback unit configured to change a user prompt that induces utterance for speech recognition into a sound source and to play the sound source. The apparatus further includes a microphone detection signal extraction unit configured to extract a microphone detection signal when a user's speech is received as a speech signal through a microphone. The apparatus also includes a user prompt removal unit configured to remove the user prompt from the microphone detection signal. The apparatus further includes a speech recognition unit configured to recognize a speech based on a value from which the user prompt is removed. The apparatus also includes a response output unit configured to output a response related to the speech based on a result of recognizing the speech.

Techniques are disclosed relating to automatically generate new music content. In some embodiments, a computing system receivers user input specifying a user-defined music control element. The computing system may train a machine learning model to change both composition and performance parameters based on user adjustments to the user-defined music control element. In embodiments in which composition and performance subsystems are on different devices, one device may transmit configuration information to another device, where the configuration information specifies how to adjust parameters based on user input to the user-defined music control element. Disclosed techniques may facilitate centralized learning for human-like music production while allowing individualized customization for individual users. Further, disclosed techniques may allow artists to define their own abstract music controls and make those controls available to end-users.

The invention relates to the technical field of artificial intelligence, and provides an interactive dialogue method and device, equipment and a storage medium, and the method comprises the steps: collecting a voice signal inputted by a user, and carrying out the recognition of the voice signal, and obtaining first text content corresponding to the voice signal; extracting voice emotion representation of the voice signal; extracting a text emotion representation of the first text content; fusing the speech emotion representation and the text emotion representation to obtain emotion information; according to the first text content, determining second text content which has a dialogue with the first text content; and outputting dialogue voice according to the emotion information and the second text content. According to the method, the voice emotion representation and the text emotion representation are fused, so that the real emotion state of the user is fully known, and the emotion information matched with the emotion state or the speaking style of the user is obtained. According to the emotional information and the second text content, the dialogue voice matched with the emotional state or the speaking style of the user is output, and the authenticity of the interactive dialogue is improved.

A computer assisted method for classifying digital audio files based on features of a digital audio signal comprised in the file, comprising: storing the audio file in a digital memory; determining a portion (p) of drop of the audio file, for example the portion with the highest subjective loudness; classifying said audio file based on features of said drop.

The invention relates to the technical field of recommendation systems, in particular to a music recommendation method based on a knowledge graph and reinforcement learning. The method comprises the following steps: S1, extracting information of users and songs and interaction records of the users and the songs from a music data set, and performing knowledge extraction; s2, constructing nodes and edges of a knowledge graph according to entities, relationships and attributes of knowledge extraction, storing and managing the knowledge graph by using a graphic database, and performing knowledge representation; s3, designing a reasonable prompt template and generating prompt input according to behavior characteristics and requirements of the user, and constructing a user music preference model after model training; s4, constructing a user preference music knowledge graph according to the user music preference model and actual historical interaction behaviors of the user and the music; and S5, proposing a recommendation strategy based on reinforcement learning, and updating and optimizing the recommendation strategy according to user feedback.

The invention discloses a personalized music recommendation method and system based on deep learning and brain wave signals, and belongs to the technical field of machine learning, and the method comprises the steps: collecting physiological signals when a user wears an electroencephalogram device and a heart rate monitoring device, and combining the time domain and frequency domain characteristics of heart rate variability data, a feature vector is formed through dimensionality reduction of an auto-encoder; a sample similarity matrix is calculated according to the feature vectors after dimension reduction, spherical clustering grouping is executed, and a state action value table of each category is constructed and initialized; inputting the feature vectors into the constructed deep network, playing back and storing interaction data by adopting priority experience, and executing parameter updating and gradient cutting; and acquiring scores of recommended music and physiological signal changes of a user, re-clustering based on newly added data, and dynamically adjusting a clustering structure. According to the scheme, the matching degree of music recommendation and the current physiological state of the user can be improved, and recommendation individuation and effectiveness are enhanced.

The invention discloses an intelligent music recommendation method based on emotional perception and acoustic features, and relates to the technical field of intelligent recommendation systems and emotional computation.The method comprises the steps that physiological signals, music acoustic features and historical behavior data of a user are collected; preprocessing the multi-source features and mapping the multi-source features to the same dimension to construct a fusion matrix; building a double-branch deep learning model, fusing features through an attention mechanism and training parameters; an evolutionary algorithm is adopted to optimize hyper-parameter screening optimal combination; generating a recommendation list matched with the real-time emotion and preference; and continuously collecting user interaction data, and regularly and incrementally training the dynamic update model. According to the method, emotion and behavior dual-drive recommendation is achieved by fusing physiological signals and acoustic features, emotion perception is accurate, recommended content fits the real-time mood, model optimization is efficient, recommendation precision and diversity are remarkably improved, and the music consumption experience of a user is greatly improved.

Techniques for diarization using embedding pruning are disclosed. A set of audio content segments and their associated tokens are accessed by a speaker enumeration module of a speech processing engine. The speaker enumeration module uses various pruning criteria to prune audio content segments from the set to result in a pruned set of audio content segments. The pruned set of audio content segments is analyzed using a clustering process to determine a number of speakers. The number of speakers is used in a second clustering process to identify speakers in the original set of audio content segments prior to pruning. A transcription of the original audio content with speaker labels is generated using the number of speakers identified for the pruned set of audio content segments.

The invention relates to the technical field of archive management, and discloses an archive management method based on artificial intelligence, comprising the following steps: collecting various types of archive data, including audio data, image data and text data; according to the method, through the steps of preprocessing, feature extraction, standardization and the like, archive data of different modalities are converted into unified multi-modal feature representation, data isomerism elimination and feature effective extraction are achieved, the multi-modal fusion method can fully utilize complementarity between data of different modalities, and the multi-modal fusion efficiency is improved. The overall expression ability of data is improved, a user can convert voice into a text and calculate semantic information of the text in a voice query mode, the semantic-level retrieval mode is more accurate and flexible than traditional keyword matching, a target file in a file database can be quickly retrieved based on input of a deep learning model, and the user experience is improved. And the retrieval efficiency and accuracy are improved.

The invention discloses an engine abnormal sound detection method, device and equipment and a medium. The engine abnormal sound detection method comprises the following steps: acquiring actual noise audio data of a to-be-detected engine; processing the actual noise audio data to generate actual noise audio feature data; establishing a standard audio database; wherein the standard audio database comprises standard noise audio feature data under various combination conditions of various influence parameters; determining matched noise audio feature data according to the feature information of the to-be-detected engine and the standard noise audio feature data; according to similarity parameters of the matched noise audio feature data and the actual noise audio feature data, abnormal sound faults of the engine are recognized; and when the abnormal sound fault of the engine is identified, the abnormal sound fault type of the engine is diagnosed according to the actual noise audio feature data.

Implementations described herein relate to methods, systems, and computer-readable media to automatically answer a call. In some implementations, a method includes receiving a call from a caller device at a client device. The method further includes determining, based on an identifier associated with the call, whether the call matches auto answer criteria, and yin response to determining that the call matches the auto answer criteria, answering the call without user input and without alerting a user of the client device. The method further includes generating a call embedding for the call based on received audio of the call, comparing the call embedding with spam embeddings to determine whether the call is a spam call, and in response to determining that the call is a spam call, terminating the call.