Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Audio visualization model training and audio visualization method, device and equipment

An audio and model technology, applied in the field of audio and video, can solve problems such as difficult to meet diverse user needs, detachment, etc., to achieve the effect of satisfying emotional interaction needs

Active Publication Date: 2021-07-23
HANGZHOU NETEASE CLOUD MUSIC TECH CO LTD
View PDF6 Cites 7 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, this technology is out of the user's interest and preference, and it is not driven by the user's personalized preference for video matching, so it is difficult to meet the diverse needs of users.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Audio visualization model training and audio visualization method, device and equipment
  • Audio visualization model training and audio visualization method, device and equipment
  • Audio visualization model training and audio visualization method, device and equipment

Examples

Experimental program
Comparison scheme
Effect test

Embodiment approach

[0314] As an optional implementation, the device also includes:

[0315] The knowledge map building module is used to construct the knowledge map in the following manner:

[0316] Define entity types, entity attribute information, edges corresponding to different types of association relationships and rules for determining each type of association relationship, the entity types include video types and audio types;

[0317]Extracting entities of different entity types from the source database as nodes according to the defined entity types and entity attribute information, and extracting attribute information of the nodes from related information of the nodes;

[0318] According to the rules for judging various types of association relationships, it is determined whether there is an association relationship between different nodes, and when it is determined that there is an association relationship, the different nodes are connected by corresponding types of edges according to t...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides a method, a device and equipment for providing audio visualization model training and audio visualization. The method comprises the following steps: acquiring a training sample comprising user information, a user history playing video, a target audio, a target video and a relation label about whether the target audio is associated or not; inputting the training sample into an audio visualization model, and performing feature extraction on the target audio to obtain a first feature representation of the target audio; performing feature extraction on the user information and the user historical playing video to obtain user features and user interest expression features, performing feature extraction on the target video to obtain second feature representation, and performing joint processing on the user features, the user interest expression features and the second feature representation to obtain third feature representation; determining a similarity between the first feature representation and the third feature representation; and according to the similarity and the relation label in the training sample, updating the parameters of the audio visualization model. According to the invention, personalized video matching can be carried out on the same audio, and diversified user requirements are met.

Description

technical field [0001] The present invention relates to the field of audio and video technology, in particular to an audio visualization model training and audio visualization method, device and equipment. Background technique [0002] During the audio playback process, the user completes the aesthetic experience process of audio works from sensibility to rationality through sound perception, emotional feeling, image association and rational perception. Audio has the characteristics of image thinking. Accompanied by emotions, through imagination and association, images such as audio images, life scenes, and artistic conceptions are obtained, and audio visualization is derived from this. Audio visualization mainly realizes the interpretation of music emotions with video animation, and integrates audio material and video. [0003] An audio playback scenario introduced in related technologies is to automatically match the audio with dynamic video according to the audio current...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/64G06F16/61
CPCG06F16/64G06F16/61
Inventor 展丽霞肖强孔昭阳董家骥李勇
Owner HANGZHOU NETEASE CLOUD MUSIC TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com