Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Audio visualization model training and audio visualization method, device and equipment

An audio and model technology, applied in the field of audio and video, can solve problems such as difficulty in meeting diverse user needs and separation, and achieve the effect of meeting the needs of emotional interaction

Active Publication Date: 2022-05-20
HANGZHOU NETEASE CLOUD MUSIC TECH CO LTD
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, this technology is out of the user's interest and preference, and it is not driven by the user's personalized preference for video matching, so it is difficult to meet the diverse needs of users.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Audio visualization model training and audio visualization method, device and equipment
  • Audio visualization model training and audio visualization method, device and equipment
  • Audio visualization model training and audio visualization method, device and equipment

Examples

Experimental program
Comparison scheme
Effect test

Embodiment approach

[0314] As an optional implementation, the device also includes:

[0315] The knowledge map building module is used to construct the knowledge map in the following manner:

[0316] Define entity types, entity attribute information, edges corresponding to different types of association relationships and rules for determining each type of association relationship, the entity types include video types and audio types;

[0317]Extracting entities of different entity types from the source database as nodes according to the defined entity types and entity attribute information, and extracting attribute information of the nodes from related information of the nodes;

[0318] According to the rules for judging various types of association relationships, it is determined whether there is an association relationship between different nodes, and when it is determined that there is an association relationship, the different nodes are connected by corresponding types of edges according to t...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The present invention provides a method, device and equipment for providing audio visualization model training and audio visualization, including: obtaining training samples including user information, user history playback video, target audio, target video, and whether the target audio is associated with a relationship label; The sample is input into the audio visualization model, and feature extraction is performed on the target audio to obtain the first feature representation of the target audio; feature extraction is performed on user information and user history playback videos to obtain user features and user interest expression features, and feature extraction is performed on the target video to obtain the first feature representation Two feature representations, the user features, user interest expression features and the second feature representation are jointly processed to obtain the third feature representation; determine the similarity between the first feature representation and the third feature representation; according to the similarity and the relationship in the training samples label, to update the parameters of the audio visualization model. The present invention can carry out personalized video matching on the same audio, so as to meet various user demands.

Description

technical field [0001] The present invention relates to the field of audio and video technology, in particular to an audio visualization model training and audio visualization method, device and equipment. Background technique [0002] During the audio playback process, the user completes the aesthetic experience process of audio works from sensibility to rationality through sound perception, emotional feeling, image association and rational perception. Audio has the characteristics of image thinking. Accompanied by emotions, through imagination and association, images such as audio images, life scenes, and artistic conceptions are obtained, and audio visualization is derived from this. Audio visualization mainly realizes the interpretation of music emotions with video animation, and integrates audio material and video. [0003] An audio playback scenario introduced in related technologies is to automatically match the audio with dynamic video according to the audio current...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/64G06F16/61
CPCG06F16/64G06F16/61
Inventor 展丽霞肖强孔昭阳董家骥李勇
Owner HANGZHOU NETEASE CLOUD MUSIC TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com