Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Sound-based method and apparatus for generating AR content and storage medium

A sound and content technology, applied in the computer field, can solve problems such as voice application limitations, and achieve the effect of intelligent service and experience

Active Publication Date: 2018-12-21
SAMSUNG ELECTRONICS CHINA R&D CENT +1
View PDF13 Cites 21 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0012] In view of this, the present invention provides a method, storage medium and device for generating AR content based on sound, so as to solve the limitation problem of voice application in at least one of the above-mentioned existing AR technologies

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Sound-based method and apparatus for generating AR content and storage medium
  • Sound-based method and apparatus for generating AR content and storage medium
  • Sound-based method and apparatus for generating AR content and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0101] Embodiment 1: Video Business Conference

[0102] The recognition result of the scene is a video conference. The voice input in the video conference is obtained, and the natural language understanding technology is used to analyze nouns, charts, time, abbreviations and other semantic information. Text summarization and other technologies are further processed, and finally generated such as Figure 2-Figure 5 The displayed AR content is displayed to the user.

[0103] figure 2 It mainly involves charts: in the video conference, the voice talks about the data and the total amount, such as the total sales volume, and the sales volume of A\B\C. Use NLP technology to analyze voice, after intelligent calculation, use electronic chart automatic generation technology to intelligently generate electronic chart in AR mode, and the user can save the electronic chart after sending a save command.

[0104] image 3 It mainly involves time: when the user talks about a certain mon...

Embodiment 2

[0107] Embodiment 2: Tour commentary

[0108] The scene is tourism explanation, obtain different voices input by tour guides, use natural language understanding technology to analyze semantic information, and then use image recognition technology to generate AR images related to the voice content of tour guides in real time.

[0109] Figure 6 It mainly involves the introduction of cultural relics: when the voice of the tour guide talks about the evolution of historical relics over time, NLP technology is used to intelligently analyze the voice content, and then image recognition technology is used to superimpose the transformed image of cultural relics on the original cultural relics image in the form of AR content.

[0110] Figure 7 It mainly involves the introduction of historic sites: when the voice of the tour guide introduces the ancient sites and the life and work of the working people here in ancient times, NLP technology is used to intelligently analyze the voice, a...

Embodiment 3

[0111] Embodiment Three: Sports Commentary

[0112] The scene recognition result is a live sports game, and the commentator voice explains the game situation, such as the name of the ball holder, number, record, offensive route, etc. Use NLP technology to intelligently analyze the commentator's voice, and combine face recognition technology and intelligent search technology to zoom in on the target player, such as Figure 8 As shown, AR displays commentary content and player encyclopedia information.

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides a sound-based method and apparatus for generating an AR content and a storage medium. The method comprises: step 11, acquiring real-time voice data in a current AR environment;S131, determining whether the real-time voice data include a speaker; and if so, converting a speech content of the speaker in the real-time voice data into a text and extracting key semantic information; S132, extracting an artificial non-speaking sound or natural sound in the real-time voice data; S133, locating sound source locations of the speaker and the artificial non-speaking voice or natural sound; S15, carrying out scene matching based on the key semantic information and artificial non-speaking voice or natural sound; and S17, generating an AR content according to the key semantic information, the sound source locations, the scene or a real scene image in the AR environment and displaying the AR content. With the method provided by the invention, the AR content can be dynamicallygenerated based on the input voice and the AR intelligent display of the auditory information can be realized; and thus the convenient and intelligent services and experience can be provided for users.

Description

technical field [0001] The present invention relates to the field of computers, in particular to a method, storage medium and device for generating AR content based on sound. Background technique [0002] The typical augmented reality technology AR (Augmented Reality) is a technology that calculates the position and angle of the camera image in real time and adds corresponding images, videos, and 3D models. The goal of this technology is to nest the virtual world on the screen. Show and interact in the real world. The AR system has three outstanding features: real-world and virtual information integration; real-time interactivity; adding and positioning virtual objects in a three-dimensional scale space. AR technology can be widely used in many fields. [0003] The principle of AR implementation, firstly, the camera and the sensor collect the video or image of the real scene, which is transmitted to the background processing unit to analyze and reconstruct it, and combine ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G10L15/26G10L21/10G06F3/16
CPCG06F3/167G10L15/26G10L21/10
Inventor 张薇葛玲玲许馨月李文轩姚巍吴楠楠
Owner SAMSUNG ELECTRONICS CHINA R&D CENT
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com