Click-and-read and question answering method and system based on image recognition and tracking technology

A technology of image recognition and question answering system, applied in the direction of character and pattern recognition, electric-operated teaching aids, instruments, etc., can solve the problems of inability to recognize images, high cost of recognition, poor affinity, etc., and achieve good accuracy and affinity , enhance emotional communication, and improve the reading experience

Active Publication Date: 2019-03-08
昆明微想智森科技股份有限公司
View PDF21 Cites 10 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] The system and method combine OCR technology, so that the point-and-click system can be applied to point-reading of ordinary books, and the applicable area is greatly improved, but it has the following problems: 1) It is necessary to recognize the depth of the book page and the camera, and the depth of the finger from the page , the depth recognition error of the monocular camera is very large, and the recognition accuracy is not ideal; the depth recognition accuracy of the binocular camera will be higher, but the recognition cost is higher; 2) The voice of the point-to-read playback is extracted from the page through OCR technology After the text in the image, the voice synthesized by the machine, the OCR technology cannot recognize the image, and the highest text recognition accuracy rate can only reach about 95%; the voice synthesized by the machine is unnatural, and the affinity is poor; 3) Due to the occlusion of the book by the hand, the image collected in real time is incomplete, and the recognition conversion from image to text will have errors

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Click-and-read and question answering method and system based on image recognition and tracking technology
  • Click-and-read and question answering method and system based on image recognition and tracking technology
  • Click-and-read and question answering method and system based on image recognition and tracking technology

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0046] Such as figure 1 As shown, this embodiment discloses a point-to-read question-and-answer system based on image recognition and tracking technology, which includes the following components:

[0047] 1) The pointing and reading pointing piece 1 is used to click on the pages of the publication. Various objects that are convenient for image recognition and clicking can be used, such as fingers, prefabricated pointing and reading sticks, and user-made pointing and reading magic wands.

[0048] 2) The camera device 2 is set directly above or obliquely above the publication page, and is used to collect real-time images of the page and the pointing device 1 for pointing and reading.

[0049] 3) point-reading question-and-answer device 3, for identifying and extracting image features, tracking point-reading pointing piece 1 position and judging click event; In the question-and-answer mode, the question audio is played first, and then it is judged whether the user has clicked th...

Embodiment 2

[0075] On the basis of Embodiment 1, this embodiment provides a simple and easy method for judging the click of a pointing piece, which can improve the accuracy of clicking, including horizontal tracking and height tracking, wherein:

[0076] 1) Horizontal Tracking

[0077] Track the coordinates of the point-and-read pointer 1 on the page coordinate system. When one or more points at the front end of the point-and-read pointer 1 enter the coordinate range of the trigger area, it is judged that the point-to-read pointer 1 has entered the trigger area.

[0078] 2) Altitude Tracking

[0079] 2.1) If image 3 As shown, in the process of point reading, the camera device 2 fixed on the top of the publication takes real-time photos of the point reading pointing piece 1 and the publication page, and the position tracking module 3.3 of the point reading question answering system calculates the point reading pointing piece 1 recognition area in real time (for The ratio of the area rec...

Embodiment 3

[0088] Such as figure 2 As shown, on the basis of Embodiment 2, this embodiment provides a dual-function pointing and reading pointing device based on image recognition and tracking technology, including a hand-held part 1.1 for the user to hold and a recognition part 1.2 for image recognition (i.e. The recognition area in the previous article).

[0089] The front and back sides of the identification part 1.2 are respectively set as the first identification surface 1.3 and the second identification surface 1.4, and the first identification surface 1.3 and the second identification surface 1.4 are respectively provided with characteristic images which can be distinguished from each other and are convenient for image identification. The surface 1.3 and the second recognition surface 1.4 respectively correspond to different functions, and can be used to switch between different point-to-read audio or point-to-read / question-and-answer modes. In order to avoid confusion, prompts ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a click-and-read and question answering method and system based on image recognition and tracking technology. The method comprises a click-and-read mode and a question answering mode. When the click-and-read mode is set, if a user clicks a triggering area on a page by a click-and-read pointing component, a pre-recorded click-and-read audio corresponding to the trigger areais played. When the question answering mode is set, a pre-recorded question audio corresponding to a current page is played, whether the user clicks the triggering area by the click-and-read pointingcomponent is judged, and a judgment result voice is returned. The system comprises a camera device, a click-and-read question answering device, a storage device and an audio playback device. Accordingto the invention, a book reader can be helped to obtain a better reading experience, and the degree of knowledge grasp of the book reader can be checked through questions and answers.

Description

technical field [0001] The present invention relates to a point reading method and system, in particular to a point reading question answering method and system based on image recognition and tracking technology. Background technique [0002] The reading pen is a new generation of intelligent reading and learning tools developed by using optical code recognition technology and digital voice technology. It can simultaneously realize many functions such as point reading, repeat reading, follow-up reading, recording, and entertainment. The technical principle is to first encode the content of the book through OID, and use special printing technology to print the code on the special book, and then the user can scan the book code with a point reading pen to recognize and play the corresponding voice. When the user uses the reading pen to scan the content of the book, the sound content emitted by the reading pen is combined with the content on the scanned book, and augmented reali...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G09B5/04G09B7/02G06K9/60
CPCG09B5/04G09B7/02G06V10/20
Inventor 刘博许炯俞竣腾柳清侬继泽
Owner 昆明微想智森科技股份有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap