Book point reading method and system based on deep learning

A deep learning and book point technology, applied in the field of book point reading, can solve the problems of cumbersome operation, fixed image and text recognition and poor accuracy, so as to improve accuracy, simplify point reading detection and recognition methods, and avoid point reading errors. Effect

Pending Publication Date: 2020-06-30
暗物智能科技(广州)有限公司
View PDF13 Cites 8 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] Therefore, the technical problem to be solved by the present invention is to overcome the defects of cumbersome operation, fixed image and character recognition and poor accuracy in the prior art, so as to provide a method and system for point reading of books based on deep learning

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Book point reading method and system based on deep learning
  • Book point reading method and system based on deep learning
  • Book point reading method and system based on deep learning

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0028] The book point reading method based on deep learning provided by the embodiment of the present invention can be applied to the field of text detection and recognition, such as figure 1 As shown, including the following steps:

[0029] Step S11: Obtain a finger point reading image taken by the image acquisition device.

[0030] Finger point reading recognition is divided into finger detection and text detection. Traditional finger detection requires gestures to draw a rectangle in the text area to determine the point reading area, and recognize the rectangular area based on algorithms such as skin color segmentation, and finally detect the text in the rectangular area. However, in the embodiment of the present invention, the image acquisition device is used to capture the text that needs to be clicked, without using a point-reading pen, only a finger is needed to point to the area that needs to be clicked, and it is simpler than the traditional finger swipe area detection.

[0...

Embodiment 2

[0071] The embodiment of the present invention provides a book point reading system based on deep learning, such as Image 6 Shown, including:

[0072] The image acquisition module 1 acquires the finger point reading image taken by the image acquisition device; this module executes the method described in step S1 in the embodiment 1, which will not be repeated here.

[0073] Fingertip position and text detection module 2, which inputs the finger point reading image into the finger detection model and text detection model at the same time, respectively detects the fingertip position and all text areas in the image; this module executes the step S2 described in embodiment 1 The method will not be repeated here.

[0074] The text area cutting module 3 combines the detected fingertip position and the text area to perform affine transformation on the text area to cut the text area; this module executes the method described in step S3 in embodiment 1, and will not be repeated here. .

[00...

Embodiment 3

[0080] The embodiment of the present invention provides a computer device, such as Figure 7 As shown, it includes: at least one processor 401, such as a CPU (Central Processing Unit, central processing unit), at least one communication interface 403, memory 404, and at least one communication bus 402. Among them, the communication bus 402 is used to implement connection and communication between these components. The communication interface 403 may include a display (Display) and a keyboard (Keyboard), and the optional communication interface 403 may also include a standard wired interface and a wireless interface. The memory 404 may be a high-speed RAM memory (Ramdom Access Memory, volatile random access memory), or a non-volatile memory (non-volatile memory), such as at least one disk memory. Optionally, the memory 404 may also be at least one storage device located far away from the aforementioned processor 401. The processor 401 can execute the book point reading method b...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a book point reading method and system based on deep learning, and the method comprises the steps: enabling an obtained finger point reading image photographed by an image collection device to be inputted into a finger detection model and a text detection model at the same time, and respectively detecting the fingertip position and all text regions in the image; conductingaffine transformation and cutting on the character area according to the detected fingertip position and the detected character area; inputting the cut character area into a character recognition model to recognize character information; carrying out sentence segmentation or segmentation processing on the text information by utilizing punctuation or segmentation character information; and outputting individual characters, words, sentences or text segments by voice according to a preset click-to-read demand. Finger point reading images are shot through the image acquisition device, so that theexpansibility of point reading content is improved; the finger detection model, the text detection model and the text recognition model are trained, point reading of single Chinese characters, words,sentences and text segments is achieved, a traditional point reading detection and recognition method is simplified, and the accuracy of character detection and recognition is improved.

Description

Technical field [0001] The invention relates to the technical field of book point reading, in particular to a book point reading method and system based on deep learning. Background technique [0002] In recent years, with the rapid development of computer vision and deep learning, technologies such as taking photos for literacy, taking photos to search for questions, and reading supplementary studies have been widely used in intelligent education. Among them, the point-reading machine is a popular learning aid tool. It uses finger detection and text detection functions to perceive the location of the content pointed to by the user’s finger, and the text information content of the location area can be identified based on the location. So as to complete the process of human-computer interaction. However, the dot reading machine in the prior art has the following disadvantages: first, it is necessary to use gestures to draw a rectangle on the text area to determine the dot reading...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06K9/34G06N3/04G06N3/08G09B5/06
CPCG06N3/08G09B5/062G06V30/153G06V10/267G06N3/044G06N3/045
Inventor 黄炜恒张俊怡罗丹陈添水陈崇雨
Owner 暗物智能科技(广州)有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap