Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

A saccade path prediction method and apparatus based on machine learning

A machine learning and prediction device technology, applied in the field of image processing, can solve the problems of predicting fixation points relying on static saliency maps, insufficient prediction saccade paths, etc., achieving good universality and scalability, and eliminating the dependence of saliency maps. Effect

Active Publication Date: 2019-03-08
XIDIAN UNIV
View PDF10 Cites 8 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] The embodiment of the present invention provides a machine learning-based glance path prediction method and device, which solves the problem in the prior art that the gaze point prediction is too dependent on the static saliency map, and the technical problem that the glance path prediction in natural scene pictures is insufficient , to eliminate the dependence of the model on the saliency map, and take into account the timing between gaze points, and achieve good technical results on multiple public data sets

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A saccade path prediction method and apparatus based on machine learning
  • A saccade path prediction method and apparatus based on machine learning
  • A saccade path prediction method and apparatus based on machine learning

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0060] figure 1 It is a schematic flowchart of a saccade path prediction method based on machine learning in an embodiment of the present invention. Such as figure 1 As shown, the method includes:

[0061] Step 110: Obtain an image data set to be processed, wherein each image information in the image data set has corresponding truth information;

[0062] Specifically, the image data set to be processed refers to a set of multiple pictures waiting to be processed, and the corresponding truth value information refers to the coordinates of the gaze point of the corresponding image as a label.

[0063] Step 120: Make training samples of the image data set according to the truth information;

[0064] Further, the preparation of training samples of the image data set according to the truth value information specifically includes: processing the truth value information to obtain eye movement data information of N observers; The eye movement data of the observer is processed by the boundary;...

Embodiment 2

[0099] The effect of the present invention will be further described below in conjunction with simulation experiments.

[0100] 1. Simulation conditions:

[0101] In the simulation experiment of the present invention, the computer system used is Ubuntu 16.04, the machine learning framework is TensorFlow, the version is 1.1.0, the Python version is 2.7, the vector of the embedded matrix is ​​V×M, and V is based on different data sets. Corresponding adjustments, M takes 512, C takes 16, which means 8 fixation points.

[0102] 2. Simulation content:

[0103] In the simulation experiment of the present invention, the picture name and the Arabic numerals are mapped to form a dictionary, an experiment is designed for each data set, the training set pictures and the test set pictures are selected according to the numbers, and the corresponding eye movement data sets are processed to obtain labels. Use samples to train the LSTM network, use the gradient descent optimization algorithm RMSProp...

Embodiment 3

[0108] Based on the same inventive concept as the saccade path prediction method based on machine learning in the foregoing embodiment, the present invention also provides a saccade path prediction device based on machine learning, such as Figure 4 Shown, including:

[0109] A first obtaining unit, the first obtaining unit is configured to obtain a to-be-processed image data set, wherein each image information in the image data set has corresponding truth information;

[0110] A first production unit, the first production unit is configured to produce training samples of the image data set according to the truth information;

[0111] A second obtaining unit, the second obtaining unit is configured to obtain image feature representation information of the image information according to the image information;

[0112] A first construction unit, the first construction unit is configured to construct and train an LSTM network according to the image feature representation information and t...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides a saccade path prediction method and device based on machine learning, which relates to the computer technical field. The method comprises the following steps of obtaining an image data set to be processed, wherein each image information in the image data set has corresponding truth value information; obtaining an image data set to be processed; preparing a training sample of the image data set according to the truth value information; obtaining image feature representation information of the image information according to the image information; constructing and trainingan LSTM network according to the image feature representation information and the eye movement data sample; and predicting a scanning path according to the LSTM network. The invention solves the problem that the predicted fixation is too dependent on the static saliency graph in the prior art, and the technical problem that the predicted saccade path is insufficient in the natural scene pictures,thereby eliminating the dependence of the model on the saliency graph and achieving the good technical effect on a plurality of common data sets in consideration of the temporality between the fixation points.

Description

Technical field [0001] The present invention relates to the technical field of image processing, in particular to a method and device for saccade path prediction based on machine learning. Background technique [0002] With the rapid development of information technology, mankind has entered an era of large-scale data growth. Digital images and videos have become important carriers of information. Massive image data is an important part of obtaining information. How to effectively select the best from images Valuable information has gradually become a hot spot in the field of image processing. [0003] The problem of predicting the gaze point in the prior art is too dependent on a static saliency map, and the prior art also has many shortcomings in predicting the saccade path in natural scene pictures. Summary of the invention [0004] The embodiment of the present invention provides a saccade path prediction method and device based on machine learning, which solves the problem tha...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06K9/62
CPCG06T7/0002G06T2207/20081G06T2207/20084
Inventor 齐飞高帅石光明夏朝辉
Owner XIDIAN UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com