A method for visual fixation point extraction

Abstract

The present invention discloses a method for extracting visual fixation points, the specific process of which is as follows: for an original image X, a sensor is used to extract a focused image block s t ; Then use CNN to evaluate the network output of the focused image block s t The classification possibility prediction is based on the classification possibility p c The information entropy H(p c ) is used as the difference between the focused image block and the surrounding image, and c represents the focused image block s t The information entropy is used as the feedback of the environment, and the feedback loop uses reinforcement learning to calculate the accumulated reward of each action. The accumulated reward of the agent guides the search for the next focused image block. The present invention uses the powerful feature extraction ability of deep learning technology combined with the environmental interaction ability of reinforcement learning technology to realize the visual gaze point focus extraction function.

Description

Technical Field

[0001] The present invention relates to the technical field of machine learning, and in particular to a method for extracting visual fixation points. Background Art

[0002] "Visual attention" is the autonomous behavior of an intelligent system to maintain its own purpose, which is the essence of the attention problem. Attention ability is the first reaction to a large amount of information after perceiving external information. Whether the information of external stimuli meets the purpose of the system will inevitably become the basis for determining whether the system pays attention to this information. Information related to the target needs attention. In the field of machine learning, there are methods to seek suitable mathematical models to simulate the selective attention function of the human visual system.

[0003] At present, the following three mathematical models for extracting salient parts of images have been formed: A. Space-based visual attention model: It is a bottom-up data-driven model that is driven only by perceptual data to guide the viewpoint of a person to the salient area in the scene, which usually has a strong contrast with the surrounding area. The idea of ​​this model is to first extract features (color, brightness, motion, texture, etc.) through a filtering algorithm based on the input image, analyze and fuse the feature map to obtain an interest map, and finally select the only attention target from multiple candidate areas to be noticed in the interest map through a certain competition mechanism. B. Object-based visual attention model: It is a top-down task-oriented driven model determined by human "cognitive factors", such as knowledge, expectations and current goals. The idea of ​​the model is to perform multi-scale processing of the input image using a Gaussian pyramid to form a coarse-scale, coarse-scale, and fine-scale scene map grouping, calculate the salience of each group and subgroup, and start attention competition. The model naturally realizes a hierarchical selection and attention mode to achieve attention transfer. C. Top-down combined with bottom-up visual focusing method: The factors that guide attention allocation in the human visual system are combined by these two parts to jointly act on the visual focusing result. In recent years, a variety of computational models have been proposed to simulate human visual attention mechanisms, including models based on cognition, Bayesian, decision theory, information theory, graph models, frequency domain analysis, and pattern classification. Computational modeling of visual attention has made great progress and has been applied to target detection and recognition, and video quality assessment.

[0004] These algorithms have great influence in the field of visual focus mechanism research. Although they have achieved certain results in related experiments, it is difficult for visual attention models to simulate the saliency of objects in complex scenes in the objective world. For example, the points that attract people's attention in the scene include visual color, texture, sound, taste, etc. The calculation speed, execution effect, and robustness of each model are still far behind the human visual system. In particular, the visual focus technology of the top-down task-oriented driven model determined by human cognitive prior knowledge and emotional psychological factors has poor application effect. Therefore, the research on visual attention modeling methods is mostly focused on the bottom-up visual attention process driven by data, while the top-down visual attention based on task-driven is difficult to model and analyze, so the related research methods and theories are relatively limited. However, the top-down task-driven visual attention model that permeates human prior knowledge and prejudgment consciousness is more in line with the human visual focus mechanism. How to further simulate the attention area selection mechanism of saliency feature extraction in the process of visual information perception, establish a calculation and analysis model that is more in line with the visual characteristics of the human eye, and improve the efficiency of information processing in the robot vision system is the goal. Summary of the invention

[0005] In view of the shortcomings of the prior art, the present invention aims to provide a method for extracting visual gaze points, which applies the powerful feature extraction capability of deep learning technology and the environmental interaction capability of reinforcement learning technology to achieve the visual focusing function.

[0006] In order to achieve the above object, the present invention adopts the following technical solution:

[0007] A method for extracting visual fixation points, the specific process is as follows:

[0008] (1) For the original image X, use the perceptron to extract the focused image block s t ;

[0009] (2) Then use the CNN evaluation network to output the focused image block s t The classification possibility prediction;

[0010] (3) Using focused image block s t The classification probability p c The information entropy H(p c ) judges the difference between the focused image block and the surrounding image, and the information entropy H(p c ) is calculated as follows:

[0011]

[0012] c represents the focused image patch s t Category of;

[0013] (4) Information entropy is used as feedback from the environment. The feedback loop uses reinforcement learning to calculate the accumulated reward of each action. The accumulated reward of the agent guides the search for the next focused image block.

[0014] Furthermore, in step (4), the agent adopts the Qlearning algorithm based on the different-strategy temporal difference learning algorithm in the value function method to iteratively calculate; the strategy refers to the mapping from state to action, and the strategy is changed according to the value function to establish the reinforcement learning objective function Seek the optimal parameter θ to maximize the expected cumulative return, π θ Represents the strategy; the optimal strategy is obtained according to the reinforcement learning results, guiding the current state s t The action to take t , that is, the focus position of the next move; output the next focus position l t+1 , enter the loop of extracting the next focused image block; the current state s t It refers to the current focused image block. Each focused image block extracted by the sensor is recorded as (s0, s1, s2...s t ....s T ), a trajectory is completed in T+1 steps; R is the reward to the decision maker, and T is the number of steps.

[0015] Furthermore, in step (1), the process of extracting the focused image block by the perceptron is as follows: obtaining the focused image block s by function mapping t .

[0016] Furthermore, in step (2), the CNN evaluation network is used to output the focused image block s t The process of predicting the classification possibility is:

[0017] 1) Determine the focused image block category c;

[0018] 2) Focus on the predicted probability p of the image block classification possibility c =P(c|s t ), the CNN output is a softmax layer, which does not necessarily have output at each step. It can be set to output a classification possibility prediction probability after K steps; using p c When c is the maximum value, it is used as the category of the input focused image block, that is, the category of the focused image block

[0019] The beneficial effect of the present invention is that in the method of the present invention, a captured frame of image or a video (continuous multiple frames of image) is used to select a focused image block in the image space through a visual focusing method based on reinforcement learning, and then an understanding of the environmental field of view is obtained through the understanding of a series of focused image blocks. On this basis, combined with the subjective task requirements of the robot, the visual system bionic eye can be driven to move to a certain posture, thereby achieving the purpose of imitating the human eye focusing and staring at the area of ​​interest. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0021] The present invention will be further described below in conjunction with the accompanying drawings. It should be noted that this embodiment is based on the technical solution and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to this embodiment.

[0022] The existing visual focus model is still far inferior to the visual attention process of the human eye in terms of processing speed and execution effect. The method in this embodiment starts from the perspective of machine learning, adopts deep learning and reinforcement learning technology and combines them to realize the problem of visual focus point selection and transfer.

[0023] When human vision sees things, it focuses on a certain point in the scene (video sequence) and outputs a series of image blocks of the focus area. The selection of the area is a sequential decision problem, which can be expressed in the Markov decision process framework. To simulate the way humans see things, the robot vision system processes the input in chronological order, processes different positions in the picture at a time, and gradually combines the information of these parts to establish an understanding of the scene or a dynamic interval representation of the environment. On the one hand, it can reduce the interference of unnecessary information and reduce the impact of noise, and then it can also reduce the amount of calculation.

[0024] Since deep reinforcement learning has the general intelligence to solve complex problems, using reinforcement learning to define problems and optimize goals, and deep learning to solve problems such as feature extraction, state representation, and strategy representation will achieve better expected results than traditional mathematical modeling methods. Task driving based on visual focus can be regarded as a control problem.

[0025] This embodiment provides a method for extracting visual gaze points. The specific process is as follows: Figure 1 As shown:

[0026] (1) For the original image X, use the perceptron to extract the focused image block s t ;

[0027] (2) Then use the CNN evaluation network to output the focused image block s t The classification possibility prediction;

[0028] (3) Using the focused image block s t The classification probability p c The information entropy H(p c ) Judge the focused image patch s t The difference from the surrounding images, information entropy H(p c ) is calculated as follows:

[0029]

[0030] c represents the focused image patch s t Category.

[0031] (4) Information entropy is used as feedback from the environment. The feedback loop uses reinforcement learning to calculate the accumulated reward of each action. The agent’s accumulated reward guides the search for the next focused image block.

[0032] It should be noted that the agent uses the Q-learning algorithm based on the different-strategy time-difference learning algorithm in the value function method for iterative calculation. The strategy refers to the mapping from state to action. The strategy is changed according to the value function to establish the reinforcement learning objective function. Seek the optimal parameter θ to maximize the expected cumulative return, π θ Indicates strategy.

[0033] According to the reinforcement learning results, the optimal strategy is obtained, guiding the current state s t (i.e. the current focused image block) the action to be taken a t , that is, the focus position of the next move, output the next focus position l t+1 , entering the loop of extracting the next focused image block.

[0034] It should be noted that the decision process is described by a tuple (S, A, P, R, T), where the state s represents the focused image block collected at different times, and the focused image block extracted by the perceptron is recorded as (s0, s1, s2...s t ....s T ), a trajectory is completed in T+1 steps. Action A is the position of the agent in the scene (may be represented by map position or clock position); state transition probability p = P(s t+1 |s t ,a), R is the observed return of the decision, and T is the number of steps.

[0035] Furthermore, the process of extracting the focused image block by the perceptron is as follows: obtaining the focused image block s through function mapping t , l tis the focus position. Similar to the retinal variation rate imaging effect, high-resolution imaging is performed near the gaze point and low-resolution imaging is performed in the periphery.

[0036] Furthermore, the CNN evaluation network is used to output the focused image patch s t The process of predicting the classification possibility is:

[0037] 1) Determine the category c of the focused image block;

[0038] 2) Focus on the predicted probability p of the image block classification possibility c =p(c|s t ), using p c When c is the maximum value, it is regarded as the category of the focused image block, that is, the category of the focused image block

[0039]

[0040] Furthermore, the calculation of the return value R is as follows:

[0041]

[0042] y* is the label of the focused image patch, i.e., the category. t ) is the probability p that the focused image patch belongs to category c c Information entropy.

[0043] Q(s t ,a t )←Q(s t ,a t )+α[r+γmaxQ(s t+1 ,a t+1 )-Q(s t ,a t )

[0044] Q(s,a)=f Q (s t ,θ Q ),θ Q are the parameters of the Q network, which is a CNN deep learning network.

[0045] The stochastic gradient descent method is used to optimize the cost function of the Q network.

[0046] For those skilled in the art, various corresponding changes and modifications can be made according to the above technical solutions and concepts, and all of these changes and modifications should be included in the protection scope of the claims of the present invention.

Claims

1. A method for extracting visual fixation points, characterized in that: The specific process is: (1) For the original image X, use the perceptron to extract the focused image block s t ; (2) Then use the CNN evaluation network to output the focused image block s t The classification possibility prediction process is: 1) Determine the focused image block category c; 2) Focus on the predicted probability p of the image block classification possibility c =P(c|s t ), the CNN output is a softmax layer, which does not necessarily have output at each step. After setting K steps, a classification possibility prediction probability is output; using p c When c is the maximum value, it is used as the category of the input focused image block, that is, the category of the focused image block (3) Using focused image block s t The classification probability p c The information entropy H(p c ) judges the difference between the focused image block and the surrounding image, and the information entropy H(p c ) is calculated as follows: c represents the focused image patch s t Category of; (4) Information entropy is used as feedback from the environment. The feedback loop uses reinforcement learning to calculate the accumulated reward of each action. The accumulated reward of the agent guides the search for the next focused image block. The intelligent agent adopts the Q-learning algorithm based on the different-strategy time-difference learning algorithm in the value function method to iteratively calculate; the strategy refers to the mapping from state to action, and the strategy is changed according to the value function to establish the reinforcement learning objective function Seek the optimal parameter θ to maximize the expected cumulative return, π θ Represents the strategy; the optimal strategy is obtained according to the reinforcement learning results, guiding the current state s t The action to take t , i.e. the focus position of the next move; Output the next focus position l t+1 , enter the loop of extracting the next focused image block; the current state s t It refers to the current focused image block. Each focused image block extracted by the sensor is recorded as (s0, s1, s2...s t ....s T ), a trajectory is completed in T+1 steps; R is the reward to the decision maker, and T is the number of steps.

2. The method according to claim 1, characterized in that In step (1), the process of extracting the focused image block by the perceptron is as follows: the focused image block s is obtained by function mapping t , l t is the focus position.

Images