103 results about "Visual attentiveness" patented technology

The invention is a system and method for determining visual attention, and supports the eye tracking measurements with other physiological signal measurements like emotions. The system and method of the invention is capable of registering stimulus related emotions from eye-tracking data. An eye tracking device of the system and other sensors collect eye properties and/or other physiological properties which allows a subject's emotional and visual attention to be observed and analyzed in relation to stimuli.

The present invention relates to a system and method for attentional selection. More specifically, the present invention relates to a system and method for the automated selection and isolation of salient regions likely to contain objects, based on bottom-up visual attention, in order to allow unsupervised one-shot learning of multiple objects in cluttered images.

A method to augment, virtualize, or present mixed reality of content to display a larger amount of content on a head worn device (HWD) including: creating a set of virtual screens in the display of the HWD deployed in a manner to both surround and positionally shift from a movement of the viewer to provide an immersive viewing experience wherein the set of virtual screens include: at least a primary virtual screen and one or more secondary virtual screens; enabling the immersive viewing experience by generating dynamic virtual screen arrangements; connecting a first sensor attached to a head of the viewer and a second sensor attached to a torso of the viewer for generating sensed data of rotating differences of the positionally shift of the viewer; and configuring the content on each of the virtual screens to determine a particular virtual screen having a visual attention of the viewer.

The invention discloses a visual dialogue generation method based on a double visual attention network. The method comprises the following steps: 1, preprocessing text input in a visual dialogue and constructing a word list; 2, feature extraction of a dialogue image and feature extraction of a dialogue text; 3, attention processing is performed on the historical dialogue information based on the current problem information; 4, independent attention processing of the double visual features is carried out; 5, attention processing of mutual intersection of the double visual features; 6, optimizing the visual features; 7, performing multi-modal semantic fusion and decoding to generate an answer feature sequence; 8, optimizing parameters of a visual dialogue generation network model based on the double visual attention networks; 9, prediction answer generation. According to the invention, more complete and reasonable visual semantic information and finer-grained text semantic information can be provided for the intelligent agent, so that the reasonability and accuracy of answers predicted and generated by the intelligent agent to questions are improved.

The invention belongs to the technical field of cross-media generation, and particularly relates to a video description generation system based on a graph convolution network. The video description generation system comprises a video feature extraction network, a graph convolution network, a visual attention network and a sentence description generation network. The video feature extraction network performs sampling processing on videos to obtain video features and outputs the video features to the graph convolution network. The graph convolution network recreates the video features accordingto semantic relations and inputs the video features into sentence descriptions to generate a recurrent neural network; and the sentence description generation network generates sentences according tofeatures of video reconstruction. The features of a frame-level sequence and a target-level sequence in the videos are reconstructed by adopting graph convolution, and the time sequence information and the semantic information in the videos are fully utilized when description statements are generated, so that the generation is more accurate. The invention is of great significance to video analysisand multi-modal information research, can improve the understanding ability of a model to video visual information, and has a wide application value.

An image processing apparatus including a region of interest (ROI) configuration unit may generate a visual attention map according to visual characteristics of a human in relation to an input three dimensional (3D) image. A disparity adjustment unit may adjust disparity information, included in the input 3D image, using the visual attention map. Using the disparity information adjusted result, a 3D image may be generated and displayed which reduces a level of visual fatigue a user may experience in viewing the 3D image.

The invention provides an Alzheimer's disease rehabilitation training and ability evaluation system based on virtual reality. The system comprises a rehabilitation training module, an ability evaluation module and a feedback module, wherein the rehabilitation training module is used for conducting rehabilitation training on an Alzheimer's disease patient in the aspects of cognition and social contact; the ability evaluation module is used for collecting the visual attention information, the behavior mode information and social communication ability information generated by the rehabilitation training module, carrying out comprehensive evaluation according to the information and generating a comprehensive evaluation result; and the feedback module is used for feeding back the evaluation result to the rehabilitation training module and changing the content and process of rehabilitation training. According to the invention, the behavior mode of the testee is amplified by using the virtualreality technology, the data is analyzed and processed by combining an artificial intelligence technology, a comprehensive evaluation result can be obtained by comparing the data with a standard control group and an Alzheimer's disease control group, and the content and process of rehabilitation training can be adjusted by combining real-time feedback and stage feedback to adapt to the current state of the Alzheimer's disease patient.

The invention discloses an airborne photoelectric video target intelligent detection and identification method. On the basis of a YOLOv3 model, rectangular convolution is adopted to extract strip-shaped target features such as a bridge, expansion convolution is adopted to expand a receptive field and reserve spatial structure information of a multi-scale target, a visual attention mechanism is introduced into a sampling branch on a feature pyramid to endow the model with different weights of target features of different regions and different channels, and a convolution mode of a residual module is improved into depth separable convolution to reduce calculation complexity. The method has the following advantages: high aerial photography target detection precision is kept, and meanwhile, high aerial photography target detection and recognition speed can be achieved on an airborne embedded system.

The invention provides a multi-scale self-attention target detection method based on weight sharing. A device for the method is composed of a weight-shared multi-scale convolutional network and a visual self-attention module. According to the method, multi-scale, an attention mechanism, fine-grained feature extraction and lightweight of a model can be considered to a certain extent; a residual block and expansion convolution technology is innovatively fused into a multi-scale convolution network; the method has the capability of realizing deep feature extraction while the lightweight of the model is ensured; the visual attention mechanism is introduced into the multi-scale convolution network, so that the network can pay attention to key areas in the image, and computing resources are saved. The method is wide in adaptability and high in robustness, and can be used for various target detection tasks. The method is experimented on a well-known data set, experimental results show that the method has high accuracy while ensuring the lightweight of the model, average accuracy of 73.6 is obtained, and effectiveness of the method is proved.

An apparatus for coding an image includes a setting unit that sets quantization width for each coded block of an image frame of image data. The apparatus also includes a visual attention calculating unit that calculates a level of visual attention to a first element for each coded block of the image frame; and a perceptual distortion calculating unit that calculates a level of perceptual distortion of a second element whose distorted image quality is easily visually perceived, for each coded block of the image frame. The apparatus also includes a correcting unit that corrects the quantization width based on the level of visual attention and the level of perceptual distortion; and a quantizing unit that quantizes the image data based on the corrected quantization width.

The invention discloses a visual attention recognition method and system, a storage medium and a processor. The method comprises the following steps: acquiring first image information of a target areathrough a first camera; identifying a face predetermined area of the first target object from the first image information; analyzing the predetermined area of the face by using a first model to obtain the facial feature points of the first target object, then determining the head posture and the visual direction of the first target object according to the facial feature points, training the firstmodel through machine learning by using multiple sets of data, wherein each set of data in the multiple sets of data comprises at least one of the following: an image and the facial feature points identified in the image; and determining an object of visual attention of the first target object according to the head posture and the visual direction. The technical problem that the attention of theuser cannot be accurately recognized in the prior art is solved.

The invention discloses a video behavior identification method based on a spatio-temporal feature fusion deep learning network, which adopts two independent networks to extract time and space information of a video respectively, adds LSTM learning video time information to each network on the basis of a CNN, and fuses the time and space information with a certain strategy. The accuracy of the FSTFN on a data set is improved by 7.5% compared with a network model without introducing a space-time network proposed by Tran, the accuracy on the data set is improved by 4.7% compared with a common double-flow network model, a segmentation mode is adopted for a video, each video sample samples a network composed of a plurality of segments and inputs the segments into a CNN and an LSTM, and by covering the time range of the whole video. The problem of long-term dependence in video behavior recognition is solved, a visual attention mechanism is introduced to the tail end of the CNN, the weight ofa non-visual subject in a network model is reduced, the influence of the visual subject in a video image frame is improved, and the spatial features of the video are better utilized.

The invention discloses a multi-screen display system and a mouse switching control method, and the mouse switching control method is applied to the multi-screen display system comprising a main display screen and at least one extended display screen, and comprises the steps: obtaining a user image collected by each camera; the cameras are installed on the main display screen and the extended display screen respectively. Inputting each user image into a neural network model, and predicting a screen currently concerned by a user by utilizing the neural network model to obtain a prediction result; and according to the prediction result, controlling to switch the mouse to the screen which the user pays attention to currently. According to the multi-screen display system and the mouse switching control method, on the basis of visual attention self-learning, the current user operation screen is predicted by analyzing the visual attention pointing of the current user, the mouse is automatically switched to the corresponding screen position, and the user experience is improved.

The invention provides a multi-exposure image fusion method based on an attention generative adversarial network. The thought of the attention mechanism is highly matched with the detail weighting problem in multi-exposure fusion, the weight of each input image can be adaptively selected by applying channel attention, and the weights of different spatial positions are adaptively selected by usingspatial attention. The technology has a wide application prospect in various multimedia vision fields. According to the algorithm, a new attention generative adversarial network is designed to be usedfor a multi-exposure image fusion task, and a visual attention mechanism is introduced into the generative network, so that the network can be helped to adaptively learn weights of different input images and different spatial positions to achieve a better fusion effect.

The invention provides an intelligent screening method and an intelligent screening system for abnormal tongue image. The intelligent screening method and the intelligent screening system have the beneficial effects that: by constructing a knowledge platform of traditional Chinese medicine tongue diagnosis and an expert auxiliary diagnosis system, an objective and quantitative measurement means isprovided for tongue diagnosis information judgment, and the diagnosis correctness is improved; a fine-grained image classification algorithm based on an attention mechanism is adopted, and the attention mechanism reflects the perception difference of a human vision system to the surrounding environment, that is, attention is focused on a salient region in the environment; a convolutional neural network is used for simulating visual attention properties of human beings, so that information of classification targets is fully utilized, and the accuracy rate of tongue image fine-grained image classification can be improved; under the inspiration of the visual attention mechanism, spatial position features are introduced into classification and modeling is carried out in the process of utilizing the fine-grained image classification algorithm, namely, a plurality of key regions of a tongue image are positioned, modeling is carried out according to the spatial relationship among all parts of the tongue image, and the integrity and discrimination of part features can be improved.

The invention discloses a teaching quality evaluation method based on eye movement tracking, and the method comprises the steps: calculating a visual attention score of a student according to the correlations between a visual path of the student and a cursor path in a teaching process of a teacher, a display path of course contents in the teaching process of the teacher, and a teaching content sequence in the teaching process of the teacher; and generating the attention mode of the student according to the relationship between the attention score and the preset attention score threshold, so that the attention of the student in the online learning process is visualized, and the learning efficiency of the student can be quantified. Furthermore, according to the attention mode, the quantitative index and the assessment result of the previous course of the same subject of the student, the attention mode, the quantitative index and the assessment result of the future course of the subject of the student are predicted, and prediction of future learning quality and assessment results of the student is realized. In addition, the invention also discloses a teaching quality evaluation deviceand system based on eye movement tracking, and a computer readable storage medium.

The invention relates to the technical field of image retrieval and computer vision, in particular to a fine-grained image retrieval method based on visual attention mechanism weighting. The method comprises the following steps of: image preprocessing: setting the length of the longest side of an image to be 500 pixels; feature extraction: inputting the image into a convolutional neural network, and then selecting and outputting the features of the last convolutional layer; target feature selection: firstly, optimizing a local activation graph, and then selecting a local feature vector according to an activation graph result, so as to realize more accurate target feature selection; feature weighted aggregation: evaluating the importance degree of each feature, so as to enable the weightedfine-grained local features to still be embodied during pooling aggregation and improve the precision of fine-grained retrieval; and performing image retrieval, and calculating cosine similarity between the characteristic vectors of the queried image and a database image. An image feature extraction and coding detail graph is shown in figure 1. According to the method, fine-grained image retrievalcan be realized, and the retrieval accuracy is improved.

The invention discloses a road scene semantic segmentation method based on multi-model fusion. The method comprises the following steps: building a multi-classification model and a binary-classification model; respectively carrying out end-to-end training on the multi-classification model and the two-classification model to obtain an optimal weight value enabling a loss function to be minimum; performing multi-classification prediction and dichotomy prediction on the road scene image by using the optimal weight value to form a preliminary segmentation result image; performing image post-processing on a preliminary segmentation result image formed by binary classification prediction; and fusing the preliminary segmentation result image formed by multi-classification prediction and the segmentation result image after image processing. According to the multi-classification model, visual attention is added to a feature fusion part on the basis of an original HRNet, so that effective feature maps obtain larger fusion weights, invalid or poor-effect feature maps obtain smaller fusion weights, the pixel representation capability of the multi-classification model is improved, and a better segmentation result is obtained.

The invention discloses a novel efficient iris image quality evaluation method based on a deep neural network. A feature extraction model is used to extract a feature map of an iris image in an inputimage, then a reconstruction model is used to estimate an iris effective area thermodynamic diagram from the feature map of the iris image, finally a quality prediction model uses the iris effective area as an area of interest, and the overall quality score of the iris image is calculated from the feature map. According to the method, other processes such as preprocessing or segmentation and positioning do not need to be carried out on the collected eye image;, the deep neural network can be directly used for extracting the global features of the eye image, the thermodynamic diagram of the iris effective area is automatically estimated according to the extracted features, the global features of the iris and the thermodynamic diagram of the iris effective area are combined through a visualattention mechanism, and quality evaluation is carried out on the iris image. The iris image quality evaluation method provided by the invention is simple in process, high in calculation speed and high in robustness and adaptability.