230 results about "Visual attentiveness" patented technology

The invention relates to a strong supervision change detection method based on a convolutional neural network and a visual attention model, and belongs to the field of remote sensing and computer vision. A pair of remote sensing paste images at different moments is read into the model, the model comprises a CNNTTransConv module, a mixed double-attention module and a multi-scale strong supervision module, the model is optimized through a loss function, the remote sensing images at different moments pass through the optimized model, a change graph is obtained, and a change detection task is achieved. The system has the advantages that the feature extraction module improves the expression ability of global features; the hybrid double attention module enhances the ability of the network to obtain more obvious change information, and the multi-scale strong supervision module supervises output features of different scales without introducing additional parameters, thereby further enhancing the ability of the network to distinguish change features. The method realizes the most advanced performance, and can be widely applied to remote sensing change detection tasks of various scenes.

The invention provides an end-side multi-modal large model accelerated reasoning method and system, and the method comprises the steps: carrying out the two-stage screening and rearrangement of visual tokens based on the CLS attention and text-to-visual attention in a visual encoder and pre-filling stage, and constructing a sparse attention and sparse key value cache; in a decoding stage, an important neuron set is judged according to activation gating or historical statistics, only a corresponding feedforward network weight is pulled and calculated, missed weights are loaded on demand through asynchronous I / O, and hot neurons are maintained in a high-speed memory to utilize model sparsity, so that video memory / memory occupancy and calculation overhead are remarkably reduced on an end side; throughput and time delay performance are improved. According to the method, the internal memory and computing resources required by reasoning of the multi-modal large language model are reduced from two dimensions by utilizing the endogenous sparsity of the end-side large language model in input and the model, so that a higher reasoning speed is achieved by utilizing fewer resources on the premise of keeping the size of the model unchanged, and the performance of the whole system is improved.

The invention discloses a self-adaptive teaching strategy adjustment method based on sentiment analysis and a computer device. The method comprises the following steps: obtaining a target emotion feature data stream according to multi-modal student emotion data, and then extracting visual deep features, audio deep features and physiological deep features from the target emotion feature data stream; processing the visual attention weight, the audio attention weight, the physiological attention weight, the visual deep feature, the audio deep feature and the physiological deep feature according to a preset fusion strategy to obtain a multi-dimensional fusion feature, and analyzing the feature to obtain basic emotion recognition information of the individual student; analyzing the basic emotion recognition information through a built personalized emotion model of each student individual to obtain corresponding emotion state evaluation information; and on the basis of the constructed teaching knowledge graph and the emotional state evaluation information, generating a personalized teaching adjustment strategy so as to dynamically adjust an actual teaching scheme. According to the method, dynamic adjustment can be realized, and personalized teaching requirements are met.

The invention belongs to the technical field of intelligent education, and particularly relates to a classroom attention detection method and system based on multi-modal data fusion. Aiming at the problems of high equipment cost, low multi-source data fusion efficiency, insufficient privacy protection and the like in the prior art, the invention provides the following solutions: collecting face, eye movement, posture, voice signals and heart rate variability data of a student through a sensor; multi-modal data synchronization is realized by adopting a time sequence alignment algorithm; respectively extracting a visual attention feature, a voiceprint matching feature and a physiological wake-up feature by using a lightweight deep learning model; constructing a multi-modal data fusion network, and dynamically adjusting a feature weight in combination with a classroom scene; attention anomaly detection is realized by adopting a hybrid model, and real-time early warning is output through edge computing equipment. The method has the beneficial effects that the hardware cost is greatly reduced while the detection precision is ensured, and the privacy of students is effectively protected; a dynamic weight distribution mechanism improves the adaptability of different teaching scenes.

The invention provides an accurate transportation control method and system for a remediation agent for in-situ remediation of underground water, and relates to the technical field of underground water pollution remediation, and the method comprises the steps: collecting an image of a drilling rock core of a polluted site and sensor data, extracting features through a visual attention network and a time sequence attention network, and carrying out the feature extraction; the features are input into a conditional probability diffusion model and fused to generate hydrogeologic feature vectors, a high-precision hydrogeologic feature model is constructed in combination with space attention mapping and a generative diffusion model, the high-precision hydrogeologic feature model is input into a multi-region collaborative simulation environment for optimization calculation, and monitoring data are fused through edge calculation nodes. And training the lightweight control model to execute repair medicament transport control.

The invention provides a 3D modeling method based on a digital twin city, and belongs to the technical field of 3D city modeling. Through a multi-source semantic data fusion acquisition means, space-time semantic tags are added to various types of data, the problem of multi-source data isomerism is solved, and semantic unification and efficient fusion of different types of data are realized; by means of hierarchical dynamic modeling means, a building component network is constructed based on triple association of geometry, functions and semantics, so that a model structure better conforms to the logic of the building industry, and the reasonability and efficiency of modeling are improved; a virtual-real two-way mapping means is applied, incremental updating of the model is triggered through a cross-modal semantic comparison algorithm, limitation of traditional static modeling is broken through, and dynamic synchronization of the virtual model and the physical world is achieved; an intelligent optimization means is adopted, a multi-constraint-condition generative adversarial network is combined with a visual attention mechanism, it is ensured that generated textures conform to building geometric features and material specifications, and meanwhile intelligent distribution of rendering resources is achieved.

The invention relates to the technical field of data processing, and discloses an automatic visual large-screen display processing method based on a control model and data driving, which comprises the following steps: optimizing large-screen information display, and switching parameters through layout density, visual attention and scenes; dynamically adjusting a data display strategy, and calculating data freshness, abnormal fluctuation and correlation parameters; user interaction experience is improved, and hot area response and multi-dimensional drilling parameters are optimized; enhancing information readability, and calculating graphic cognition and color conflict parameters; the system performance is stabilized, and the rendering frame rate and the data throughput are monitored; the business value, the attention decision response and the abnormity discovery timeliness are measured; dynamic weight, abnormal fusing, self-learning and multi-dimensional evaluation systems are adopted in intelligent optimization. According to the method, dynamic weight distribution, an abnormal fusing mechanism, self-learning optimization and a multi-dimensional evaluation system are combined, an intelligent, real-time and precise large-screen display system is achieved, and information display reasonability, interaction experience and system stability are improved.

The invention relates to the technical field of artificial intelligence, can be applied to business scenes such as agent autonomous decision making, financial science and technology and medical health, and discloses a task processing method and device based on visual attention enhancement, equipment and a medium. Visual hierarchical features are extracted, a double fovea attention module processes and fuses high-level visual features, a side suppression network obtains enhanced visual features, and a cross-modal fusion module generates fusion features by taking the enhanced visual features as query vectors and taking language components and action components as key and value vectors; and fusing the feature input decision network to generate target category and position information, generating feedback information based on actual label difference, and updating module parameters to complete a target task. According to the invention, through combination of a bionic vision mechanism and multi-modal attention fusion, the visual feature extraction and background suppression capability is improved, and the target capture efficiency and recognition precision in a complex scene can be improved.

The invention discloses an aluminum ingot surface defect detection method and system based on visual acquisition, and relates to the technical field of aluminum product detection, and the method comprises the steps: constructing an illumination adjustment model, and generating an optimal illumination parameter according to a preprocessed aluminum ingot preliminary image and physical condition information; a high-quality image with the optimal illumination parameter is collected, global evaluation and local focusing are carried out on the collected high-quality image, and detailed information of a high-attention area is generated; performing defect judgment on the high-attention region based on the detailed information of the high-attention region; performing comprehensive evaluation based on a defect judgment result; according to the invention, the dynamic generation of the optimal illumination parameter is realized, and the image acquisition quality is optimized; meanwhile, a bionic visual attention mechanism is introduced, a high-attention area is accurately positioned, and the recognition capability of fine defects is enhanced. The problem that in the prior art, image quality and tiny defect detection are insufficient is solved, and the accuracy and efficiency of aluminum ingot surface defect detection are remarkably improved.

The invention provides an automatic software testing method and system based on a visual attention mechanism, and relates to the technical field of automatic testing, and the method comprises the steps: obtaining an initial interface of to-be-tested software, carrying out the preprocessing, inputting an enhanced interface image into an interface detection model, and recognizing control information through the analysis of the enhanced interface image; inputting the control information into a pre-trained attention mechanism model, generating semantic association and context information between controls based on the control information, determining an execution sequence and execution parameters of test actions, sequentially simulating manual test actions, and performing test operation on the controls according to the execution parameters; interface feedback information of the to-be-tested software is obtained, text content in the interface feedback information is extracted based on a character recognition algorithm, semantics corresponding to the text content are determined in combination with a pre-constructed semantic understanding model and compared with a preset expected result, and a test conclusion corresponding to the test operation is generated.

The invention relates to a visual attention mobility anti-attack method based on self-adaptive token fine tuning, which comprises the following steps of: constructing a feature important matrix by utilizing a proxy model middle layer, and generating a semantic-guided sparse mask; performing token-level point multiplication on the sparse mask and the initialized disturbance, and performing linear superposition on the sparse mask and the initialized disturbance and an original image to serve as initial input data; inputting the input data into the agent model for iterative training; in each round of iterative training, sparse masks are continuously updated, the number of discarded tokens is gradually increased, and gradient variance between layers in an agent module is reduced through gradient back propagation by using a gradient penalty factor and a balance factor. Meanwhile, self-adaptive truncation factor sets are designed for different modules of the agent model to weaken the attention mechanism of the model, and through T times of iterative training, a target loss function is maximized to obtain optimal mobility disturbance. Experiments prove that the method has better attack effect and mobility on the attack of the target model.

The invention discloses a super-resolution image reconstruction method based on a multi-scale large-kernel convolution double-residual neural network, which is suitable for the field of image processing, and comprises the following steps: cutting a data set, inputting a cut original low-resolution image into a preprocessing module, carrying out image normalization and data enhancement operation, and carrying out image reconstruction; generating a preprocessed low-resolution image; the preprocessed low-resolution images form a distorted image block data set, and a training set, a verification set and a test set are formed; according to an existing distorted image block data set, a super-resolution image reconstruction method based on a multi-scale large-kernel convolution double-residual neural network is constructed; and inputting the data set into the constructed multi-scale large-kernel convolution double-residual neural network to extract semantic features, and amplifying a feature map by using an up-sampling module of the model to generate a super-resolution image. According to the method, a multi-scale large-kernel convolution and double-residual structure is introduced, a visual attention mechanism is used in the neural network, the extracted features better conform to human visual perception features, and super-resolution image reconstruction is more accurate.

The invention provides a visual token removal method for video understanding model reasoning acceleration, which is based on a visual attention contraction phenomenon, dynamically evaluates the importance of visual tokens and screens and removes redundant tokens by analyzing the distribution and change of attention weights of the visual tokens in a reasoning pre-filling stage. Therefore, calculation overhead is reduced and reasoning efficiency is improved. The method comprises the following steps: carrying out frame sampling and region division on video input to obtain image blocks with space-time structures; the method comprises the following steps: extracting visual features by using a visual encoder and a projector and generating a visual token; splicing the visual token with the system token and the text token to form multi-modal input; tracking the attention change trend of the text to the vision to evaluate the importance of the visual token; the recognition of unimportant visual tokens is accelerated through a grouping and aggregation mode, and the unimportant visual tokens are gradually removed in reasoning. While the output quality is maintained, the method significantly reduces the reasoning time and memory occupation.

The invention provides a VR scene adaptive rendering method based on AI and multi-modal perceptual interaction, and the method comprises the steps: firstly obtaining a perceptual signal set of a current interaction stage of a user through a multi-modal interaction interface, including user interaction actions, visual attention and scene physical feedback signals, and then analyzing the set to obtain a sensing signal set of the current interaction stage of the user; the method comprises the following steps: generating user concerned area features and scene dynamic sensitive area features, then determining a rendering area priority sequence and a detail retention reference through a rendering resource allocation model based on the user concerned area features and the scene dynamic sensitive area features, and adjusting a resource scheduling rule and a quality control rule of a VR rendering system according to the result. And executing the adjusted rule to generate the current frame of rendering picture, and updating the decision logic of the rendering resource allocation model according to the change trend of the interactive feedback of the user to the current frame of picture and the physical feedback of the scene, thereby realizing the dynamic self-adaption of VR scene rendering, improving the rendering efficiency and quality, and enhancing the user immersion.

The invention discloses a robot automatic grabbing path planning method based on visual identification, and relates to the technical field of intelligent grabbing. The method comprises the following steps: acquiring multi-view visual data of a target scene, and generating a scene three-dimensional compact reconstruction model through a multi-modal image fusion algorithm; performing target detection and feature extraction on the model, and screening an optimal capture point in combination with a visual attention mechanism; constructing a dynamic environment obstacle probability map, updating an obstacle state through time sequence visual tracking, and quantifying an interference weight; an initial grabbing path is planned based on an improved fast expansion random tree algorithm, path smoothness constraints and robot joint movement limit parameters are introduced, and path nodes are optimized through a Bezier curve; visual servo feedback and path deviation prediction are fused, path parameters are corrected in real time, and a continuous movement track is generated. The method effectively adapts to the dynamic environment, gives consideration to path safety, smoothness and mechanical arm motion characteristics, and remarkably improves the grabbing success rate and operation reliability.

The invention relates to the technical field of liquid crystal display, and discloses a method and system for reducing power consumption of a liquid crystal panel sequential control chip and the chip, and the method comprises the steps: obtaining historical watching data of a user, and constructing and training a user visual preference prediction model; collecting visual attention data and display content data of a user in real time, and determining a region of interest on the liquid crystal panel in combination with the prediction model; dividing a high-resolution area, a medium-resolution area and a low-resolution area of the liquid crystal panel, and setting corresponding pixel density and refresh frequency threshold values; performing visual saliency analysis on the display content, and determining a key salient region and a non-key salient region; and local refreshing is carried out in the regions with different resolutions, wherein the refreshing frequency of the key salient region is higher than that of the non-key salient region. The system comprises a plurality of modules for realizing the method. The chip adopts the system and the method, so that the power consumption of the liquid crystal panel time sequence control chip can be effectively reduced.

The invention discloses an artistic material aesthetic evaluation method and system fusing a design rule and a visual attention mechanism, and the method comprises the steps: simulating a human visual attention path through extracting the multi-level visual information of an image, and carrying out the collaborative analysis through combining four interpretable aesthetic dimensions, namely composition, color, content and image quality, extracting local and overall aesthetic representations; each expert module outputs intermediate aesthetic indexes according to visual features and classic design rules; under the guidance of a visual attention mechanism, the intermediate aesthetic indexes are fused into a unified comprehensive aesthetic representation; the model outputs aesthetic scores of the nine local areas, and the overall aesthetic evaluation score of the image is obtained through weighted average calculation. According to the method, an end-to-end multi-task joint training strategy is adopted, and the expression ability and evaluation precision of aesthetic features of the art image are effectively enhanced.

A multi-modal large model illusion detection and inhibition method based on attention time sequence difference comprises the following steps: inputting text lexical elements of cue words and visual lexical elements of images into a multi-modal large model, and obtaining an internal attention graph of the decoding stage of the multi-modal large model; then calculating the attention proportion of the visual lexical units at the current generation moment, and making a difference between the attention proportion and the proportion at the previous moment; and if the difference value exceeds a set threshold value, determining that the lexical elements are visual related lexical elements. When the visual related lexical elements are recognized, performing secondary forward propagation of primary visual enhancement to obtain more accurate output; and if not, directly entering the next step of generation. According to the method, visual related lexical elements in text generation are recognized and refined through attention time sequence difference, two-time forward propagation is adopted, the visual attention of second-time forward propagation is enhanced based on a visual attention graph of first-time forward propagation, and illusion can be recognized and corrected on the premise that the language expression ability is not reduced.

The invention provides an intelligent prestress tensioning method and device for a fabricated building component, relates to the technical field of stress detection and analysis, and aims to perform three-dimensional simulation testing and visual attention monitoring to determine basic tensioning information, train a self-correction module, control an intelligent tensioning pump station and realize intelligent prestress tensioning of the fabricated building component. The device is used for assisting an upper industrial personal computer in numerical control management of a jack tensioning full period, synchronously monitoring and receiving sensing data to carry out deviation positioning and deviation correction decision making, and generating a deviation correction strategy to carry out numerical control compensation management of a feedback target, and is used for solving the problems that in the prior art, tensioning intelligence and flexibility are insufficient; the technical problems that the adaptive capacity under the complex working condition is poor, and full-period effective operation conforming to the actual operation condition cannot be achieved are solved. Effective construction information is determined through simulation and attention analysis, stress sensing layout is carried out, dynamic and static analysis and decision making are carried out in combination with a self-correction module, the intelligence and flexibility of tensioning control are improved, and reasonable and effective operation is carried out according to the actual operation condition.

The invention discloses a visual language large model illusion relieving method based on attention hijacking detection, and the method comprises the steps: selecting target attention layers from an autoregression language decoder of a visual language large model according to actual demands, and carrying out the processing of each target attention layer, and calculating a hijacking person evaluation score of each instruction mark based on the visual information amount transmitted to each generation mark by each instruction mark, taking the instruction marks with higher hijacking person evaluation scores as attention hijacking persons to form an attention hijacking person set, and then performing attention distraction processing so as to realize hallucination relief. According to the method, the instruction marks continuously distorting visual perception of the visual language large model in the decoding process are identified, and the visual attention of the instruction marks is shielded, so that the interference of the instruction marks on a visual area with relatively weak discrimination is isolated, the context attention perception ability of the model is recovered, and the occurrence of illusion is reduced.

The invention relates to the technical field of computer vision, and provides a text image matching method and system based on large language model entropy increase, and the method comprises the steps: enabling data set image features to pass through a visual attention encoder to obtain original image codes; constructing a hypergraph based on original image coding and generating hypergraph image features through a hypergraph neural network; performing residual connection on the hypergraph image features and original image codes to construct a visual hypergraph adapter; inducing a large language model to generate a synonym material through a prompt learning method by utilizing a first text feature generated by the image feature description, and obtaining an entropy-increased fusion text feature; encoding the fused text feature and the first text feature; constructing a hypergraph based on multi-dimensional feature coding, and generating hypergraph text features through a hypergraph neural network; performing dimension reduction on the supergraph text features, and then combining with original corpus feature codes to construct a text supergraph adapter; and calculating a similarity matrix based on the visual hypergraph adapters and the text hypergraph adapters to obtain a similarity ranking.

The invention relates to the technical field of intelligent illumination control, and discloses an intelligent enhancement method and system for the brightness of an LED light-emitting module, and the method comprises the steps: collecting environment illumination data and LED module operation state data, and carrying out the preprocessing; acquiring cultural relic material information, performing exhibit change detection, determining an illumination constraint threshold, calculating effective illumination and tracking accumulated exposure; determining an illumination safety boundary and dynamically modulating an LED spectrum; performing hierarchical coordination by adopting a three-layer game decision framework; a three-dimensional light field representation framework is constructed through audience behavior perception and visual attention prediction, and space illumination optimization is carried out; environment sudden changes and audience behaviors are detected, and quick response adjustment is executed; analyzing lighting space-time distribution characteristics, performing dynamic power distribution and generating an intelligent dimming curve; according to the invention, environmental perception, cultural relic protection constraint, light attenuation compensation, multi-module cooperation and intelligent decision can be comprehensively considered.

Systems and methods for collecting visual attention information are described. The methods comprise the steps of displaying a browser window on a display (210); estimating a user's (220) gaze point within the display (210); transforming the geometry of the estimated gaze point and / or the geometry of the browser window to a common coordinate system; identifying an object (215) in the browser window that corresponds to the estimated gaze point; and storing the estimated gaze point and data corresponding to the identified object (215).

A system and computer-implemented method of attention tracking and includes obtaining a video stream of a predefined area, the predefined area including at least one person, detecting a person image of the person in the predefined area, detecting objects in the predefined area, extracting gaze points and determining a possible attention target of the detected person, continuously tracking of attention direction of the person, determining in real time an attention score based on the attention target, and dynamically displaying, in a display device, the video stream together with attention information, including the attention score, for the person. The system and method provides real time determining of a personalized attention score for an individual student or an overall attention score for a group of students attending a lecture in a classroom environment.

The embodiment of the invention relates to the technical field of bionic robots, in particular to a bionic robot vision collaboration method based on multiple vision modules, a robot vision device, a vision unit device and a robot. The method comprises the steps that S1, a task instruction is received and analyzed, and task semantics are obtained; s2, acquiring main view image data and extracting low-level visual features; s3, searching a bionic attention weight mapping table to obtain advanced visual feature requirements and initial feature weights; s4, generating an initial dynamic attention thermodynamic diagram, and determining a high-attention area and a secondary-attention area; s5, acquiring high-resolution image data, and updating low-level visual features of the front view image data; and S6, performing weighted fusion, updating the dynamic attention thermodynamic diagram in real time, and forming a perception-action closed loop. According to the method, by simulating a visual attention mechanism of task-driven selective focusing from top to bottom and saliency perception from bottom to top according to tasks, the improvement of the visual collaborative bionic degree of the robot is realized.

The invention belongs to the field of spatial design optimization, and particularly relates to an exhibition streamline optimization method and system based on a viewpoint thermodynamic diagram and a spatial syntax, and the method comprises the steps: firstly constructing an initial three-dimensional layout model, carrying out the dynamic discretization of the initial three-dimensional layout model into visual or path-connected spatial units, and calculating the integration degree, understandability and other indexes of the units in combination with the spatial syntax. Generating a theoretical streamline sequence; collecting viewpoint data of the test group along a preset path through eye movement tracking, and generating a viewpoint thermodynamic value distribution diagram in combination with a thermodynamic diagram algorithm, group interest distribution and visual attention characteristics; based on the theoretical streamline and the thermodynamic diagram, the coupling coordination degree of the space unit is calculated through a coupling algorithm, a preset optimization strategy is called after a defect unit is recognized, and an optimized exhibition streamline scheme is generated by combining the initial model and simulation algorithm adjustment. And the exhibition viewing experience and the space use efficiency are effectively improved.

A method and system for tracking visual attention are disclosed. By generating at least one visual stimulus with a characteristic modulation (the characteristic modulation is applied to a high spatial frequency (HSF) component of the visual stimulus) and displaying the visual stimulus or each visual stimulus via a graphical user interface (GUI) of a display, a response can be induced in the brain of a user. By receiving a neural signal of the user from a neural signal capture device such as an EEG device, the user's focus point (when the user views the visual stimulus) can be determined based on the neural signal, because the neural signal includes information associated with the characteristic modulation of the visual stimulus to which the user's visual attention is directed.

The invention discloses a model generation image evaluation method and device and a storage medium, and relates to the technical field of image processing. According to the method, the image generated based on the text cue word is obtained, wherein the image comprises the image element corresponding to the text cue word; filtering the image to obtain a filtering response value of each image pixel in the image, and generating a visual attention map of the image based on the filtering response values, the visual attention map representing frequency domain energy distribution of the image; and determining a visual saliency value of the image element according to the visual attention map, and determining a layout score of the image according to the visual saliency value of the image element. According to the method, spatial relations such as distances, alignment modes and hierarchical structures among elements are quantitatively evaluated through visual saliency values, and the defect that a previous model cannot accurately evaluate and adjust image layout is overcome.

The invention discloses an automobile wire harness quality inspection method based on visual inspection, and belongs to the field of visual inspection and automobile wire harness detection.The method comprises the steps that a normal sample image set is constructed based on visual images of qualified automobile wire harnesses; self-adaptive preprocessing is carried out; inputting a pre-trained visual attention network, extracting key visual features of qualified wire harnesses, and constructing a normal feature library; collecting a visual image of a to-be-detected wire harness, performing adaptive preprocessing, and extracting key visual features of the to-be-detected image; calculating the feature deviation degree between the key visual features of the to-be-detected image and the normal feature library, and judging whether defects exist or not through the feature deviation degree; dynamically updating a normal feature library and a judgment threshold value through an online self-calibration module; and for the to-be-detected image which is judged to have the defect, positioning a defect area through a visual attention thermodynamic diagram, matching a preset defect feature template library, determining a defect type and outputting a quality inspection conclusion. According to the invention, appearance and assembly precision defects can be covered.