128 results about "Visual attentiveness" patented technology

The invention provides an end-side multi-modal large model accelerated reasoning method and system, and the method comprises the steps: carrying out the two-stage screening and rearrangement of visual tokens based on the CLS attention and text-to-visual attention in a visual encoder and pre-filling stage, and constructing a sparse attention and sparse key value cache; in a decoding stage, an important neuron set is judged according to activation gating or historical statistics, only a corresponding feedforward network weight is pulled and calculated, missed weights are loaded on demand through asynchronous I / O, and hot neurons are maintained in a high-speed memory to utilize model sparsity, so that video memory / memory occupancy and calculation overhead are remarkably reduced on an end side; throughput and time delay performance are improved. According to the method, the internal memory and computing resources required by reasoning of the multi-modal large language model are reduced from two dimensions by utilizing the endogenous sparsity of the end-side large language model in input and the model, so that a higher reasoning speed is achieved by utilizing fewer resources on the premise of keeping the size of the model unchanged, and the performance of the whole system is improved.

The invention provides a 3D modeling method based on a digital twin city, and belongs to the technical field of 3D city modeling. Through a multi-source semantic data fusion acquisition means, space-time semantic tags are added to various types of data, the problem of multi-source data isomerism is solved, and semantic unification and efficient fusion of different types of data are realized; by means of hierarchical dynamic modeling means, a building component network is constructed based on triple association of geometry, functions and semantics, so that a model structure better conforms to the logic of the building industry, and the reasonability and efficiency of modeling are improved; a virtual-real two-way mapping means is applied, incremental updating of the model is triggered through a cross-modal semantic comparison algorithm, limitation of traditional static modeling is broken through, and dynamic synchronization of the virtual model and the physical world is achieved; an intelligent optimization means is adopted, a multi-constraint-condition generative adversarial network is combined with a visual attention mechanism, it is ensured that generated textures conform to building geometric features and material specifications, and meanwhile intelligent distribution of rendering resources is achieved.

The invention relates to the technical field of artificial intelligence, can be applied to business scenes such as agent autonomous decision making, financial science and technology and medical health, and discloses a task processing method and device based on visual attention enhancement, equipment and a medium. Visual hierarchical features are extracted, a double fovea attention module processes and fuses high-level visual features, a side suppression network obtains enhanced visual features, and a cross-modal fusion module generates fusion features by taking the enhanced visual features as query vectors and taking language components and action components as key and value vectors; and fusing the feature input decision network to generate target category and position information, generating feedback information based on actual label difference, and updating module parameters to complete a target task. According to the invention, through combination of a bionic vision mechanism and multi-modal attention fusion, the visual feature extraction and background suppression capability is improved, and the target capture efficiency and recognition precision in a complex scene can be improved.

The invention provides a VR scene adaptive rendering method based on AI and multi-modal perceptual interaction, and the method comprises the steps: firstly obtaining a perceptual signal set of a current interaction stage of a user through a multi-modal interaction interface, including user interaction actions, visual attention and scene physical feedback signals, and then analyzing the set to obtain a sensing signal set of the current interaction stage of the user; the method comprises the following steps: generating user concerned area features and scene dynamic sensitive area features, then determining a rendering area priority sequence and a detail retention reference through a rendering resource allocation model based on the user concerned area features and the scene dynamic sensitive area features, and adjusting a resource scheduling rule and a quality control rule of a VR rendering system according to the result. And executing the adjusted rule to generate the current frame of rendering picture, and updating the decision logic of the rendering resource allocation model according to the change trend of the interactive feedback of the user to the current frame of picture and the physical feedback of the scene, thereby realizing the dynamic self-adaption of VR scene rendering, improving the rendering efficiency and quality, and enhancing the user immersion.

The invention discloses a robot automatic grabbing path planning method based on visual identification, and relates to the technical field of intelligent grabbing. The method comprises the following steps: acquiring multi-view visual data of a target scene, and generating a scene three-dimensional compact reconstruction model through a multi-modal image fusion algorithm; performing target detection and feature extraction on the model, and screening an optimal capture point in combination with a visual attention mechanism; constructing a dynamic environment obstacle probability map, updating an obstacle state through time sequence visual tracking, and quantifying an interference weight; an initial grabbing path is planned based on an improved fast expansion random tree algorithm, path smoothness constraints and robot joint movement limit parameters are introduced, and path nodes are optimized through a Bezier curve; visual servo feedback and path deviation prediction are fused, path parameters are corrected in real time, and a continuous movement track is generated. The method effectively adapts to the dynamic environment, gives consideration to path safety, smoothness and mechanical arm motion characteristics, and remarkably improves the grabbing success rate and operation reliability.

The invention discloses an artistic material aesthetic evaluation method and system fusing a design rule and a visual attention mechanism, and the method comprises the steps: simulating a human visual attention path through extracting the multi-level visual information of an image, and carrying out the collaborative analysis through combining four interpretable aesthetic dimensions, namely composition, color, content and image quality, extracting local and overall aesthetic representations; each expert module outputs intermediate aesthetic indexes according to visual features and classic design rules; under the guidance of a visual attention mechanism, the intermediate aesthetic indexes are fused into a unified comprehensive aesthetic representation; the model outputs aesthetic scores of the nine local areas, and the overall aesthetic evaluation score of the image is obtained through weighted average calculation. According to the method, an end-to-end multi-task joint training strategy is adopted, and the expression ability and evaluation precision of aesthetic features of the art image are effectively enhanced.

A multi-modal large model illusion detection and inhibition method based on attention time sequence difference comprises the following steps: inputting text lexical elements of cue words and visual lexical elements of images into a multi-modal large model, and obtaining an internal attention graph of the decoding stage of the multi-modal large model; then calculating the attention proportion of the visual lexical units at the current generation moment, and making a difference between the attention proportion and the proportion at the previous moment; and if the difference value exceeds a set threshold value, determining that the lexical elements are visual related lexical elements. When the visual related lexical elements are recognized, performing secondary forward propagation of primary visual enhancement to obtain more accurate output; and if not, directly entering the next step of generation. According to the method, visual related lexical elements in text generation are recognized and refined through attention time sequence difference, two-time forward propagation is adopted, the visual attention of second-time forward propagation is enhanced based on a visual attention graph of first-time forward propagation, and illusion can be recognized and corrected on the premise that the language expression ability is not reduced.

The invention relates to the technical field of intelligent illumination control, and discloses an intelligent enhancement method and system for the brightness of an LED light-emitting module, and the method comprises the steps: collecting environment illumination data and LED module operation state data, and carrying out the preprocessing; acquiring cultural relic material information, performing exhibit change detection, determining an illumination constraint threshold, calculating effective illumination and tracking accumulated exposure; determining an illumination safety boundary and dynamically modulating an LED spectrum; performing hierarchical coordination by adopting a three-layer game decision framework; a three-dimensional light field representation framework is constructed through audience behavior perception and visual attention prediction, and space illumination optimization is carried out; environment sudden changes and audience behaviors are detected, and quick response adjustment is executed; analyzing lighting space-time distribution characteristics, performing dynamic power distribution and generating an intelligent dimming curve; according to the invention, environmental perception, cultural relic protection constraint, light attenuation compensation, multi-module cooperation and intelligent decision can be comprehensively considered.

The embodiment of the invention relates to the technical field of bionic robots, in particular to a bionic robot vision collaboration method based on multiple vision modules, a robot vision device, a vision unit device and a robot. The method comprises the steps that S1, a task instruction is received and analyzed, and task semantics are obtained; s2, acquiring main view image data and extracting low-level visual features; s3, searching a bionic attention weight mapping table to obtain advanced visual feature requirements and initial feature weights; s4, generating an initial dynamic attention thermodynamic diagram, and determining a high-attention area and a secondary-attention area; s5, acquiring high-resolution image data, and updating low-level visual features of the front view image data; and S6, performing weighted fusion, updating the dynamic attention thermodynamic diagram in real time, and forming a perception-action closed loop. According to the method, by simulating a visual attention mechanism of task-driven selective focusing from top to bottom and saliency perception from bottom to top according to tasks, the improvement of the visual collaborative bionic degree of the robot is realized.

The invention belongs to the field of spatial design optimization, and particularly relates to an exhibition streamline optimization method and system based on a viewpoint thermodynamic diagram and a spatial syntax, and the method comprises the steps: firstly constructing an initial three-dimensional layout model, carrying out the dynamic discretization of the initial three-dimensional layout model into visual or path-connected spatial units, and calculating the integration degree, understandability and other indexes of the units in combination with the spatial syntax. Generating a theoretical streamline sequence; collecting viewpoint data of the test group along a preset path through eye movement tracking, and generating a viewpoint thermodynamic value distribution diagram in combination with a thermodynamic diagram algorithm, group interest distribution and visual attention characteristics; based on the theoretical streamline and the thermodynamic diagram, the coupling coordination degree of the space unit is calculated through a coupling algorithm, a preset optimization strategy is called after a defect unit is recognized, and an optimized exhibition streamline scheme is generated by combining the initial model and simulation algorithm adjustment. And the exhibition viewing experience and the space use efficiency are effectively improved.

The invention discloses a model generation image evaluation method and device and a storage medium, and relates to the technical field of image processing. According to the method, the image generated based on the text cue word is obtained, wherein the image comprises the image element corresponding to the text cue word; filtering the image to obtain a filtering response value of each image pixel in the image, and generating a visual attention map of the image based on the filtering response values, the visual attention map representing frequency domain energy distribution of the image; and determining a visual saliency value of the image element according to the visual attention map, and determining a layout score of the image according to the visual saliency value of the image element. According to the method, spatial relations such as distances, alignment modes and hierarchical structures among elements are quantitatively evaluated through visual saliency values, and the defect that a previous model cannot accurately evaluate and adjust image layout is overcome.

The invention discloses an automobile wire harness quality inspection method based on visual inspection, and belongs to the field of visual inspection and automobile wire harness detection.The method comprises the steps that a normal sample image set is constructed based on visual images of qualified automobile wire harnesses; self-adaptive preprocessing is carried out; inputting a pre-trained visual attention network, extracting key visual features of qualified wire harnesses, and constructing a normal feature library; collecting a visual image of a to-be-detected wire harness, performing adaptive preprocessing, and extracting key visual features of the to-be-detected image; calculating the feature deviation degree between the key visual features of the to-be-detected image and the normal feature library, and judging whether defects exist or not through the feature deviation degree; dynamically updating a normal feature library and a judgment threshold value through an online self-calibration module; and for the to-be-detected image which is judged to have the defect, positioning a defect area through a visual attention thermodynamic diagram, matching a preset defect feature template library, determining a defect type and outputting a quality inspection conclusion. According to the invention, appearance and assembly precision defects can be covered.

The invention discloses a super-resolution image reconstruction method based on a multi-scale large separable kernel convolutional neural network, which is suitable for the field of image processing, and comprises the following steps: cutting a data set, inputting a cut original low-resolution image into a preprocessing module, carrying out image normalization and data enhancement operation, and carrying out image reconstruction; generating a preprocessed low-resolution image; the preprocessed low-resolution images form a distorted image block data set, and a training set, a verification set and a test set are formed; constructing a super-resolution image reconstruction method based on a multi-scale large separable kernel convolutional neural network according to an existing distorted image block data set; and inputting the data set into the constructed multi-scale large separable kernel convolutional neural network to extract semantic features, and amplifying a feature map by using an up-sampling module of the model to generate a super-resolution image. According to the method, a multi-scale large separable kernel convolution structure is introduced, a visual attention mechanism is used in the neural network, the extracted features better conform to human visual perception features, and super-resolution image reconstruction is more accurate.

The invention discloses a visual large language model illusion phenomenon elimination method, and belongs to the technical field of visual large language model processing. The hallusion elimination method solves the problems that an existing hallusion elimination method needs to consume long time in the reasoning stage, needs to depend on manual data annotation and consumes a large amount of computing resources. According to the method, the visual perception ability is enhanced by using the attention state of the model in the description state of answering the visual information, and in order to make the model fully pay attention to the input visual information, the attention head output which describes the sensitivity of the instruction is optimized into the output which describes the query in the reasoning process; according to the method, disturbance is applied to a specific attention head, so that the model obtains sufficient visual attention again, the visual perception ability of the model is effectively enhanced, the effect that the illusion phenomenon eliminating ability of the model is improved by utilizing the inherent fine-grained visual perception ability of the visual language model is achieved, and no extra training or tool is needed. The method can be applied to visual large language model processing.

The invention relates to the technical field of flight simulation, in particular to an intelligent flight simulation and evaluation system for a training plane, and the system comprises a multi-modal data real-time collection module which is used for collecting operation input data, flight state parameters and visual attention data in real time; the implicit mental model deconstruction module is used for constructing an implicit mental model representing the individual decision preference of the pilot; the pilot cognitive state quantification module is used for calculating and generating a mental model deviation degree, a scene awareness entropy and decision recovery time; the self-adaptive scene generation module is used for deciding whether to inject a cognitive disturbance scene into the simulated flight environment or not, and manipulation input data, flight state parameters and visual attention data generated by the response of a pilot to the cognitive disturbance scene are captured by the multi-modal data real-time acquisition module to form a closed loop; according to the system, real-time deconstruction and quantitative evaluation of the pilot internal mental model are realized.

The invention discloses a vehicle-based video pushing method and system and a storage medium, and belongs to the technical field of data processing. When the vehicle accesses the WiFi network, the server obtains an interest tag of a driver of the vehicle; the server searches for a push video matched with the interest tag in a network video library, and pushes a video clip of a predetermined duration in the push video to the vehicle through a WiFi network; the vehicle obtains vehicle state information, environment information and driver state information, and generates a safety level according to the vehicle state information, the environment information and the driver state information; the vehicle obtains visual attention information and user interaction behavior information, and an attention weight is generated according to the visual attention information and the user interaction behavior information; and the vehicle determines a playing mode according to the safety level and the attention weight, and controls playing of the video clip according to the playing mode. According to the invention, the video is pushed in advance when the WiFi is accessed, the pushing flow is saved, and the driving safety and the watching experience can also be considered.

The invention discloses an autism child attention prediction method based on a Mamb-Unet structure, and the method comprises the steps: constructing a multi-modal data set for autism child attention prediction, constructing an autism child attention prediction model based on the Mamb-Unet structure, and training the model; and performing atypical visual attention prediction on a to-be-detected image by using the trained model to obtain a prediction result. The model comprises a preprocessing module, an encoder part, a bottleneck part, a decoder part and an output part which are connected in sequence, wherein an efficient self-adaptive visual state space block, a frequency domain attention module, a conditional visual state space block and a conditional attention fusion module are arranged; the output of the encoder portion is also propagated through a hopping connection to the decoder portion. According to the method, the autism spectrum disorder atypical visual attention prediction accuracy is remarkably improved.

The invention discloses an image enhancement method based on a human eye attention perception mechanism, and belongs to the technical field of image processing, and the method comprises the steps: obtaining the gaze focus distribution data of human eyes in an original image, and obtaining a visual attention distribution diagram based on the gaze focus distribution data; based on the visual attention distribution map, obtaining a distance weight between each pixel in the image and the fixation focus, and based on the distance weight between each pixel and the fixation focus, obtaining a regional human eye saliency grading matrix; based on the regional human eye saliency grading matrix and the visual attention distribution diagram, obtaining a layered processing regional map; and the original image is processed according to the region priority based on the hierarchical processing region map, and image enhancement based on a human eye attention perception mechanism is realized. According to the invention, image quality and computing resource consumption can be effectively balanced, targeted image enhancement is realized, and the visual experience of a user is improved.

This invention relates to the field of image feature enhancement technology, providing an image feature enhancement method and system based on learnable unary function gating. The method includes: dividing the input image into image patches and mapping them to visual tokens; in a multi-head attention layer, calculating a visual attention aggregation score based on an attention weight matrix to quantify the degree of abnormal attention received by the image patch; inputting the normalized visual token, aggregation score, and two-dimensional position code into a gating module composed of a learnable unary function to generate a gating matrix; after each attention head completes SDPA output and before multi-head stitching, performing element-wise gating modulation using the gating matrix, and stitching and projecting to obtain the enhanced image features. This invention, through the synergy of aggregation score and learnable unary function, suppresses abnormal attention propagation from background noise, enhances the expression of key features of small targets, and improves the feature discriminativeness and task adaptability of the visual Transformer.

The invention relates to the technical field of intelligent interaction systems and automobile customer service marketing, and discloses a traffic automobile customer service marketing method and system based on a large language model.The traffic automobile customer service marketing method based on the large language model.The traffic automobile customer service marketing method based on the large language model.The traffic automobile customer service marketing method based on the large language modelcomprises the steps that an eye movement tracking algorithm is used for collecting fixation point data of a driver, and an eye movement track data set is generated; constructing a visual attention heat map, and forming a user attention distribution model; multi-medium characteristic parameters are calculated, and a medium characteristic model is established; generating and adapting personalized marketing content by using a large language model, and outputting multi-medium compatible marketing information; intelligent attention guidance is realized according to the user attention distribution model, the medium characteristic model and the multi-medium compatible marketing information; through the eye movement tracking technology, accurate perception and quantitative analysis of the visual attention of the driver are realized, so that the marketing system can grasp the attention point of the user in real time, and the technical problem that a traditional interface cannot perceive the actual attention point of the user is solved.

A communication robot includes an auditory information processing portion configured to recognize a volume of voice collected by a sound collection portion and generate an auditory attention map by projecting a sound position in a three-dimensional space onto a two-dimensional attention map in which the robot is located at a center, a visual information processing portion configured to generate a visual attention map using a face detection result obtained by detecting a face of a person from an image captured by an imaging portion and a motion detection result obtained by detecting a motion of the person, an attention map generation portion configured to generate an attention map by integrating the auditory attention map and the visual attention map, and a motion processing portion configured to control eyeball movements and motions of the communication robot using the attention map.

An information processing device displays a content image on a display, determines whether or not a user's gaze is directed toward a work medium for the user to perform a task, and performs, in a case where it is determined that the user's gaze is directed toward the work medium, a process for changing a mode of the content image displayed on the display so that a level of visual attention of the user decreases.

The invention belongs to the technical field of artificial intelligence, and particularly relates to a video large language model illusion relieving method and system. The method comprises the following steps: inputting a video and a text into a video big language model, and starting generation of a plurality of candidate replies in parallel; in the process of generating each candidate reply, monitoring a visual attention degeneration point; upon detecting the visual attention recession point, stopping generation of the corresponding candidate reply, and calculating a timing attention collapse value of the corresponding candidate reply; and selecting the candidate reply with the maximum time sequence attention collapse value to continuously generate until the candidate reply is finished. According to the method, the video large language model can give more attention to the global content of the video, perception illusion is reduced, so that correct reply is made, additional training of the model is not needed, and the calculation cost is low.

The invention discloses a cloud mobile phone video stream compression method and related equipment, and relates to the technical field of cloud computing, and the method comprises the steps: obtaining original image data and interface element metadata of a cloud mobile phone screen; based on a visual attention model and the interface element metadata, determining dynamic visual weight distribution of each region; performing dynamic blocking on the original image data based on a three-level region division mechanism to obtain a differential region division result; according to the dynamic vision weight distribution and the differentiated region division result, differentiated compression parameters of all the regions are determined; and based on the differential compression parameters and a dynamic boundary buffering technology, executing video coding operation of each region. According to the method and the device, the image quality of the high-vision weight region is guaranteed, the bandwidth occupation of the low-attention region is effectively reduced, the blocking effect and misjudgment problems are solved, and the user experience in a low-bandwidth scene is optimized.

This invention provides a method and system for precise transport control of remediation agents in in-situ groundwater remediation, relating to the field of groundwater pollution remediation technology. The method includes: acquiring images and sensor data from borehole cores at contaminated sites; extracting features through visual attention networks and temporal attention networks; fusing these features into a conditional probability diffusion model to generate hydrogeological feature vectors; constructing a high-precision hydrogeological feature model by combining spatial attention mapping and a generative diffusion model; inputting this model into a multi-regional collaborative simulation environment for optimization calculation; fusing monitoring data through edge computing nodes; and training a lightweight control model to execute the transport control of remediation agents.

The invention discloses a visual attention calculation method based on image reconstruction, and the method comprises the steps: firstly obtaining a complex scene image, carrying out the preprocessing, obtaining an image data set, building an image reconstruction self-coding and decoding model based on SimMIM, carrying out the model training, inputting an image to be subjected to the visual attention calculation after the training, and carrying out the image reconstruction, and splicing the reconstructed images to obtain a complete reconstructed image, performing quality evaluation, and finally introducing an error fusion strategy based on global weight search to obtain a visual attention distribution diagram of a complex scene. According to the method, the local image reconstruction capability of the image self-coding model is utilized, the reconstruction difficulty or credibility of different areas of the image can be evaluated, a new view angle is provided for judging identification and understanding of difficult areas in a complex scene, and by fusing the pixel-level error and the structure / perception-level error between the image blocks, the reconstruction difficulty or credibility of the different areas of the image can be evaluated. The reconstruction quality of a local image area can be evaluated more comprehensively and finely, and visual attention distribution of a complex scene is obtained.

This invention discloses a method for studying visual attention mechanisms based on EEG microstates, comprising: collecting EEG signals from subjects while watching videos and preprocessing them; extracting saliency maps from the videos, and extracting statistical features from the perspectives of spatial saliency information in the local temporal domain and spatiotemporal saliency information changes in the global time series, respectively, to obtain sIQR features and tsIQR features; converting the preprocessed EEG signals into EEG topology map sequences and performing spatial clustering to extract EEG microstate templates, determining the number of templates and selecting the final microstate templates, and then backfitting them to the EEG signals to obtain microstate sequences; extracting microstate features and depth features from the microstate sequences, examining the statistical differences between microstate features and sIQR and tsIQR features, constructing decoding models based on microstate features or depth features, and using the decoding models to decode segments and videos respectively to obtain segment labels and video labels.

The invention relates to the technical field of enterprise propaganda, and discloses a personalized recommendation method for virtual digital humans in enterprise propaganda. According to the method, multi-modal interaction data such as visual attention data and voice feedback data in the interaction process of a user and a virtual digital human are collected in real time, and an original interaction flow is generated; performing multi-dimensional fusion analysis on the original interaction flow, analyzing a dependency relationship among different dimensions, identifying a hidden association between a user interest mode and a virtual digital human performance feature, labeling an analysis result as an initial interest index, and generating an interest labeling data set with confidence; optimizing model adaptability and dynamically updating a model state by utilizing a fusion analysis result; matching the real-time interaction data with the dynamic interest evolution model, and identifying recommendation candidates and opportunities to obtain recommendation contexts; and mining a resource library based on a matching result, identifying a deep recommendation strategy and an adjustment signal, and tracing the strategy to identify a core factor and an optimization direction, thereby realizing accurate and adaptive personalized recommendation.

Aspects of the present disclosure are directed to gesture-based user interfaces (UIs) for artificial reality (XR) messaging applications. By supplementing or replacing “gaze to tap” user interfaces with “ultra low friction” (ULF) gestures, a user is not required to repeatedly remove his focus from the real world to gaze at menu options in order to select them. The ULF gestures can include, for example, a single pinch motion to tell a messaging application to start recording a voice message. Releasing the pinch stops the recording and allows for editing, while a snap (or tug right) can stop the recording and send the message immediately. A tug left can delete the message unsent. Adding these ULF gestures to the messaging application's UI allows the user to fully engage with the application while maintaining visual focus on the real world, thus encouraging the user to remain connected through the XR system.

The application relates to the technical field of video recognition, and discloses a course video key frame intelligent recognition method based on an AI visual attention mechanism, which comprises the following steps: acquiring a visual saliency feature map of a video frame and calculating a global attention gravity center coordinate, constructing a spatial second moment tensor by using the visual saliency feature map to determine an anisotropy coefficient, then performing nonlinear weighted processing on the trajectory distribution density in a space-time trajectory space, calculating a second acceleration residual based on the processed trajectory evolution process, and determining a key frame in combination with the trajectory distribution density and the second acceleration residual. The application uses an anisotropy regulation mechanism to suppress non-content dynamic interference, checks the integrity of teaching content generation through the second acceleration residual, solves the problem of lag in semantic turning point capture under a dynamic background, and enhances the semantic density of extracted sequences and the recognition stability.