91 results about "Visual space" patented technology

The invention belongs to the technical field of electrical equipment detection, and discloses a defect image enhancement method fusing reasoning and generation, which integrates visible light, infrared and laser radar data through a multi-modal feature fusion network, breaks through the limitation that a contrast file CN114281093A only depends on a visible light image, and improves the detection accuracy. The dynamic attention mechanism can flexibly deploy visual, spatial and semantic feature weights according to defect types, key features can still be captured in complex environments such as strong light and shielding, and meanwhile, the spatial form of the defects is analyzed by means of three-dimensional point cloud; by means of the design, missing detection caused by insufficient characteristics of tiny parts such as hardware fittings and pins is effectively avoided. Aiming at the problem of distortion of a sample generated by a traditional data enhancement method in a comparison file, the sample quality is guaranteed through double mechanisms of reasoning constraint and physical verification, defect features output by a reasoning model directly constrain feature distribution of the generated sample, and meanwhile, a material mechanics rule is introduced to verify the physical rationality of the generated sample.

The invention relates to the field of drawing recognition, in particular to a PDF drawing data extraction method and system based on intelligent recognition. Comprising the following steps: reading an internal structure of a PDF engineering drawing to obtain a native text stream, a vector path and a grating image; identifying the native text flow through a shunt preprocessing framework to form structured text data; rendering the vector path and the grating image to obtain a background image; analyzing the structured text data by utilizing the intelligent recognition model through the character recognition and extraction sub-model, obtaining drawing metadata and recording the position, and obtaining a character recognition result; analyzing the background image through a graphic element recognition and classification sub-model, recognizing and classifying component elements, and obtaining a graphic recognition result; and performing fusion according to the visual space corresponding relation to form a drawing analysis result. According to the method, the adaptive capacity of engineering drawings with various sources and different qualities is improved through the shunting preprocessing framework and the intelligent identification model.

The invention relates to a vision-semantic collaborative perception method and system for an automatic driving vehicle. Comprising a visual-semantic feature coding module and a visual-semantic feature fusion module. The visual-semantic feature encoding module comprises a text semantic encoder and a visual space-time encoder; a visual space-time encoder extracts single-frame semantic and structural features based on a 2D visual encoding model, and a 3D visual encoding model is adopted for modeling to form 3D frame-level fusion visual perception features; and the visual-semantic feature fusion module comprises a time converter, a cross converter and a context enrichment converter, and is used for adaptively adjusting a fusion weight based on a dynamic weight fusion strategy to generate required visual-semantic feature fusion information. According to the invention, the multi-modal weight is automatically adjusted, the environment perception error and decision delay are reduced, the adaptability and response precision of the automatic driving vehicle to the trunk line logistics complex traffic scene are improved, and the safe and stable operation of the intelligent driving system in the complex road environment is ensured.

The invention discloses a mathematical formula identification coding method, and particularly relates to the technical field of formula coding. The method comprises the following steps: acquiring mathematical formula image data to be identified, and performing symbol boundary extraction and preprocessing to generate symbol feature expression data; and performing visual spatial layout analysis and symbol type semantic classification based on the symbol feature expression data to generate formula layout structure data and symbol semantic classification data. And through graph structure analysis based on a topological relation, determining an inter-symbol topological relation of the mathematical formula, and generating symbol topological relation data. And deriving a dimension constraint relationship and an operator dependency relationship between symbols through a mathematical meta-knowledge mining technology, and generating mathematical meta-knowledge constraint data. Initial mathematical formula structure expression data is generated through decoding of structure and semantic constraint fusion, and a formula coding sequence is generated through formula consistency verification and semantic constraint reconstruction. According to the invention, the accuracy and efficiency of mathematical formula identification can be effectively improved.

The invention relates to the technical field of computer vision, and discloses a figure interaction detection method based on spatial fine-grained context interaction feature fusion, which comprises the following steps of: firstly, performing target detection on an input image to obtain a target detection result set and a person-object pairing feature; performing gridding projection on the image to obtain image global features, and inputting the image global features into a spatial fine-grained feature learning module to obtain spatial fine-grained features; inputting the spatial fine-grained features and the human-object pairing features into a spatial context interaction feature fusion module to obtain spatial context interaction features, and inputting the spatial context interaction features and the spatial fine-grained features into a visual encoder to obtain enhanced human-object pairing features; obtaining an interaction category score according to the feature and a text embedding feature of a character interaction category; and iteratively optimizing the character interaction detection model until convergence. According to the method, the description of local details of a visual space is enhanced, the context relationship between a person and an object is enhanced, and the accuracy of person interaction detection is improved.

The invention relates to the technical field of urban planning and spatial information, in particular to an urban landscape style assessment method and system based on three-dimensional reconstruction, and the method comprises the following steps: S1, generating a live-action three-dimensional model of a target urban area; s2, dividing the target urban area into a plurality of space grid units on a horizontal plane; s3, extracting three-dimensional visual feature parameters in each vision field range; s4, calculating a visual space oppression index according to the geometrical relationship of the buildings in the visual field range; s5, calculating a landscape style and feature comprehensive evaluation value of the space grid unit; and S6, mapping the comprehensive evaluation value of the landscape style to the geographic position of the landscape style, and generating an urban landscape style evaluation thermodynamic diagram. According to the method, through three-dimensional modeling, visual feature extraction and comprehensive quantitative evaluation, three-dimensional, standardized and visual evaluation of urban landscape styles and features is realized, and scientific support is provided for urban planning and styles and features management.

The invention discloses a low-cost robot imitation learning method and system based on a human video. The method comprises the following steps: S1, data acquisition; s2, data extraction and physical alignment are carried out to eliminate man-machine physical differences; the step is divided into two parallel processing modules of action space alignment and visual space alignment; s3, data set construction: mixing the aligned human data with real robot teleoperation data, carrying out balanced sampling, and constructing a mixed data set Dmix; and S4, cooperative training: constructing a strategy network based on diffusion Transform for training. According to the method, data can be acquired only through the monocular RGB camera, expensive robot teleoperation data are replaced with cheap and easily available human videos, and the data acquisition threshold is greatly reduced. Through a visual alignment strategy of random color grid rendering, a network can learn neglect skin color textures and pay attention to geometric structures without a complex generative model, so that the robot can be seamlessly migrated to robots in different forms.

The invention discloses a vehicle-mounted audio augmented reality system and method based on virtual-real fusion space anchoring, and relates to the technical field of intelligent cabin man-machine interaction, and the system comprises a controller, an augmented reality display device, a distributed loudspeaker array, and a sensor group. The controller receives the virtual image pixel coordinates, the eyeball position coordinates and the vehicle state semantic identifier, and stores a cabin three-dimensional digital model. The system converts a dynamic pixel coordinate into a three-dimensional virtual sound source coordinate by utilizing perspective inverse projection through vision-space mapping logic; the retrieval model anchors the semantic identifier to the physical component coordinates through the semantic-space mapping logic. The audio rendering module calculates a speaker drive gain based on the virtual sound source coordinates to synthesize a virtual sound source. By constructing a unified cabin three-dimensional digital model and a double-channel mapping mechanism, spatial alignment of an audio sound image, an AR visual track and a physical part of a vehicle body is realized, and audio-visual perception splitting is eliminated.

The invention relates to the technical field of photovoltaic safety monitoring, and discloses a photovoltaic power station fire hazard monitoring system based on computer vision. The system comprises an image acquisition unit, a visual feature extraction unit, a feature fusion unit, a hidden danger weight distribution unit and a hidden danger tracing unit. The image acquisition unit acquires real-time visible light image data and infrared thermal imaging data; a visual feature extraction unit extracts a visual space feature vector and a heat distribution feature vector; the feature fusion unit performs multi-modal fusion on the two types of vectors to generate a fusion monitoring feature set; the hidden danger weight distribution unit calculates a hidden danger association degree score set through a pre-trained convolutional neural network model, wherein the set comprises association strength of abnormal heat distribution and visual abnormality of different regions; and the hidden danger tracing unit generates a fire hidden danger positioning result based on association intensity distribution joint positioning. The system can comprehensively capture the state of the photovoltaic power station, improves the accuracy and comprehensiveness of fire hazard monitoring, and assists the safe operation of the photovoltaic power station.

The invention discloses a laboratory safety assessment method based on machine vision, and relates to the technical field of laboratory safety monitoring, and the method comprises the steps: carrying out the standard processing of collected laboratory comprehensive data, obtaining a standard monitoring image, carrying out the feature element setting of the standard monitoring image, and obtaining a feature recognition matrix; constructing a virtual visual space, performing supplementary positioning on the virtual visual space to obtain a virtual laboratory model and an image acquisition node, and performing feature acquisition on the standard monitoring image through a feature recognition matrix based on the virtual visual space to obtain a monitoring feature picture; performing periodic comparison on the monitoring feature pictures to obtain a difference comparison result, and performing safety judgment on the laboratory according to the difference comparison result to obtain an anomaly evaluation result; laboratory safety is greatly improved, accident loss is reduced, and the safety management level of a laboratory is improved.

Implementations for robot control are provided. A method involves, based on vision data depicting an environment of a robot and a natural language instruction for the robot, determining an affordance plan for performing a task. The affordance plan comprises a sequence of intermediate representations of the robot in visual space, such as end effector poses. An action input prompt is assembled with data indicative of the vision data, the natural language instruction, and the affordance plan. The action input prompt is processed using one or more generative models to generate action output indicative of one or more actions to be performed by the robot. Subsequently, a robot control signal is generated based on the one or more actions. This provides a spatially precise and dimensionally concise form of guidance for robot manipulation tasks, which can improve performance and generalization.

The application discloses a kind of double-branch zero sample remote sensing scene classification method based on knowledge graph, it is related to scene identification technical field, by analyzing the spatial distribution information of local area ground object composition, automatically constructs a kind of " scene-landscape-ground object " three-level knowledge graph;On this basis, the double-branch supervision zero sample remote sensing scene classification network DBSS designed from global and local two branches respectively to the mapping of semantic vector to visual space is supervised, to promote visual space can fully reflect the correlation structure contained in semantic space;Through a large number of experiments on UCM, AID and NWPU data set, it shows that the class average accuracy and overall accuracy of AKG-DBSS for the classification of invisible class scene can reach 98% and 59.56% respectively, the standard deviation is less than 6.91%, significantly better than other prior art methods, with good scalability.

The invention provides an intraoperative gauze counting and tracking method based on visual perception, and relates to the technical field of visual pattern recognition. A collaborative perception framework integrating multispectral physical fingerprint recognition, visual space-time tracking and adaptive entangled state filtering is constructed; the core problem of target identity confirmation and persistent tracking in the prior art is solved. High-robustness fluorescent fingerprints are introduced to serve as identity anchor points, deep coupling and mutual correction of physical identities and spatial-temporal trajectories are achieved through an adaptive fusion algorithm, and finally absolute identity confirmation and high-precision and high-robustness continuous tracking of each target are achieved. According to the method, the recognition accuracy in a seriously polluted and shielded environment is improved, error accumulation in a long-term tracking process is effectively inhibited, and the self-adaptive capability and reliability of the whole system in a dynamic complex scene are enhanced.

This invention discloses a dynamic monitoring and alarm system for electrical fire hazards based on big data analysis, belonging to the field of electrical safety monitoring and intelligent early warning technology. It includes modules for visual space modeling and interference prediction, temporal thermal anomaly extraction, structural contour recognition and consistency judgment, temperature disturbance assessment and credibility grading, composite credibility modeling and hotspot correction, and dynamic scoring and identification model update. The visual space modeling and interference prediction module acquires the field-of-view projection model of the thermal imaging monitoring area and establishes a high-risk pixel distribution model for reflection interference based on camera installation parameters and spatial geometric features. This invention constructs an intelligent identification closed-loop system from perception to decision-making, integrating spatial modeling, thermal behavior analysis, and dynamic credibility assessment to achieve accurate identification and continuous adjustment of false hotspots, effectively avoiding false alarms and missed alarms, and significantly improving the accuracy and system stability of electrical fire early warning.

The invention relates to a spatial position instruction fine tuning method based on a multi-modal large language model, and the method comprises the following steps: S1, converting a spatial position reasoning data set into a visual instruction format through employing a dialogue template, and obtaining a visual spatial position reasoning data set; s2, acquiring a large language model InternVL as a multi-modal large language model, performing pre-training on the general data set to obtain a pre-training model, reasoning the data set based on the visual spatial position, adjusting parameters of the pre-training model by adopting a low-rank adaptation method to obtain a trained large language model, and outputting a description corresponding to a spatial task by the large language model; and S3, introducing a text-based large language model, and optimizing the description corresponding to the space task based on the large language model. Compared with the prior art, the method has the advantages that the ability of the multi-modal large language model in understanding and generating context rich description is fully utilized, and the ability of the model in generating accurate and detailed description is enhanced.

The application provides a belt deviation monitoring method based on visual space mapping, and belongs to the technical field of belt deviation monitoring, solves the problem that the existing belt deviation monitoring technology is complex in calculation and cannot directly reflect the specific deviation of the belt, the belt target detection module is used to convert a video stream into an image frame, and the belt in the image frame is detected to draw the contour of the belt, the coordinate mapping module and the center point calculation module are used to calculate the corresponding coordinates of the world coordinate system of the center of the belt in the image frame, the angle correction module is used to detect the change of the angle of the belt, the center coordinates of the world coordinate system are corrected according to the change angle of the belt, and the specific deviation of the belt is obtained by using the deviation calculation module. The mapping relationship between the image coordinates and the space coordinates is used, the specific deviation of the belt is obtained by using the matrix calculation mode, the fuzzy classification of the monitoring result is abandoned, and the deviation of the belt in the working process is monitored in real time.

The invention discloses a self-adaptive braking energy recovery control method based on multi-mode perception fusion, and belongs to the technical field of new energy automobile control. The method comprises the following steps: synchronously acquiring RGB image flow of a front-view camera of a vehicle and dynamics state data of a CAN bus of a chassis, and performing space-time alignment by adopting an improved linear interpolation method; respectively extracting visual spatial features and dynamic time sequence features through an improved ResNet-18 network and a time domain convolutional network; performing cross attention operation to generate fusion features by taking the dynamic features as Query and the visual features as Key / Value; inputting the fusion features into a multi-task prediction head, and synchronously outputting a braking condition category and a regenerative braking distribution coefficient lambda; and the vehicle control unit calculates target electro-hydraulic braking force according to the total braking torque of the driver and lambda, and issues and executes the target electro-hydraulic braking force after ABS activation and battery SOC and temperature safety rule correction. The braking intention recognition precision and the energy recovery efficiency are improved at the same time under the complex working condition.

The application belongs to the technical field of power equipment detection, and discloses a defect image enhancement method fusing reasoning and generation, which integrates visible light, infrared and laser radar data through a multi-modal feature fusion network, breaks through the limitation of relying only on visible light images in the contrast file CN114281093A, and dynamically adjusts the weights of visual, spatial and semantic features according to the defect type, so that key features can still be captured in complex environments such as strong light and shielding, and the spatial form of the defect is analyzed with the help of three-dimensional point cloud; this design effectively avoids the missed detection of small components such as hardware pins due to insufficient features; in view of the problem that the generated samples are distorted in the traditional data enhancement method in the contrast file, the sample quality is guaranteed through a double mechanism of "reasoning constraint + physical verification", the defect features output by the reasoning model directly constrain the feature distribution of the generated samples, and the physical rationality of the generated samples is verified by introducing the law of material mechanics.

The invention discloses a multi-modal space-time alignment safe driving emotion recognition method based on sensitive word guidance. Sensitive word features are extracted through driving monitoring videos and voice data. A plurality of candidate time intervals in the video where emotions may be present are determined by analyzing the temporal attention of the sensitive word-speech text. And adaptively determining a key time frame with a driving emotion by means of sensitive word-visual time cross attention. In a key time frame, component features are extracted in a multi-space region of a face component, an alignment relationship between the component features and voice sensitive words is analyzed, and a key space region with driving emotion is adaptively determined by means of sensitive word-visual space cross attention. According to the method, the space-time asynchronous condition of the sensitive words in the multi-modal data is fully considered, the candidate time interval, the key time frame and the key space area are sequentially analyzed, the key clues of the driving emotion are gradually searched, and the accuracy of safe driving emotion recognition can be effectively improved.

The application provides a method for evaluating visual space ability based on non-immersive virtual reality. The method is used for a computer device, and the method comprises the following steps: displaying a path learning interface through a display screen, so that a subject performs path learning; the path learning interface displays a two-dimensional plane map; the map has a specified path marked with a starting point and an ending point; a virtual three-dimensional space is displayed through the display screen, so that the subject performs a three-dimensional virtual space walking test; when the subject performs the three-dimensional virtual space walking test, the subject starts from the starting point and walks along the street to the ending point according to the specified path remembered by the subject, or returns to the starting point from the ending point; the eye movement track and the mouse action of the subject are used to calculate an evaluation index of the subject; and the visual space ability of the subject is evaluated according to the evaluation index. The detection paradigm of the application can evaluate the visual space ability of subjects of different ages or different cognitive function levels, and realizes a visual space test close to a real scene.

The invention relates to the technical field of video fusion, and discloses a video fusion method and system based on a shadow map and a program product, and the method comprises the following steps: creating a virtual perspective camera in a three-dimensional scene, and generating the shadow map according to a visual cone of the virtual perspective camera; pixels in the three-dimensional scene are converted from a visual space coordinate system of a physical world camera to a virtual perspective camera coordinate system, NDC coordinates of the pixels are obtained, the NDC coordinates are aligned with texture coordinates of the shadow map, and sampling texture coordinates are obtained; judging whether the pixels in the visual range of the virtual perspective camera are in the visual area of the shadow map or not, and obtaining the color of the current pixel of the three-dimensional scene; and obtaining the color of the current pixel of the fused three-dimensional scene. According to the invention, the problems of shielding area processing errors and the like in the prior art are solved.

Methods, systems, devices and computer software / program code products enable the generation of synthetic images based on actual images captured by one or more physical cameras. The generated synthetic images provide visual spatial awareness to enable adaptive behavior in a dynamic environment, including robotics and simulation.

The invention discloses a multi-modal driver abnormal state detection method and system based on a three-expert network. The method comprises the following steps: acquiring an acquired face or upper body video image, an electroencephalogram time sequence signal and a myoelectricity time sequence signal of a driver; the collected three modal signals are preprocessed; inputting the preprocessed three modal signals into a three-expert network for feature extraction to respectively obtain a visual space feature sequence, an electroencephalogram time domain feature sequence and a myoelectricity activity feature sequence; and carrying out feature fusion on the visual space feature sequence, the electroencephalogram time domain feature sequence and the myoelectricity activity feature sequence by adopting a cross-modal attention fusion mechanism to obtain a fusion feature vector, and outputting a driver state classification result after global average pooling, full connection layer and focus loss function optimization are carried out on the fusion feature vector. According to the method, deep cross-modal fusion of visual, electroencephalogram and myoelectricity three-modal signals is realized, and the recognition accuracy and robustness of the abnormal state of the driver are remarkably improved.

The invention provides a deep learning-based ocarina fingering real-time error correction system and method, and the method comprises the steps: carrying out the image preprocessing of a real-time video stream when a target user practices an ocarina, and obtaining a standard image sequence; for each frame of standard image in the standard image sequence, determining the coverage confidence of each fingertip of the target user and the corresponding sound hole in the standard image, and determining the static fingering characteristics of the finger of the target user in the standard image for the coverage state of the corresponding sound hole according to all the coverage confidence; determining a static fingering feature of a target user finger to a corresponding sound hole coverage state in each frame of standard image; performing time sequence context analysis on all the static fingering features to obtain a fingering track fusing fingering conversion spatio-temporal context in the ocarina practicing process of the target user; and outputting a fingering identification result of the target user practicing the ocarina based on the fingering track. By adopting the scheme of the invention, the ocarina fingering dynamic identification fusing the visual space information and the playing time sequence context can be realized.

The application discloses a kind of multi-modal space-time alignment safe driving emotion recognition methods based on sensitive word guide.The application extracts sensitive word features by driving monitoring video and voice data.With the analysis sensitive word-voice text time attention, determine the multiple candidate time intervals that possibly exist emotion in video.With the sensitive word-visual time cross attention, adaptively determine the key time frame that exists driving emotion.In key time frame, extract component features in the multiple space regions of human face component, analyze its alignment relationship with voice sensitive word, with the sensitive word-visual space cross attention, adaptively determine the key space region that exists driving emotion.The application fully considers the space-time asynchronization of sensitive word in multi-modal data, sequentially analyzes candidate time interval, key time frame, key space region, gradually finds the key clue of driving emotion, and can effectively improve the accuracy of safe driving emotion recognition.

The invention provides a power grid image recognition method based on unmanned aerial vehicle edge calculation and related equipment, and relates to the technical field of image processing, and the method comprises the steps: fitting a line trend curve based on historical inspection point positions of a to-be-inspected line, and constructing a virtual line simulation flight corridor along the line trend curve based on a view field capture range; extracting a tower head contour feature center in the inspection image, calculating a space relative position vector of the tower head contour feature center relative to the virtual line-imitating flight corridor, and generating flight attitude correction data; identifying a rigid support assembly in the inspection image, constructing an assembly skeleton topology, and generating a flexible line tracking window along the connection direction of the assembly skeleton topology; and calculating a form distortion value of the component skeleton topology and texture gray scale distribution data in the flexible circuit tracking window, and generating defect identification data according to the form distortion value and the texture gray scale distribution data. According to the method, the virtual line-imitating gallery and the visual space mapping closed loop are constructed, and the flexible wire area is locked by using the rigid topology constraint, so that the positioning precision of edge end inspection and the defect identification efficiency are improved.

The invention discloses a bionic visual information perception method and system based on semantic driving. The method comprises the steps that scene image information containing target content is acquired; performing semantic analysis on the scene image information to extract high-level semantic information, and calculating a high-level semantic weight based on the high-level semantic information; identifying visual features of the scene image information, and fusing the visual features and the high-level semantic weight to screen out key information pixels from the scene image information; mapping the key information pixel into a corresponding cortical electrode number based on a topological corresponding relation between a preset visual space and a visual cortical layer, and generating a stimulation vector corresponding to the cortical electrode number; and outputting the cortical electrode number and the stimulation vector to bionic visual stimulation equipment so as to realize perception of the scene image information by a user. Through a semantic-driven key information pixel screening and structured expression mechanism, efficient transmission of facial expressions and text information is realized under the limitation of a low-pixel channel.

The invention provides an efficient and accurate integrated imaging light field three-dimensional salient target detection method. According to the method, feature extraction and fusion are carried out on four-dimensional light field data through a visual state space coding module, and three-dimensional saliency detection and optical reconstruction are achieved. Firstly, light field data of a three-dimensional scene are obtained through a lens array and recorded in a two-dimensional micro-image array. Then, the micro-image array is divided into sub-micro-image sequences, and the sub-micro-image sequences are input into a visual space state module for multi-scale feature coding and modeling; a salient region is modeled by a saliency-guided Mama module under a selective scanning mechanism, and the robustness and space consistency of features are enhanced through multi-path scanning. And after decoding by the hybrid attention module, the recovery and detail enhancement of the light field salient features are completed, and a high-quality salient micro-image array is generated. And finally, optical reconstruction is completed in combination with a lens array, a three-dimensional salient target with real depth features is obtained, and the balance between the detection precision and the calculation efficiency is realized.

The invention discloses a visual space adjusting system. The visual space adjusting system comprises a transcranial magnetic stimulation therapeutic apparatus and a visual space adjusting device, wherein the transcranial magnetic stimulation therapeutic apparatus comprises a stimulation coil; a camera; the camera adjusting holder is used for fixing the camera and adjusting the camera; the upper computer is respectively communicated with the camera and the camera adjusting holder, the upper computer is provided with a readable storage medium, a program or an instruction is stored on the readable storage medium, and the visual space adjusting method is realized when the program or the instruction is executed by the processor. The visual space adjustment method comprises the following steps of S1, performing first adjustment on a camera based on spatial change data of a first target object relative to the camera, and obtaining first spatial data of the camera; and S2, performing second adjustment on the camera based on the first spatial data of the camera and the second spatial data of the second target object.