63 results about "Visual space" patented technology

The invention belongs to the technical field of electrical equipment detection, and discloses a defect image enhancement method fusing reasoning and generation, which integrates visible light, infrared and laser radar data through a multi-modal feature fusion network, breaks through the limitation that a contrast file CN114281093A only depends on a visible light image, and improves the detection accuracy. The dynamic attention mechanism can flexibly deploy visual, spatial and semantic feature weights according to defect types, key features can still be captured in complex environments such as strong light and shielding, and meanwhile, the spatial form of the defects is analyzed by means of three-dimensional point cloud; by means of the design, missing detection caused by insufficient characteristics of tiny parts such as hardware fittings and pins is effectively avoided. Aiming at the problem of distortion of a sample generated by a traditional data enhancement method in a comparison file, the sample quality is guaranteed through double mechanisms of reasoning constraint and physical verification, defect features output by a reasoning model directly constrain feature distribution of the generated sample, and meanwhile, a material mechanics rule is introduced to verify the physical rationality of the generated sample.

The invention relates to the field of drawing recognition, in particular to a PDF drawing data extraction method and system based on intelligent recognition. Comprising the following steps: reading an internal structure of a PDF engineering drawing to obtain a native text stream, a vector path and a grating image; identifying the native text flow through a shunt preprocessing framework to form structured text data; rendering the vector path and the grating image to obtain a background image; analyzing the structured text data by utilizing the intelligent recognition model through the character recognition and extraction sub-model, obtaining drawing metadata and recording the position, and obtaining a character recognition result; analyzing the background image through a graphic element recognition and classification sub-model, recognizing and classifying component elements, and obtaining a graphic recognition result; and performing fusion according to the visual space corresponding relation to form a drawing analysis result. According to the method, the adaptive capacity of engineering drawings with various sources and different qualities is improved through the shunting preprocessing framework and the intelligent identification model.

The invention discloses a mathematical formula identification coding method, and particularly relates to the technical field of formula coding. The method comprises the following steps: acquiring mathematical formula image data to be identified, and performing symbol boundary extraction and preprocessing to generate symbol feature expression data; and performing visual spatial layout analysis and symbol type semantic classification based on the symbol feature expression data to generate formula layout structure data and symbol semantic classification data. And through graph structure analysis based on a topological relation, determining an inter-symbol topological relation of the mathematical formula, and generating symbol topological relation data. And deriving a dimension constraint relationship and an operator dependency relationship between symbols through a mathematical meta-knowledge mining technology, and generating mathematical meta-knowledge constraint data. Initial mathematical formula structure expression data is generated through decoding of structure and semantic constraint fusion, and a formula coding sequence is generated through formula consistency verification and semantic constraint reconstruction. According to the invention, the accuracy and efficiency of mathematical formula identification can be effectively improved.

The invention relates to the technical field of computer vision, and discloses a figure interaction detection method based on spatial fine-grained context interaction feature fusion, which comprises the following steps of: firstly, performing target detection on an input image to obtain a target detection result set and a person-object pairing feature; performing gridding projection on the image to obtain image global features, and inputting the image global features into a spatial fine-grained feature learning module to obtain spatial fine-grained features; inputting the spatial fine-grained features and the human-object pairing features into a spatial context interaction feature fusion module to obtain spatial context interaction features, and inputting the spatial context interaction features and the spatial fine-grained features into a visual encoder to obtain enhanced human-object pairing features; obtaining an interaction category score according to the feature and a text embedding feature of a character interaction category; and iteratively optimizing the character interaction detection model until convergence. According to the method, the description of local details of a visual space is enhanced, the context relationship between a person and an object is enhanced, and the accuracy of person interaction detection is improved.

The invention discloses a low-cost robot imitation learning method and system based on a human video. The method comprises the following steps: S1, data acquisition; s2, data extraction and physical alignment are carried out to eliminate man-machine physical differences; the step is divided into two parallel processing modules of action space alignment and visual space alignment; s3, data set construction: mixing the aligned human data with real robot teleoperation data, carrying out balanced sampling, and constructing a mixed data set Dmix; and S4, cooperative training: constructing a strategy network based on diffusion Transform for training. According to the method, data can be acquired only through the monocular RGB camera, expensive robot teleoperation data are replaced with cheap and easily available human videos, and the data acquisition threshold is greatly reduced. Through a visual alignment strategy of random color grid rendering, a network can learn neglect skin color textures and pay attention to geometric structures without a complex generative model, so that the robot can be seamlessly migrated to robots in different forms.

The invention discloses a vehicle-mounted audio augmented reality system and method based on virtual-real fusion space anchoring, and relates to the technical field of intelligent cabin man-machine interaction, and the system comprises a controller, an augmented reality display device, a distributed loudspeaker array, and a sensor group. The controller receives the virtual image pixel coordinates, the eyeball position coordinates and the vehicle state semantic identifier, and stores a cabin three-dimensional digital model. The system converts a dynamic pixel coordinate into a three-dimensional virtual sound source coordinate by utilizing perspective inverse projection through vision-space mapping logic; the retrieval model anchors the semantic identifier to the physical component coordinates through the semantic-space mapping logic. The audio rendering module calculates a speaker drive gain based on the virtual sound source coordinates to synthesize a virtual sound source. By constructing a unified cabin three-dimensional digital model and a double-channel mapping mechanism, spatial alignment of an audio sound image, an AR visual track and a physical part of a vehicle body is realized, and audio-visual perception splitting is eliminated.

Implementations for robot control are provided. A method involves, based on vision data depicting an environment of a robot and a natural language instruction for the robot, determining an affordance plan for performing a task. The affordance plan comprises a sequence of intermediate representations of the robot in visual space, such as end effector poses. An action input prompt is assembled with data indicative of the vision data, the natural language instruction, and the affordance plan. The action input prompt is processed using one or more generative models to generate action output indicative of one or more actions to be performed by the robot. Subsequently, a robot control signal is generated based on the one or more actions. This provides a spatially precise and dimensionally concise form of guidance for robot manipulation tasks, which can improve performance and generalization.

This invention discloses a dynamic monitoring and alarm system for electrical fire hazards based on big data analysis, belonging to the field of electrical safety monitoring and intelligent early warning technology. It includes modules for visual space modeling and interference prediction, temporal thermal anomaly extraction, structural contour recognition and consistency judgment, temperature disturbance assessment and credibility grading, composite credibility modeling and hotspot correction, and dynamic scoring and identification model update. The visual space modeling and interference prediction module acquires the field-of-view projection model of the thermal imaging monitoring area and establishes a high-risk pixel distribution model for reflection interference based on camera installation parameters and spatial geometric features. This invention constructs an intelligent identification closed-loop system from perception to decision-making, integrating spatial modeling, thermal behavior analysis, and dynamic credibility assessment to achieve accurate identification and continuous adjustment of false hotspots, effectively avoiding false alarms and missed alarms, and significantly improving the accuracy and system stability of electrical fire early warning.

The invention relates to a spatial position instruction fine tuning method based on a multi-modal large language model, and the method comprises the following steps: S1, converting a spatial position reasoning data set into a visual instruction format through employing a dialogue template, and obtaining a visual spatial position reasoning data set; s2, acquiring a large language model InternVL as a multi-modal large language model, performing pre-training on the general data set to obtain a pre-training model, reasoning the data set based on the visual spatial position, adjusting parameters of the pre-training model by adopting a low-rank adaptation method to obtain a trained large language model, and outputting a description corresponding to a spatial task by the large language model; and S3, introducing a text-based large language model, and optimizing the description corresponding to the space task based on the large language model. Compared with the prior art, the method has the advantages that the ability of the multi-modal large language model in understanding and generating context rich description is fully utilized, and the ability of the model in generating accurate and detailed description is enhanced.

The application provides a belt deviation monitoring method based on visual space mapping, and belongs to the technical field of belt deviation monitoring, solves the problem that the existing belt deviation monitoring technology is complex in calculation and cannot directly reflect the specific deviation of the belt, the belt target detection module is used to convert a video stream into an image frame, and the belt in the image frame is detected to draw the contour of the belt, the coordinate mapping module and the center point calculation module are used to calculate the corresponding coordinates of the world coordinate system of the center of the belt in the image frame, the angle correction module is used to detect the change of the angle of the belt, the center coordinates of the world coordinate system are corrected according to the change angle of the belt, and the specific deviation of the belt is obtained by using the deviation calculation module. The mapping relationship between the image coordinates and the space coordinates is used, the specific deviation of the belt is obtained by using the matrix calculation mode, the fuzzy classification of the monitoring result is abandoned, and the deviation of the belt in the working process is monitored in real time.

The invention discloses a self-adaptive braking energy recovery control method based on multi-mode perception fusion, and belongs to the technical field of new energy automobile control. The method comprises the following steps: synchronously acquiring RGB image flow of a front-view camera of a vehicle and dynamics state data of a CAN bus of a chassis, and performing space-time alignment by adopting an improved linear interpolation method; respectively extracting visual spatial features and dynamic time sequence features through an improved ResNet-18 network and a time domain convolutional network; performing cross attention operation to generate fusion features by taking the dynamic features as Query and the visual features as Key / Value; inputting the fusion features into a multi-task prediction head, and synchronously outputting a braking condition category and a regenerative braking distribution coefficient lambda; and the vehicle control unit calculates target electro-hydraulic braking force according to the total braking torque of the driver and lambda, and issues and executes the target electro-hydraulic braking force after ABS activation and battery SOC and temperature safety rule correction. The braking intention recognition precision and the energy recovery efficiency are improved at the same time under the complex working condition.

The application belongs to the technical field of power equipment detection, and discloses a defect image enhancement method fusing reasoning and generation, which integrates visible light, infrared and laser radar data through a multi-modal feature fusion network, breaks through the limitation of relying only on visible light images in the contrast file CN114281093A, and dynamically adjusts the weights of visual, spatial and semantic features according to the defect type, so that key features can still be captured in complex environments such as strong light and shielding, and the spatial form of the defect is analyzed with the help of three-dimensional point cloud; this design effectively avoids the missed detection of small components such as hardware pins due to insufficient features; in view of the problem that the generated samples are distorted in the traditional data enhancement method in the contrast file, the sample quality is guaranteed through a double mechanism of "reasoning constraint + physical verification", the defect features output by the reasoning model directly constrain the feature distribution of the generated samples, and the physical rationality of the generated samples is verified by introducing the law of material mechanics.

The application provides a method for evaluating visual space ability based on non-immersive virtual reality. The method is used for a computer device, and the method comprises the following steps: displaying a path learning interface through a display screen, so that a subject performs path learning; the path learning interface displays a two-dimensional plane map; the map has a specified path marked with a starting point and an ending point; a virtual three-dimensional space is displayed through the display screen, so that the subject performs a three-dimensional virtual space walking test; when the subject performs the three-dimensional virtual space walking test, the subject starts from the starting point and walks along the street to the ending point according to the specified path remembered by the subject, or returns to the starting point from the ending point; the eye movement track and the mouse action of the subject are used to calculate an evaluation index of the subject; and the visual space ability of the subject is evaluated according to the evaluation index. The detection paradigm of the application can evaluate the visual space ability of subjects of different ages or different cognitive function levels, and realizes a visual space test close to a real scene.

The invention relates to the technical field of video fusion, and discloses a video fusion method and system based on a shadow map and a program product, and the method comprises the following steps: creating a virtual perspective camera in a three-dimensional scene, and generating the shadow map according to a visual cone of the virtual perspective camera; pixels in the three-dimensional scene are converted from a visual space coordinate system of a physical world camera to a virtual perspective camera coordinate system, NDC coordinates of the pixels are obtained, the NDC coordinates are aligned with texture coordinates of the shadow map, and sampling texture coordinates are obtained; judging whether the pixels in the visual range of the virtual perspective camera are in the visual area of the shadow map or not, and obtaining the color of the current pixel of the three-dimensional scene; and obtaining the color of the current pixel of the fused three-dimensional scene. According to the invention, the problems of shielding area processing errors and the like in the prior art are solved.

Methods, systems, devices and computer software / program code products enable the generation of synthetic images based on actual images captured by one or more physical cameras. The generated synthetic images provide visual spatial awareness to enable adaptive behavior in a dynamic environment, including robotics and simulation.

The invention discloses a multi-modal driver abnormal state detection method and system based on a three-expert network. The method comprises the following steps: acquiring an acquired face or upper body video image, an electroencephalogram time sequence signal and a myoelectricity time sequence signal of a driver; the collected three modal signals are preprocessed; inputting the preprocessed three modal signals into a three-expert network for feature extraction to respectively obtain a visual space feature sequence, an electroencephalogram time domain feature sequence and a myoelectricity activity feature sequence; and carrying out feature fusion on the visual space feature sequence, the electroencephalogram time domain feature sequence and the myoelectricity activity feature sequence by adopting a cross-modal attention fusion mechanism to obtain a fusion feature vector, and outputting a driver state classification result after global average pooling, full connection layer and focus loss function optimization are carried out on the fusion feature vector. According to the method, deep cross-modal fusion of visual, electroencephalogram and myoelectricity three-modal signals is realized, and the recognition accuracy and robustness of the abnormal state of the driver are remarkably improved.

The invention provides a deep learning-based ocarina fingering real-time error correction system and method, and the method comprises the steps: carrying out the image preprocessing of a real-time video stream when a target user practices an ocarina, and obtaining a standard image sequence; for each frame of standard image in the standard image sequence, determining the coverage confidence of each fingertip of the target user and the corresponding sound hole in the standard image, and determining the static fingering characteristics of the finger of the target user in the standard image for the coverage state of the corresponding sound hole according to all the coverage confidence; determining a static fingering feature of a target user finger to a corresponding sound hole coverage state in each frame of standard image; performing time sequence context analysis on all the static fingering features to obtain a fingering track fusing fingering conversion spatio-temporal context in the ocarina practicing process of the target user; and outputting a fingering identification result of the target user practicing the ocarina based on the fingering track. By adopting the scheme of the invention, the ocarina fingering dynamic identification fusing the visual space information and the playing time sequence context can be realized.

The application discloses a kind of multi-modal space-time alignment safe driving emotion recognition methods based on sensitive word guide.The application extracts sensitive word features by driving monitoring video and voice data.With the analysis sensitive word-voice text time attention, determine the multiple candidate time intervals that possibly exist emotion in video.With the sensitive word-visual time cross attention, adaptively determine the key time frame that exists driving emotion.In key time frame, extract component features in the multiple space regions of human face component, analyze its alignment relationship with voice sensitive word, with the sensitive word-visual space cross attention, adaptively determine the key space region that exists driving emotion.The application fully considers the space-time asynchronization of sensitive word in multi-modal data, sequentially analyzes candidate time interval, key time frame, key space region, gradually finds the key clue of driving emotion, and can effectively improve the accuracy of safe driving emotion recognition.

The invention provides a power grid image recognition method based on unmanned aerial vehicle edge calculation and related equipment, and relates to the technical field of image processing, and the method comprises the steps: fitting a line trend curve based on historical inspection point positions of a to-be-inspected line, and constructing a virtual line simulation flight corridor along the line trend curve based on a view field capture range; extracting a tower head contour feature center in the inspection image, calculating a space relative position vector of the tower head contour feature center relative to the virtual line-imitating flight corridor, and generating flight attitude correction data; identifying a rigid support assembly in the inspection image, constructing an assembly skeleton topology, and generating a flexible line tracking window along the connection direction of the assembly skeleton topology; and calculating a form distortion value of the component skeleton topology and texture gray scale distribution data in the flexible circuit tracking window, and generating defect identification data according to the form distortion value and the texture gray scale distribution data. According to the method, the virtual line-imitating gallery and the visual space mapping closed loop are constructed, and the flexible wire area is locked by using the rigid topology constraint, so that the positioning precision of edge end inspection and the defect identification efficiency are improved.

The invention discloses a bionic visual information perception method and system based on semantic driving. The method comprises the steps that scene image information containing target content is acquired; performing semantic analysis on the scene image information to extract high-level semantic information, and calculating a high-level semantic weight based on the high-level semantic information; identifying visual features of the scene image information, and fusing the visual features and the high-level semantic weight to screen out key information pixels from the scene image information; mapping the key information pixel into a corresponding cortical electrode number based on a topological corresponding relation between a preset visual space and a visual cortical layer, and generating a stimulation vector corresponding to the cortical electrode number; and outputting the cortical electrode number and the stimulation vector to bionic visual stimulation equipment so as to realize perception of the scene image information by a user. Through a semantic-driven key information pixel screening and structured expression mechanism, efficient transmission of facial expressions and text information is realized under the limitation of a low-pixel channel.

The invention discloses a visual spatial position memory normal-form method and system capable of realizing confidence feedback. The method is executed by a visual spatial position memory paradigm system capable of realizing confidence feedback, and comprises the following steps: firstly, displaying a memory interface containing a target fixation point and a randomly generated stimulation point on a display screen of the system; when display reaches a first preset duration, the stimulation point disappears, and a memory maintenance interface is entered; and after the display of the maintained interface reaches a second preset duration, receiving a target memory position point input by the subject and a confidence interval thereof. And then, calculating and outputting a memory score of the subject according to the spatial position relationship between the stimulation point and the confidence interval and a preset formula. By introducing confidence interval feedback, high-precision and quantifiable evaluation of the memory space position of the subject is realized.

The application discloses a display device adaptive adjustment method and system based on environmental interaction, and belongs to the technical field of display and human-computer interaction. The prior art relies on a single-point environmental illumination sensor, and it is difficult to accurately represent the spatial lighting distribution, screen reflection and human eye visual perception difference in a real viewing scene. The application collects scene information of a viewing environment, screen physical parameters and user position parameters, establishes a background wall brightness distribution model, a screen reflection model and an environmental contrast representation model; a visual spatial frequency model corresponding to a display stimulus is constructed; in combination with the environmental brightness representation and the visual perception model, a target optimization function containing visibility, comfort and stability constraints is established; the target display brightness is obtained by solving, and smooth adjustment and closed-loop feedback correction are performed. The application can adaptively optimize the display brightness according to multiple light sources, complex backgrounds and user visual characteristics, significantly improve the viewing comfort and detail visibility, and is suitable for various display terminals.

A system for visualizing and comparing high-dimensional text embeddings from a language model is described. The system can receive embedding vectors for input concepts from a language model, where the embedding vectors are obtained for the language model without modifying or retraining the language model. The system can project the embedding vectors into a low-dimensional visual space defined by one or more conceptual pole pairs, where each conceptual pole pair includes predefined anchor embeddings representing divergent ends of a semantic dimension, and position the input concepts at points in the visual space using similarity measures for the embedding vector of each input concept relative to the anchor embeddings of each conceptual pole pair. The system can also generate an interactive graphical visualization of the plurality of input concepts in the visual space, where the interactive graphical visualization displays each input concept at its respective point in the visual space.

The invention discloses an emergency command visual scheduling system and method based on multi-source police service data fusion, particularly relates to the technical field of data processing and information retrieval, and is used for solving the problem that an existing command system lacks an accurate feedback channel from an execution end to a command end, so that command intention and execution actual effect are disjointed. The method comprises the following steps: constructing a police affair dynamic mapping object corresponding to a physical police force unit in a visual space, responding to a defined instruction area, generating a graphical scheduling instruction, receiving execution feedback data of an associated police force unit, and carrying out global mutual verification and trajectory optimization on multi-source feedback data based on a cooperative relationship constraint analyzed from the instruction, so as to obtain a multi-source multi-source dynamic mapping object. The method comprises the following steps: generating a trusted execution trajectory set, analyzing an instruction into a dynamic constraint condition, calculating a satisfaction state sequence based on a trajectory, determining a matching degree by analyzing a sequence entropy value, and finally evaluating an execution deviation feature according to the matching degree and carrying out association labeling in a visual space, thereby forming a complete command and control closed loop.

This invention discloses a language model-driven zero-shot object detection method and system. The method includes: training a supervised detection model using visible class data from a dataset; extracting semantic vectors from a large language model based on data class names to generate external knowledge; extracting visual features of visible class images using the supervised detection model, and training a generative adversarial network (GAN) based on pseudo-visual features synthesized from external knowledge; using the GAN to synthesize pseudo-visual features for invisible class data, training an invisible visual feature classifier, and obtaining an updated supervised detection model through parameter fusion, thereby achieving zero-shot object detection of image data. Through the technical solution of this invention, visual features with rich semantics and discriminative power can be generated, improving the understanding and expression of visual content, better aligning the visual space with the semantic space, and solving the problem of semantic confusion in zero-shot object detection.

The application discloses a supply chain warehouse resource dynamic allocation system and method based on artificial intelligence, relates to the technical field of warehouse management, and comprises a management center, wherein the management center is connected with a warehouse collection module, a resource processing module, a dynamic analysis module and an intelligent deployment module; a virtual visual space is constructed for the supply chain warehouse, warehouse resource data and order task data are collected; warehouse resource data is classified and constructed according to order task data, and a resource category storage library is obtained; the available warehouse of a target user is subjected to capacity visualization according to the resource category storage library, and an alternative resource category capacity graph is obtained; the alternative resource category capacity graph is subjected to demand constraint and sorting through the virtual visual space, a warehouse resource capacity sequence is obtained, the target user is subjected to scheme simulation optimization in the virtual visual space according to the warehouse resource capacity sequence, and an optimal dynamic allocation scheme is obtained; and the warehouse operation efficiency and the space utilization rate are greatly improved.

The invention provides a stockpiling plan making method and system based on space visualization, which are applied to the technical field of port stockpiling scheduling, and by constructing a multi-level visual interaction environment corresponding to a stockpiling physical space, the stockpiling plan making method and system fuse the stockpiling space, container attributes and stockpiling rules into the same graphical interaction environment. A traditional plan making process depending on text and experience is converted into a process that a user visually generates a stockpiling plan through graphic operations such as area delimiting and plan group associating, the stockpiling plan is automatically checked based on a preset stockpiling rule, and accurate visual feedback is provided for found abnormal states and positions. According to the method, a user can directly carry out graphical adjustment and calibration in the same visual space based on visual feedback, so that conversion from static table operation to dynamic, closed-loop and visual interactive decision making is realized, and the intuition, efficiency and complex environment adaptability of plan making are remarkably improved.

The application discloses a kind of block rendering mode graphics processing method and system.The graphics processing system includes geometry processing system and fragment processing system;Geometry processing system is used to carry out geometry processing to primitive, and visible primitive is blocked to the multiple tiles M of screen visual space;Fragment processing system is used to render the multiple tiles M, and generate the rendering image of multiple tiles M;Fragment processing system includes post-processing module, and post-processing module is used to start the pixel filtering processing of the pixel in the first pixel set Pin0 of target tile M0 in the first time after the rendering image of target tile M0 is generated, before the rendering image of multiple tiles M is all generated.This application can effectively improve the processing efficiency of overall image pixel filtering, and will not produce additional pixel shading workload.

The application provides a kind of based on 3D virtual space's realistic embodied AI interaction system and its implementation method, belong to artificial intelligence and virtual reality technical field, the system includes: open object visual space positioning system, for identifying the environmental information under current visual angle / panoramic field of view by visual language model;Hierarchical memory and narrative memory generation system, for establishing hierarchical memory architecture, store the memory information corresponding to conversation;Director engine and context assembly system, for assembling structured context information, task reasoning is carried out by director engine;Natural language driven multi-track generative animation system, for arranging time sequence to multiple track instructions by large language model, and compiling into target instruction;In front-end analysis and execution the target instruction, generate the animation sequence of the role.The AI interaction system of the application has visual perception, autonomous space movement, emotional state expression and continuous behavior control ability, and improves the sense of reality.

The invention relates to the technical field of substation hidden danger monitoring, and discloses a substation surrounding hidden danger identification method and system based on multi-source visual space fusion. The method at least comprises the following steps: carrying out hidden danger identification on a standardized multi-source monitoring data set according to a data source to obtain a candidate hidden danger identification result corresponding to visual data of each source; performing spatial mapping on each candidate hidden danger recognition result to obtain spatial distribution characteristics of each candidate hidden danger target in a station area plane coordinate system; grouping and merging the unified candidate set in combination with each spatial distribution feature to obtain a plurality of hidden danger event objects; and based on each observation set, calculating an uncertainty score corresponding to each inspection device according to a predefined uncertainty score index, and performing gating fusion on each observation set according to each uncertainty score to obtain a target hidden danger identification result of each hidden danger event object. The method can efficiently and accurately discover potential safety hazards around the transformer substation.