173 results about "Scene segmentation" patented technology

The invention relates to the technical field of image processing, in particular to an image fusion driven anti-collision identification method and system. The method comprises the following steps: acquiring multi-source image data, and performing semantic guidance image fusion according to the multi-source image data to obtain image fusion data; performing scene segmentation according to the image fusion data to obtain scene segmentation data, and performing target detection on the scene segmentation data to obtain scene target data; performing confidence hierarchical space positioning according to the scene target data to obtain target positioning data, and performing motion trail prediction according to the target positioning data to obtain motion trail prediction data; and performing collision area propagation risk assessment according to the motion trail prediction data to obtain collision risk data. By fusing image perception depth, semantic understanding dimension and trajectory evolution modeling, the method has higher environmental adaptability, weak target recovery capability and system interpretability.

The invention provides a semantic information reasoning method guided by a remote sensing scene graph, which can be applied to the technical field of remote sensing image processing. The method comprises the following steps: segmenting a remote sensing image to generate a scene segmentation image; executing target detection to generate an image block set; performing fine-grained analysis on the image global features and the image block set to generate a scene description text; performing grammar analysis, extracting a triple of objects, object attributes and relation information among the objects, and generating a remote sensing scene graph related to the problem; encoding the problem text into an embedded vector, projecting the embedded vector to a visual feature space, processing the embedded vector through a frozen self-attention layer and a learnable gating layer, and outputting an image-text interaction feature; carrying out attention fusion and double gating balance on the question coding features, and outputting a text code guided by a scene graph; and fusing the interaction features and the text codes, and reasoning local semantic information of the target area. The invention further provides a semantic information reasoning device and equipment guided by the remote sensing scene graph and a medium.

The invention discloses a building construction intelligent safety monitoring method based on the Internet of Things, and the method comprises the following steps: obtaining construction environment data, structure state data and operation behavior image data, and carrying out the preprocessing; environment state modeling, local structure strain and behavior recognition and operation scene segmentation are carried out through the edge intelligent processing unit; feature fusion is carried out, and a fusion situation vector is constructed; performing high-frequency anomaly identification and emergency preliminary screening, and generating an edge preliminary early warning result and a high-risk data fragment; constructing a safety evolution trajectory crossing a time window, and fusing historical data to generate a risk semantic map; cloud semantic reasoning operation is executed, and a final risk level judgment result and a corresponding trigger source identifier are generated; and automatically triggering a safety response instruction according to a risk level judgment result. According to the method, the Internet of Things and intelligent semantic analysis are fused, multi-source safety monitoring and self-adaptive response are realized, and the method has the advantages of high real-time performance, global perception and continuous optimization.

The invention discloses an infrared image enhancement method based on scene segmentation. The method comprises the following steps: (1) inputting an original infrared image into an infrared image scene segmentation network based on edge feature guidance to obtain a scene soft segmentation result; (2) decomposing the infrared image into a background layer image and a detail layer image by adopting bilateral filtering; (3) performing adaptive enhancement on the detail layer image based on characteristics of different scenes; (4) enhancing the background layer image by adopting contrast limited adaptive histogram equalization; and (5) carrying out weighted fusion on the enhanced detail layer image and the background layer image. According to the infrared image enhancement method based on scene segmentation, the semantic segmentation algorithm is introduced, the infrared image is divided into different scene areas, and the adaptive infrared image detail enhancement parameters are set for the detail information and noise features of each scene area, so that the quality of the infrared image can be effectively improved.

A method includes obtaining, from an image sensor, image data of a real-world scene; obtaining, from a depth sensor, sparse depth data of the real-world scene; and passing the image data to a first neural network to obtain one or more object regions of interest (ROIs) and one or more feature map ROIs. Each object ROI includes at least one detected object. The method also includes passing the image data and sparse depth data to a second neural network to obtain one or more dense depth map ROIs; aligning the one or more object ROIs, one or more feature map ROIs, and one or more dense depth map ROIs; and passing the aligned ROIs to a fully convolutional network to obtain a segmentation of the real-world scene. The segmentation contains one or more pixelwise predictions of one or more objects in the real-world scene.

The invention belongs to the technical field of substation inspection, and provides a substation inspection robot multi-sensor fusion obstacle avoidance method and system, and the technical scheme is as follows: obtaining multi-modal sensor data, including laser point cloud data, millimeter wave radar point cloud data, ultrasonic radar observation data and visual image data; preprocessing the obtained multi-modal sensor data to obtain effective observation data, and determining initial obstacle data based on the effective observation data; time synchronization and fusion are carried out on the obtained multiple kinds of initial obstacle data, the fused data serve as obstacle data, scene segmentation is carried out in combination with multiple kinds of sensor data in the fusion process, confidence coefficient decision rules are dynamically set according to different scenes, and obstacle information is determined in combination with the set confidence coefficient decision rules; and generating an optimal obstacle avoidance route based on the determined obstacle information. The problem that a traditional single sensor cannot cope with a complex scene of substation inspection business is solved, and the overall stable operation capacity of the robot is improved.

The invention discloses a double-arm clothes folding robot control method and device, equipment and a storage medium. The method comprises the following steps: determining three-dimensional point cloud data of a target workbench according to a collected image through a visual perception assembly; determining a scene label graph and a desktop edge line set through a scene division assembly according to the three-dimensional point cloud data, determining a clothing mask of the to-be-processed clothing according to the scene label graph through a clothing classification assembly, and determining a clothing description vector according to the clothing mask; determining a low-risk action set and a protection area mask from the candidate clothes folding action set through a safety protection component according to the scene label graph, the desktop edge line set and the clothes mask; determining a target clothes folding action sequence through an action planning component according to the clothes description vector, the low-risk action set and the protection area mask; and folding the to-be-processed clothes through the robot control assembly according to the target clothes folding action sequence. According to the scheme, the clothes folding efficiency and reliability of the robot are improved.

The invention belongs to the technical field of computer vision, embedded edge calculation and intelligent video monitoring, and discloses a yolk-based night smoke and fire identification optimization method, which can effectively detect flames at night or in a scene with disordered light. A multi-source data fusion and lightweight scene segmentation technology is adopted, an on-site perception model is built, and the model is combined with a small sample learning mechanism, so that night common interference factors such as vehicle lamps and street lamps can be identified; then, a federal learning algorithm and a time sequence attention module are used for modeling the dynamic characteristics of the smoke and fire, and an obtained characteristic model has high generalization ability; then, the system accurately screens out real smoke and fire candidate targets through self-adaptive threshold matching and pixel-level interference elimination operation; and finally, based on a multi-factor decision model and a dynamic edge transmission technology, rapid alarm of smoke and fire identification is realized. The whole system can also automatically optimize the whole process parameters of the method through rule mining and reinforcement learning.

The invention discloses a photovoltaic panel fine identification segmentation method based on photovoltaic power station scene knowledge constraint, and the method comprises the following steps: carrying out the thinning of a photovoltaic power station scene remote sensing image Iscene; the method comprises the following steps of: making a scene segmentation sample set Sscenelabel of the photovoltaic power station; performing training and knowledge reasoning on the photovoltaic power station scene model; the method comprises the following steps: manufacturing a photovoltaic panel instance segmentation sample set Spanellabel; a photovoltaic panel identification model Mpanel is trained; reasoning a photovoltaic panel result set of a single photovoltaic power station scene; and carrying out multi-scene photovoltaic panel achievement integration. The photovoltaic panel fine identification cascade model formed by the method does not depend on remote sensing image time phase change and territorial scope, the defect that a traditional method is low in false target discrimination capability is overcome, and the identification reliability of the photovoltaic panel in a complex scene is improved; the management efficiency is improved and the management cost is reduced for the management and maintenance of the photovoltaic array in the photovoltaic power industry, and a good support effect of a remote sensing technology is exerted.

The embodiments of the present disclosure provide an image scene segmentation and apparatus, and a device and a storage medium. The method includes: obtaining an intermediate scene segmentation image by performing scene initial segmentation and scene initial fusion on an obtained target image; detecting, from the intermediate scene segmentation image, segmentation blocks to be processed; and obtaining a target scene segmentation image of the target image by performing segmentation correction on the segmentation blocks to be processed.

The invention discloses a three-dimensional scene segmentation method based on 3D Gaussian Splitting, the core idea is to make full use of the relation between different views in the step-by-step segmentation process, which relates to two complementary aspects: 1, the target is to enhance the algorithm for aggregating a 2D segmentation mask into 3D segmentation, especially to improve the precision and integrity in a scene with a small number of lenses; this function of a small number of lenses is crucial to providing cues for new views because the cues depend on segmentation predictions from preceding views. Second, by retaining the reliable cues used in the previous view, the new view can avoid cueing inconsistent targets. On the basis, a simple and effective lifting algorithm based on Gaussian center perspective is provided, the completeness of three-dimensional segmentation is guaranteed, boundary distinguishing is enhanced, and accurate three-dimensional segmentation is achieved. Besides, a prompt propagation strategy is introduced, and the reliable prompt from the adjacent view is effectively utilized by restoring the 2D prompt into the scene, so that accurate segmentation is realized.

The invention relates to the technical field of information monitoring, in particular to a bidding informatization platform data security monitoring and analysis method, which comprises the following steps of: acquiring bidding platform data, dividing user behaviors into a plurality of monitoring points according to operation objects, operation quantity and operation types of the bidding platform data in a continuous time period, acquiring a user behavior sequence formed by the monitoring points; performing scene segmentation on the user behavior, and determining an abnormal behavior relative to a historical behavior baseline under each scene segmentation dimension; merging each abnormal behavior into a plurality of association sets, and determining a behavior association result of each association set and the operation object; determining association paths under different scene segmentation dimensions according to a behavior association result in a monitoring point distribution form; for the intersection of the association paths under all scene segmentation dimensions, updating a path list after combination of the association paths, and taking the updated path list as an output early warning target; the response speed and the processing efficiency of the platform to the abnormal behavior are realized.

The invention belongs to the technical field of artificial intelligence, and discloses a hidden space world model construction method for a robot grabbing operation scene and related equipment thereof. The method comprises the steps of performing semantic segmentation on a scene image through an image semantic segmentation network to generate a semantic segmentation mask of each object; processing the semantic segmentation mask through a multi-task scene understanding network to generate implicit representation; based on the implicit representation of each object at the current moment and the action information of the mechanical arm, predicting implicit representation at the next moment through a state transition network; and according to the predicted implicit representation at the next moment, generating a scene segmentation image reconstruction result, an object existence judgment result and a contact relation judgment result through a decoder in the multi-task scene understanding network architecture, and constructing an implicit space world model of the scene. Based on the method, efficient characterization and dynamic prediction of the robot grabbing operation scene are achieved, and the generalization ability and the multi-task cooperation efficiency of the model in a complex environment are improved.

The invention discloses a video generation model training data set processing method, and belongs to the field of video processing. The method comprises the following steps: S1, storing a video collected in a specific service direction in a single specified folder, wherein the video has no fixed limitation in the aspects of duration, resolution and frame rate; s2, extracting the file name, the frame rate, the duration, the resolution ratio and the aspect ratio of the video in the S1, and recording the data in a CSV file; s3, performing the following screening on each video according to the information in the S2: deleting the video when the video resolution is lower than 360P; deleting the video when the aspect ratio of the video is less than 0.7; when the video frame rate is lower than 20 frames per second, deleting; and S4, performing scene segmentation on the video screened in the step S3 by adopting a PySceneDetect algorithm. According to the method, representative and dynamically changing core frames can be uniformly extracted from a large number of original videos, so that the diversity and quality of a video data set are improved, and high-quality training data are provided for a video generation model.

The invention relates to a self-adaptive perception strategy switching method and system for long-distance automatic driving, and the method comprises the steps: carrying out the analysis of a public data set through employing a multi-target joint optimization clustering method, generating a standard scene prototype, and constructing an initial scene-optimal strategy mapping database; acquiring multi-dimensional scene features of a navigation route, performing global optimal scene division and recognition on a target road section, and generating a perception strategy map between continuous cells; querying a scene-optimal strategy mapping database continuously optimized by reinforcement learning, and selecting an optimal perception strategy for each cell based on a Q value maximization principle; when the vehicle enters the new cell, the self-adaptive switching control module automatically activates the preloaded optimal sensing strategy; and evaluating a strategy execution effect, generating an instant reward, transmitting an experience tuple containing a state, an action and the reward to the cloud, and continuously optimizing the Q-value database. The method and the system can improve the perception performance and the resource utilization efficiency of the automatic driving system.

The invention discloses an orthograph generation method and device, equipment, a storage medium and a product. The method comprises the following steps: determining a corresponding scene sparse point cloud and a camera external parameter according to an acquired scene aerial image sequence of a target scene and a camera internal parameter; performing region segmentation on the target scene according to the camera internal reference and the camera external reference to obtain scene segmentation sub-regions; performing Gaussian splash processing on the scene sparse point cloud in each scene segmentation sub-region based on a preset two-dimensional Gaussian splash algorithm to generate a local two-dimensional Gaussian scene corresponding to each scene segmentation sub-region; rendering each local two-dimensional Gaussian scene based on an orthographic camera model to generate a corresponding local scene Gaussian orthographic graph; and splicing the Gaussian orthographs of the local scenes according to the external parameters of the camera to generate an orthograph corresponding to the target scene. The problems that a traditional method is limited by high-precision topographic information requirements and high-precision DSM generation costs too many computing resources are solved, and the high-precision orthograph can be quickly generated.

A deep neural network(s) (DNN) may be used to perform panoptic segmentation by performing pixel-level class and instance segmentation of a scene using a single pass of the DNN. Generally, one or more images and / or other sensor data may be stitched together, stacked, and / or combined, and fed into a DNN that includes a common trunk and several heads that predict different outputs. The DNN may include a class confidence head that predicts a confidence map representing pixels that belong to particular classes, an instance regression head that predicts object instance data for detected objects, an instance clustering head that predicts a confidence map of pixels that belong to particular instances, and / or a depth head that predicts range values. These outputs may be decoded to identify bounding shapes, class labels, instance labels, and / or range values for detected objects, and used to enable safe path planning and control of an autonomous vehicle.

The invention provides a scene segmentation and editing method and system based on three-dimensional Gaussian sputtering, and the method comprises the steps: carrying out the two-dimensional instance segmentation of a three-dimensional scene multi-view two-dimensional image, obtaining a two-dimensional instance mask, initializing the three-dimensional reconstruction through three-dimensional Gaussian sputtering, and obtaining a three-dimensional Gaussian body set; associating cross-view consistent semantic features extracted by the pre-training visual language model for each Gaussian body, constructing and iterating a three-dimensional instance prototype library based on a two-dimensional instance mask, and performing conversion to obtain a cross-view consistent two-dimensional supervision signal; adding a learnable instance identity code for each Gaussian body, constructing a multi-dimensional loss function in combination with a two-dimensional supervision signal, a spatial neighborhood relationship and cross-view consistent semantic features, and optimizing learnable parameters containing the instance identity codes; and realizing Gaussian body instance grouping based on the optimized instance identity codes to complete instance-level scene segmentation and editing. According to the method, the technical problems of instance identity association and high-quality instance level generation under the conditions of sparse view angle and close adjacent objects are solved.

The invention belongs to the technical field of medical information processing, and relates to a laparoscopic surgery full-granularity recognition system and method based on feature extraction and task segmentation. Firstly, a video stream composed of multiple frames of images of an endoscope is input into the system, and then the spatial feature of each frame of image is obtained through a multi-frame spatial feature extraction module; the spatial features of all the frames are then input into a multi-frame time feature fusion module to obtain time features fused with different time lengths, and then the time features are used for outputting operation stages, operation steps and operation ternary body labels; meanwhile, only the spatial features of the current frame are extracted from the spatial features of all the frames, and an operation scene segmentation label is output through a single-frame feature restoration module. According to the laparoscopic surgery full-granularity recognition system and method based on feature extraction and task segmentation, computing resources can be greatly reduced, and compared with an existing single-granularity surgery scene technology, the laparoscopic surgery full-granularity recognition system and method based on feature extraction and task segmentation can output surgery scene information of multiple granularities at a time by means of the mutual coordination effect of multiple tasks.

A method and processing device for providing a scene segmentation map for a total field of view of an image capturing device comprises obtaining a scene segmentation map for the total field of view; receiving an indication that a new focus value has been set for the image capturing device for acquiring images of a current field of view; comparing the new focus value with a stored focus value associated with a focus region in the current field of view, wherein the stored focus value represents a previously used focus value; upon the new focus value deviating from the stored focus value more than a trigger threshold, triggering a scene segmentation map update process; and otherwise maintaining the scene segmentation map.

The invention discloses an ultrahigh-resolution immersive scene rendering method based on hexagonal picture segmentation. The method comprises the following steps: 1, determining a hexagonal segmentation style of an immersive display surface; 2, unfolding the immersive display surface into a plane; 3, acquiring 3D point information of each hexagonal surface, respectively calculating the width W and the height H of the enveloping quadrilateral surface, and calculating a local coordinate system V of the hexagonal surface S; 4, using W, H and V to construct a view cone and a viewpoint matrix of the three-dimensional scene; 5, setting a shade M of a plane expansion view for the hexagonal surface S; 6, calling a rendering scene of the 3D scene for the mask region by using information of the view cone and the viewpoint matrix, and rendering the mask region into a picture at the position S of the hexagonal surface; and 7, repeating the steps 5-6, and sequentially carrying out the same treatment on each hexagonal surface. According to the method and the device, the ultrahigh-resolution picture can be rendered in a scene segmentation form, and the method and the device are suitable for ultrahigh-resolution picture rendering under immersive display equipment.

The invention relates to an indoor point cloud scene open type semantic segmentation method based on text prompt. On the basis of large-scale indoor scene point cloud data, a 3D U-Net network structure based on Mama blocks is adopted to extract local features of a point cloud scene, global features of the scene point cloud data are extracted through multi-stage Mama blocks and down-sampling, and scene detail information is recovered through up-sampling and jump connection to fuse the local-global features of the point cloud data; generating a text title prompt of a scene multi-view view, associating a projection matrix between a 2D view and a 3D scene with a point cloud scene, enabling text prompt features to be aligned with corresponding point cloud data features, and loading a category text embedded weight into a scene segmentation head to perform a semantic segmentation task; in network training, binary classification loss is added on the basis of semantic segmentation loss so as to balance the recognition and understanding capability of the scene semantic segmentation network on a basic category and a new category; and finally, open semantic segmentation of the indoor point cloud scene is realized.

The invention relates to an automatic generation method and device of a plot abstract and electronic equipment. The method comprises the following steps: performing scene segmentation and key plot anchor point identification on each single-set original script of a target video, and generating a corresponding single-set draft abstract by using a large language model; performing multi-dimensional quality evaluation and screening processing on each single-set draft abstract, and converting the single-set draft abstract into a structured data object; sequentially arranging all the data objects to obtain a single-set abstract sequence of the target video; inputting the single-set abstract sequence into a large language model to enable the large language model to generate a whole draft abstract of the target video in combination with a preset narrative structure guide prompt; and performing multi-dimensional quality evaluation on the whole drama draft abstract, and optimizing the whole drama draft abstract based on an evaluation result to obtain a drama abstract of the target video. According to the method and the device, the technical problem that the effect and the reliability of an existing method in automatic abstracting of the long series are poor is solved.

The invention discloses a three-dimensional point cloud processing method based on skeleton articulation point attention convolution, belongs to the technical field of three-dimensional computer vision, and aims to solve the technical problems that traditional convolution operation is difficult to adapt to point cloud disorder and existing dynamic convolution is high in memory consumption. The method is realized through the following scheme: pre-defining a learnable skeleton joint point system in a three-dimensional space, designing a multi-layer perceptron comprising position codes to calculate attention weights of point cloud points and skeleton nodes, and fusing preset node weights to generate a spatial self-adaptive dynamic convolution kernel; introducing a weight consistency constraint to guarantee the continuity of a physical space convolution kernel; a feature dimension reduction strategy is adopted to reduce the memory consumption by 64%, the accuracy rate of a ModelNet40 classification task reaches 93.9%, the accuracy rate of an S3DIS scene segmentation task reaches 70.5% mIoU, and the method can be applied to scenes of automatic driving laser radar real-time processing, industrial part three-dimensional defect detection and the like.

The invention discloses an offshore scene segmentation method based on condition invariant semantic segmentation, and belongs to the field of computer vision and semantic segmentation. According to the method, style mutual conversion of a source domain image and a target domain image is realized through a data loading stage, and a multi-style view pair is generated; a CISS network model is constructed, the CISS network model is composed of a MiT-B5 encoder and a context sensing decoder, the encoder extracts high-level features insensitive to visual conditions by means of discrete wavelet transform, and the decoder achieves feature reconstruction without information loss through inverse discrete wavelet transform and generates a category probability graph; designing a joint loss function including pixel-level cross entropy loss and feature-level invariance loss, and balancing loss weight in combination with a dynamic weighting strategy; and training and optimizing the model by adopting an AdamW optimizer in combination with a linear learning rate warm-up strategy. Experiments show that the method can effectively cope with visual condition changes in a real marine environment, and the robustness of the semantic segmentation model in different marine environments is improved.

The invention discloses a process planning model pre-training method and system based on multi-processing scene segmented imitation learning, and relates to the field of computer-aided manufacturing, comprising the steps of: for various processing scenes comprising different geometric features, material characteristics and processing requirements, generating processing paths of various strategies by using a mature process database of CAM software; virtual trial cutting is carried out through a processing process world model to generate a large number of initial states, rich and diversified training samples are provided for reinforcement learning pre-training, and on the basis, large-scale pre-training is carried out by adopting a segmented behavior cloning method to obtain a process autonomous planning base agent which learns processing strategies in different processing scenes; in the face of a new processing task, a processing path can be quickly generated through transfer learning. According to the method, the generalization ability is remarkably improved, meanwhile, geometric-physical-control collaborative planning is achieved, and an intelligent solution is provided for efficient planning of the modern manufacturing industry technology.

The invention discloses a scene segmentation method and system based on multi-modal small sample learning, and relates to the technical field of computer vision. The method comprises the following steps: acquiring an automatic driving video stream, and extracting image frames in the video stream; training and testing a scene segmentation model by using the data set to obtain a trained scene segmentation model, performing semantic feature extraction on the data set by using a lightweight backbone network, and then performing double-axis self-attention feature extraction and information correction processing on the image frame after semantic feature extraction by using the scene segmentation model to obtain a trained scene segmentation model; and finally, fusing the corrected multi-source features to obtain a prediction output graph. And performing scene segmentation processing on a real-time video stream to be segmented by using the trained scene segmentation model. According to the method, the understanding capability of semantic information in a complex scene can be effectively enhanced while the lightweight of the model is guaranteed, and the segmentation precision and generalization capability under the small sample condition are remarkably improved.

The invention provides a multi-modal data fusion-based power transmission line detection method and related equipment. The method comprises the following steps: performing scene segmentation on a space region of a power transmission line based on geographic information and remote sensing image data to obtain a plurality of scene regions with space continuity; obtaining original multi-modal data of the power transmission line; screening from the original multi-modal data based on a preset distance condition to obtain target multi-modal data of the target area in the target time period; obtaining a graph structure representing a relationship between the target multi-modal data based on the target multi-modal data and the corresponding position information; performing feature extraction on the target multi-modal data to obtain target multi-modal features; performing feature fusion on the target multi-modal features based on the graph structure to obtain global fusion features; and determining the operation state of the target position of the power transmission line at the target moment based on the global fusion feature. The operation state of the power transmission line can be accurately and reliably determined.

The invention discloses a semi-supervised adaptive unstructured scene segmentation method, and relates to the technical field of computer vision. The method comprises the following steps: acquiring label-free image data of an unstructured scene; inputting the weakly enhanced image into a teacher model to generate a first prediction result; inputting the strong enhancement image into the student model to generate a second prediction result; calculating the JS divergence between the first prediction result and the second prediction result, and dynamically adjusting the attenuation coefficient of the index moving average according to the JS divergence; updating teacher model parameters according to the adjusted index moving average attenuation coefficient to obtain an unstructured scene segmentation model; and inputting a to-be-predicted unstructured scene image into the trained unstructured scene segmentation model, and outputting an unstructured scene segmentation result. According to the method, the teacher model weight updating rate is reduced in the model divergence significant stage such as the initial training stage or the difficult sample processing stage so as to suppress noise propagation, and the prediction precision of unstructured scene segmentation is effectively improved.

This invention discloses a 3D scene segmentation and interaction method based on 3D Gaussian, comprising: Step 1, instance discovery: inputting 3D Gaussian scene data, dividing the 3D Gaussian points into structurally coherent instance-level Gaussian groups to obtain refined instances; Step 2, instance and scene semantic assignment: selecting representative viewpoint images of refined instances and inputting them into a visual-language model to generate semantic description labels for the instances; filtering instance pairs in the scene through spatial geometric relationships, and clarifying the spatial relationship description of instance pairs through a large model, constructing a static scene graph that integrates geometric proximity relationships and instance semantic relationships, forming a structured description of all instances; Step 3, natural language-driven instance localization and interaction: receiving user input commands, combining the instance semantic description, the static scene graph, and the real-time viewpoint direction relationship during the query to perform multi-dimensional matching, determining the target instance ID, and executing interactive operations.