271 results about "Real time vision" patented technology

The invention discloses a multi-modal fusion real-time environment monitoring visual robot system, and relates to the technical field of real-time vision. The system comprises a multi-mode sensing module, and is equipped with various sensors such as a binocular stereo camera and a laser radar to collect environment data. The heterogeneous data preprocessing unit cleans and downsamples data such as images and point clouds; the space-time alignment fusion module realizes multi-source data space-time registration and synchronization; the environment semantic understanding engine constructs an environment semantic graph through a multi-branch network in combination with an attention mechanism; the abnormal event detection unit identifies abnormity based on a historical data model; the path planning and decision-making module integrates multiple targets to generate an optimal path; the autonomous movement execution module controls the robot to move and operate; and the cloud cooperative control center supports model updating and remote intervention. According to the invention, through cooperative work of all the modules, full-process intelligentization of environment monitoring data acquisition, processing, analysis and decision making is realized.

The invention provides a robot navigation method and system based on visual identification, and relates to the technical field of computer vision, firstly, a continuous visual information set of the surrounding environment of a robot is obtained, the continuous visual information set comprises environment visual images of different directions of the advancing direction of the robot, and the object appearance features and the spatial arrangement relation are recorded; then establishing association mapping between the continuous visual information set and a preset navigation area to obtain a double-association mapping result; generating a semantic anchor point navigation path based on a double-correlation mapping result, wherein the semantic anchor point navigation path comprises a semantic anchor point sequence from the current position to the target position; real-time visual semantic features in the navigation process are obtained through a real-time visual collection module, and a semantic anchor navigation path is dynamically updated; finally, a final navigation execution path is output according to the updated path, the robot is driven to complete navigation operation, and the navigation capability and the intelligent level of the robot in a complex dynamic environment are improved.

A system for real-time analysis of emotional feedback during motivational speeches, consisting of: a series of multimodal sensors, including at least one visual sensor configured to capture facial expressions of spectators, at least one directional microphone configured to capture the audio responses of the audience, and optionally one or more physiological sensors configured to capture biometric signals from spectators; an edge-based processing unit that is communicatively coupled to the arrangement of multimodal sensors, wherein the edge-based processing unit comprises the following: (a) a feature extraction module configured to extract visual features from captured facial images, acoustic features from voice responses, and physiological features from biometric signals; (b) an emotion inference machine configured to process the features using a deep learning-based emotion recognition model comprising a convolutional neural network (CNN) for classifying facial expressions, a recurrent neural network (RNN) for classifying voice emotions, and a multimodal late fusion layer configured to compute a composite emotion state vector representing the aggregated emotions of the audience; (c) a timestamp and speech alignment module configured to correlate the calculated composite emotion state vector with segmented portions of a live motivational speech based on real-time speech-to-text transcription and semantic analysis; and (d) a session-based storage unit configured to log time-indexed emotional state vectors and corresponding speech segments for post-event analysis; A speaker feedback interface comprising a portable display or a podium-mounted visualization panel, wherein the interface is configured to display visual indicators of emotional feedback in real time, the indicators being derived from the emotional state vector and including at least emotional trend graphs, threshold alerts, or engagement indices.

The invention discloses an automobile production line gluing system based on visual inspection, and relates to the technical field of industrial automation and intelligent control, and the automobile production line gluing system comprises a gluing data acquisition module, a gluing feature extraction module, a compensation correction module, a gluing decision module and a closed-loop management and control module; the gluing data acquisition module preprocesses assembly line gluing track images and glue gun nozzle three-dimensional coordinate data, the gluing feature extraction module constructs a gluing feature sequence table, the compensation correction module outputs glue outlet opening degree compensation amount and nozzle track correction amount, and the gluing decision-making module outputs glue gun pressure adjusting amount and nozzle movement speed. And the closed-loop management and control module detects pressure abnormity and speed stability in real time, optimizes control parameters and generates a process quality map. Through real-time visual detection and dynamic closed-loop control, the glue line width precision is improved, the track compensation response is shortened, the glue gun speed fluctuation coefficient and glue waste are reduced, the gluing qualification rate is improved, the production cost is reduced, and the automobile manufacturing and production requirements are met.

The invention discloses a distribution line unmanned aerial vehicle edge end real-time visual identification and target detection method and system based on lightweight AI. The method is based on a YOLOv8 architecture, and constructs a complete lightweight detection framework comprising a feature extractor, an enhancement module and a simplified detection head by introducing a space structure maintaining assembly, a separated large kernel convolution and a weighted reconstruction feature pyramid. A cross-dimension semantic relation model is constructed by utilizing a combined attention structure, multi-level feature fusion is realized by reconstructing a feature pyramid network, model training is performed by adopting a hybrid optimization function and a dynamic sample adjustment mechanism, and the model is deployed on a mobile computing platform after being optimized by a hierarchical knowledge transfer method. According to the method, the detection speed is remarkably increased while high precision is kept, and real-time identification and anomaly analysis of the power line element are effectively realized.

The invention relates to the technical field of compartment remnant real-time detection systems, and discloses a real-time visual identification and detection system for remnant in a compartment of a subway last station. In the system, an image acquisition module acquires a monitoring video stream and generates a carriage image sequence set; the feature extraction module processes the image to obtain a time sequence association partition labeling set; the time sequence association module analyzes the time sequence stable section image frame and generates a feature matching overlay analysis result; the abnormity judgment module identifies abnormal points to form a remnant abnormal point set; and the risk output module is used for marking the point locations with the left risks and generating a real-time detection and risk early warning result. The system can adapt to the dynamic environment of subway carriages, the efficiency and accuracy of remnant detection are improved, and real-time identification and risk early warning are achieved.

The invention discloses a power transmission channel forest fire risk early warning method and system based on spatial-temporal feature fusion, and relates to the technical field of risk early warning. The method comprises the following steps: constructing a power transmission channel risk grid and fusing historical multi-source data to generate a static flammability background map; the method comprises the following steps: collecting real-time visual and micrometeorological data, and after space-time alignment processing, respectively extracting visual risk features and environmental flammability trend features by using a double-branch space-time feature extraction network; inputting the static background and the dynamic characteristics into a space-time fusion module to obtain a comprehensive forest fire risk index; when the index exceeds a threshold, triggering a forest fire spreading deduction model corrected by combining a power transmission corridor effect, predicting fire spreading, evaluating a line tripping probability, and issuing a graded early warning and emergency strategy; according to the method, the defects of serious data islands and inaccurate early warning are overcome, the crossing from pure fire point monitoring to risk situation dynamic prediction is realized, and the accuracy of forest fire early warning and the intelligent level of power grid prevention and control are remarkably improved.

The invention discloses an edge spraying compensation control method based on intelligent visual feedback. The method comprises the following steps: performing space scanning on a to-be-sprayed workpiece to obtain spraying image data, and establishing a space corresponding relation between a spray gun motion coordinate and a workpiece surface coordinate; registering the real-time spraying image by using the corresponding relation, identifying a geometric boundary line, brightness gradient change and a coating adhesion area of the edge of the workpiece, and generating edge feature data; according to the difference between the edge feature data and the target spraying image, the spraying coverage deviation and the boundary overlapping error are calculated, the coating thickness change trend, the optical density change rate and the boundary direction offset are extracted, and the spraying gun posture adjustment amount, the spraying distance correction amount, the spraying pressure correction amount and the path speed correction amount are generated; the angle, the spraying distance, the pressure and the movement speed of the spray gun are dynamically adjusted according to the parameters; according to the method, real-time visual feedback and self-adaptive compensation control in the spraying process are achieved, and the uniformity and consistency of the coating on the edge of the workpiece are effectively improved.

The invention provides a dynamic data traceability method and system oriented to network security, and relates to the technical field of network security and data governance crossing, and the method comprises the steps: carrying out the real-time visual monitoring of a user interaction interface of a data transaction platform, and capturing the visual behavior information related to data calling; extracting feature parameters based on the visual behavior information to obtain a visual feature data set, and performing structured conversion on the visual feature data set to obtain a structured call log record containing visual behavior associated information; based on the structured call log record, extracting a time sequence, a frequency and a spatial distribution feature of a call behavior as basic indexes; a plurality of feature sampling states are dynamically selected in a monitoring time window, and a dynamic feature evaluation set is constructed based on the feature sampling states. According to the invention, the accuracy and controllability of network security protection in a data transaction scene can be improved.

The invention provides an intelligent driving control method and system for an unmanned ship with a body based on a world model, and relates to the technical field of artificial intelligence, and the method comprises the steps: collecting a real-time visual image, navigation state data and control motion data of the unmanned ship, carrying out the preprocessing and synchronous alignment of the data, and obtaining a real-time visual image; standardized vision, state and action input and a standardized data packet are obtained; the standardized vision, state and action input is mapped into latent representation, autoregression calculation is carried out by predicting a backbone network, and a multi-step latent state and latent action sequence is predicted; and finally obtaining a future state sequence and a future action sequence based on the predicted potential state and potential action sequence. According to the invention, interpretable autonomous intelligent driving is realized.

The embodiment of the invention provides a ceramic diaphragm automatic positioning system based on high-precision visual feedback and closed-loop control, and the system comprises a parameter reference setting module which is used for setting a theoretical reference point and a visual detection parameter; the real-time visual detection module is used for collecting an image of the to-be-stacked film and outputting an image coordinate of the mark point; the core operation module is used for converting the image coordinates of the mark points into actual coordinates based on the visual detection parameters, reading a theoretical reference point of the current to-be-stacked diaphragm and calculating a position error vector between the actual coordinates and the theoretical reference point, and the position error vector comprises a translation compensation amount and a rotation compensation amount; and the motion control and closed loop feedback module is used for receiving the position error vector and generating a control instruction to drive the workbench to move. According to the system, through parameterized reference setting, real-time visual detection, intelligent error calculation and real-time motion compensation, the technical limitation is effectively overcome, and the lamination alignment precision and efficiency are remarkably improved.

The invention provides a self-adaptive contour extraction and target recognition system and method for hand-eye calibration, and the system is characterized in that a hand-eye calibration system which integrates intelligent sampling planning, self-adaptive visual perception, online quality evaluation and closed-loop feedback control is constructed; the core defects of the prior art are overcome. According to the system, through deep fusion of real-time visual perception and motion control in a traditional calibration process, stable, accurate and adaptive extraction and processing of target contour features are realized. The method has the beneficial effects that the calibration robustness and precision in unstructured environments such as illumination variation, shielding and complex textures are remarkably improved, meanwhile, the safety and efficiency of the sampling process are ensured, and the method is suitable for high-dynamic and multi-interference industrial field application.

The invention discloses an unmanned aerial vehicle high-altitude cleaning path precise planning method and system based on visual navigation, and the method comprises the following steps: constructing an initial three-dimensional grid model of a to-be-cleaned building, dividing cleaning regions according to the initial three-dimensional grid model, and setting priorities; the unmanned aerial vehicle collects visual data of a cleaning area through a multi-mode visual sensor, wherein the visual data comprise surface texture, depth information and a stain distribution thermodynamic diagram; performing local correction on the initial three-dimensional grid model based on the real-time visual data to generate a high-precision dynamic three-dimensional grid model; generating a cleaning path by adopting an adaptive grid traversal algorithm according to the priority of the cleaning area and the coverage range of the unmanned aerial vehicle cleaning head; obstacles in the path are detected in real time through a visual feature matching algorithm, and when the obstacles are detected, path re-planning is carried out to avoid the obstacles until all the areas are cleaned. According to the method, the modeling precision and the dynamic adaptability are improved, and the intelligence and the pertinence of path planning are realized.

The invention relates to the technical field of perovskite thin film detection, in particular to a perovskite blade coating thin film defect detection device and method based on real-time visual monitoring, and the device comprises a support, an image acquisition unit, an illumination unit and a self-adaptive adjustment system which are arranged in a glove box and located at a coating machine, a flash evaporation box and a heating stage respectively; the camera and the lens are installed on the adjusting frame, and the first driving piece achieves height adjustment. The light source is installed on the supporting frame and controlled by the second driving piece to ascend and descend. The image analysis unit evaluates the definition, contrast and illumination uniformity of the acquired image, and the control unit automatically adjusts the height and angle of the light source and the position of the camera according to the result so as to keep the image quality stable; according to the scheme, real-time monitoring is carried out in the working procedures of coating, flash evaporation, heat treatment and the like, the comprehensiveness, real-time performance and accuracy of thin film detection are effectively improved, influences caused by environmental fluctuation are avoided, and the accuracy and consistency of defect recognition are guaranteed.

The disclosed invention introduces an advanced Visual Alarm and Guidance System designed to significantly enhance safety in industrial environments. This innovative system integrates state-of-the-art computer vision and light projection technologies, along with trajectory prediction algorithms, to dynamically identify potential hazards and guide workers in real-time out of potential collision pathways. It marks a considerable advancement over traditional safety methods by not only detecting imminent dangers but also providing clear, visual navigation aids. This system is adaptable to various high-risk settings, seamlessly integrates with existing infrastructures, and offers a novel approach to real-time, visual safety management in industrial settings.

The invention provides a defect detection system and method for bracket processing, and relates to the technical field of defect visual inspection. A visual inspection device is used for collecting a to-be-detected image in real time; performing pixel registration on the to-be-detected image to obtain defect edge uniformity in a defect area on the surface of the bracket, and determining a line form boundary on a defect position on the surface of the bracket according to the defect edge uniformity and an adjacent identification value between adjacent pixel points in defect pixel points; performing texture screening on the defect extension information to obtain gray scale transaction deviations on defect extension abrupt change points in a defect area on the surface of the support, and determining a neighborhood span index in an area to be detected on the surface of the support according to all the gray scale transaction deviations; and determining a pixel incongruous trend according to the grain form boundary and the neighborhood span index, and carrying out visual defect detection on the bracket processing surface. According to the method, accurate real-time visual detection can be carried out on the bracket processing surface defects in a bracket processing dynamic detection scene, so that the accuracy of bracket defect detection is improved.

The invention provides an AI-powered speech, language therapy, and language learning system that utilizes Natural Language Processing (NLP), Convolutional Neural Networks (CNNs), and Generative Adversarial Networks (GANs) to deliver personalized, real-time therapy and learning for individuals with speech disorders or those seeking to improve language proficiency. The system analyzes user speech, language comprehension, and facial expressions, providing immediate feedback on pronunciation, fluency, articulation, and sentence structure. A GAN-generated avatar interacts with the user, mimicking human expressions and offering dynamic, engaging sessions. The platform adapts exercises based on user performance using personalized algorithms to ensure continuous progress. Additionally, it securely stores user data in compliance with privacy regulations, making it accessible through web and mobile platforms. This invention improves upon existing speech therapy and language learning solutions by integrating real-time visual and auditory feedback with AI-driven personalization, offering a more immersive and effective experience.

The invention relates to the technical field of medical instruments, and discloses a tracheotomy anatomical structure image recognition method and system, and the method comprises the steps: obtaining real-time visual image data and spatial positioning data of an endoscope, carrying out the time synchronization, carrying out the mode recognition of the spatial positioning data, and locking the remarkable anatomical features in the real-time visual image data, and according to the characteristics and the spatial positioning data, calculating a geometric transformation relationship between an endoscope visual coordinate system and a spatial positioning coordinate system, and finally fusing the data and displaying anatomical structure information and a surgical tool position in real time. The method can effectively solve the problem that in the prior art, an endoscope is irregularly deformed in the narrow trachea with physiological bending of a patient.

The invention provides an unmanned aerial vehicle biological control method and system based on real-time visual perception and accurate delivery decision. The method comprises the following steps: collecting farmland environment data through multiple sensors, and performing low-illumination enhancement on a visible light image; utilizing an improved target detection network to synchronously identify diseases and insect pests and position a putting point; the two-dimensional image, thermal infrared and three-dimensional point cloud features are fused to realize accurate positioning of the target; a visual language model is introduced for cross-modal reasoning, target disambiguation and priority analysis are completed in combination with a task instruction, and a collision-free delivery sequence is generated; and finally, planning an optimal flight path, and realizing precise drug delivery through trajectory optimization and closed-loop control. The intelligent level and the operation precision of pest control in a complex farmland environment are improved.

The invention discloses a real-time visual processing method and system based on ESN-CV coprocessing, and relates to the technical field of visual processing, and the method comprises the steps: firstly, synchronously collecting a camera image and road surface humidity data of a physical sensor, and extracting an image reflection intensity distribution matrix as a visual feature; then humidity time sequence data and visual features are input into an echo state network, a dynamic weight coefficient matrix is generated through spatio-temporal feature fusion, the dynamic weight coefficient matrix is injected into a predefined convolutional layer of a target detection network, kernel weight parameters are adjusted in an element-by-element superposition mode, and the feature extraction capacity of a high-sensitivity area is enhanced. And after a detection result is output, the system triggers closed-loop feedback through a confidence coefficient deviation value: when the deviation exceeds a limit, the actual offset is calculated by combining a high-precision map, an error correction vector is generated, the state of the ESN reserve pool is updated by utilizing a Hadamard product, and the weight generation logic of the next frame is optimized in real time.

The invention relates to the technical field of intelligent wearing, and discloses an intelligent interaction system and method based on AR glasses. According to the method, an environment scene is intelligently divided, a plurality of interaction areas are generated, then an initial interaction task instruction is received through a user interface, AR glasses are started to execute a task, initial environment data are collected in the execution process, and historical interaction records are obtained. Then, the interaction priority and the attention level of each interaction area are calculated by using the data, so that a customized navigation path is generated. When the AR glasses interact according to the path, position information of the AR glasses is collected in real time, interference source characteristics are obtained, whether the AR glasses enter an interference influence range is detected, if yes, an offset alarm is triggered, whether the AR glasses deviate from a preset path is judged based on the position information, and if not, a path correction command is generated. In addition, real-time visual data can be captured, an abnormal mode or a specific target is analyzed and recognized, and if the abnormal mode or the specific target is recognized, the position information and the corresponding visual data are stored.

The invention relates to the technical field of image updating, in particular to a multi-modal image real-time updating method and system for temporal bone surgery. The method comprises the following steps: acquiring a real-time multi-modal image, performing pixel-by-pixel frequency domain reconstruction optimization, and constructing a frequency spectrum enhanced fusion image; performing reverse geometric transformation compensation on the spectrum enhancement fusion image to obtain a space alignment optimization image; performing real-time visual contrast enhancement on the space alignment optimization image, and constructing a visual enhancement image; carrying out inter-frame difference calculation on the vision enhanced image, carrying out real-time increment updating optimization, and constructing an increment updating image sequence; and performing multi-level cache rendering management and parallel execution based on the incremental updating image sequence. The temporal bone surgery safety and efficiency are improved through real-time and efficient image updating.

The invention belongs to the technical field of medical robots and minimally invasive surgery, and provides a robot control system and method for blue laser vaporization surgery of prostatic hyperplasia. Mapping the nuclear magnetic image volume data to a deformation field under an ultrasonic acquisition coordinate system, and deforming the preoperative nuclear magnetic image to an intra-operative ultrasonic space by using the deformation field to complete image registration; fusing the registered image with a stereoscopic vision system; the spatial depth of the surface of the target tissue is obtained from the endoscopic image so as to supplement navigation information; according to the utility model, the prostate deformation and probe posture change adaptive capacity in an operation is improved, the real-time visual closed-loop regulation and control capacity is improved, and the characteristics of small light spots, shallow heat diffusion, excellent hemostasis and the like of blue laser are combined, so that the vaporization and hemostasis precision in a tiny blood vessel dense area is ensured, and the problems of large tissue trauma and the like are avoided.

The invention discloses a real-time visual language navigation method based on frontier exploration and neural relationship inference. The method comprises the following steps: constructing an initial occupation grid map and a confidence value map; based on the occupied grid map, identifying a boundary between the explored area and the unexplored area, and generating a candidate leading edge waypoint set; a vision-language model is used for detecting instance objects existing in the current visual field, and points closest to the robot are extracted to serve as a candidate instance route point set; calculating visual language comprehensive scores of all waypoints in the candidate frontier waypoint set and the candidate instance waypoint set, and selecting the waypoint with the highest score as a next navigation target; and based on the selected target waypoint, calling a local path planner to generate a collision-free path, driving the robot to move, and updating the occupied grid map and the confidence value map in a rolling manner. Task specific training is not needed, vision and language semantic information can be efficiently fused, and intelligent exploration and target navigation in an unknown environment are achieved.

The invention discloses an automatic rectification method and system for an intelligent patrol point of power transformation equipment, and relates to the technical field of intelligent patrol of the power transformation equipment, and the method comprises the steps: extracting the structural features of a preset patrol point reference map of the power transformation equipment based on a power transformation equipment feature enhancement detection segmentation algorithm, and building a feature library of the patrol point reference map; the method comprises the following steps: extracting real-time visual features of a current frame image in an intelligent patrol process of power transformation equipment; matching the real-time visual features with a feature library of the inspection point location reference map by using a power transformation equipment multi-dimensional feature matching technology, and judging whether the current inspection point location deviates or not according to a matching result; if it is judged that the current patrol point position deviates, based on the visual feature corresponding relation between the reference image and the current frame image, deviation parameters for driving a camera to move are calculated; and generating a camera adjustment instruction according to the obtained offset parameter. The method avoids the hidden trouble of missing faults due to point location offset, and is suitable for large-scale inspection requirements in a complex transformer substation environment.

The invention discloses an intelligent door lock real-time visual analysis method and system based on edge calculation, and relates to the technical field of door locks, and the method comprises the steps: collecting a user face image in real time through a multi-mode sensor integrated with an intelligent door lock, and synchronously obtaining a distance value and dynamic motion information between a face and a camera; judging whether a living body detection and recognition process is triggered or not; when a living body detection and recognition process is triggered, extracting image features, depth information and Doppler frequency shift dynamic parameters of a face region; generating a composite living body feature vector containing a three-dimensional structure, texture details and motion characteristics; and performing dual verification on the generated composite living body feature vector. The method has the advantages that by introducing the Doppler frequency shift detection technology of the millimeter wave radar, the specific dynamic micro-motion characteristics of the living body are creatively fused into the living body authentication system, the defect that traditional living body detection excessively depends on static textures is effectively overcome, and the biological motion fingerprint authentication capacity of the intelligent door lock is improved.

The invention discloses a feature matching method based on double-branch feature extraction and channel feature enhancement, which is used for improving the feature expression capability and global perception capability of a lightweight feature matching network. The method comprises the following steps: after preprocessing an input image, respectively extracting local details and global structure features through a parallel backbone network and a dynamic channel kernel feature extraction branch; a channel feature enhancement module is used for carrying out progressive strength enhancement on multiple levels; a self-adaptive convolution kernel is generated for each channel through a dynamic channel kernel attention module, and spatial attention modeling of the key area is achieved; a multi-scale feature fusion module is adopted to fuse local and global features, and finally a dense feature map, a key point heat map and a reliability heat map are output; according to the method, the calculation efficiency is ensured, the feature matching precision and robustness in a complex scene are remarkably improved, and the method is suitable for lightweight application scenes of real-time vision SLAM, augmented reality and unmanned aerial vehicle navigation.

According to the photovoltaic power station intelligent cleaning system with cooperation of the unmanned vehicle and the unmanned aerial vehicle, through systematic cooperation of the intelligent control center, the unmanned vehicle and the unmanned aerial vehicle, whole-process intelligence and high efficiency of photovoltaic power station cleaning operation are achieved. A digital twinborn model is constructed through unmanned vehicle environment scanning, a global optimal operation scheme is generated, and a decision foundation of accurate operation is laid; the unmanned aerial vehicle depends on the real-time visual perception and self-adaptive cleaning technology, the crossing from extensive cleaning to precise on-demand operation is achieved, and the cleaning effect and the resource utilization efficiency are remarkably improved; a dynamic docking supply mechanism established between the unmanned vehicle and the unmanned aerial vehicle thoroughly breaks through the cruising bottleneck of the unmanned aerial vehicle, an uninterrupted operation cycle is formed, and continuous self-optimization of an algorithm is realized through a data feedback closed loop. The automation degree, the cleaning quality, the operation efficiency and the economical efficiency of photovoltaic power station cleaning operation are comprehensively improved, and the operation and maintenance problem faced by a large-scale photovoltaic power station is effectively solved.

The embodiment of the invention provides a humanoid robot control method and system and related equipment, and belongs to the technical field of robots. The method comprises the following steps: in response to a first electroencephalogram signal of a target user, collecting real-time visual data, real-time tactile data and real-time action data of the humanoid robot; performing data fusion on the first electroencephalogram signal, the real-time visual data, the real-time tactile data and the real-time action data to obtain real-time fusion data; and inputting the real-time fusion data into the trained end-to-end model, and outputting an action instruction to control the humanoid robot to execute actions according to the action instruction. According to the embodiment of the invention, the precision and real-time performance of humanoid robot control can be improved.

This invention discloses a deep learning-based robot visual recognition and decision-making control method, belonging to the field of control system technology. It includes constructing a thought decision tree based on input commands and real-time visual images, obtaining the linear relationship between the middle-level leaf nodes and the top leaf nodes, and using this relationship to associate current visual features with task intent. By constructing a multi-level thought decision tree structure, combined with multi-modal feature fusion and a deep learning engine, this invention achieves deep fusion and unified representation of multi-source heterogeneous data. Furthermore, by constructing a fusion search engine and semantic alignment mechanism, it effectively integrates text commands with visual images and other modal information, enabling accurate parsing of task intent and collaborative processing of environmental perception, and improving the real-time response and decision-making accuracy of service robots for complex tasks.