668 results about "Visual feedback" patented technology

A robotic surgical system includes a robotic manipulator configured to perform surgical procedures under direct surgeon control. A surgical camera system captures real-time intraoperative video. An external imaging interface receives multimodal imaging data, including preoperative and intraoperative data from at least one of magnetic resonance imaging (MRI), computed tomography (CT), ultrasound, and fluoroscopy. An artificial intelligence (AI module has a trained neural network and a deep learning model trained on multi-institutional annotated surgical datasets, The AI module is configured to execute one or more of: fuse acquired video and imaging data into temporally and spatially coherent anatomical visualizations; generate continuously updating overlays aligned with the surgical field, with segmented anatomical features; projected tissue boundaries, proximity indicators for instruments, and predictive deformation trends; provide dynamic predictive trend visualization indicating zones of future anatomical complexity or risk; register and align preoperative imaging data with intraoperative imaging data in real time; adapt overlay presentation in response to tissue deformation without actuating the robotic manipulate or; and passively augment visual feedback without initiating any autonomous actuation of surgical instruments.

The invention provides a picking mechanical arm operation method and system based on visual feedback. The picking mechanical arm operation method comprises the steps that firstly, a multi-dimensional visual perception set including the shape contour, surface physical characteristic visual representation, connection part space distribution, growth environment dynamic interference and the like of a picked object is obtained; a mechanical arm collaborative decision-making model is constructed based on the multi-dimensional visual perception set, stress distribution characteristics of connection parts are analyzed, and optimal contact area parameters and mechanical arm posture collaborative parameters are determined; according to the parameters, generating a sub-time sequence action instruction set containing action information in the contact stage, the clamping stage and the separation stage; when the mechanical arm is controlled to execute the instruction set, a dynamic visual feedback set of the picking scene is obtained in real time; and finally, input parameters of the collaborative decision model are updated based on feedback, and instruction set parameters are adjusted, so that the mechanical arm completes picking operation on the premise of keeping the integrity of the picked object, and the intelligent level and the picking operation quality of the picking mechanical arm are improved.

The invention discloses an edge spraying compensation control method based on intelligent visual feedback. The method comprises the following steps: performing space scanning on a to-be-sprayed workpiece to obtain spraying image data, and establishing a space corresponding relation between a spray gun motion coordinate and a workpiece surface coordinate; registering the real-time spraying image by using the corresponding relation, identifying a geometric boundary line, brightness gradient change and a coating adhesion area of the edge of the workpiece, and generating edge feature data; according to the difference between the edge feature data and the target spraying image, the spraying coverage deviation and the boundary overlapping error are calculated, the coating thickness change trend, the optical density change rate and the boundary direction offset are extracted, and the spraying gun posture adjustment amount, the spraying distance correction amount, the spraying pressure correction amount and the path speed correction amount are generated; the angle, the spraying distance, the pressure and the movement speed of the spray gun are dynamically adjusted according to the parameters; according to the method, real-time visual feedback and self-adaptive compensation control in the spraying process are achieved, and the uniformity and consistency of the coating on the edge of the workpiece are effectively improved.

The invention provides a humanoid robot hand-eye coordination intelligent calibration control system and a humanoid robot hand-eye coordination intelligent calibration control method, and the humanoid robot hand-eye coordination intelligent calibration control system integrates high-precision contour extraction, visual feedback closed-loop control, multi-sensor fusion and dynamic path planning. The robot hand-eye calibration method has the beneficial effects that the calibration efficiency and precision can be remarkably improved, the problem of calibration parameter drift caused by system time-varying characteristics and environmental interference is effectively solved through continuous tracking of a target contour and real-time online compensation of pose deviation, and the calibration accuracy is improved. Therefore, the reliability and the absolute positioning precision of the robot in long-term complex operation tasks are guaranteed, and the self-adaptive capability of a robot system is enhanced.

The embodiment of the invention provides a ceramic diaphragm automatic positioning system based on high-precision visual feedback and closed-loop control, and the system comprises a parameter reference setting module which is used for setting a theoretical reference point and a visual detection parameter; the real-time visual detection module is used for collecting an image of the to-be-stacked film and outputting an image coordinate of the mark point; the core operation module is used for converting the image coordinates of the mark points into actual coordinates based on the visual detection parameters, reading a theoretical reference point of the current to-be-stacked diaphragm and calculating a position error vector between the actual coordinates and the theoretical reference point, and the position error vector comprises a translation compensation amount and a rotation compensation amount; and the motion control and closed loop feedback module is used for receiving the position error vector and generating a control instruction to drive the workbench to move. According to the system, through parameterized reference setting, real-time visual detection, intelligent error calculation and real-time motion compensation, the technical limitation is effectively overcome, and the lamination alignment precision and efficiency are remarkably improved.

The invention relates to the technical field of digital twinning, in particular to a real-time interactive visualization system and method based on digital twinning, and the system comprises a data collection module which is used for configuring a multi-protocol adapter deployed on a physical equipment layer to collect the operation data of physical equipment, and transmitting the operation data of the physical equipment to an edge computing node. According to the method, the general data adapter is used for driving the three-dimensional rendering engine to execute real-time rendering based on physics, so that the virtual scene can dynamically reflect details such as illumination change and material aging of a physical entity, and high-fidelity visual feedback is provided. According to the method, a closed-loop control mechanism is established in combination with a multi-modal interaction framework, pure data monitoring is converted into operable bidirectional interaction, operation of a user on a visual interface can be converted into feedback instructions in real time to reversely adjust physical equipment, and the response efficiency of remote operation and maintenance is improved.

The invention provides a self-adaptive accurate rust removal method and system based on unmanned aerial vehicle visual feedback, and relates to the technical field of large metal structure surface maintenance and anticorrosion treatment.The rust removal method comprises the steps that a global path is planned, and a global flight waypoint sequence is generated; a local image of the current to-be-derusted area is collected; the corrosion characteristics of the current to-be-derusted area are analyzed, and initial derusting parameters and initial derusting time are set; after each derusting operation is finished, a frame of intermediate result image is collected; based on the difference between the intermediate result image and the historical image, performing quantitative evaluation on the effect of the previous rust removal operation, and performing feature analysis on the residual rust region; self-adaptively adjusting and generating rust removal parameters of the next stage; and the steps of derusting operation, image collection, evaluation and parameter adjustment are repeatedly executed, and after the termination condition is met, the unmanned aerial vehicle is switched to the next to-be-derusted area.

The invention relates to a personalized exercise rehabilitation risk early warning method and device based on IMU dynamic instability assessment, and the method comprises the steps: collecting multi-dimensional exercise data of a subject, carrying out the posture fusion, and obtaining the limb posture, joint angle change rate and angular momentum of the subject; performing time sequence mode decomposition to extract a single gait cycle and a specific jump landing attitude; calculating a biomechanical feature vector of each motion primitive; inputting the single gait cycle, the specific jump landing attitude and the biomechanical feature vector into a hybrid injury induced motion model, and outputting a multi-dimensional injury risk index vector; and comparing the multi-dimensional injury risk index vector with historical health data and a moving target dynamic adjustment threshold value of the subject, and if the threshold value is exceeded, providing real-time auditory, tactile or visual feedback to the subject. According to the method, the influence of movement speed and rhythm change is overcome, dynamic instability in movement can be more sensitively reflected, and personalized risk assessment in a real sense is realized.

The invention discloses a multi-size wafer pre-positioning method and system based on visual identification, and belongs to the technical field of wafer pre-positioning, and the method specifically comprises the steps: collecting a to-be-positioned wafer image of a stage region, employing a visual feature extraction model to recognize the outer edge features of a to-be-positioned wafer, constructing an outer edge candidate curve, and carrying out the recognition of the outer edge candidate curve; performing multi-circle fitting on the outer edge candidate curve, determining a size parameter of a to-be-positioned wafer by adopting a hierarchical confidence rule, generating an eccentric correction path of the minimum rotation amount based on the size parameter of the to-be-positioned wafer, predicting the offset of the wafer relative to the center of the carrying table, and establishing a rotation compensation sequence; performing segmented rotation and micro-displacement adjustment on the carrying table according to the eccentric correction path, and performing pre-positioning on the wafer to be positioned under the condition of multiple sizes; the method does not need to depend on a template with a fixed size or a mechanical positioning structure, can automatically analyze the outer edge features of wafers with different sizes, and completes eccentric prediction and rotation compensation based on visual feedback.

The invention relates to the technical field of building automation and robots, and discloses a floor robot cutting system based on self-adaptive visual guidance. The technical problems that an existing floor cutting robot excessively depends on a preset path, and the adaptability is poor and the cutting precision is low when the actual working conditions that errors exist between on-site marking lines and design drawings and the ground is uneven exist are solved. An actual cutting datum line is captured and recognized in real time through a visual perception module carried on a robot body, and structured path information is generated; and then the self-adaptive control module dynamically plans and corrects the motion trail of the robot according to the deviation between the visual feedback information and the current pose of the robot so as to guide the cutting execution mechanism to accurately work, so that the accuracy loss caused by environment change and construction errors in a traditional scheme is effectively overcome, and the cutting efficiency is improved. And the operation precision, the field environment adaptability and the automation level of automatic floor cutting are remarkably improved.

The invention relates to the technical field of computer aided design and BIM (Building Information Model), discloses a BIM ancient building component parameterization design method and system fused with a historical form and system, and aims to solve the problems of parameter distortion, low modeling efficiency, high rework rate and the like caused by disjunction of a historical building rule and a BIM in the prior art. The method comprises the following steps: acquiring a historical shape and system standard text data set; a component-region-geometry triple knowledge graph is constructed; outputting Bezier curve control points conforming to regional characteristics through a lightweight curvature parameter generation network; generating an initial model in the BIM; a dynamic geometric constraint engine is used for verifying whether parameters such as the roll-killing radian and curvature continuity are out of tolerance or not in real time, and visual feedback and correction suggestions are triggered; and the model is automatically updated after user confirmation. The modeling precision and efficiency are remarkably improved, the rework rate is reduced, and the requirements of ancient building protection engineering specifications are met.

The invention discloses a vision-language-action robot control method and system based on task stage semantic enhancement, and the method comprises the steps: firstly constructing a task stage semantic system, and disassembling a task into stages with definite semantics; constructing a task stage observation module based on a pre-trained vision-language model, deducing a task state and abnormality in real time, and generating stage semantic features; modulation parameters are generated through a lightweight network, and dynamic modulation is carried out on the vision-language features to strengthen the vision feedback weight; and finally, fusing the modulated characteristics and the body state characteristics, and outputting a control instruction through an action generation network to form closed-loop control. According to the method, the multi-modal information weight can be dynamically balanced, task abnormity can be identified in real time, error correction is triggered, the false completion phenomenon is remarkably reduced, the task execution reliability, robustness and generalization ability of the robot in a complex dynamic environment are improved, and the method is suitable for various robot systems needing high-credibility operation.

The invention discloses an engineering measurement data intelligent analysis method and system, and relates to the technical field of engineering data analysis, and the method comprises the steps of data access and fingerprint identification, space-time alignment and dynamic reference construction, semantic feature extraction and causal atlas construction, multi-modal data fusion and intelligent diagnosis, abnormity traceability and causal chain generation, and visual feedback and early warning output. According to the method, the total station, the unmanned aerial vehicle, the GNSS and the optical fiber sensor are used for collecting multi-source data and generating fingerprints, constructing a dynamic reference profile, extracting semantic features and combining construction logs to form a causal atlas, multi-modal fusion and anomaly diagnosis can be achieved, a trigger source can be traced, a causal chain is output, finally, risks are visually marked in a three-dimensional BIM model, and early warning is pushed. The integrity, real-time performance and interpretability of monitoring data are effectively improved, and construction safety and management decision support are enhanced.

The invention discloses a welding wire production control system and method based on visual feedback, and belongs to the technical field of precision machining. The system comprises a multi-mode intelligent sensing subsystem, a self-adaptive model prediction controller and a safety and exception handling subsystem, and a two-stage control architecture of a quality closed loop and an execution closed loop is constructed. According to the method, the microcosmic roughness, macroscopic geometry, defects and other multi-dimensional quality characteristics of the surface of the welding wire are obtained in real time through multi-modal sensing and fused into a comprehensive quality index; carrying out rolling optimization by adopting a model prediction control algorithm for updating parameters on line, and generating a control instruction; meanwhile, trend early warning based on statistical process control and sudden fault diagnosis based on model residual analysis are operated in parallel, and active safety protection is achieved. According to the method, the problems of unstable quality, control lag and passive safety caused by supplied material fluctuation and tool time variation in the welding wire flaking machining process are solved, and self-adaptive accurate control and intelligent safety protection in the machining process are achieved.

The invention discloses a large-ink-load ceramic anilox roller laser engraving system and an engraving method thereof, relates to the field of ceramic roller laser engraving, and aims to solve the problems of limited ink load increase, low production efficiency and the like in the prior art. The system comprises a laser head, a ceramic roller and an ink cell observer, and further comprises an ink cell image acquisition module, a test point positioning module, an engraving control module, an analysis module and an auxiliary scanning planning module. The laser head comprises a laser, a high-frequency torque motor and the like, the high-frequency torque motor can drive a reflecting mirror to move so as to adjust a laser beam track, and the method comprises the steps of generating net-shaped data, positioning an ink cell and a test point, matching laser power to engrave the test point, comparing a data output strategy, dynamically adjusting an auxiliary scanning track and then engraving; double scanning tracks are adopted, the laser power and tracks are adaptively controlled in combination with real-time visual feedback, fine engraving of ink cells is achieved, the ink loading amount, the ink transfer uniformity and the production efficiency are improved, and ceramic damage is avoided.

The invention discloses an automatic high-precision assembly system and method based on visual feedback, belongs to the technical field of automatic assembly, and solves the problem that an existing scheme is insufficient in the aspects of automatic integration and actual usability. Comprising an embedded control unit, an XYZ Cartesian sliding table, a limiting sensor, a visual sensor, a visual mark and an upper computer processing unit, the embedded control unit is connected with the upper computer processing unit, and the embedded control unit is used for receiving a control instruction and feeding back state information. The embedded control unit is connected with a motor driver of the XYZ Cartesian sliding table and used for sending a motion control signal, and the embedded control unit is connected with the limiting sensor and used for receiving a switch signal. Working space boundary recognition and automatic calibration of the camera-robot hand-eye relation are achieved, the configuration and deployment process of a visual guidance system on the XYZ sliding table is greatly simplified, the requirement for professional skills of operators is lowered, and deployment efficiency is improved.

The embodiment of the invention provides a three-dimensional scanning detection method, which is applied to the technical field of three-dimensional scanning, and comprises the following steps: setting at least one reference area on a preset model, and definitely defining a key position which needs to be preferentially concerned by a user and ensures the integrity of scanning data; then, a user is guided to scan based on the set reference area, verification information is generated in real time in the scanning process, and the verification information displays the achievement condition of the accumulation degree of the scanning data in the reference area relative to the saturation threshold value of the scanning data in a visual mode; therefore, a user is directly notified whether the scanning data of each reference area is complete or not, and the user is guided to carry out supplementary scanning on the incomplete area in time. According to the scheme, the verification of the scanning data integrity is advanced from a traditional post detection link to a real-time scanning process, so that a user can actively ensure the scanning integrity of a key area based on visual feedback, and the detection misjudgment and reworking risks caused by missing scanning or incomplete data are fundamentally avoided.

The invention relates to the technical field of education and teaching. The invention provides a classroom teaching dynamic feedback and evaluation system integrated with multi-modal emotion calculation. The system comprises a multi-modal data acquisition module used for acquiring multi-source data in a classroom teaching environment and sending the multi-source data to an emotion-cognitive calculation center; the emotion-cognition calculation center is used for carrying out fusion analysis on the multi-source data, identifying the emotion state and the cognition state of the student, carrying out attribution analysis through an emotion-cognition coupling model, generating a coupling state label for describing the specific reason of the learning dilemma, and sending the coupling state label to the dynamic intervention engine; the dynamic intervention engine is used for matching and generating individual-level, group-level and system-level teaching intervention instructions from the teaching strategy knowledge base according to the received coupling state labels; and the visual feedback and evaluation module is connected with the emotion-cognition calculation center and the dynamic intervention engine, and is used for providing real-time classroom state visual display for the teacher and pushing personalized learning suggestions to the student terminal.

The invention relates to the technical field of visual inspection, in particular to a sneaker appearance flaw intelligent detection system and method based on small sample learning, and the system comprises a flaw intelligent detection center, a flaw analysis unit, a check acquisition unit, a difficulty evaluation unit and a visual feedback unit. According to the method, defect type identification and bounding box coordinate output of the sports shoes are realized in a mode of combining the model and the 2D image, meanwhile, along with precision analysis of the bounding box coordinates, data support is provided for follow-up, a defect area is obtained based on analysis of the bounding box coordinates, and meanwhile, 3D and spectral image analysis is performed on the defect area, so that the detection accuracy of the sports shoes is improved. The obtained flaw type is checked with the flaw type analyzed by the 2D image, so that the consistency of flaw results is ensured, the error rate of flaw detection results is reduced, and repair feasibility judgment processing is carried out, so that repair difficulty classification of the sneakers with flaws is facilitated, namely, objective classification is realized in combination with multi-index weighted fusion.

The invention relates to the technical field of circular screen equipment interaction, and discloses a circular screen audio playing interface optimization method and system, a storage medium and equipment, and the method comprises the steps: obtaining physical parameters of a circular screen, and building a polar coordinate system with the center of the screen as an original point; dividing a screen display area into a plurality of functional areas according to radial distances; based on the polar coordinate system and the partitioning result, carrying out adaptive layout on a plurality of elements of an audio playing interface; a touch response area of the interaction element is expanded, and multiple interaction gestures including a rotation gesture are recognized and responded; dynamic visual feedback is triggered in the user interaction process, and according to the method, the adaptation problem of a traditional rectangular UI on the circular screen device is comprehensively solved through the technical means of establishing a special coordinate system for the circular screen, optimizing the interface element layout, fusing multi-modal interaction gestures, enriching the visual animation effect, achieving dynamic self-adaptive optimization of the interface and the like.

Systems and methods are provided for performing EMG signal operations. The system accesses one or more blocks of EMG data that were generated based on a plurality of EMG channels of an EMG communication device based on one or more subthreshold activity (STA) of one or more muscles associated with speech production. The system processes the one or more blocks of the EMG data and computes a metric for each block of the one or more blocks of the EMG data that have been processed. The system determines that the metric representing at least one EMG channel of the plurality of EMG channels transgresses the threshold for detection of the STA and generates audible or visual feedback to indicate that the metric representing the at least one EMG channel of the plurality of EMG channels transgresses the threshold for detection of the STA.

The invention relates to the technical field of robots, in particular to a mobile robot remote operation system and method based on mixed reality, and the method comprises an operator end and a robot end, the operator end completes a complex remote operation task based on visual feedback provided by a man-machine cooperation interaction interface, and the robot end completes a remote operation task based on visual feedback provided by the man-machine cooperation interaction interface. The virtual reality controller integrates a motion state and an operation mode of the mobile robot, and the mixed reality server generates visual feedback according to visual information of a remote operation visual angle of the mobile robot; the robot end comprises a mobile robot, a collection device on the mobile robot and a robot server, and after the mobile robot receives a remote control instruction sent by the virtual reality controller to the robot end, posture and button information of the mobile robot are input into the robot server to drive the mobile robot to move. Therefore, the problems of low real-time performance and accuracy of remote control, poor operation efficiency and flexibility and the like caused by limitation of bandwidth and system time delay in related technologies are solved.

The invention, which relates to the technical field of industrial equipment state monitoring, discloses a belt conveyor fault detection and early warning system comprising a multi-mode sensing module, an edge calculation module, a feature fusion and analysis module, a risk level determination module, an intelligent early warning module and a visual feedback module. Multi-source data such as images, sound, tension, temperature and vibration in the running process of a conveyor are collected, features are extracted and preprocessed through edge calculation, and a structured feature vector is constructed; through deep fusion and fault identification model analysis, operation state identification and trend score generation, and in combination with a dynamic threshold algorithm, a risk level is determined, the method has a hierarchical response mechanism, supports user feedback driven model online learning and optimization, and improves the accuracy and adaptive ability of fault identification; the system has the advantages of being comprehensive in detection, timely in response and high in intelligent degree, and is suitable for various complex conveying environments.

The invention relates to the technical field of electric energy metering, and discloses a power consumption management method and system of an electric energy metering box. The method comprises the following steps: acquiring an original electric signal containing a plurality of electric parameters, and generating a steady-state and transient-state data block through filtering noise reduction, variable-point detection and time domain segmentation processing; based on the data block, obtaining a standardized feature vector through feature extraction, feature sorting and normalization processing; carrying out load identification and time alignment on the feature vector by utilizing transfer learning and confidence judgment, and generating a load type identifier with a timestamp; the load access time is further analyzed and standardized, and structured intermediate data is generated through space-time alignment and data fusion; and finally, a power consumption management report is formed through visualization processing. According to the invention, the precision and reliability of non-intrusive load monitoring can be effectively improved, fine-grained and automatic analysis and visual feedback of power consumption behaviors of users are realized, and the method has the beneficial effects of high data processing efficiency, high identification accuracy and visual and convenient management.

The embodiment of the invention provides an infrared touch control device, projection equipment and an infrared touch control method, which are used for solving the technical problem of insufficient convenience of the existing key type interaction mode. The infrared touch device comprises a frame shell, an infrared laser transmitter, an infrared camera and a function processing circuit. According to the infrared touch device, the projector is used for projecting a control menu picture, an invisible infrared light plane is formed through the infrared laser transmitter, the control menu picture and the infrared light plane jointly form a virtualized interaction interface, and then interaction behaviors of a user are sensed in real time through the infrared camera. Therefore, seamless fusion of the interactive behavior and visual feedback of the user is realized, and the instant response touch function provided by the physical touch screen is reproduced by adopting different technical paths, so that the interactive process is simpler, more visual, more natural and more convenient, and the user experience is greatly optimized.

The invention provides a multimedia interaction control method and system based on a man-machine feedback closed loop, and relates to the technical field of financial science and technology, and the method comprises the steps: obtaining the multi-mode original feedback data of a user in an interaction environment in real time; preprocessing the multi-modal original feedback data to obtain preprocessed multi-modal feedback data; performing space-time coupling processing on the preprocessed multi-modal feedback data to generate multi-modal fusion features; dynamically determining a plurality of feature reference points based on feedback data of different modalities in a feature space corresponding to the multi-modal fusion features; the feature reference points at least comprise a user pupil center coordinate point in the visual feedback data, a fundamental frequency energy peak point in the voice feedback data and a pressure distribution gravity center point in the tactile feedback data. According to the method, space-time cooperative processing, dynamic feature quantification and closed-loop adaptive compensation of multi-modal feedback in a financial scene can be realized, and the instruction precision and information response timeliness of financial interaction are improved.

The invention discloses a robot grabbed target recognition and positioning method and system based on a knowledge graph, and the method comprises the steps: constructing a triple knowledge graph containing object appearance features and grabbed point pose parameters through multi-source data integration, and achieving a structured node network; environment perception data of a robot vision module is obtained, vector representation of a target object is determined through processing, if the similarity exceeds a threshold value, a matching node is inquired through the entity linking technology, recognition ambiguity is solved, and an accurate result is obtained; based on a result, retrieving pose parameters in the knowledge graph, performing coordinate conversion to generate a fine-grained grabbing sequence, combining real-time visual feedback, adopting a dynamic adjustment algorithm to optimize a path, and updating the knowledge graph through simulation evaluation to adapt to a new scene. According to the method, the core function of the knowledge graph in grabbing optimization is highlighted, the accuracy, robustness and adaptive capacity of robot operation are improved, and the method is suitable for the fields of industrial automation and the like.

The invention relates to the technical field of first-aid patient information processing, in particular to a severe trauma early evaluation and processing system based on virtual-real combination interaction, which comprises a scene risk sensing module, a dynamic physiological extraction module, an image feature analysis module, an abnormal grade judgment module and a priority scheduling grading module. According to the invention, through abnormal time period positioning, multi-terminal data synchronization and fusion of multi-source data, feature analysis is carried out under a unified time sequence, continuous triggering of risk early warning and hierarchical response is realized, a wound index is automatically generated, geographical location information is linked, seamless information circulation and task pushing among multi-terminal platforms are realized, and the efficiency of risk early warning and hierarchical response is improved. Automatic synchronization, intelligent grading and scheduling response of data from a site to a hospital are promoted, system advantages are established in the aspects of multi-link cooperation, information closed loop and early trauma judgment, physiological process visual feedback, grading task closed loop and real-time resource matching are further supported, and the early recognition and processing efficiency of critical patients is improved.

The invention relates to the field of natural language semantic analysis, in particular to a natural language-driven CIM scene interaction method and system, and the method comprises the steps: receiving a natural language interaction instruction inputted by a user side in real time; analyzing the natural language interaction instruction, and extracting scene data in the natural language interaction instruction; calling a semantic mapping library based on the extracted scene data, and mapping the scene data into corresponding CIM platform functions and space elements through the semantic mapping library; according to the CIM platform function and the space element, generating an interface calling instruction of the CIM platform, sending the interface calling instruction to the CIM platform, and executing a corresponding space element operation through the CIM platform; an operation result returned by the CIM platform is received, multi-modal feedback information is generated based on the operation result, and the multi-modal feedback information comprises visual feedback carried out in the CIM scene; visually feeding back synchronous natural language explanation feedback; and outputting the multi-mode feedback information to the user side.

The invention discloses a flexible material automatic feeding and accurate positioning control system based on machine vision, and belongs to the technical field of machine vision and intelligent decision making. Comprising a sensing and identifying module, a decision-making and planning module and a control and execution module. The perception and recognition module recognizes a material contour and quantifies a deformation parameter of the material by using a deep learning model through an image acquisition module and a flexible material recognition and state analysis module; the decision-making and planning module selects an optimal grabbing strategy from a feeding strategy library according to the deformation state and plans a collision-free path; the control and execution module realizes smooth and stable grabbing through the self-adaptive grabbing control module in combination with a visual feedback and impedance control algorithm, and performs real-time tracking and dynamic pose compensation on materials through the accurate positioning and placing module by utilizing a visual servo technology; the industrial problems of grabbing damage, placement drifting and the like caused by the fact that flexible materials are prone to deformation are solved, and high-precision and high-stability automatic feeding is achieved.