Collaborative robot closed-loop operation method based on scene perception and thinking chain reasoning

Abstract

The invention belongs to the technical field of intelligent and robot operation, and discloses a collaborative robot closed-loop operation method based on scene perception and thinking chain reasoning, which comprises the following steps of: firstly, generating structured semantic representation containing confidence by utilizing a visual language model of LoRA efficient fine tuning and fusing multi-view data, and providing semantic understanding for a robot; secondly, utilizing a three-dimensional diffusion model to generate explicit geometric priori under the condition of no real CAD, and correspondingly completing PnP initial pose estimation in combination with 2D-3D; then, carrying out pose optimization by adopting a tracking-refining strategy, and triggering reinitialization based on geometric prior when a residual error exceeds a threshold value; and finally, decomposing a global task by utilizing a thinking chain reasoning module, and realizing'perception-reasoning-execution 'closed-loop control based on visual servo in cooperation with an execution module. The method realizes effective combination of large model reasoning and physical execution, has strong robustness under dynamic disturbance, and is suitable for intelligent manufacturing and man-machine cooperation scenes.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of embodied intelligence and robot operation, and relates to a collaborative robot closed-loop operation method based on scene perception and thought chain reasoning, more particularly to a robot operation method based on closed-loop perception, reasoning and execution integration of visual language model, which can realize stable execution of long-term tasks in open and dynamic environments, and is suitable for application scenarios such as intelligent manufacturing, human-robot collaboration and service robots. BACKGROUND

[0002] With the rapid development of artificial intelligence and robot technology, the application demand of robots in industrial manufacturing, logistics transportation, medical care and home service fields is increasing. Realizing long-term stable operation of robots in complex environments has become a key link to promote robots from the laboratory to actual deployment. Traditional robot operation methods mostly rely on precise modeling and manually set rules. Such methods have certain effect in structured scenarios, but are prone to failure in open and dynamic environments due to environmental changes and target disturbances, making it difficult to meet the needs of long-term task execution.

[0003] In recent years, with the development of deep learning and large models, visual language action (VLA) methods, as an important development direction of embodied intelligence, have gradually attracted attention. Such methods attempt to use large-scale pre-trained visual language models to directly map natural language instructions and perception information to control actions. Although such methods can reduce the design of artificial rules and improve the generalization ability of robots in multi-task conditions, there are two main problems: first, the end-to-end mapping method is difficult to adapt to environmental disturbances in long-term tasks. Once a deviation occurs in the early steps, errors will accumulate in subsequent tasks, eventually leading to task failure; second, such models lack sufficient perception of environmental details, usually relying on long and tedious prompt word reasoning space constraints and object attributes, which are prone to perceptual hallucinations, resulting in unstable execution results.

[0004] To improve interpretability and robustness, some research proposes a hierarchical framework that separates perception, planning and execution into independent modules. The perception module obtains object information through target detection, segmentation and other methods, the planning module performs task decomposition and action reasoning based on language models, and the execution module calls motion planning and control algorithms to complete specific actions. The hierarchical architecture alleviates the vulnerability of end-to-end models to some extent, but introduces new problems: first, the separation of perception and reasoning makes it difficult for spatial information to be fed back to task planning in real time; second, the planning results are usually generated once before execution, lacking an online error correction mechanism that cannot be corrected in time when disturbances or environmental changes occur; third, due to the lack of closed-loop feedback, small deviations will accumulate gradually, making it impossible to guarantee global consistency in long-term tasks.

[0005] In addition, most of the existing perception modules are static reasoning, that is, only the target pose and constraint conditions are obtained in the initial stage, and there is a lack of continuous perception ability in the task execution process. This makes the robot unable to dynamically adjust the operation strategy according to the latest environmental information when facing the target object being moved, the light changing or the external interference, further aggravating the risk of task failure. Therefore, the prior art still lacks a method capable of unifying perception and reasoning and forming a closed-loop feedback mechanism during execution, thereby ensuring the stability and robustness of the task in a dynamic scene. SUMMARY

[0006] In view of the deficiencies of the prior art in the operation of collaborative robots in an open dynamic environment, the present application provides a somatic intelligent collaborative robot operation method based on scene perception and thought chain reasoning.

[0007] To achieve the above-mentioned purpose, according to one aspect of the present application, a collaborative robot closed-loop operation method based on scene perception and thought chain reasoning is provided, comprising the following steps: S1 Constructing a collaborative robot operation model: including a scene perception module, a pose optimization module, a thought chain reasoning module and an execution module, obtaining multi-view images and task instructions, and completing data preprocessing; S2 Scene perception module: based on the data obtained in step S1, using a visual language model to extract image and task language features, performing multi-modal feature fusion to obtain scene semantic representation; S3 Prior generation and initialization: under the condition of no real CAD model, generating explicit CAD prior by fusing overhead view and palm view observations through a three-dimensional diffusion model, combining projection geometry constraints and two-dimensional-three-dimensional correspondence to solve the first frame pose and complete initial pose estimation; S4 Pose optimization module: on the basis of step S3, using a tracking-refinement strategy for pose iterative optimization, introducing symmetry constraints, scale normalization and regularization constraints, and re-weighting the candidate pose according to the semantic constraints for pose optimization; When the re-projection error or residual exceeds the threshold, the explicit CAD prior triggers re-initialization to eliminate drift; S5 Thought chain reasoning module: on the basis of step S4, inputting the scene semantic representation in step S2 into the thought chain reasoning module, using chain reasoning to decompose the global task, generating subtasks and performing geometric and semantic consistency checks; S6 Execution Module: Based on the subtask of step S5, it calls the inverse kinematics solution and motion planning interface of the collaborative robot controller to convert the end target pose output in step S5 into joint control quantities / trajectory and send it out for execution. During the execution process, it continuously senses and performs real-time fine-tuning of the trajectory using visual servoing, monitors projection errors and constraint conflicts, and triggers online error correction and replanning when the error exceeds the domain or is disturbed. At the same time, it feeds the perception results back to the thought chain reasoning module. S7 completes the current subtask and determines the global goal achievement status; if not achieved, it returns to step S6-S7 for closed-loop iteration until the task is completed.

[0008] Preferably, the data preprocessing in step S1 includes timestamp synchronization, calibration and distortion correction, image normalization, target segmentation mask generation, and instruction text label structuring.

[0009] Preferably, the visual language model in step S2 is Qwen2-VL, and the scene semantic representation is a structured semantic representation with confidence, wherein the confidence is used for candidate pose selection or weighting.

[0010] Preferably, the structured semantic representation is a triple, which includes an object bounding box, an operable region box, and an operation constraint for the end effector. The operation constraint includes at least one or more of the following: gripping direction and gripper opening / closing range.

[0011] Preferably, the triple expression of the structured semantic representation is:

[0012] in, For the outer frame of the object, This is the operable area bounding box. This represents the set of operational constraints acting on the gripper and dexterous hand, including the gripping direction, gripper opening and closing range, and finger bending range.

[0013] Preferably, the pose of the first frame in step S3 is obtained by solving the PnP equation, the expression of which is:

[0014] in, The pose of the first frame. It represents a two-dimensional to three-dimensional correspondence. This is the first frame image. The mask for the first frame. This is the depth map of the first frame. This is the camera intrinsic parameter matrix.

[0015] Preferably, in step S4, the symmetry constraint is used to filter the optimal solution by traversing the symmetry group of the object using the Chamfer distance, and its expression is:

[0016] in, Represents the symmetrical chamfer distance between point sets. Denotes the symmetric group, It is through symmetric operations Transformed CAD priors This represents the distance between two points in three-dimensional Euclidean space.

[0017] As a preferred option, in step S4, the scale normalization and regularization constraints first unify the target diameter to a unit scale, and then introduce lightweight external regularization and fine-tuning terms for scale-sensitive categories to represent the difference between the actual and estimated scales and attitudes. The regularization and fine-tuning terms are expressed as follows:

[0018] in, , These represent the true scale and the estimated scale, respectively. and These represent the actual pose and the predicted pose, respectively. This is the weighting factor.

[0019] Preferably, the chain reasoning in step S5 adopts a closed-loop approach of "decomposition-execution-feedback-replanning".

[0020] Preferably, the geometric and semantic consistency check in step S5 is achieved by calculating the difference between the geometric state description and the semantic state representation. When the difference exceeds a preset threshold, a subtask update is triggered. The calculation formula is as follows:

[0021] in, The threshold representing the difference between the geometric state description and the semantic state representation. This represents the geometric state description obtained from the pose optimization module. This represents the semantic state representation obtained by reasoning from a visual language model.

[0022] Preferably, in step S6, the trajectory is fine-tuned by dynamically correcting the deviation through visual servoing, and this execution architecture can be directly integrated into the existing industrial robot control stack.

[0023] In summary, the technical solutions conceived by this invention have the following beneficial effects compared with the prior art: (1) By introducing structured semantic representation in the perception stage, combining generative 3D priors and projective geometry in the estimation stage, integrating geometric and semantic constraints in the optimization stage, using chain reasoning to achieve dynamic task decomposition and replanning in the reasoning stage, and achieving closed-loop operation based on inverse kinematics analytical control in the execution stage, the existing methods can effectively solve the problems of pose estimation and initialization without CAD, drift accumulation during execution, and lack of online correction capability under external disturbances.

[0024] (2) By efficiently fine-tuning the structured triplet and LoRA parameters, the operable area and action constraints can be explained and cost-effective, avoiding illusions.

[0025] (3) A collaborative robot operation method that can still operate reliably in the absence of a real CAD model, realizing the pose initialization and dynamic optimization of the object; and by introducing dual constraints of geometric consistency and semantic consistency, the accuracy of pose estimation and the feasibility of operation are improved.

[0026] (4) Establish an overall framework that combines scene perception and thought chain reasoning, so that the robot has embodied semantic understanding ability, can output structured operational constraint representation, and continuously decompose and replan the task during execution to ensure global consistency of long-term tasks and adaptability to disturbances.

[0027] (5) All actions are executed through inverse kinematics analysis control and motion planning, which is easy to integrate into the industrial control stack and is friendly to engineering implementation. Attached Figure Description

[0028] Figure 1 This is a schematic diagram of the collaborative robot operation method based on scene perception and thought chain reasoning technology in an embodiment of the present invention.

[0029] Figure 2 This is a schematic diagram of a method for efficient fine-tuning of scene perception in an embodiment of the present invention.

[0030] Figure 3 This is a schematic diagram of the process of generating and initializing the collaborative robot in an embodiment of the present invention.

[0031] Figure 4 This is an example of a data acquisition experimental platform in an embodiment of the present invention. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0033] Existing collaborative robots suffer from the following problems in open and dynamic environments: First, end-to-end solutions for motion generation are prone to perceptual and semantic illusions under changes in lighting, viewpoint, and occlusion; second, pose tracking is susceptible to accumulated errors due to disturbances; and third, the lack of feedback-driven online replanning during execution makes it difficult to maintain global goal consistency. To address these issues, this invention proposes the following technical solutions.

[0034] Specifically, the collaborative robot closed-loop operation method based on scene perception and thought chain reasoning provided by this invention is mainly aimed at embodied intelligence applications. By introducing structured semantic representation through LoRA fine-tuning in the perception stage, combining generative 3D priors and projective geometry in the estimation stage, incorporating geometric and semantic constraints in the optimization stage, using chain reasoning to achieve dynamic task decomposition and replanning in the reasoning stage, and realizing "perception-reasoning-execution" closed-loop operation based on inverse kinematics analytical control in the execution stage, this method effectively solves the problems of existing methods, such as difficulty in pose estimation and initialization without CAD, easy drift accumulation during execution, and lack of online correction capability under external disturbances. This method effectively improves the operational stability and generalization of collaborative robots in open dynamic environments.

[0035] Combination Figure 1 The present invention provides a closed-loop operation method for collaborative robots based on scene perception and thought chain reasoning, comprising the following steps: S1 constructs a collaborative robot operation model, including a scene perception module, a pose optimization module, a thought chain reasoning module, and an execution module, to acquire multi-view images and task instructions and complete data preprocessing. In this embodiment, data preprocessing refers to the standardization processing of multi-view images and task instructions before they enter scene perception / pose optimization, including timestamp synchronization, calibration and distortion correction, image normalization, target segmentation mask generation, and instruction text labeling and structuring; specifically, it includes: multi-view data timestamp synchronization, camera calibration and distortion correction, resolution unification, cropping / scaling, pixel and other image normalization; target detection and segmentation to obtain masks and perform multi-view alignment, and text cleaning, synonym normalization and structured encoding of task instructions.

[0036] S2 Scene Perception Module: Based on the data obtained in step S1, it uses a visual language model to extract image and task language features, and performs multimodal feature fusion to obtain scene semantic representation; The visual language model is Qwen2-VL, and the scene semantic representation is a structured semantic representation with confidence. The structured semantic representation is a triple, which includes the object bounding box, the operable region box, and the operation constraints for the end effector. The operation constraints include at least one or more of the following: gripping direction and gripper opening and closing range.

[0037] The triple expression for the structured semantic representation is:

[0038] in, For the outer frame of the object, This is the operable area bounding box. This represents the set of operational constraints acting on the gripper and dexterous hand, including gripping direction, gripper opening and closing range, and finger flexion range. The supervision labels are constructed from extended HANDAL and YCB datasets, employing low-rank adaptation LoRA technology for efficient parameter fine-tuning to ensure fast and reliable output of constraint information; its weight matrix is ​​decomposed as follows:

[0039] in, For frozen pre-trained weights, , For low-rank adaptors, , These are the input and output dimensions, respectively. It is a low-rank order.

[0040] S3 Prior Generation and Initialization: In the absence of a real CAD model, explicit CAD priors are generated by fusing top-down and palm-view observations through a 3D diffusion model. The first frame pose is solved by combining projection geometric constraints and 2D-3D correspondence to complete the initial pose estimation. The pose of the first frame is obtained by solving the PnP equation, the expression of which is:

[0041] in, The pose of the first frame. It represents a two-dimensional to three-dimensional correspondence. This is the first frame image. The mask for the first frame. This is the depth map of the first frame. This is the camera intrinsic parameter matrix.

[0042] S4 Pose Optimization Module: Based on step S3, a tracking-refinement strategy is adopted for iterative pose optimization. Symmetry constraints, scale normalization and regularization constraints are introduced, and candidate poses are reweighted according to semantic constraints to optimize pose. When the reprojection error or residual exceeds the threshold, explicit CAD prior triggers reinitialization to eliminate drift. The following constraints are introduced during the optimization process: 1. Symmetry Constraint: The optimal solution is selected by traversing the symmetry group of the object using Chamfer distance. That is, the optimal equivalent solution is selected by traversing the symmetry group of the object and using Chamfer distance. The expression is as follows:

[0043] in, Represents the symmetrical chamfer distance between point sets. Denotes the symmetric group, It is through symmetric operations Transformed CAD priors This represents the distance between two points in three-dimensional Euclidean space.

[0044] 2. Scale constraint: Normalize the object diameter and apply scale regularization to improve the stability and consistency of cross-class estimation. That is, for targets of different categories, first unify the target diameter to a unit scale so that the estimation process is carried out under a unified normalized scale. At the same time, for scale-sensitive categories, introduce lightweight external regularization and fine-tuning terms to represent the difference between the actual and estimated scales and poses. The regularization and fine-tuning terms are expressed as follows:

[0045] in, , These represent the true scale and the estimated scale, respectively. and These represent the actual pose and the predicted pose, respectively. This is the weighting factor.

[0046] S5 Thinking Chain Reasoning Module: Based on step S4, the scene semantic representation in step S2 is input into the thinking chain reasoning module. The global task is decomposed using chain reasoning, sub-tasks are generated, and geometric and semantic consistency checks are performed. That is, a sub-task pipeline is generated based on the closed-loop method of "decomposition-execution-feedback-replanning", and geometric-semantic alignment is guaranteed through consistency verification.

[0047] In each step First, the current environmental state is perceived by the all-view camera. and combine it with the current subtask Input the model to predict the action policy , and object Then, combining the perception information from the palm and the full-view camera, the object bounding box is analyzed and updated in real time. Operation position bounding box Operational constraints and position , parameters and Composition of execution instruction tuples After the execution layer controls the arms to complete the movement, the environment state is updated. It also provides feedback on the completion status of the current subtask and monitors external dynamic disturbances. The inference module is in the global objective... Initial task sequence Historical sub-tasks and the latest status Predict the next subtask under the combined effect of the new inference; According to the original plan When there is inconsistency, the system uses Replace the atomic task and update the task sequence to This is then used as a constraint input for subsequent predictions, and the process repeats until the global task is completed.

[0048] Furthermore, the geometric and semantic consistency check is achieved by calculating the difference between the geometric state description and the semantic state representation. When the difference exceeds a preset threshold, a subtask update is triggered; the calculation formula is as follows:

[0049] in, The threshold representing the difference between the geometric state description and the semantic state representation. This represents the geometric state description obtained from the pose optimization module. This represents the semantic state representation obtained by reasoning from a visual language model.

[0050] S6 Execution Module: Based on the subtask of step S5, it calls the inverse kinematics solution and motion planning interface of the collaborative robot controller to convert the end-effector pose output in step S5 into joint control variables / trajectory and send it for execution; the inverse kinematics solution is implemented using the controller's built-in algorithm module. During execution, continuous perception and visual servoing are used to fine-tune the trajectory in real time, monitoring projection errors and constraint conflicts. When the error exceeds the domain or is disturbed, online error correction and replanning are triggered; at the same time, the perception results are fed back to the thought chain reasoning module; the fine-tuned trajectory is dynamically corrected for deviations through visual servoing, and this execution architecture can be directly integrated into the existing industrial robot control stack.

[0051] Further, the execution instruction tuple is defined as:

[0052] in, For action strategy, For the set of operational constraints, For the bounding box of the operation area, For the bounding box of the object, For the bounding box of the operation area, To predict pose, For the target object.

[0053] Furthermore, the task sequence update rule is expressed as follows:

[0054] in, For the replaced subtask, when compared with the original task When inconsistent, This will serve as a new input, ensuring that the task execution process can adapt to environmental disturbances and maintain the consistency of global objectives.

[0055] The execution module is entirely based on analytical control and is used to call inverse kinematics calculation and motion planning to generate paths and track trajectories based on subtasks and operational constraints.

[0056] S7 completes the current subtask and determines the global goal achievement status; if not achieved, it returns to step S6-S7 for closed-loop iteration until the task is completed.

[0057] Combination Figure 1 See also Figure 4 An example of a data acquisition experimental platform is provided. The overall process of this invention includes steps S1-S7, specifically: Combination Figure 2 The system operates on S1 and S2 to acquire multi-view images and task instructions, which are then fused into a multimodal visual language model with highly efficient parameter fine-tuning. The model outputs a structured scene semantic representation, including object bounding boxes, operable regions, and operational constraints for the end effector, along with confidence values.

[0058] Combination Figure 3 Operation S3, in the absence of a real CAD model, uses a 3D diffusion model to fuse top-down and palm-view perspectives to generate explicit CAD priors, and then uses projection geometric constraints to solve the pose of the first frame based on sparse 2D-3D correspondence. Operation S4 updates the pose frame by frame using a "tracking-refinement" strategy based on the initialization of the first frame; when the reprojection error or residual exceeds the threshold, reinitialization is triggered based on CAD priors.

[0059] Operation S5 inputs the structured semantic representation and candidate poses into the inference module, generating a sequence of subtasks based on a closed-loop approach of "decomposition-execution-feedback-replanning".

[0060] Operation S6 generates the execution trajectory using inverse kinematics analytical control and motion planning based on the sub-task and operational constraints. During execution, the perception results are fed back in real time, updating the inference end and forming a closed loop of perception-inference-execution. Reprojection errors and task constraint conflicts are monitored in real time during execution. When the accumulated error exceeds a threshold... At that time, a re-initialization based on CAD priors is triggered, and S4 is invoked to update the subtask sequence. This design ensures robust execution even under disturbances, occlusion, or target offset.

[0061] Operation S7 checks whether the global goal has been achieved after each subtask is completed. If it has, the process terminates; otherwise, it returns to S4-S6 to continue the closed-loop iteration until the goal is achieved or the safety interruption strategy is triggered.

[0062] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A closed-loop operation method for collaborative robots based on scene perception and thought chain reasoning, characterized in that, Includes the following steps: S1 constructs a collaborative robot operation model, which includes a scene perception module, a pose optimization module, a thought chain reasoning module, and an execution module, to acquire multi-view images and task instructions and complete data preprocessing. S2 Scene Perception Module: Based on the data obtained in step S1, it uses a visual language model to extract image and task language features, and performs multimodal feature fusion to obtain scene semantic representation; S3 Prior Generation and Initialization: In the absence of a real CAD model, explicit CAD priors are generated by fusing top-down and palm-view observations through a 3D diffusion model. The first frame pose is solved by combining projection geometric constraints and 2D-3D correspondence to complete the initial pose estimation. S4 Pose Optimization Module: Based on step S3, a tracking-refinement strategy is adopted for pose iterative optimization. Symmetry constraints, scale normalization and regularization constraints are introduced, and candidate poses are reweighted according to semantic constraints to optimize pose. When the reprojection error or residual exceeds the threshold, explicit CAD priors trigger reinitialization to eliminate drift. S5 Thinking Chain Reasoning Module: Based on step S4, the scene semantic representation in step S2 is input into the thinking chain reasoning module, and the global task is decomposed by chain reasoning to generate sub-tasks and perform geometric and semantic consistency checks. S6 Execution Module: Based on the subtask of step S5, it calls the inverse kinematics solution and motion planning interface of the collaborative robot controller to convert the end target pose output in step S5 into joint control quantities / trajectory and send it out for execution. During the execution process, it continuously senses and performs real-time fine-tuning of the trajectory using visual servoing, monitors projection errors and constraint conflicts, and triggers online error correction and replanning when the error exceeds the domain or is disturbed. At the same time, it feeds the perception results back to the thought chain reasoning module. S7 completes the current subtask and determines the achievement of the global goal; if not, it returns to step S6-S7 for closed-loop iteration until the task is completed.

2. The collaborative robot closed-loop operation method based on scene perception and thought chain reasoning as described in claim 1, characterized in that, The data preprocessing described in step S1 includes timestamp synchronization, calibration and distortion correction, image normalization, target segmentation mask generation, and instruction text label structuring.

3. The collaborative robot closed-loop operation method based on scene perception and thought chain reasoning as described in claim 1, characterized in that, In step S2, the visual language model is Qwen2-VL, and the scene semantic representation is a structured semantic representation with confidence, which is used for candidate pose selection or weighting. The structured semantic representation is a triple, which includes an object bounding box, an operable region box, and an operation constraint for the end effector. The operation constraint includes at least one or more of the following: gripping direction and gripper opening / closing range.

4. The collaborative robot closed-loop operation method based on scene perception and thought chain reasoning as described in claim 3, characterized in that, The triple expression for the structured semantic representation is: in, For the outer frame of the object, This is the operable area bounding box. This represents the set of operational constraints acting on the gripper and dexterous hand, including the gripping direction, gripper opening and closing range, and finger bending range.

5. The collaborative robot closed-loop operation method based on scene perception and thought chain reasoning as described in claim 1, characterized in that, In step S3, the pose of the first frame is obtained by solving the PnP equation, the expression of which is: in, The pose of the first frame. It represents a two-dimensional to three-dimensional correspondence. This is the first frame image. The mask for the first frame. This is the depth map of the first frame. This is the camera intrinsic parameter matrix.

6. The collaborative robot closed-loop operation method based on scene perception and thought chain reasoning as described in claim 1, characterized in that, In step S4, the symmetry constraint is used to traverse the symmetry group of the object and select the optimal solution. The expression is as follows: in, Represents the symmetrical chamfer distance between point sets. Denotes the symmetric group, It is through symmetric operations Transformed CAD priors This represents the distance between two points in three-dimensional Euclidean space.

7. The collaborative robot closed-loop operation method based on scene perception and thought chain reasoning as described in claim 1, characterized in that, In step S4, scale normalization and regularization constraints first unify the target diameter to a unit scale, and then introduce lightweight external regularization and fine-tuning terms for scale-sensitive categories to represent the difference between the actual and estimated scales and attitudes. The regularization and fine-tuning terms are expressed as follows: in, , These represent the true scale and the estimated scale, respectively. and These represent the actual pose and the predicted pose, respectively. This is the weighting factor.

8. The collaborative robot closed-loop operation method based on scene perception and thought chain reasoning as described in claim 1, characterized in that, In step S5, the chain reasoning adopts a closed-loop approach of "decomposition-execution-feedback-replanning".

9. A closed-loop operation method for a collaborative robot based on scene perception and thought chain reasoning as described in claim 8, characterized in that, In step S5, the geometric and semantic consistency check is achieved by calculating the difference between the geometric state description and the semantic state representation. When the difference exceeds a preset threshold, a subtask update is triggered. The calculation formula is as follows: in, The threshold representing the difference between the geometric state description and the semantic state representation. This represents the geometric state description obtained from the pose optimization module. This represents the semantic state representation obtained by reasoning from a visual language model.

10. A closed-loop operation method for a collaborative robot based on scene perception and thought chain reasoning as described in claim 1, characterized in that, In step S6, the trajectory is fine-tuned by dynamically correcting deviations through visual servoing, and this execution architecture can be directly integrated into the existing industrial robot control stack.

Images