A fisher information based robot object observation angle prediction method and device

Abstract

The application belongs to the field of information science and technology and robot, and relates to a mechanical arm object observation angle prediction method and device based on Fisher information. The method comprises the following steps: adopting a mobile object-fixed camera perception paradigm to define a world coordinate system, a camera coordinate system and a mechanical arm end effector coordinate system, and initializing a three-dimensional reconstruction model; constructing color Fisher information gain and depth Fisher information gain, and performing weighted fusion to obtain total Fisher information gain; in each iteration, selecting an observation view angle capable of maximizing the total Fisher information gain from a candidate view angle set as an optimal observation view angle; performing preprocessing on the RGB image and the depth image collected under the optimal observation view angle; and incorporating the observation data after preprocessing into a training set, incrementally updating the three-dimensional reconstruction model until the iteration termination condition is met. The application can realize efficient and autonomous selection of an optimal observation view angle by a robot in a grasping object state, so as to improve object perception accuracy.

Description

Technical Field

[0001] This invention belongs to the field of information science and technology and robotics, specifically relating to a method and device for predicting the observation angle of a robotic arm based on Fisher information. Background Technology

[0002] Robotics, as an important indicator of a nation's scientific and technological development level, has been widely researched and applied since its inception. Among the various capabilities of robots, arm manipulation is the foundation and core skill for performing tasks in complex, unstructured environments. To achieve precise and efficient object manipulation, robots must be able to accurately perceive and acquire the three-dimensional geometric information of target objects. High-quality object perception and 3D reconstruction are the key foundations for subsequent decision-making, path planning, and control execution. Traditional robot perception strategies mainly include two modes. The first is the Static Camera-Static Object (SCSO) mode, in which both the camera and the object are fixed. The robot can only acquire information from a single perspective, and the data quality is highly dependent on the prior model under that perspective, lacking sufficient generalization ability and only applicable to specific objects. The second is the Moving Camera-Static Object (MCSO) mode, which achieves multi-view observation by moving or rotating a camera fixed on the top or wrist of the robot. However, in open, unstructured, and complex scenarios, when an object has been grasped by a robotic arm, the MCSO strategy often requires complex motion planning to move the entire robotic arm or external platform, which appears unnatural and inefficient.

[0003] The aforementioned traditional strategies, especially those employing fixed or random sensor parameters (such as camera pose), often lead to problems such as missing important viewpoints of objects and low data utilization efficiency, ultimately affecting the quality of 3D reconstruction of the target object. Furthermore, existing active perception algorithms largely rely on heuristic gains based on geometric overlap or visibility for viewpoint selection. These methods lack a direct measure of the uncertainty of the internal parameters of the 3D reconstruction model (such as 3D Gaussian reconstruction). During data acquisition, calibration residuals, dynamic background interference, and scale blurring due to monocular depth often cause "ghosting" or geometric distortion in the reconstruction model, making it difficult to achieve high-quality object representation under sparse viewpoints. Therefore, a more intelligent and efficient perception strategy is urgently needed, enabling robots to achieve high-quality 3D reconstruction with the fewest possible viewpoints by selectively and actively observing the object in their grasp. Summary of the Invention

[0004] The purpose of this invention is to provide a method and device for predicting the object observation angle of a robotic arm based on Fisher information, which aims to solve the problems of low data utilization efficiency and poor 3D reconstruction quality in existing robot perception strategies, so as to enable the robot to efficiently and autonomously select the best observation angle when grasping an object, thereby improving the accuracy of object perception.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: A method for predicting the object observation angle of a robotic arm based on Fisher information includes the following steps: The world coordinate system, camera coordinate system, and robotic arm end effector coordinate system are defined using a moving object-fixed camera perception paradigm. Images are acquired from a predefined set of initial viewpoints to initialize the 3D reconstruction model. Construct the color Fisher information gain and the depth Fisher information gain and then perform a weighted fusion to obtain the total Fisher information gain; In each iteration, the observation view that maximizes the total Fisher information gain is selected from the candidate view set as the best observation view. Preprocessing is performed on the RGB and depth images acquired at the optimal viewing angle. The preprocessing includes foreground image separation, global scale-offset alignment, and dynamic correction of camera extrinsic parameters. The preprocessed observation data from the best observation perspective is incorporated into the training set, and the 3D reconstruction model is updated incrementally until the iteration termination condition is met. The trained 3D reconstruction model is used to complete the robotic arm's operation on the target object.

[0006] Furthermore, in the moving object-fixed camera perception paradigm, the camera coordinate system is fixed relative to the world coordinate system, and the viewing angle is determined by the pose of the end effector of the robotic arm.

[0007] Furthermore, the construction of color Fisher information gain and depth Fisher information gain includes: constructing negative log-likelihood loss functions based on the color rendering function and the depth rendering function, respectively, and approximating the color and depth Fisher information matrices by using the diagonal terms of the Hessian matrix corresponding to the loss function.

[0008] Furthermore, by combining the forward and inverse kinematics models of the robotic arm, joint physical limits, and anti-collision constraints, uniform or gridded discrete sampling calculations are performed within the reachable manifold workspace of the robotic arm's end effector to obtain the candidate viewpoint set.

[0009] Furthermore, the foreground image separation includes: using a monocular depth estimation network to extract initial monocular depth map sequences from the acquired RGB images of each viewpoint; performing pixel-level statistical significance difference calculations on the multi-frame depth image sequences in the time dimension; setting a dynamic significance threshold to determine regions where the depth value changes significantly over time as moving foreground and static regions as background, thereby generating a high-precision foreground mask to eliminate background interference.

[0010] Furthermore, the global scale-offset alignment includes: establishing a linear geometric alignment model containing scale factor and depth offset, using a nonlinear optimization objective function based on least squares method to solve for the optimal scale factor and offset, and uniformly aligning each local monocular predicted depth map to a global coordinate system with absolute physical scale.

[0011] Furthermore, the dynamic correction of camera extrinsic parameters includes: extracting and associating shared two-dimensional feature points from a sequence of observation images from multiple frames at different viewpoints; parameterizing the camera extrinsic parameter residuals to be optimized into rigid body transformation error vectors in Lie algebra space; establishing reprojection error models from three-dimensional spatial points to two-dimensional image planes at various viewpoints; and iteratively updating the rigid body transformation error vectors in Lie algebra space using the Gauss-Newton method or the LM algorithm until the reprojection error is minimized, thereby obtaining the corrected high-precision camera extrinsic parameter pose.

[0012] A robotic arm object observation angle prediction device based on Fisher information, comprising: The modeling initialization module is used to define the world coordinate system, camera coordinate system, and robotic arm end effector coordinate system using the moving object-fixed camera perception paradigm, and to acquire images from a predefined set of initial viewpoints to initialize the 3D reconstruction model. The Fisher information calculation module is used to construct the color Fisher information gain and the depth Fisher information gain and then perform a weighted fusion to obtain the total Fisher information gain. The optimal viewpoint iterative selection module is used to select the observation viewpoint that maximizes the total Fisher information gain from the candidate viewpoint set as the best observation viewpoint in each iteration. The image preprocessing module is used to perform preprocessing on the RGB image and depth image acquired under the best viewing angle. The preprocessing includes foreground image separation, global scale-offset alignment and dynamic correction of camera extrinsic parameters. The incremental update module for the reconstruction model is used to incorporate the preprocessed observation data from the best viewing angle into the training set and incrementally update the 3D reconstruction model until the iteration termination condition is met. The task execution module is used to complete the robotic arm's operation tasks on the target object using the trained 3D reconstruction model.

[0013] The present invention also provides a computer device including a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing the methods described above.

[0014] The present invention also provides a computer-readable storage medium storing a computer program that, when executed by a computer, implements the above-described method.

[0015] The beneficial effects of this invention are as follows: This invention proposes a MOSC (Moving Object-Fixed Camera) perception paradigm, which effectively avoids the motion planning limitations of the traditional MCSO mode in handheld operation scenarios and significantly improves perception efficiency in confined spaces. Based on Fisher information, it performs analytical-level sensitivity measurement on the underlying parameters of the 3D reconstruction model, breaking the limitations of existing empirical heuristic gain evaluation and achieving high technical performance under extremely sparse sampling. Based on the characteristics of MOSC data, it designs preprocessing algorithms and dynamic correction of extrinsic parameters to ensure highly consistent geometric reconstruction, significantly enhancing the robot's robustness and representation accuracy in unstructured complex environments. Attached Figure Description

[0016] Figure 1 This is a flowchart of the steps of a robotic arm object observation angle prediction method based on Fisher information according to the present invention.

[0017] Figure 2 This is a schematic diagram comparing the existing MCSO sensing strategy with the MOSC sensing strategy of the present invention, where (a) is the MCSO mode and (b) is the MOSC mode, and "fixed" in the figure indicates a fixed state.

[0018] Figure 3 This is a schematic diagram of the multi-frame joint depth estimation and optimization process.

[0019] Figure 4 This is a diagram showing the before-and-after comparison of camera pose after optimization using multiple frames of extrinsic parameters.

[0020] Figure 5 This is a block diagram of a robotic arm object observation angle prediction device based on Fisher information according to the present invention. Detailed Implementation

[0021] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but this does not constitute any limitation on the present invention.

[0022] The main contents and innovations of this invention include: 1. Problem Modeling and Viewpoint Initialization under the MOSC Strategy. This invention innovatively adopts a moving object-fixed camera perception strategy and proposes a Moving Object-Fixed Camera (MOSC) perception paradigm (e.g., Figure 2 (As shown). That is, the robot actively perceives objects by adjusting the posture of the end effector of the gripper arm holding the target object, within the fixed camera's viewpoint. This design establishes the kinematic constraints between the end effector and the fixed camera, replacing the overall camera movement with the high-degree-of-freedom motion of the end effector, and establishing a set of candidate viewpoints covering the reachable manifold space.

[0023] 2. Fisher Information Modeling Combining Color and Depth Data. This invention constructs a Fisher Information Matrix (FIM) for color and depth by analytically deriving the loss function of the 3D reconstruction model. This matrix, as an approximation of the Hessian matrix, quantifies the contribution of the observation perspective to the convergence of the model's underlying parameters, establishing an analytical mapping between pixel-level observations and model parameter uncertainties. This allows for precise guidance to the robot in selecting the observation perspective that maximizes the benefits for the current model.

[0024] 3. Foreground separation and scale alignment based on multi-frame joint optimization. Addressing the characteristic of object movement while the background remains stationary in MOSC mode, this invention proposes a separation strategy that integrates multi-frame depth consistency (e.g., Figure 3 As shown in the figure, the foreground mask is accurately extracted and background noise is removed by utilizing the significant difference in depth. At the same time, to address the scale ambiguity in monocular depth estimation, a global scale-migration joint correction algorithm is adopted to ensure that multi-view data are fused in a unified physical scale space.

[0025] 4. Dynamic Correction and Model Update of Camera Extrinsic Parameters in Lie Algebra Space. Considering the data characteristics under the MOSC strategy, this method models the dynamic calibration residuals of camera extrinsic parameters and refines the pose by minimizing the reprojection error of multi-view feature points in Lie algebra space. Subsequently, data from selected views are included in the training set for incremental model updates until model performance converges. During observation, to address pose deviations caused by robotic arm vibration or calibration residuals, this invention establishes a reprojection error model in Lie algebra space. By identifying feature matching relationships between multiple views, the camera extrinsic parameters are dynamically optimized online. Then, the refined data is included in the training set to drive incremental parameter updates in the reconstruction model, thereby achieving high-quality 3D reconstruction under extremely sparse viewpoints.

[0026] In one embodiment, the present invention provides a robotic arm object observation angle prediction method based on Fisher information, such as... Figure 1 As shown, it includes the following steps: The moving object-fixed camera (MOSC) perception paradigm is used to define the world coordinate system, camera coordinate system, and robotic arm end effector coordinate system. Images are acquired from a predefined set of initial viewpoints to initialize the 3D reconstruction model. Construct the color Fisher information gain and the depth Fisher information gain and then perform a weighted fusion to obtain the total Fisher information gain; In each iteration, the observation view that maximizes the total Fisher information gain is selected from the candidate view set as the best observation view. Preprocessing is performed on the RGB and depth images acquired at the optimal viewing angle. The preprocessing includes foreground image separation, global scale-offset alignment, and dynamic correction of camera extrinsic parameters. The preprocessed observation data from the best observation perspective is incorporated into the training set, and the 3D reconstruction model is updated incrementally until the iteration termination condition is met. The trained 3D reconstruction model is used to complete the robotic arm's operation on the target object.

[0027] In the moving object-fixed camera perception paradigm, the camera coordinate system is fixed relative to the world coordinate system, and the viewing angle is determined by the pose of the end effector of the robotic arm.

[0028] In one embodiment, constructing the color Fisher information gain and the depth Fisher information gain includes: constructing negative log-likelihood loss functions based on the color rendering function and the depth rendering function, respectively, and approximating the color and depth Fisher information matrices by using the diagonal terms of the Hessian matrix corresponding to the loss function.

[0029] In one embodiment, the candidate viewpoint set is obtained by combining the forward and inverse kinematics model of the robotic arm, joint physical limits, and anti-collision constraints, and performing uniform or gridded discrete sampling calculations within the reachable manifold workspace of the robotic arm's end effector.

[0030] In one embodiment, the foreground image separation includes: using a monocular depth estimation network to extract an initial monocular depth map sequence from the acquired RGB images from each viewpoint; performing pixel-level statistical significance difference calculation on the multi-frame depth image sequence in the time dimension; setting a dynamic significance threshold to determine regions where the depth value changes significantly over time as moving foreground and static regions as background, thereby generating a high-precision foreground mask to eliminate background interference.

[0031] In one embodiment, the global scale-offset alignment includes: establishing a linear geometric alignment model containing scale factor and depth offset, using a nonlinear optimization objective function based on least squares to solve for the optimal scale factor and offset, and uniformly aligning the various local monocular predicted depth maps to a global coordinate system with absolute physical scale.

[0032] In one embodiment, the dynamic correction of camera extrinsic parameters includes: extracting and associating shared two-dimensional feature points in a sequence of observation images from multiple frames at different viewpoints; parameterizing the residuals of the camera extrinsic parameters to be optimized into rigid body transformation error vectors in Lie algebra space; establishing reprojection error models from three-dimensional spatial points to two-dimensional image planes at various viewpoints; and iteratively updating the rigid body transformation error vectors in Lie algebra space using the Gauss-Newton method or the LM algorithm until the reprojection error is minimized, thereby obtaining the corrected high-precision camera extrinsic parameter pose.

[0033] In one embodiment, a robotic arm object observation angle prediction method based on Fisher information according to the present invention specifically includes the following steps: (1) Problem modeling and perspective initialization.

[0034] Figure 2 This is a schematic diagram comparing the existing MCSO sensing strategy with the MOSC sensing strategy of this invention. This invention defines a world coordinate system. Fixed camera coordinate system and the coordinate system of the robotic arm end effector Determine that the camera coordinate system is fixed relative to the world coordinate system, and the viewing angle... The pose of the robotic arm end effector The transformation relationship between the coordinate systems is shown in formula (1): in, This represents the transformation from the camera coordinate system to the end effector coordinate system. This represents the transformation from the end effector of the robotic arm and its target object to the world coordinate system. This represents the transformation from the camera coordinate system to the world coordinate system. In MOSC mode, It is a constant. As a variable. Perspective. Depend on Decision. From a predefined initial set of perspectives. Collection in China Zhang images are used for the initialization of the 3D reconstruction model, to This represents the initialized 3D reconstruction model.

[0035] (2) Calculate Fisher's information gain objective function.

[0036] Constructing the total Fisher information gain As a metric for viewpoint selection, it is defined as the Fisher information gain of the image's color. And depth Fisher information gain The weighted sum is shown in formula (2): in and For the preset weights, For model parameters, For color rendering functions, This is a depth rendering function.

[0037] For color information, the loss function Defined as the negative log-likelihood loss function, as shown in formula (3): in, Indicates camera pose. express Image observation data acquired from the perspective of [viewpoint] Indicates the given model parameters and position Under these conditions, the model renders the actual observed image. The probability of.

[0038] Color Fisher Information Matrix It can be approximated as the diagonal terms of a Hessian matrix. Depth Fisher information. The modeling method is the same as that for color information, the difference being that the depth Fisher information uses the actual depth value. and depth rasterization function The loss is calculated as shown in formula (4): in, Indicates the given model parameters and position Under these conditions, the model renders a depth image. The probability of.

[0039] (3) Iterative selection of the best observation angle.

[0040] In each iteration From the perspective of candidate set Choose the next best perspective This satisfies the objective function of maximizing the total Fisher information gain, as shown in Equation (5): This step ensures that the selected viewpoint can maximize the elimination of the current model. Uncertainty.

[0041] Among them, the candidate view set By combining the forward and inverse kinematics models of the robotic arm, joint physical limits, and anti-collision constraints, uniform or gridded discrete sampling calculations are performed within the reachable manifold workspace of the robotic arm's end effector.

[0042] (4) Data preprocessing and point cloud initialization.

[0043] This invention preprocesses the acquired data to improve accuracy. The acquired data refers to the RGB color image sequence containing the target object and the end effector of the robotic arm, along with their corresponding initial depth image sequence, captured by a fixed camera at each selected viewing angle. New data needs to be acquired in each iteration.

[0044] (4.1) First, foreground image separation is performed, using a strategy that fuses depth estimation from multiple frames (e.g., Figure 3 As shown in the figure, it can accurately extract foreground objects (the grasped object and the end of the robotic arm) and overcome the limitations of single-frame segmentation methods.

[0045] The strategy for fusing depth estimates from multiple frames specifically includes the following steps: a) The acquired RGB images from each viewpoint are processed using a monocular depth estimation network to extract the initial monocular depth map sequence.

[0046] b) Based on the physical motion characteristics of the MOSC mode, perform pixel-level statistical significance calculation (such as depth change rate or variance) on the time dimension of multi-frame depth image sequences.

[0047] c) Set a dynamic saliency threshold to identify regions where the depth value changes significantly over time as moving foreground (target object and robotic arm) and static regions as background, thereby generating a high-precision foreground mask to eliminate background interference.

[0048] (4.2) Next, global scale-offset alignment is performed. A multi-frame joint optimization strategy is adopted to solve the scale ambiguity of monocular depth estimation and align the local depth map to a unified global coordinate system.

[0049] The multi-frame joint optimization strategy specifically includes the following steps: a) Extract the predicted depth data output by the monocular depth estimation network and match it with the physical observation depth (or reference depth) between multiple frames of images.

[0050] b) Establish a linear geometric alignment model for the global scale factor s and the depth offset t.

[0051] c) Construct a nonlinear optimization objective function based on the least squares method: in and They represent the first i The scale factor and offset of the image. p These are key pixels in the depth map. This represents the predicted depth for each frame. This represents the corresponding true depth map.

[0052] d) Solve the objective function to obtain the optimal scale factor and offset, and align the local monocular predicted depth maps to a global coordinate system with absolute physical scale.

[0053] (4.3) Finally, camera extrinsic parameter optimization is performed using a feature point matching-based process for a fixed camera coordinate system. Relative to the world coordinate system The extrinsic parameters are corrected, and the camera pose before and after optimization is compared, for example... Figure 4 As shown. This step refines the pose by minimizing the reprojection error of multi-view feature points, and specifically includes the following steps: a) Use feature extraction and matching algorithms to extract and associate shared two-dimensional feature points in a sequence of observation images from multiple frames at different viewpoints.

[0054] b) Parameterize the camera extrinsic residuals to be optimized into rigid body transformation error vector ε in the Lie algebra space se(3).

[0055] c) Based on the corresponding three-dimensional spatial coordinates, establish a reprojection error model from three-dimensional spatial points to two-dimensional image planes from various perspectives.

[0056] d) Using the Gauss-Newton method or the Levenberg-Marquardt (LM) algorithm, the error vector ε is iteratively updated in the Lie algebra space until the reprojection error is minimized, thereby outputting the corrected high-precision camera extrinsic pose.

[0057] Point cloud initialization refers to using the initial two-dimensional image data acquired in the previous steps, the scale-aligned depth map, and the camera pose optimized and calibrated by extrinsic parameters, to back-project the pixels on the two-dimensional image into three-dimensional space, generating an initial three-dimensional point cloud set.

[0058] (5) Update the three-dimensional reconstruction model.

[0059] Choose the perspective The preprocessed data is incorporated into the observation dataset (as a training set), and the 3D reconstruction model of the target object is updated using a 3D reconstruction algorithm to obtain a new 3D reconstruction model. Repeat steps (3) to (5) until the preset number of iterations or reconstruction quality index is reached.

[0060] After the training process in steps (1) to (5) is completed, the robot uses the three-dimensional reconstruction model of the target object to perform object manipulation.

[0061] The three-dimensional reconstruction model in this invention can be a scene model based on explicit neural rendering representations such as 3D Gaussian Splatting (3DGS).

[0062] The 3D reconstruction algorithm in this invention can be an algorithm that projects a set of 3D Gaussian primitives into a 2D image using differentiable rasterization technology, and uses backpropagation and gradient descent algorithms to incrementally iteratively optimize the 3D model parameters based on the loss value between the actual image and the rendered image (i.e., the aforementioned negative log-likelihood loss).

[0063] Experiments demonstrate that, with only approximately 18 carefully selected viewpoint images actively acquired (equivalent to 18.75% of the total sampled data), the peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) of the rendered images from the 3D reconstruction model of this invention can reach or approach the upper limit performance (e.g., achieving over 80% of the full sampling performance) of training with 96 densely sampled images. This data significantly verifies that this method effectively ensures high-quality 3D reconstruction accuracy while greatly reducing data acquisition redundancy.

[0064] Another embodiment of the present invention provides a robotic arm object observation angle prediction device based on Fisher information, such as... Figure 5 As shown, it includes: The modeling initialization module is used to define the world coordinate system, camera coordinate system, and robotic arm end effector coordinate system using the moving object-fixed camera perception paradigm, and to acquire images from a predefined set of initial viewpoints to initialize the 3D reconstruction model. The Fisher information calculation module is used to construct the color Fisher information gain and the depth Fisher information gain and then perform a weighted fusion to obtain the total Fisher information gain. The optimal viewpoint iterative selection module is used to select the observation viewpoint that maximizes the total Fisher information gain from the candidate viewpoint set as the best observation viewpoint in each iteration. The image preprocessing module is used to perform preprocessing on the RGB image and depth image acquired under the best viewing angle. The preprocessing includes foreground image separation, global scale-offset alignment and dynamic correction of camera extrinsic parameters. The incremental update module for the reconstruction model is used to incorporate the preprocessed observation data from the best viewing angle into the training set and incrementally update the 3D reconstruction model until the iteration termination condition is met. The task execution module is used to complete the robotic arm's operation tasks on the target object using the trained 3D reconstruction model.

[0065] The above division of modules is merely illustrative. In practical applications, the functions described above can be assigned to different functional modules as needed to complete all or part of the functions described in the aforementioned method. The specific working process of each module can be found in the corresponding process in the aforementioned method embodiments. Each module can be implemented entirely or partially through software, hardware, or a combination thereof.

[0066] Another embodiment of the present invention provides a computer device (computer, server, etc.) including a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing steps of the method of the present invention.

[0067] Another embodiment of the present invention provides a computer-readable storage medium (such as ROM / RAM, disk, optical disk) that stores a computer program, which, when executed by a computer, implements the steps of the method of the present invention.

[0068] Another embodiment of the present invention provides a computer program product, the computer program product including a computer program, which, when executed by a computer, implements the steps of the method of the present invention.

[0069] The above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Those skilled in the art can modify or make equivalent substitutions to the technical solutions of the present invention. The scope of protection of the present invention is defined by the claims.

Claims

1. A method for predicting the object observation angle of a robotic arm based on Fisher information, characterized in that, Includes the following steps: The world coordinate system, camera coordinate system, and robotic arm end effector coordinate system are defined using a moving object-fixed camera perception paradigm. Images are acquired from a predefined set of initial viewpoints to initialize the 3D reconstruction model. Construct the color Fisher information gain and the depth Fisher information gain and then perform a weighted fusion to obtain the total Fisher information gain; In each iteration, the observation view that maximizes the total Fisher information gain is selected from the candidate view set as the best observation view. Preprocessing is performed on the RGB and depth images acquired at the optimal viewing angle. The preprocessing includes foreground image separation, global scale-offset alignment, and dynamic correction of camera extrinsic parameters. The preprocessed observation data from the best observation perspective is incorporated into the training set, and the 3D reconstruction model is updated incrementally until the iteration termination condition is met. The trained 3D reconstruction model is used to complete the robotic arm's operation on the target object.

2. The method according to claim 1, characterized in that, In the moving object-fixed camera perception paradigm, the camera coordinate system is fixed relative to the world coordinate system, and the viewing angle is determined by the pose of the end effector of the robotic arm.

3. The method according to claim 1, characterized in that, The construction of color Fisher information gain and depth Fisher information gain includes: constructing negative log-likelihood loss functions based on the color rendering function and the depth rendering function respectively, and approximating the color and depth Fisher information matrices by using the diagonal terms of the Hessian matrix corresponding to the loss function.

4. The method according to claim 1, characterized in that, By combining the forward and inverse kinematics models of the robotic arm, joint physical limits, and anti-collision constraints, uniform or gridded discrete sampling calculations are performed within the reachable manifold workspace of the robotic arm's end effector to obtain the candidate viewpoint set.

5. The method according to claim 1, characterized in that, The foreground image separation includes: using a monocular depth estimation network to extract initial monocular depth map sequences from the acquired RGB images from each viewpoint; performing pixel-level statistical significance difference calculations on the multi-frame depth image sequences in the time dimension; setting a dynamic significance threshold to determine regions where the depth value changes significantly over time as moving foreground and static regions as background, thereby generating a high-precision foreground mask to eliminate background interference.

6. The method according to claim 1, characterized in that, The global scale-offset alignment includes: establishing a linear geometric alignment model with scale factor and depth offset, using a nonlinear optimization objective function based on least squares to solve for the optimal scale factor and offset, and aligning the various local monocular predicted depth maps to a global coordinate system with absolute physical scale.

7. The method according to claim 1, characterized in that, The dynamic correction of camera extrinsic parameters includes: extracting and associating shared two-dimensional feature points from a sequence of multiple frames of observed images from different perspectives; parameterizing the residuals of the camera extrinsic parameters to be optimized into rigid body transformation error vectors in Lie algebra space; establishing reprojection error models from three-dimensional spatial points to two-dimensional image planes from various perspectives; and iteratively updating the rigid body transformation error vectors in Lie algebra space using the Gauss-Newton method or the LM algorithm until the reprojection error is minimized, thereby obtaining the corrected high-precision camera extrinsic pose.

8. A robotic arm object observation angle prediction device based on Fisher information, characterized in that, include: The modeling initialization module is used to define the world coordinate system, camera coordinate system, and robotic arm end effector coordinate system using the moving object-fixed camera perception paradigm, and to acquire images from a predefined set of initial viewpoints to initialize the 3D reconstruction model. The Fisher information calculation module is used to construct the color Fisher information gain and the depth Fisher information gain and then perform a weighted fusion to obtain the total Fisher information gain. The optimal viewpoint iterative selection module is used to select the observation viewpoint that maximizes the total Fisher information gain from the candidate viewpoint set as the best observation viewpoint in each iteration. The image preprocessing module is used to perform preprocessing on the RGB image and depth image acquired under the best viewing angle. The preprocessing includes foreground image separation, global scale-offset alignment and dynamic correction of camera extrinsic parameters. The incremental update module for the reconstruction model is used to incorporate the preprocessed observation data from the best viewing angle into the training set and incrementally update the 3D reconstruction model until the iteration termination condition is met. The task execution module is used to complete the robotic arm's operation tasks on the target object using the trained 3D reconstruction model.

9. A computer device, characterized in that, It includes a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing the method of any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a computer, implements the method according to any one of claims 1 to 7.

Images