Machine vision-based mechanical dog quality distribution analysis method

Abstract

This invention discloses a machine vision-based method for analyzing the mass distribution of a robotic dog, comprising the following steps: obtaining complete 3D digital model data of the robotic dog; generating an overall mass mapping matrix of the robotic dog based on material properties and geometric volume parameters; generating an optimized depth map using an improved ZoeDepth model; backprojecting the optimized depth map into dense 3D point cloud data of the robotic dog; forming a coarse registration result for the robotic dog; obtaining the final error measurement result using IPC and Chamfer Distance algorithms; obtaining an updated 3D digital model of the robotic dog; forming a 3D mass distribution dataset; generating a heat map of the robotic dog's mass distribution, a scatter plot of the robotic dog's centroid, and a vector map of the robotic dog's centroid, and performing visualization and data storage, thereby achieving non-destructive testing and high-precision modeling analysis of the internal mass distribution of the robotic dog, with advantages such as no need for physical sensors, high modeling accuracy, and excellent visualization effects.

Description

Technical Field

[0001] This invention relates to the field of intelligent robot structure inspection and quality analysis, and in particular to a machine vision-based method for analyzing the mass distribution of a mechanical dog. Background Technology

[0002] Quadrupedal robotic dogs, as an important branch of mobile robotics, have been widely used in recent years in scenarios such as security patrol, complex environment detection, logistics handling, and rescue. Compared with traditional wheeled or tracked mobile platforms, robotic dogs have stronger terrain adaptability and posture flexibility. However, their multi-degree-of-freedom joint structure and complex dynamic characteristics determine that the entire machine is highly dependent on mass distribution during movement. The mass distribution of the robotic dog directly affects gait stability, motion control accuracy, and energy consumption efficiency during movement, turning, and load changes. Abnormal mass distribution in key components may lead to problems such as center of gravity shift, gait instability, and increased vibration, and in severe cases, even falls or system failures. Therefore, how to achieve accurate analysis and dynamic monitoring of the overall and local mass distribution of the robotic dog has become a key technical issue to ensure its safety and reliability.

[0003] Currently, existing technologies primarily assess the state of robotic robots through sensor measurements or visual inspection. Some research institutions utilize multi-sensor fusion methods, acquiring posture information via inertial measurement units, pressure sensors, and joint encoders to achieve motion state perception. However, these methods are often limited to monitoring external motion parameters and struggle to reflect changes in the internal structural mass distribution. For example, while the quadruped robot state monitoring system based on multi-sensor fusion published by Beijing Institute of Technology can achieve posture detection and tilt angle estimation, it does not involve mass distribution estimation; the mobile robot visual inspection system proposed by Tsinghua University is mainly used for surface deformation and appearance defect identification, lacking the ability to infer internal mass characteristics and perform structural mechanics analysis. Traditional methods often fail to obtain complete mass mapping relationships when dealing with complex structures, densely connected components, or multi-material mechanical systems, making it difficult to visualize and accurately model dynamic mass distribution. Summary of the Invention

[0004] One objective of this invention is to propose a machine vision-based method for analyzing the quality distribution of mechanical dogs. This invention fully utilizes machine vision, an improved ZoeDepth model, 3D reconstruction, and point cloud registration techniques. Addressing the problems of strong sensor dependence, poor structural adaptability, insufficient error correction accuracy, and lack of visualization feedback in existing mechanical dog quality detection methods, this invention constructs a systematic analysis process from visual data acquisition, model reconstruction, error correction, quality analysis, and visualization.

[0005] The machine vision-based method for analyzing the mass distribution of a mechanical dog according to an embodiment of the present invention includes the following steps:

[0006] Establish a three-dimensional digital model of the mechanical dog and obtain complete three-dimensional digital model data of the mechanical dog;

[0007] Based on the material properties and geometric volume parameters in the three-dimensional digital model data of the mechanical dog, the mass parameters of each component are calculated, and the overall mass mapping matrix of the mechanical dog is generated.

[0008] Based on the 3D digital model of the mechanical dog, preset viewpoint parameters are set, and an optimized depth map is generated using the improved ZoeDepth model.

[0009] The optimized depth map is back-projected into dense 3D point cloud data for the robotic dog based on the camera's intrinsic parameters.

[0010] An initial coordinate alignment operation is performed on the 3D digital model of the robot dog and the dense 3D point cloud data of the robot dog to complete the coarse registration of the 3D model of the robot dog and the point cloud, thus forming the coarse registration result of the robot dog.

[0011] The IPC and ChamferDistance algorithms were used to perform fine registration and error calculation on the coarse registration results of the mechanical dog, and the final error measurement results were obtained.

[0012] The local feature parameters of the three-dimensional digital model of the mechanical dog are updated based on the final error measurement results. The assembly deviation area and deformation area in the mechanical dog structure are identified and dynamically corrected to obtain the updated three-dimensional digital model of the mechanical dog.

[0013] Using the updated 3D digital model of the robotic dog and the overall mass mapping matrix of the robotic dog, the overall centroid coordinates, local mass parameters and principal inertial axis parameters of the robotic dog are calculated to form a 3D mass distribution dataset;

[0014] Based on the 3D mass distribution dataset, a heat map of the mechanical dog's mass distribution, a scatter plot of the mechanical dog's centroid, and a vector map of the mechanical dog's centroid are generated, and then visualized and stored.

[0015] Optionally, the 3D digital model data of the mechanical dog is obtained by importing the 3D model file of the mechanical dog and performing automatic topology analysis to extract the geometric dimensions, assembly relationships and spatial pose information of each component of the mechanical dog. The assembly relationship refers to the connection relationship formed by the torso structure, limb joint units, camera system units, sensor brackets, battery compartment and signal receiving module.

[0016] Optionally, the generation of the overall mass mapping matrix of the mechanical dog specifically includes:

[0017] Import the 3D digital model data of the mechanical dog into the preset quality modeling system, perform topological structure analysis and spatial discretization processing on the 3D digital model data of the mechanical dog, extract the geometric dimensions, assembly relationships and spatial pose parameters of each structural component of the mechanical dog, and establish a component index table.

[0018] The material database is invoked, and a corresponding material property density value is assigned to each component in the component index table. The single component mass value is calculated based on the component's geometric volume parameters. The single component mass value is equal to the product of the material property density value and the geometric volume parameters.

[0019] A mapping relationship is established between the mass value of each component and its corresponding spatial centroid coordinates. Based on the spatial coordinate system of the mechanical dog's three-dimensional digital model, an overall mass mapping matrix of the mechanical dog is constructed. The mapping relationship is established by constructing a mass-position correspondence index table based on the mass value of each component in the mechanical dog's three-dimensional digital model and its corresponding spatial centroid coordinates. The mass values ​​within the spatial coordinate units are weighted and accumulated in the mechanical dog's three-dimensional coordinate system. Each spatial unit of the mass mapping matrix represents the distribution of local mass density at the three-dimensional coordinate point. The local mass density is obtained by accumulating the mass of the components at the corresponding positions.

[0020] Optionally, the generation of the optimized depth map specifically includes:

[0021] Based on the preset viewpoint parameters set by the 3D digital model of the mechanical dog, a corresponding rendered image of the mechanical dog is generated. The rendered image of the mechanical dog is then input into the improved ZoeDepth model. The improved ZoeDepth model includes a feature extraction module, a depth fusion and scale calibration module, and a depth refinement module. The feature extraction module refers to a visual encoding unit that performs multi-level feature extraction on the rendered image of the mechanical dog to obtain a 2D feature mapping tensor of the mechanical dog. The depth fusion and scale calibration module refers to performing scale normalization and global consistency optimization on the 2D feature mapping tensor of the mechanical dog to output a relative depth map. The depth refinement module refers to post-processing the relative depth map to generate an optimized depth map.

[0022] In the feature extraction module, the rendered image of the mechanical dog is input into a visual coding network based on the Swin Transformer. Multi-scale feature extraction and fusion are performed using hierarchical convolutional embedding and windowed self-attention mechanism to obtain the two-dimensional feature mapping tensor of the mechanical dog.

[0023] In the depth fusion and scale calibration module, based on the mechanical dog's two-dimensional feature mapping tensor, the relative depth value of each pixel is calculated according to the camera focal length parameters, baseline length and pixel disparity value, and consistent normalization is performed at the global scale to output a relative depth map.

[0024] In the depth refinement module, edge refinement, noise suppression, and smoothing constraints are applied to the relative depth map to generate an optimized depth map.

[0025] Optionally, obtaining the dense 3D point cloud data of the mechanical dog specifically includes:

[0026] Perform back-projection calculation on each pixel in the optimized depth map. The back-projection calculation refers to converting two-dimensional depth information into three-dimensional camera coordinate points.

[0027] The coordinates of the 3D camera are transformed to the coordinate system of the 3D digital model of the mechanical dog using the camera intrinsic parameters. The coordinates of the 3D points in the coordinate system of the 3D digital model of the mechanical dog are obtained by rotation matrix and translation vector. The camera intrinsic parameters include camera focal length parameters and principal point coordinate parameters.

[0028] The coordinates of three-dimensional points in the coordinate system of the three-dimensional digital model of the mechanical dog are spatially aggregated to generate dense three-dimensional point cloud data of the mechanical dog.

[0029] Optionally, the formation of the coarse registration result of the mechanical dog specifically includes:

[0030] Voxel downsampling and normal estimation operations are performed on the 3D digital model of the robotic dog and the dense 3D point cloud data of the robotic dog, respectively. The local geometric feature vector of each sampling point is extracted to generate the sampling point set of the 3D digital model of the robotic dog.

[0031] Based on the spatial distribution characteristics of the sampling point set of the three-dimensional digital model of the mechanical dog and the dense three-dimensional point cloud data of the mechanical dog, the spatial centroid vector is calculated and a preliminary spatial alignment reference benchmark is established.

[0032] Based on the preliminary spatial alignment reference, the spatial covariance matrix is ​​calculated, and the initial rotation matrix and initial translation vector are obtained through the singular value decomposition method. A coarse registration homogeneous transformation matrix is ​​constructed between the 3D digital model of the robot dog and the dense 3D point cloud data of the robot dog. The spatial covariance matrix is ​​obtained by subtracting the corresponding centroid position from the coordinates of each point of the sampling point set of the 3D digital model of the robot dog and the centroid position of the dense 3D point cloud data of the robot dog, and then calculating the outer product of the coordinate vectors of the centered point set and summing and averaging all point pairs.

[0033] By performing rotation and translation transformations on the dense 3D point cloud data of the robotic dog in space using a coarse registration homogeneous transformation matrix, a preliminary spatial alignment result is obtained, forming the coarse registration result of the robotic dog. The coarse registration result of the robotic dog includes a set of spatial transformation parameters, an aligned point cloud dataset, and preliminary error evaluation indicators.

[0034] Optionally, obtaining the final error measurement result specifically includes:

[0035] The initial rotation matrix and initial translation vector in the coarse registration result of the mechanical dog are used as the initial transformation parameters for iterative nearest point registration to generate initial point cloud data;

[0036] For each point in the initial point cloud data, a nearest neighbor correspondence is established with the sampling point set of the 3D digital model of the mechanical dog using the IPC method, forming a feature correspondence set of two point sets.

[0037] Calculate the centroid vectors of the sampling point set of the 3D digital model of the mechanical dog and the dense 3D point cloud data of the mechanical dog based on the feature correspondence set, and obtain the updated rotation matrix and translation vector;

[0038] The updated rotation matrix and translation vector are used to perform spatial transformation on the dense 3D point cloud data of the mechanical dog to obtain iterated point cloud data. The average spatial residual between the iterated point cloud data and the sampling points of the 3D digital model of the mechanical dog is calculated to form the iteration error.

[0039] The convergence of the iterative registration process is determined according to the preset convergence conditions. When the error change between two consecutive iterations is less than the convergence threshold, the final registered point cloud set is formed. The optimal registered point cloud set includes the final rotation matrix, translation vector and transformed point cloud data.

[0040] Based on the final registered point cloud set and the sampling point set of the mechanical dog's 3D digital model, the Chamfer distance and root mean square residual are calculated to obtain the final error measurement result;

[0041] The final error measurement results include the final registration point cloud set, the sampling point set of the mechanical dog's 3D digital model, the final registration point cloud set, the Chamfer distance, and the root mean square residual.

[0042] Optionally, obtaining the updated 3D digital model of the mechanical dog specifically includes:

[0043] Based on the final error measurement results, residual calculation is performed on the corresponding point pairs in the sampling point set of the mechanical dog's 3D digital model and the final registration point cloud set to generate a residual vector and a corresponding scalar residual. The residual vector is the spatial coordinate difference between the final registration point and the sampling point of the mechanical dog's 3D digital model, and the scalar residual is the Euclidean distance of the residual vector.

[0044] Based on the residual vector and scalar residual, the error values ​​of each registration point pair are mapped to the corresponding spatial positions under the final registration point cloud set, generating a residual distribution mapping map of the mechanical dog structure. According to the spatial distribution characteristics of the residual distribution mapping map, assembly deviation threshold and deformation threshold are set to define assembly deviation region and deformation region respectively. When the local residual value exceeds the assembly deviation threshold, the region is marked as assembly deviation region. When the average residual value of the neighborhood exceeds the deformation threshold, the region is marked as deformation region, forming a spatial constraint set.

[0045] The local feature parameters of the mechanical dog's 3D digital model are dynamically updated by using a set of spatial constraints. Within the assembly deviation region, the coordinates of the sampling points are corrected according to the displacement update coefficient. Within the deformation region, the corresponding sampling point set is smoothed using a Laplace smoothing process according to the smoothing coefficient, thus obtaining the updated 3D digital model of the mechanical dog.

[0046] Optionally, the formation of the three-dimensional quality distribution dataset specifically includes:

[0047] The updated 3D digital model of the mechanical dog is spatially discretized to generate a set of voxels. The set of voxels includes voxel spatial coordinates, voxel volume and voxel mass. The voxel mass parameter is calculated from the mass density of the corresponding region in the overall mass mapping matrix of the mechanical dog.

[0048] The overall centroid coordinates of the mechanical dog are calculated based on a set of voxels, where the overall centroid coordinates are the weighted average of the spatial coordinates of all voxels.

[0049] The inertia tensor matrix of the mechanical dog is calculated based on the spatial distribution characteristics of the voxel set. Each component of the inertia tensor matrix is ​​calculated by the product of the squares of the voxel mass and the relative coordinates.

[0050] Eigenvalue decomposition is performed on the inertia tensor matrix to obtain the principal moments of inertia and principal axis direction parameters of the robot dog, forming the principal axis parameters. The principal moments of inertia refer to the rotational characteristics of the robot dog in different principal axis directions, and the principal axis direction parameters refer to the direction of the principal axis in three-dimensional space.

[0051] The overall space is partitioned using the updated 3D digital model of the mechanical dog. The mass, local centroid, mass density, and mass weight parameters of each local region are calculated to form local mass parameters. The overall centroid coordinates, principal inertial axis parameters, and local mass parameters are then combined to form a 3D mass distribution dataset.

[0052] Optionally, the mass distribution heatmap of the mechanical dog is a visual representation of local mass concentration areas, mass distribution uniformity, and structural equilibrium state by mapping the mass density parameters of each voxel in the three-dimensional mass distribution dataset to color gradients. The scatter plot of the mechanical dog's centroid is used to analyze the centroid shift phenomenon of assembly errors and structural imbalance by comparing the spatial distance between the local centroid and the overall centroid. The centroid vector diagram of the mechanical dog is used to perform aggregation analysis on the vector set to identify the direction of inertial deviation, mass unevenness areas, and potential deformation trends in the mechanical dog structure.

[0053] The beneficial effects of this invention are:

[0054] This invention constructs a machine vision-based method for detecting and analyzing the mass distribution of a robotic dog, overcoming the limitations of traditional methods relying on physical measurements and sensor detection. It achieves digital, intelligent, and visual assessment of the robotic dog's mass status. This method integrates 3D digital modeling, mass estimation, and an improved ZoeDepth model, enabling accurate modeling and static solution of the overall and local mass distribution of the robotic dog without the need for actual image acquisition or physical intervention. The system generates a mass mapping matrix by establishing the geometric information and material properties of each component in the 3D model of the robotic dog. Combining the ICP and Chamfer Distance algorithms, it automatically corrects model errors and accurately perceives local structural changes, effectively obtaining the overall centroid coordinates and principal inertial axis distribution of the robotic dog, significantly improving modeling accuracy and dynamic matching capabilities.

[0055] Compared to previous technologies limited to attitude detection or surface appearance recognition, this invention achieves non-destructive identification and dynamic analysis of the internal quality state of a structure. It enables real-time monitoring and automatic correction of the robot's mass offset, structural strain, and local assembly errors under complex task conditions. Furthermore, this invention introduces a multi-dimensional offset comparison and quality anomaly identification mechanism, combined with a mass distribution visualization feedback module, dynamically fusing the component's centroid position, offset heatmap, and 3D model. This allows operators to intuitively judge the robot's mass change trends and stability levels under different operating conditions. Through this mechanism, the system can not only achieve dynamic structural diagnosis and load prediction in a virtual simulation environment, but also provide structural optimization and predictive maintenance basis for robots in complex scenarios such as inspection, rescue, and handling.

[0056] Therefore, this invention has advantages such as no need for physical sensors, high diagnostic accuracy, strong structural adaptability, good result readability, and high system versatility, which can significantly improve analysis efficiency and application flexibility while ensuring detection accuracy. Through the integrated design of visual computing and quality modeling, this invention provides a new technical path for structural health monitoring, dynamic balance control, and life cycle management of robotic dogs, and has significant engineering application value and promotional significance. Attached Figure Description

[0057] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0058] Figure 1 This is an overall flowchart of the machine vision-based mechanical dog mass distribution analysis method proposed in this invention;

[0059] Figure 2 This is a schematic diagram of the module structure of the ZoeDepth model, which is based on the machine vision-based mechanical dog mass distribution analysis method proposed in this invention. Detailed Implementation

[0060] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0061] refer to Figure 1-2 The machine vision-based method for analyzing the mass distribution of a mechanical dog includes the following steps:

[0062] Establish a three-dimensional digital model of the mechanical dog and obtain complete three-dimensional digital model data of the mechanical dog;

[0063] Based on the material properties and geometric volume parameters in the three-dimensional digital model data of the mechanical dog, the mass parameters of each component are calculated, and the overall mass mapping matrix of the mechanical dog is generated.

[0064] Based on the 3D digital model of the mechanical dog, preset viewpoint parameters are set, and an optimized depth map is generated using the improved ZoeDepth model.

[0065] The optimized depth map is back-projected into dense 3D point cloud data for the robotic dog based on the camera's intrinsic parameters.

[0066] An initial coordinate alignment operation is performed on the 3D digital model of the robot dog and the dense 3D point cloud data of the robot dog to complete the coarse registration of the 3D model of the robot dog and the point cloud, thus forming the coarse registration result of the robot dog.

[0067] The IPC and ChamferDistance algorithms were used to perform fine registration and error calculation on the coarse registration results of the mechanical dog, and the final error measurement results were obtained.

[0068] The local feature parameters of the three-dimensional digital model of the mechanical dog are updated based on the final error measurement results. The assembly deviation area and deformation area in the mechanical dog structure are identified and dynamically corrected to obtain the updated three-dimensional digital model of the mechanical dog.

[0069] Using the updated 3D digital model of the robotic dog and the overall mass mapping matrix of the robotic dog, the overall centroid coordinates, local mass parameters and principal inertial axis parameters of the robotic dog are calculated to form a 3D mass distribution dataset;

[0070] Based on the 3D mass distribution dataset, a heat map of the mechanical dog's mass distribution, a scatter plot of the mechanical dog's centroid, and a vector map of the mechanical dog's centroid are generated, and then visualized and stored.

[0071] In this embodiment, the three-dimensional digital model data of the mechanical dog is obtained by importing the three-dimensional model file of the mechanical dog and performing automatic topology analysis to extract the geometric dimensions, assembly relationships and spatial pose information of each component of the mechanical dog. The assembly relationship refers to the connection relationship formed by the torso structure, limb joint units, camera system units, sensor brackets, battery compartment and signal receiving module.

[0072] In this embodiment, the generation of the overall mass mapping matrix of the mechanical dog specifically includes:

[0073] Import the 3D digital model data of the mechanical dog into the preset quality modeling system, perform topological structure analysis and spatial discretization processing on the 3D digital model data of the mechanical dog, extract the geometric dimensions, assembly relationships and spatial pose parameters of each structural component of the mechanical dog, and establish a component index table.

[0074] The material database is invoked, and a corresponding material property density value is assigned to each component in the component index table. The single component mass value is calculated based on the component's geometric volume parameters. The single component mass value is equal to the product of the material property density value and the geometric volume parameters.

[0075] A mapping relationship is established between the mass value of each component and its corresponding spatial centroid coordinates. Based on the spatial coordinate system of the mechanical dog's three-dimensional digital model, an overall mass mapping matrix of the mechanical dog is constructed. The mapping relationship is established by constructing a mass-position correspondence index table based on the mass value of each component in the mechanical dog's three-dimensional digital model and its corresponding spatial centroid coordinates. The mass values ​​within the spatial coordinate units are weighted and accumulated in the mechanical dog's three-dimensional coordinate system. Each spatial unit of the mass mapping matrix represents the distribution of local mass density at the three-dimensional coordinate point. The local mass density is obtained by accumulating the mass of the components at the corresponding positions.

[0076] In this embodiment, the generation of the optimized depth map specifically includes:

[0077] Based on the preset viewpoint parameters set by the 3D digital model of the mechanical dog, a corresponding rendered image of the mechanical dog is generated. The rendered image of the mechanical dog is then input into the improved ZoeDepth model. The improved ZoeDepth model includes a feature extraction module, a depth fusion and scale calibration module, and a depth refinement module. The feature extraction module refers to a visual encoding unit that performs multi-level feature extraction on the rendered image of the mechanical dog to obtain a 2D feature mapping tensor of the mechanical dog. The depth fusion and scale calibration module refers to performing scale normalization and global consistency optimization on the 2D feature mapping tensor of the mechanical dog to output a relative depth map. The depth refinement module refers to post-processing the relative depth map to generate an optimized depth map.

[0078] In the feature extraction module, the rendered image of the mechanical dog is input into a visual coding network based on the Swin Transformer. Multi-scale feature extraction and fusion are performed using hierarchical convolutional embedding and windowed self-attention mechanism to obtain the two-dimensional feature mapping tensor of the mechanical dog.

[0079] In the depth fusion and scale calibration module, based on the mechanical dog's two-dimensional feature mapping tensor, the relative depth value of each pixel is calculated according to the camera focal length parameters, baseline length, and pixel disparity values. Consistent normalization is then performed at the global scale to output a relative depth map.

[0080] ;

[0081] in, pixel coordinates The relative depth map For relative indexes, For camera focal length parameters, Baseline length This represents the disparity value.

[0082] In the depth refinement module, edge thinning, noise suppression, and smoothing constraints are applied to the relative depth map to generate an optimized depth map:

[0083] ;

[0084] in, This is the final depth map of the mechanical dog. For depth preservation coefficient, For smoothing adjustment coefficient, is the second-order Laplacian smoothing term of the relative depth map.

[0085] In this embodiment, obtaining the dense 3D point cloud data of the mechanical dog specifically includes:

[0086] Perform backprojection calculations on each pixel in the optimized depth map, whereby the backprojection calculations refer to converting two-dimensional depth information into three-dimensional camera coordinates:

[0087] ;

[0088] in,( , , () represents the three-dimensional coordinates of a pixel in the camera coordinate system. , Here are the focal length parameters of the camera in the x and y directions. , Principal point coordinate parameters;

[0089] The coordinates of the 3D camera are transformed to the coordinate system of the 3D digital model of the mechanical dog using the camera intrinsic parameters. The coordinates of the 3D points in the coordinate system of the 3D digital model of the mechanical dog are obtained by rotation matrix and translation vector. The camera intrinsic parameters include camera focal length parameters and principal point coordinate parameters.

[0090] The coordinates of three-dimensional points in the coordinate system of the three-dimensional digital model of the mechanical dog are spatially aggregated to generate dense three-dimensional point cloud data of the mechanical dog.

[0091] In this embodiment, the formation of the coarse registration result of the mechanical dog specifically includes:

[0092] Voxel downsampling and normal estimation operations are performed on the 3D digital model of the robotic dog and the dense 3D point cloud data of the robotic dog, respectively. The local geometric feature vector of each sampling point is extracted to generate the sampling point set of the 3D digital model of the robotic dog.

[0093] Based on the spatial distribution characteristics of the sampling point set of the three-dimensional digital model of the mechanical dog and the dense three-dimensional point cloud data of the mechanical dog, the spatial centroid vector is calculated and a preliminary spatial alignment reference benchmark is established.

[0094] Based on the preliminary spatial alignment reference, the spatial covariance matrix is ​​calculated, and the initial rotation matrix and initial translation vector are obtained through the singular value decomposition method. A coarse registration homogeneous transformation matrix is ​​constructed between the 3D digital model of the robot dog and the dense 3D point cloud data of the robot dog. The spatial covariance matrix is ​​obtained by subtracting the corresponding centroid position from the coordinates of each point of the sampling point set of the 3D digital model of the robot dog and the centroid position of the dense 3D point cloud data of the robot dog, and then calculating the outer product of the coordinate vectors of the centered point set and summing and averaging all point pairs.

[0095] By performing rotation and translation transformations on the dense 3D point cloud data of the robotic dog in space using a coarse registration homogeneous transformation matrix, a preliminary spatial alignment result is obtained, forming the coarse registration result of the robotic dog. The coarse registration result of the robotic dog includes a set of spatial transformation parameters, an aligned point cloud dataset, and preliminary error evaluation indicators.

[0096] In this embodiment, obtaining the final error measurement result specifically includes:

[0097] The initial rotation matrix and initial translation vector in the coarse registration result of the mechanical dog are used as the initial transformation parameters for iterative nearest point registration to generate initial point cloud data;

[0098] For each point in the initial point cloud data, a nearest neighbor correspondence is established with the sampling point set of the 3D digital model of the mechanical dog using the IPC method, forming a feature correspondence set of two point sets.

[0099] Calculate the centroid vectors of the sampling point set of the 3D digital model of the mechanical dog and the dense 3D point cloud data of the mechanical dog based on the feature correspondence set, and obtain the updated rotation matrix and translation vector;

[0100] The updated rotation matrix and translation vector are used to perform spatial transformation on the dense 3D point cloud data of the mechanical dog to obtain iterated point cloud data. The average spatial residual between the iterated point cloud data and the sampling points of the 3D digital model of the mechanical dog is calculated to form the iteration error.

[0101] The convergence of the iterative registration process is determined according to the preset convergence conditions. When the error change between two consecutive iterations is less than the convergence threshold, the final registered point cloud set is formed. The optimal registered point cloud set includes the final rotation matrix, translation vector and transformed point cloud data.

[0102] Based on the final registered point cloud set and the sampling point set of the mechanical dog's 3D digital model, the Chamfer distance and root mean square residual are calculated to obtain the final error metric results:

[0103]

[0104] in, To determine the Chamfer distance between the final registration point cloud and the sampling point set of the robotic dog's 3D digital model, For two points in a 3D point cloud, The square of the Euclidean distance. To minimize the computation, For the final registered point cloud set, This is the sampling point set for the three-dimensional digital model of the mechanical dog.

[0105] The final error measurement results include the final registration point cloud set, the sampling point set of the mechanical dog's 3D digital model, the final registration point cloud set, the Chamfer distance, and the root mean square residual.

[0106] In this embodiment, obtaining the updated three-dimensional digital model of the mechanical dog specifically includes:

[0107] Based on the final error measurement results, residual calculation is performed on the corresponding point pairs in the sampling point set of the mechanical dog's 3D digital model and the final registration point cloud set to generate a residual vector and a corresponding scalar residual. The residual vector is the spatial coordinate difference between the final registration point and the sampling point of the mechanical dog's 3D digital model, and the scalar residual is the Euclidean distance of the residual vector.

[0108] Based on the residual vector and scalar residual, the error values ​​of each registration point pair are mapped to the corresponding spatial positions under the final registration point cloud set, generating a residual distribution mapping map of the mechanical dog structure. According to the spatial distribution characteristics of the residual distribution mapping map, assembly deviation threshold and deformation threshold are set to define assembly deviation region and deformation region respectively. When the local residual value exceeds the assembly deviation threshold, the region is marked as assembly deviation region. When the average residual value of the neighborhood exceeds the deformation threshold, the region is marked as deformation region, forming a spatial constraint set.

[0109] The local feature parameters of the mechanical dog's 3D digital model are dynamically updated by using a set of spatial constraints. Within the assembly deviation region, the coordinates of the sampling points are corrected according to the displacement update coefficient. Within the deformation region, the corresponding sampling point set is smoothed using a Laplace smoothing process according to the smoothing coefficient, thus obtaining the updated 3D digital model of the mechanical dog.

[0110] In this embodiment, the formation of the three-dimensional quality distribution dataset specifically includes:

[0111] The updated 3D digital model of the mechanical dog is spatially discretized to generate a set of voxels. The set of voxels includes voxel spatial coordinates, voxel volume and voxel mass. The voxel mass parameter is calculated from the mass density of the corresponding region in the overall mass mapping matrix of the mechanical dog.

[0112] The overall centroid coordinates of the mechanical dog are calculated based on a set of voxels, where the overall centroid coordinates are the weighted average of the spatial coordinates of all voxels.

[0113] The inertia tensor matrix of the mechanical dog is calculated based on the spatial distribution characteristics of the voxel set. Each component of the inertia tensor matrix is ​​calculated by the product of the squares of the voxel mass and the relative coordinates.

[0114] Eigenvalue decomposition is performed on the inertia tensor matrix to obtain the principal moments of inertia and principal axis direction parameters of the robot dog, forming the principal axis parameters. The principal moments of inertia refer to the rotational characteristics of the robot dog in different principal axis directions, and the principal axis direction parameters refer to the direction of the principal axis in three-dimensional space.

[0115] The overall space is partitioned using the updated 3D digital model of the mechanical dog. The mass, local centroid, mass density, and mass weight parameters of each local region are calculated to form local mass parameters. The overall centroid coordinates, principal inertial axis parameters, and local mass parameters are then combined to form a 3D mass distribution dataset.

[0116] In this embodiment, the mass distribution heatmap of the mechanical dog is a visual representation of local mass concentration areas, mass distribution uniformity, and structural equilibrium state by mapping the mass density parameters of each voxel in the three-dimensional mass distribution dataset to color gradients. The scatter plot of the mechanical dog's centroid is used to analyze the centroid shift phenomenon of assembly errors and structural imbalance by comparing the spatial distance between the local centroid and the overall centroid. The centroid vector diagram of the mechanical dog is used to perform aggregation analysis on the vector set to identify the direction of inertial deviation, mass unevenness areas, and potential deformation trends in the mechanical dog structure.

[0117] Example 1:

[0118] This embodiment uses a certain model of quadrupedal robotic dog as the research object to verify the feasibility and superiority of a machine vision-based method for analyzing the mass distribution of the robotic dog. The robotic dog adopts a composite structure of aluminum-magnesium alloy and carbon fiber, with overall dimensions of approximately 0.9m in length, 0.45m in width, and 0.5m in height, and a total weight of approximately 32.5kg. Traditional methods rely on installing accelerometers and ground pressure detection devices to measure changes in the center of mass. This approach is complex to implement, has a slow response time, and cannot detect changes in internal mass distribution. The machine vision-based detection method provided by this invention achieves non-contact, high-precision visual analysis of the overall and local mass distribution of the robotic dog, effectively overcoming the limitations of traditional methods.

[0119] In the specific implementation, a complete 3D digital model is first established using the mechanical dog's CAD model. The geometric volume and material properties of each structural component are extracted to generate the overall mass mapping matrix of the mechanical dog. By setting a preset viewing angle, a multi-view depth map is generated using an improved ZoeDepth visual depth model, and back-projection is performed based on camera intrinsic parameters to obtain dense 3D point cloud data. Subsequently, high-precision registration between the digital model and the point cloud is achieved through IPC and Chamfer Distance algorithms, keeping the spatial error stably controlled within 0.35mm. Comparison shows that the traditional IMU-based centroid estimation error is approximately 1.2mm, while the error of this method is reduced to 0.38mm, improving accuracy by approximately 68%.

[0120] In multi-condition experiments, known mass blocks were loaded onto the main control compartment, battery compartment, and front leg modules of the mechanical dog, and the overall center of mass change was calculated using the method of this invention. The results show that under different loading conditions, the center of mass offset predicted by the method of this invention differs from the measured results by no more than 1.5%, which is far superior to the detection error of traditional sensors. The system can also automatically detect local mass unevenness caused by assembly deviations. For example, when a 0.3mm assembly gap offset is applied at the left hind leg joint, the system automatically identifies a local mass anomaly in that area, with the deviation rate increasing by 2.3%, and accurately highlights the area in red on the mass distribution heatmap.

[0121] Furthermore, the method of this invention iteratively corrects the 3D model of the robotic dog through a dynamic update mechanism, automatically adjusting local feature parameters and inertia distribution. After dynamic correction, the model's moment of inertia error decreased to 0.9%, and the stability of the center of gravity improved by 14.7%. In complex terrain simulation, attitude control based on mass distribution feedback reduced the robotic dog's stable recovery time from 4.1 seconds in the original system to 2.5 seconds, and the vibration amplitude decreased by nearly 45%. These data fully demonstrate that this invention has significant advantages in 3D mass modeling accuracy, structural stability detection, and dynamic control capabilities.

[0122] The embodiments verify that the present invention can complete accurate modeling and visualization analysis of the mass distribution of a mechanical dog structure without the need for physical sensors and external measuring equipment. This method automates the entire process from model construction and depth vision recognition to error correction and mass characteristic extraction, possessing characteristics such as high precision, reusability, and ease of deployment, providing a reliable technical foundation for mechanical dog structure optimization, assembly inspection, and operation and maintenance.

[0123] Table 1 Comparison of Mass Distribution Test Results for Mechanical Dogs

[0124]

[0125] As shown in Table 1, the machine vision-based mass distribution detection method for robotic dogs proposed in this invention exhibits high accuracy and stability under various working conditions. In the loading scenarios of the main control cabin and battery compartment, the centroid offset prediction error is controlled within 1.5%, significantly better than the approximately 7% error level of traditional detection methods. Under complex terrain conditions, the system adjusts the attitude control parameters through visual mass distribution feedback, reducing the robotic dog's recovery time by approximately 39% and significantly improving motion stability. Simultaneously, the entire detection process does not rely on any physical sensors and is entirely calculated through a visual model, greatly reducing detection complexity and maintenance costs. These results demonstrate that the method of this invention can achieve high-precision mass distribution analysis and structural health monitoring in both virtual and physical environments, providing a solid technical foundation for the self-sensing and self-correction of intelligent mechanical platforms.

Claims

1. A machine vision-based method for analyzing the mass distribution of a mechanical dog, characterized in that, Includes the following steps: Establish a three-dimensional digital model of the mechanical dog and obtain complete three-dimensional digital model data of the mechanical dog; Based on the material properties and geometric volume parameters in the three-dimensional digital model data of the mechanical dog, the mass parameters of each component are calculated, and the overall mass mapping matrix of the mechanical dog is generated. Based on the 3D digital model of the mechanical dog, preset viewpoint parameters are set, and an optimized depth map is generated using the improved ZoeDepth model. The optimized depth map is back-projected into dense 3D point cloud data for the robotic dog based on the camera's intrinsic parameters. An initial coordinate alignment operation is performed on the 3D digital model of the robot dog and the dense 3D point cloud data of the robot dog to complete the coarse registration of the 3D model of the robot dog and the point cloud, thus forming the coarse registration result of the robot dog. The IPC and Chamfer Distance algorithms were used to perform fine registration and error calculation on the coarse registration results of the mechanical dog, and the final error measurement results were obtained. The local feature parameters of the three-dimensional digital model of the mechanical dog are updated based on the final error measurement results. The assembly deviation area and deformation area in the mechanical dog structure are identified and dynamically corrected to obtain the updated three-dimensional digital model of the mechanical dog. Using the updated 3D digital model of the robotic dog and the overall mass mapping matrix of the robotic dog, the overall centroid coordinates, local mass parameters and principal inertial axis parameters of the robotic dog are calculated to form a 3D mass distribution dataset; Based on the 3D mass distribution dataset, a heat map of the mechanical dog's mass distribution, a scatter plot of the mechanical dog's centroid, and a vector map of the mechanical dog's centroid are generated, and then visualized and stored.

2. The machine vision-based mechanical dog mass distribution analysis method according to claim 1, characterized in that, The mechanical dog's three-dimensional digital model data is obtained by importing the mechanical dog's three-dimensional model file and performing automatic topology analysis to extract the geometric dimensions, assembly relationships, and spatial pose information of each component of the mechanical dog. The assembly relationship refers to the connection relationship formed by the torso structure, limb joint units, camera system units, sensor brackets, battery compartment, and signal receiving module.

3. The machine vision-based mechanical dog mass distribution analysis method according to claim 1, characterized in that, The generation of the overall mass mapping matrix of the mechanical dog specifically includes: Import the 3D digital model data of the mechanical dog into the preset quality modeling system, perform topological structure analysis and spatial discretization processing on the 3D digital model data of the mechanical dog, extract the geometric dimensions, assembly relationships and spatial pose parameters of each structural component of the mechanical dog, and establish a component index table. The material database is invoked, and a corresponding material property density value is assigned to each component in the component index table. The single component mass value is calculated based on the component's geometric volume parameters. The single component mass value is equal to the product of the material property density value and the geometric volume parameters. A mapping relationship is established between the mass value of each component and its corresponding spatial centroid coordinates. Based on the spatial coordinate system of the mechanical dog's three-dimensional digital model, an overall mass mapping matrix of the mechanical dog is constructed. The mapping relationship is established by constructing a mass-position correspondence index table based on the mass value of each component in the mechanical dog's three-dimensional digital model and its corresponding spatial centroid coordinates. The mass values ​​within the spatial coordinate units are weighted and accumulated in the mechanical dog's three-dimensional coordinate system. Each spatial unit of the mass mapping matrix represents the distribution of local mass density at the three-dimensional coordinate point. The local mass density is obtained by accumulating the mass of the components at the corresponding positions.

4. The machine vision-based mechanical dog mass distribution analysis method according to claim 1, characterized in that, The generation of the optimized depth map specifically includes: Based on the preset viewpoint parameters set by the 3D digital model of the mechanical dog, a corresponding rendered image of the mechanical dog is generated. The rendered image of the mechanical dog is then input into the improved ZoeDepth model. The improved ZoeDepth model includes a feature extraction module, a depth fusion and scale calibration module, and a depth refinement module. The feature extraction module refers to a visual encoding unit that performs multi-level feature extraction on the rendered image of the mechanical dog to obtain a 2D feature mapping tensor of the mechanical dog. The depth fusion and scale calibration module refers to performing scale normalization and global consistency optimization on the 2D feature mapping tensor of the mechanical dog to output a relative depth map. The depth refinement module refers to post-processing the relative depth map to generate an optimized depth map. In the feature extraction module, the rendered image of the mechanical dog is input into a visual coding network based on the Swin Transformer. Multi-scale feature extraction and fusion are performed using hierarchical convolutional embedding and windowed self-attention mechanism to obtain the two-dimensional feature mapping tensor of the mechanical dog. In the depth fusion and scale calibration module, based on the mechanical dog's two-dimensional feature mapping tensor, the relative depth value of each pixel is calculated according to the camera focal length parameters, baseline length and pixel disparity value, and consistent normalization is performed at the global scale to output a relative depth map. In the depth refinement module, edge refinement, noise suppression, and smoothing constraints are applied to the relative depth map to generate an optimized depth map.

5. The machine vision-based mechanical dog mass distribution analysis method according to claim 1, characterized in that, The acquisition of the dense 3D point cloud data of the mechanical dog specifically includes: Perform back-projection calculation on each pixel in the optimized depth map. The back-projection calculation refers to converting two-dimensional depth information into three-dimensional camera coordinate points. The coordinates of the 3D camera are transformed to the coordinate system of the 3D digital model of the mechanical dog using the camera intrinsic parameters. The coordinates of the 3D points in the coordinate system of the 3D digital model of the mechanical dog are obtained by using the rotation matrix and translation vector. The camera intrinsic parameters include the camera focal length parameter and the principal point coordinate parameter. The coordinates of three-dimensional points in the coordinate system of the three-dimensional digital model of the mechanical dog are spatially aggregated to generate dense three-dimensional point cloud data of the mechanical dog.

6. The machine vision-based mechanical dog mass distribution analysis method according to claim 1, characterized in that, The formation of the coarse registration result of the robotic dog specifically includes: Voxel downsampling and normal estimation operations are performed on the 3D digital model of the mechanical dog and the dense 3D point cloud data of the mechanical dog, respectively. The local geometric feature vector of each sampling point is extracted to generate the sampling point set of the 3D digital model of the mechanical dog. Based on the spatial distribution characteristics of the sampling point set of the three-dimensional digital model of the mechanical dog and the dense three-dimensional point cloud data of the mechanical dog, the spatial centroid vector is calculated and a preliminary spatial alignment reference benchmark is established. Based on the preliminary spatial alignment reference, the spatial covariance matrix is ​​calculated, and the initial rotation matrix and initial translation vector are obtained through the singular value decomposition method. A coarse registration homogeneous transformation matrix is ​​constructed between the 3D digital model of the robot dog and the dense 3D point cloud data of the robot dog. The spatial covariance matrix is ​​obtained by subtracting the corresponding centroid position from the coordinates of each point of the sampling point set of the 3D digital model of the robot dog and the centroid position of the dense 3D point cloud data of the robot dog, and then calculating the outer product of the coordinate vectors of the centered point set and summing and averaging all point pairs. By performing rotation and translation transformations on the dense 3D point cloud data of the robotic dog in space using a coarse registration homogeneous transformation matrix, a preliminary spatial alignment result is obtained, forming the coarse registration result of the robotic dog. The coarse registration result of the robotic dog includes a set of spatial transformation parameters, an aligned point cloud dataset, and preliminary error evaluation indicators.

7. The machine vision-based mechanical dog mass distribution analysis method according to claim 1, characterized in that, The specific steps to obtain the final error measurement result include: The initial rotation matrix and initial translation vector in the coarse registration result of the mechanical dog are used as the initial transformation parameters for iterative nearest point registration to generate initial point cloud data; For each point in the initial point cloud data, a nearest neighbor correspondence is established with the sampling point set of the 3D digital model of the mechanical dog using the IPC method, forming a feature correspondence set of two point sets. Calculate the centroid vectors of the sampling point set of the 3D digital model of the mechanical dog and the dense 3D point cloud data of the mechanical dog based on the feature correspondence set, and obtain the updated rotation matrix and translation vector; The updated rotation matrix and translation vector are used to perform spatial transformation on the dense 3D point cloud data of the mechanical dog to obtain iterated point cloud data. The average spatial residual between the iterated point cloud data and the sampling points of the 3D digital model of the mechanical dog is calculated to form the iteration error. The convergence of the iterative registration process is determined according to the preset convergence conditions. When the error change between two consecutive iterations is less than the convergence threshold, the final registered point cloud set is formed. The optimal registered point cloud set includes the final rotation matrix, translation vector and transformed point cloud data. Based on the final registered point cloud set and the sampling point set of the mechanical dog's 3D digital model, the Chamfer distance and root mean square residual are calculated to obtain the final error measurement result; The final error measurement results include the final registration point cloud set, the sampling point set of the mechanical dog's 3D digital model, the final registration point cloud set, the Chamfer distance, and the root mean square residual.

8. The machine vision-based mechanical dog mass distribution analysis method according to claim 1, characterized in that, The acquisition of the updated 3D digital model of the mechanical dog specifically includes: Based on the final error measurement results, residual calculation is performed on the corresponding point pairs in the sampling point set of the mechanical dog's 3D digital model and the final registration point cloud set to generate a residual vector and a corresponding scalar residual. The residual vector is the spatial coordinate difference between the final registration point and the sampling point of the mechanical dog's 3D digital model, and the scalar residual is the Euclidean distance of the residual vector. Based on the residual vector and scalar residual, the error values ​​of each registration point pair are mapped to the corresponding spatial positions under the final registration point cloud set, generating a residual distribution mapping map of the mechanical dog structure. According to the spatial distribution characteristics of the residual distribution mapping map, assembly deviation threshold and deformation threshold are set to define assembly deviation region and deformation region respectively. When the local residual value exceeds the assembly deviation threshold, the region is marked as assembly deviation region. When the average residual value of the neighborhood exceeds the deformation threshold, the region is marked as deformation region, forming a spatial constraint set. The local feature parameters of the mechanical dog's 3D digital model are dynamically updated by using a set of spatial constraints. Within the assembly deviation region, the coordinates of the sampling points are corrected according to the displacement update coefficient. Within the deformation region, the corresponding sampling point set is smoothed using a Laplace smoothing process according to the smoothing coefficient, thus obtaining the updated 3D digital model of the mechanical dog.

9. The machine vision-based mechanical dog mass distribution analysis method according to claim 1, characterized in that, The formation of the three-dimensional quality distribution dataset specifically includes: The updated 3D digital model of the mechanical dog is spatially discretized to generate a set of voxels. The set of voxels includes voxel spatial coordinates, voxel volume and voxel mass. The voxel mass parameter is calculated from the mass density of the corresponding region in the overall mass mapping matrix of the mechanical dog. The overall centroid coordinates of the mechanical dog are calculated based on a set of voxels, where the overall centroid coordinates are the weighted average of the spatial coordinates of all voxels. The inertia tensor matrix of the mechanical dog is calculated based on the spatial distribution characteristics of the voxel set. Each component of the inertia tensor matrix is ​​calculated by the product of the squares of the voxel mass and the relative coordinates. Eigenvalue decomposition is performed on the inertia tensor matrix to obtain the principal moments of inertia and principal axis direction parameters of the robot dog, forming the principal axis parameters. The principal moments of inertia refer to the rotational characteristics of the robot dog in different principal axis directions, and the principal axis direction parameters refer to the direction of the principal axis in three-dimensional space. The overall space is partitioned using the updated 3D digital model of the mechanical dog. The mass, local centroid, mass density, and mass weight parameters of each local region are calculated to form local mass parameters. The overall centroid coordinates, principal inertial axis parameters, and local mass parameters are then combined to form a 3D mass distribution dataset.

10. The machine vision-based mechanical dog mass distribution analysis method according to claim 1, characterized in that, The mass distribution heatmap of the mechanical dog is a visual representation of local mass concentration areas, mass distribution uniformity, and structural equilibrium state by mapping the mass density parameters of each voxel in the three-dimensional mass distribution dataset to color gradients. The scatter plot of the mechanical dog's centroid is used to analyze the centroid shift phenomenon caused by assembly errors and structural imbalance by comparing the spatial distance between the local centroid and the overall centroid. The centroid vector diagram of the mechanical dog is used to perform aggregation analysis on the vector set to identify the direction of inertial deviation, mass unevenness areas, and potential deformation trends in the mechanical dog structure.

Images