Viewpoint planning method, system and computer equipment for robot 3D reconstruction equipment

Abstract

The present invention discloses a viewpoint planning method, system, and computer equipment for robotic 3D reconstruction equipment. The method obtains a standard 3D CAD model of a special-shaped aeroengine blade to be measured; designs and expands a voxel state space, updates the voxel state in the state space by scanning the blade to be measured; defines the scanner depth range, calculates the scanner radius based on the blade's dimensions, uses a spherical coordinate system to represent the scanner's position, adjusts the scanner's position based on an offset angle, and constructs an action space based on adjustable parameters; converts the scanner's position and orientation into the angles of the robotic arm and rotary table to construct a seven-degree-of-freedom measurement motion control system; designs a reward function and trains the seven-degree-of-freedom measurement motion control system using a reinforcement learning algorithm. When the training termination condition is met, the trained seven-degree-of-freedom measurement motion control system is obtained and applied to actual 3D measurement tasks to generate an optimal viewpoint plan to guide robotic scanning. This achieves efficient and accurate viewpoint planning.

Description

Technical Field

[0001] The present invention belongs to the technical field of automatic three-dimensional measurement, and in particular relates to a viewpoint planning method, system and computer equipment for robot three-dimensional reconstruction equipment. Background Art

[0002] Three-dimensional (3D) measurement technology is crucial in modern industrial manufacturing, especially in the inspection and quality control of industrial components with complex geometries. With the increasing demand for precision parts across various industries, the need for accurate and efficient 3D scanning methods has also grown significantly. Traditional 3D measurement technology typically relies on a spherical coordinate system with a fixed radius. While simple, this method lacks sufficient scanning coverage for complex objects, resulting in accuracy and efficiency that fail to meet the requirements of modern industrial applications. Against this backdrop, 3D measurement robots equipped with advanced sensors, particularly structured light scanners, have gradually become mainstream in the industry due to their efficiency, flexibility, and high precision.

[0003] However, a key challenge in automated 3D measurement is viewpoint planning—selecting the optimal sensor position to maximize surface coverage and minimize scanning time. Traditional viewpoint planning methods often rely on manual intervention, guided by experienced technicians. While effective in some cases, this approach is often inefficient and cannot always determine the optimal viewpoint, resulting in suboptimal scan results. Furthermore, insufficient control over point cloud overlap can lead to poor alignment accuracy, necessitating time-consuming and error-prone manual adjustments.

[0004] In recent years, deep reinforcement learning (DRL) has attracted increasing attention as a data-driven viewpoint planning solution. Methods such as DDQN, A3C, and NBV-Net use neural networks to predict the next best viewpoint by optimizing coverage metrics. However, existing methods have three major limitations: (1) they usually operate in a discrete action space, which limits the ability to find the optimal viewpoint in high-degree-of-freedom environments; (2) the practical constraints of the measurement equipment (such as depth range limitations and robot kinematics) are often not incorporated into the planning process, resulting in the generated viewpoints being theoretically optimal but unattainable in practice; and (3) the problem of controlling the overlap rate between consecutive scans is not fully addressed, which affects the accuracy of 3D reconstruction.

[0005] Based on this, a viewpoint planning method, system and computer equipment for robot 3D reconstruction equipment are proposed. Summary of the Invention

[0006] In response to the above technical problems, the present invention provides a viewpoint planning method, system and computer equipment for a robot three-dimensional reconstruction device.

[0007] The technical solution adopted by the present invention to solve the technical problem is:

[0008] A viewpoint planning method for a robot 3D reconstruction device, the method comprising the following steps:

[0009] S100: Obtain the standard 3D CAD model of the aircraft engine special-shaped blade to be measured in the measurement system;

[0010] S200: Designing a voxel state space based on a standard 3D CAD model of the aircraft engine's special-shaped blade to be tested, expanding the voxel state space to accommodate deviations, and updating the voxel state of the state space by scanning the blade to be tested;

[0011] S300: Defining the depth range of the scanner, calculating the scanner radius based on the size of the blade to be measured, using a spherical coordinate system to represent the position of the scanner, adjusting the position of the scanner according to the offset angle to point to the center of the blade to be measured, and forming an action space based on adjustable parameters;

[0012] S400: Using mapping equations and the Rodriguez rotation formula, the scanner position and orientation are converted into the angles of the robotic arm and the rotation stage, thus constructing a seven-degree-of-freedom measurement motion control system.

[0013] S500: Design a reward function and train the seven-degree-of-freedom measurement motion control system based on the current state and action in combination with a reinforcement learning algorithm. During the training process, the seven-degree-of-freedom measurement motion control system evaluates each action based on the feedback of the reward function and updates its decision-making strategy to maximize the cumulative reward and optimize the viewpoint selection. When the training termination condition is reached, the trained seven-degree-of-freedom measurement motion control system is obtained and applied to actual 3D measurement tasks to generate the optimal viewpoint plan to guide the robot to scan.

[0014] Preferably, S200 includes:

[0015] S210: Design the state space based on the bounding box voxel set of the standard CAD model, with a voxel side length of L;

[0016] S220: To adapt to the deviation between the measured blade and the CAD model, the state space is expanded outward on the basis of the axis-aligned bounding box, and the minimum vertex coordinates of the axis-aligned bounding box are set The length, width and height of the bounding box are , the minimum vertex coordinates after expansion are , the side length is ;

[0017] S230: The state space contains unoccupied voxels, occupied voxels and unknown voxels. The scanner is on the blade being measured. After scanning, there are K voxels in the voxel space that are occupied, and the state is 1. K rays are projected from the sensor origin in the direction of these K occupied voxels. The voxel state is updated by ray casting. The area between the scanner and the occupied voxels is unoccupied voxels, and the state is 0. The voxels after the occupied voxels are considered unknown voxels, and the state is 2. The voxel state in the state space is updated after each scan.

[0018] Preferably, in S230, the scanner is set After a scan at , K voxels in the voxel space are occupied, and their center coordinates are , suppose K rays are projected from the sensor origin toward the K occupied voxels, represents the kth ray, 1≤k≤K, and the nth voxel that the kth ray passes through is represented by , 1≤n≤N means that the voxel center coordinates are , then the voxel state is determined by the following formula:

[0019] ;

[0020] The area between the scanner and the occupied voxels is empty voxels, and the voxels after the occupied voxels are unknown voxels.

[0021] Preferably, S300 includes:

[0022] S310: The depth range of the scanner is defined by the near plane and the far plane;

[0023] S320: To ensure that the blade to be measured remains within the depth range of the scanner, the radius of the scanner is calculated according to the size of the blade to be measured and the depth range of the scanner, wherein the scanner operates around the center of gravity of the blade to be measured;

[0024] S330: Use spherical coordinates to represent the scanner's position, including azimuth , pitch angle and radius To effectively scan a specific area of the object, adjust the position of the scanner and shift it up or down or left or right at an angle. , while pointing to the center of the object;

[0025] S340: According to the adjustable parameter azimuth , pitch angle ,radius and offset angle Composing action space.

[0026] Preferably, S320 specifically includes:

[0027] ;

[0028] in, are the length, width and height of the object respectively, is the scanning radius of the scanner, is the near plane, is the far plane;

[0029] The action space in S340 is specifically:

[0030] .

[0031] Preferably, the mapping equation in S400 is specifically:

[0032] ;

[0033] in, is the unit direction vector of the z-axis, and Respectively represent the vertical and horizontal offset angles of the scanner orientation. is the scanner position vector pointing to the center of the blade and is expressed as:

[0034] ;

[0035] is the Rodriguez rotation formula, which is used to express the rotation along the vector Rotation The rotation formula of the vector, the rotation angle is , the formula is as follows:

[0036] ;

[0037] A six-degree-of-freedom robotic arm and a rotating stage are used to create a seven-degree-of-freedom measuring device. The rotating stage allows the blade to be adjusted to various postures, and the robotic arm can perform scanning movements within a plane.

[0038] Scanner coordinates ( , , ) is converted into the coordinate system of the robot arm through coordinate transformation ( , , ), and calculate the angles of the robotic arm and the rotary table using the following equations:

[0039] ;

[0040] in, represents the joint angles of the robot arm, Indicates the rotation angle of the turntable; and Represent the lengths of the second and third sections of the robot arm, respectively.

[0041] Preferably, the reward function in S500 includes a coverage rate reward, an overlap rate control reward, and an overlap rate threshold reward, including:

[0042] The current state space contains a set of unoccupied voxels and a set of occupied voxels , after each scan, the set of unoccupied voxels in the result is , the set of occupied voxels is , the standard blade CAD model is re-meshed and subdivided to determine the number of occupied voxels in the model ;

[0043] Coverage Rewards Calculated after each scan, the point cloud overlap rate of the scan is also calculated , which is defined as follows:

[0044] ;

[0045] ;

[0046] in, Indicates the number of voxels.

[0047] When the target ideal overlap ratio is set to When designing the overlap rate control reward as follows:

[0048] ;

[0049] In addition, in order to enhance the control of the overlap rate of the seven-degree-of-freedom measurement motion control system, an overlap rate threshold reward is designed. :

[0050] ;

[0051] in is the overlap rate deviation threshold set, is significantly larger than The reward value;

[0052] The specific reward function is defined by the following equation:

[0053] ;

[0054] in, and is the weight; in the initial step, the reward function encourages the seven-degree-of-freedom measurement motion control system to choose the action that maximizes the model coverage. For subsequent steps, the seven-degree-of-freedom measurement motion control system minimizes the overlap deviation of the scanned point cloud while maximizing the model coverage.

[0055] Preferably, S500 includes:

[0056] S510: A soft actor-critic algorithm based on a maximum entropy framework is used to train a seven-degree-of-freedom measurement motion control system. The algorithm consists of an actor network and two independent critic networks. The actor network generates a viewpoint planning strategy and selects actions based on the current state. The two independent critic networks evaluate the Q-values of the actions and update the critic network parameters by minimizing the temporal difference error.

[0057] S520: During training, the 7DOF measurement motion control system accumulates experience through continuous interaction with the environment and stores this experience in an experience replay pool. Experience includes state, action, reward, and next state. During training, data is randomly sampled from the experience replay pool to update the actor and critic networks. The actor network is optimized using a policy gradient method to maximize the accumulated reward and policy entropy. The critic network updates its parameters by minimizing the temporal difference error. To stabilize training, the SAC algorithm uses a soft update mechanism.

[0058] S530: Through repeated iterative training, the seven-degree-of-freedom measurement motion control system gradually adjusts its viewpoint planning strategy and selects the best viewpoint to maximize the cumulative reward. When the coverage rate meets the set coverage rate or the number of actions reaches the maximum set number of actions, the training ends and the trained seven-degree-of-freedom measurement motion control system is obtained. It is applied to actual 3D measurement tasks to generate the optimal viewpoint planning to guide the robot to scan.

[0059] The viewpoint planning system for robotic 3D reconstruction equipment includes:

[0060] The standard model acquisition module of the blade to be measured is used to obtain the standard three-dimensional CAD model of the special-shaped blade of the aircraft engine to be measured in the measurement system;

[0061] The state space design module is used to design the voxel state space based on the standard 3D CAD model of the special-shaped blade of the aircraft engine to be tested, expand the voxel state space to accommodate the deviation, and update the voxel state of the state space by scanning the blade to be tested;

[0062] The motion space design module is used to define the depth range of the scanner, calculate the scanner radius based on the size of the blade to be measured, use a spherical coordinate system to represent the position of the scanner, adjust the position of the scanner according to the offset angle, point to the center of the blade to be measured, and form the motion space based on adjustable parameters;

[0063] A seven-degree-of-freedom measurement motion control system implementation module is used to convert the scanner position and orientation into the angles of the robotic arm and the rotation stage using the mapping equation and the Rodriguez rotation formula, thereby constructing a seven-degree-of-freedom measurement motion control system.

[0064] The training and viewpoint planning module is used to design a reward function and train the seven-degree-of-freedom measurement motion control system based on the current state and action in combination with a reinforcement learning algorithm. During the training process, the seven-degree-of-freedom measurement motion control system evaluates each action based on the feedback of the reward function and updates its decision-making strategy to maximize the cumulative reward and optimize the viewpoint selection. When the training termination condition is reached, the trained seven-degree-of-freedom measurement motion control system is obtained and applied to actual 3D measurement tasks to generate the optimal viewpoint plan to guide the robot for scanning.

[0065] A computer device includes a memory and a processor, wherein the memory stores a computer program and the processor implements the steps of a viewpoint planning method for a robot three-dimensional reconstruction device when executing the computer program.

[0066] The viewpoint planning method, system, and computer device described above for robotic 3D reconstruction equipment achieve efficient and accurate viewpoint planning by introducing a multi-degree-of-freedom continuous motion space and robotic kinematic constraints. This method can handle objects with complex geometries and offers superior measurement accuracy and efficiency to existing methods. Furthermore, precise overlap control and an adaptive scanning strategy give this method significant advantages in industrial applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 This is a flow chart of a viewpoint planning method for a robot 3D reconstruction device according to one embodiment of the present invention;

[0068] Figure 2 Schematic diagram of a state update method of a model during training in one embodiment of the present invention;

[0069] Figure 3 The orientation of the scanner in one embodiment of the present invention;

[0070] Figure 4 In one embodiment of the present invention, the position of the scanner is mapped to the rotation angles of the joints of the robot arm and the turntable;

[0071] Figure 5 A reinforcement learning framework structure for blade viewpoint planning in one embodiment of the present invention;

[0072] Figure 6 The spatial voxel change process of measuring a blade in one embodiment of the present invention;

[0073] Figure 7 These are the experimental comparison results of different depth enhancement methods in one embodiment of the present invention, wherein (a) is the experimental comparison result diagram of four different depth enhancement methods on blade model 1, (b) is the experimental comparison result diagram of four different depth enhancement methods on blade model 2, (c) is the experimental comparison result diagram of four different depth enhancement methods on blade model 3, and (d) is the experimental comparison result diagram of four different depth enhancement methods on blade model 4. DETAILED DESCRIPTION

[0074] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is further described in detail below with reference to the accompanying drawings.

[0075] In one embodiment, Figure 1 As shown, a viewpoint planning method for a robot 3D reconstruction device comprises the following steps:

[0076] S100: Obtain a standard 3D CAD model of the special-shaped blade of the aircraft engine to be measured in the measurement system, and set the maximum coverage rate of the blade to be measured and the maximum number of actions of the scanning device;

[0077] S200: Designing a voxel state space based on a standard 3D CAD model of the aircraft engine's special-shaped blade to be tested, expanding the voxel state space to accommodate deviations, and updating the voxel state of the state space by scanning the blade to be tested;

[0078] S300: Defining the depth range of the scanner, calculating the scanner radius based on the size of the blade to be measured, using a spherical coordinate system to represent the position of the scanner, adjusting the position of the scanner according to the offset angle to point to the center of the blade to be measured, and forming an action space based on adjustable parameters;

[0079] S400: Using mapping equations and the Rodriguez rotation formula, the scanner position and orientation are converted into the angles of the robotic arm and the rotation stage, thus constructing a seven-degree-of-freedom measurement motion control system.

[0080] S500: Design a reward function and train the seven-degree-of-freedom measurement motion control system based on the current state and action in combination with a reinforcement learning algorithm. During the training process, the seven-degree-of-freedom measurement motion control system evaluates each action based on the feedback of the reward function and updates its decision-making strategy to maximize the cumulative reward and optimize the viewpoint selection. When the training termination condition is reached, the trained seven-degree-of-freedom measurement motion control system is obtained and applied to actual 3D measurement tasks to generate the optimal viewpoint plan to guide the robot to scan.

[0081] In one embodiment, S200 includes:

[0082] S210: Design the state space based on the bounding box voxel set of the standard CAD model, with a voxel side length of L;

[0083] S220: To adapt to the deviation between the measured blade and the CAD model, the state space is expanded outward on the basis of the axis-aligned bounding box, and the minimum vertex coordinates of the axis-aligned bounding box are set The length, width and height of the bounding box are , the minimum vertex coordinates after expansion are , the side length is ;

[0084] S230: The state space contains unoccupied voxels, occupied voxels and unknown voxels. The scanner is on the blade being measured. After scanning, there are K voxels in the voxel space that are occupied, and the state is 1. K rays are projected from the sensor origin in the direction of these K occupied voxels. The voxel state is updated by ray casting. The area between the scanner and the occupied voxels is unoccupied voxels, and the state is 0. The voxels after the occupied voxels are considered unknown voxels, and the state is 2. The voxel state in the state space is updated after each scan.

[0085] Specifically, the standard CAD model of the blade to be measured is generally known, so the observation space is defined as the set of voxels in the bounding box of the CAD model. Assuming the voxel side length is L, that is, the voxel is a cube of size L×L×L. Because the blade to be measured may deviate from its standard CAD model, to ensure that the blade to be measured is completely within the bounding box, the state space needs to be partially expanded outward from its AABB bounding box. The state space is defined as the set of three spatial voxel states: unoccupied voxels, occupied voxels, and unknown voxels, represented by 0, 1, and 2, respectively.

[0086] In one embodiment, in S230, the scanner is set to After a scan at , K voxels in the voxel space are occupied, and their center coordinates are , suppose K rays are projected from the sensor origin toward the K occupied voxels, represents the kth ray, 1≤k≤K, and the nth voxel that the kth ray passes through is represented by , 1≤n≤N means that the voxel center coordinates are , then the voxel state is determined by the following formula:

[0087] ;

[0088] The area between the scanner and the occupied voxels is empty voxels, and the voxels after the occupied voxels are unknown voxels.

[0089] Specifically, the scanning and determination diagram of the voxel state is as follows: Figure 2 As shown, ray tracing is used to update the state space, which is a collection of unoccupied voxels, occupied voxels, and unknown voxels.

[0090] In one embodiment, S300 includes:

[0091] S310: The depth range of the scanner is defined by the near plane and the far plane;

[0092] S320: To ensure that the blade to be measured remains within the depth range of the scanner, the radius of the scanner is calculated according to the size of the blade to be measured and the depth range of the scanner, wherein the scanner operates around the center of gravity of the blade to be measured;

[0093] S330: Use spherical coordinates to represent the scanner's position, including azimuth , pitch angle and radius To effectively scan a specific area of the object, adjust the position of the scanner and shift it up or down or left or right at an angle. , while pointing to the center of the object;

[0094] S340: According to the adjustable parameter azimuth , pitch angle ,radius and offset angle Composing action space.

[0095] Specifically, in the device, the object to be measured must be within a specific depth range of the scanner, called the depth range (DOF). If the object exceeds this depth range, it cannot be scanned, resulting in the loss of relevant information. The depth range of the scanner is defined by two boundaries: the near plane and the near plane. and the far plane . Set the scanner to work around the center of the object, with a radius of To ensure that the measured model remains within the depth range of the scanner, the radius The following equation must be satisfied:

[0096] ;

[0097] in, are the length, width and height of the object respectively, is the scanning radius of the scanner, is the near plane, is the far plane;

[0098] The action space in S340 is specifically:

[0099] .

[0100] In one embodiment, the mapping equation in S400 is specifically:

[0101] ;

[0102] in, is the unit direction vector of the z-axis, and Respectively represent the vertical and horizontal offset angles of the scanner orientation. is the scanner position vector pointing to the center of the blade and is expressed as:

[0103] ;

[0104] is the Rodriguez rotation formula, which is used to express the rotation along the vector Rotation The rotation formula of the vector, the rotation angle is , the formula is as follows:

[0105] ;

[0106] A six-degree-of-freedom robotic arm and a rotating stage are used to create a seven-degree-of-freedom measuring device. The rotating stage allows the blade to be adjusted to various postures, and the robotic arm can perform scanning movements within a plane.

[0107] Scanner coordinates ( , , ) is converted into the coordinate system of the robot arm through coordinate transformation ( , , ), and calculate the angles of the robotic arm and the rotary table using the following equations:

[0108] ;

[0109] in, represents the joint angles of the robot arm, Indicates the rotation angle of the turntable; and Represent the lengths of the second and third sections of the robot arm, respectively.

[0110] Specifically, if Figure 3 The scanner orientation is shown in Figure 1. The scanner orientation can be adjusted vertically and horizontally to focus on a specific area around the center of the object. The scanner position is mapped to the joints of the robot arm and the rotation angle of the turntable as shown in Figure 1. Figure 4 shown.

[0111] In one embodiment, the reward function in S500 includes a coverage reward, an overlap rate control reward, and an overlap rate threshold reward, including:

[0112] The current state space contains a set of unoccupied voxels and a set of occupied voxels , after each scan, the set of unoccupied voxels in the result is , the set of occupied voxels is , the standard blade CAD model is re-meshed and subdivided to determine the number of occupied voxels in the model ;

[0113] Coverage Rewards Calculated after each scan, the point cloud overlap rate of the scan is also calculated , which is defined as follows:

[0114] ;

[0115] ;

[0116] in, Indicates the number of voxels.

[0117] When the target ideal overlap ratio is set to When designing the overlap rate control reward as follows:

[0118] ;

[0119] In addition, in order to enhance the control of the overlap rate of the seven-degree-of-freedom measurement motion control system, an overlap rate threshold reward is designed. :

[0120] ;

[0121] in is the overlap rate deviation threshold set, is significantly larger than The reward value;

[0122] The specific reward function is defined by the following equation:

[0123] ;

[0124] in, and is the weight; in the initial step, the reward function encourages the seven-degree-of-freedom measurement motion control system to choose the action that maximizes the model coverage. For subsequent steps, the seven-degree-of-freedom measurement motion control system minimizes the overlap deviation of the scanned point cloud while maximizing the model coverage.

[0125] Specifically, coverage rewards : Rewards are given based on the increase in model coverage during scanning, encouraging the robot to choose viewpoints that maximize coverage; overlap rate controls rewards : Optimize the point cloud alignment accuracy by controlling the point cloud overlap rate between consecutive scans to prevent the overlap rate from being too high, resulting in data redundancy or alignment problems; overlap rate threshold reward : If the overlap rate is controlled within the preset target range, an additional reward will be given, thereby encouraging the seven-degree-of-freedom measurement motion control system to select a viewpoint that can maintain a good overlap rate.

[0126] In one embodiment, S500 includes:

[0127] S510: A Soft Actor-Critic algorithm based on a maximum entropy framework is used to train a seven-degree-of-freedom measurement motion control system. The SAC algorithm not only optimizes cumulative rewards but also maintains the exploratory nature of the seven-degree-of-freedom measurement motion control system by maximizing policy entropy, thus avoiding premature convergence to a suboptimal policy. The algorithm consists of an actor network and two independent critic networks. The actor network generates the viewpoint planning policy and selects actions based on the current state. The two independent critic networks evaluate the Q-values of the actions and update the critic network parameters by minimizing the temporal difference error.

[0128] S520: During training, the 7DOF measurement motion control system accumulates experience through continuous interaction with the environment and stores this experience in an experience replay pool. Experience includes state, action, reward, and next state. During training, data is randomly sampled from the experience replay pool to update the actor and critic networks. The actor network is optimized using a policy gradient method to maximize the accumulated reward and policy entropy. The critic network updates its parameters by minimizing the temporal difference error. To stabilize training, the SAC algorithm uses a soft update mechanism, where the parameters of the target network are a slow moving average of the current network parameters, rather than a direct copy.

[0129] S530: Through repeated iterative training, the 7DOF measurement motion control system gradually adjusts its viewpoint planning strategy, selecting the optimal viewpoint to maximize cumulative reward. During training, the 7DOF measurement motion control system strikes a balance between exploration and exploitation, efficiently covering the environment while avoiding repeated selection of suboptimal viewpoints. Ultimately, the 7DOF measurement motion control system is able to autonomously select the optimal viewpoint in complex environments and complete the given task while maintaining high levels of exploration and robustness. The entire solution utilizes techniques such as the SAC algorithm, experience replay, soft updates, and dual Q-learning to achieve an efficient and stable reinforcement learning training process. Training ends when the coverage rate meets the set coverage rate or the number of actions reaches the maximum set number of actions, resulting in a trained 7DOF measurement motion control system. This is then applied to actual 3D measurement tasks to generate optimal viewpoint planning to guide the robot in scanning.

[0130] Specifically, if Figure 5 As shown, a reinforcement learning algorithm is used to train a 7-DOF measurement motion control system. The system consists of two main components: a sensor and an environment. The sensor consists of an action network and two critic networks. The action network determines the optimal action to take, while the critic network evaluates the action value to update the action network. The environment, on the other hand, consists of a state space, an action space, and a reward function. The system interacts within the environment and selects the optimal viewpoint. This paper uses the Soft Actor-Critic (SAC) algorithm, based on a maximum entropy framework, which optimizes reward while maintaining a high degree of policy exploration. The training process gradually adjusts the policy through continuous interaction with the environment, maximizing the cumulative reward and optimizing viewpoint selection. During training, the 7-DOF measurement motion control system evaluates each action based on feedback from the reward function and updates its decision-making policy. The trained model is then applied to a real-world 3D measurement task. Specifically, the optimal viewpoint plan generated by this framework is used to guide a robot in scanning. Based on the viewpoint planning policy learned by the 7-DOF measurement motion control system, the robot automatically selects a scanning location and performs accurate scanning in the real environment. This process significantly improves the efficiency of measuring industrial components, such as parts with complex geometries, and reduces the need for manual intervention.

[0131] The above-mentioned viewpoint planning method for robotic 3D reconstruction equipment has the following advantages over existing traditional methods and other reinforcement learning methods:

[0132] 1. Multi-DOF Continuous Action Space: Existing methods typically rely on discrete action spaces, such as spherical coordinate systems based on fixed radii, which limits the flexibility and accuracy of viewpoint selection. In contrast, the multi-DOF continuous action space proposed in this paper enables adaptive scanning in high-dimensional space, allowing more flexible viewpoint selection, better adapting to objects with complex geometries, and providing better surface coverage.

[0133] 2. Integration of Real-World Robot Kinematics: Traditional methods and some reinforcement learning approaches often ignore the practical constraints of robotic systems, such as depth range and kinematic limitations. This can result in some theoretically optimal viewpoints being unfeasible. Our method effectively integrates these practical constraints (such as the scanner's depth range and robot kinematics) into the viewpoint planning process, ensuring that the optimized viewpoints are feasible in real-world applications, thereby improving the method's practical applicability and efficiency.

[0134] 3. Precise optimization of overlap rate control: In existing methods, point cloud overlap rate control is often imprecise, which may lead to low alignment accuracy of scanned data and even require extensive manual intervention. In contrast, the dual reward mechanism proposed in this invention can precisely control the overlap rate of point clouds and improve data alignment accuracy by controlling overlap rate deviation, reducing manual intervention and thus improving the accuracy and efficiency of 3D measurement.

[0135] In detailed experiments, we evaluate the performance of the SAC reinforcement learning based viewpoint planning algorithm by using four different leaves. Figure 6 The spatial voxel evolution of these leaves is shown, with darker areas representing covered regions and lighter areas representing unknown regions. The figure also shows the model's coverage data and the overlap rate of each acquired point cloud. The ideal overlap rate was set at 0.7. Experimental results demonstrate that the proposed method effectively maintains the overlap rate deviation within 10% for all tested leaves. Furthermore, after accounting for the influence of the bottom region, the scan coverage of the test leaves can reach 100%.

[0136] like Figure 7 As shown in the figure, our proposed method, SAC, was compared with several existing reinforcement learning-based viewpoint planning methods on four different leaf models, including viewpoint selection methods using the DDQN, A3C, and DDPG algorithms. To ensure fair experimentation, all methods were tested using the same simulation environment and dataset. The goal of the scanning task was to complete a comprehensive scan of the object using the minimum number of viewpoints. The experimental results show that our proposed method outperformed the other methods in terms of reward value in every test scenario.

[0137] In one embodiment, a viewpoint planning system for a robotic 3D reconstruction device is provided, comprising:

[0138] The standard model acquisition module of the blade to be measured is used to obtain the standard 3D CAD model of the special-shaped blade of the aircraft engine to be measured in the measurement system, and set the maximum coverage rate of the blade to be measured and the maximum number of actions of the scanning device;

[0139] The state space design module is used to design the voxel state space based on the standard 3D CAD model of the special-shaped blade of the aircraft engine to be tested, expand the voxel state space to accommodate the deviation, and update the voxel state of the state space by scanning the blade to be tested;

[0140] The motion space design module is used to define the depth range of the scanner, calculate the scanner radius based on the size of the blade to be measured, use a spherical coordinate system to represent the position of the scanner, adjust the position of the scanner according to the offset angle, point to the center of the blade to be measured, and form the motion space based on adjustable parameters;

[0141] A seven-degree-of-freedom measurement motion control system implementation module is used to convert the scanner position and orientation into the angles of the robotic arm and the rotation stage using the mapping equation and the Rodriguez rotation formula, thereby constructing a seven-degree-of-freedom measurement motion control system.

[0142] The training and viewpoint planning module is used to design a reward function and train the seven-degree-of-freedom measurement motion control system based on the current state and action in combination with a reinforcement learning algorithm. During the training process, the seven-degree-of-freedom measurement motion control system evaluates each action based on the feedback of the reward function and updates its decision-making strategy to maximize the cumulative reward and optimize the viewpoint selection. When the training termination condition is reached, the trained seven-degree-of-freedom measurement motion control system is obtained and applied to actual 3D measurement tasks to generate the optimal viewpoint plan to guide the robot for scanning.

[0143] Regarding the specific definition of the viewpoint planning device system of the robot 3D reconstruction equipment, please refer to the definition of the viewpoint planning method of the robot 3D reconstruction equipment above, and will not be repeated here. The various modules in the above-mentioned viewpoint planning system of the robot 3D reconstruction equipment can be implemented in whole or in part by software, hardware, and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0144] A computer device includes a memory and a processor, wherein the memory stores a computer program, and is characterized in that when the processor executes the computer program, steps of a viewpoint planning method for a robot three-dimensional reconstruction device are implemented.

[0145] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the above-described method embodiments. Any reference to memory, storage, database, or other media used in the embodiments provided herein may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0146] The above is a detailed introduction to the method and system for viewpoint planning of a robotic three-dimensional reconstruction device provided by the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only intended to help understand the core idea of the present invention. It should be pointed out that, for those skilled in the art, without departing from the principles of the present invention, several improvements and modifications may be made to the present invention, and such improvements and modifications also fall within the scope of protection of the claims of the present invention.

Claims

1. A viewpoint planning method for a robot 3D reconstruction device, characterized in that: The method comprises the following steps: S100: Obtain the standard 3D CAD model of the aircraft engine special-shaped blade to be measured in the measurement system; S200: Designing a voxel state space based on a standard 3D CAD model of the aircraft engine's special-shaped blade to be tested, expanding the voxel state space to accommodate deviations, and updating the voxel state of the state space by scanning the blade to be tested; S300: Defining the depth range of the scanner, calculating the scanner radius based on the size of the blade to be measured, using a spherical coordinate system to represent the position of the scanner, adjusting the position of the scanner according to the offset angle to point to the center of the blade to be measured, and forming an action space based on adjustable parameters; S400: Using mapping equations and the Rodriguez rotation formula, the scanner position and orientation are converted into the angles of the robotic arm and the rotation stage, thus constructing a seven-degree-of-freedom measurement motion control system. S500: Design a reward function. Based on the current state and action, the 7-DOF measurement motion control system is trained using a reinforcement learning algorithm. During training, the 7-DOF measurement motion control system evaluates each action based on the feedback from the reward function and updates its decision-making strategy to maximize the cumulative reward and optimize viewpoint selection. When the training termination condition is reached, the trained 7-DOF measurement motion control system is obtained and applied to actual 3D measurement tasks to generate the optimal viewpoint plan to guide the robot for scanning. The specific reward function is defined by the following equation: ； in, and is the weight, For coverage bonus, To control the reward for the overlap rate, is the overlap rate threshold reward.

2. The method according to claim 1, characterized in that S200 includes: S210: Design the state space based on the bounding box voxel set of the standard CAD model, with a voxel side length of L; S220: To adapt to the deviation between the measured blade and the CAD model, the state space is expanded outward on the basis of the axis-aligned bounding box, and the minimum vertex coordinates of the axis-aligned bounding box are set The length, width and height of the bounding box are , the minimum vertex coordinates after expansion are , the side length is ; S230: The state space contains unoccupied voxels, occupied voxels and unknown voxels. The scanner is on the blade being measured. After scanning, there are K voxels in the voxel space that are occupied, and the state is 1. K rays are projected from the sensor origin in the direction of these K occupied voxels. The voxel state is updated by ray casting. The area between the scanner and the occupied voxels is unoccupied voxels, and the state is 0. The voxels after the occupied voxels are considered unknown voxels, and the state is 2. The voxel state in the state space is updated after each scan.

3. The method according to claim 2, characterized in that In S230, the scanner is set to After a scan at , K voxels in the voxel space are occupied, and their center coordinates are , suppose K rays are projected from the sensor origin toward the K occupied voxels, represents the kth ray, 1≤k≤K, and the nth voxel that the kth ray passes through is represented by , 1≤n≤N means that the voxel center coordinates are , then the voxel state is determined by the following formula: ； The area between the scanner and the occupied voxels is empty voxels, and the voxels after the occupied voxels are unknown voxels.

4. The method according to claim 3, characterized in that S300 includes: S310: The depth range of the scanner is defined by the near plane and the far plane; S320: To ensure that the blade to be measured remains within the depth range of the scanner, the radius of the scanner is calculated according to the size of the blade to be measured and the depth range of the scanner, wherein the scanner operates around the center of gravity of the blade to be measured; S330: Use spherical coordinates to represent the scanner's position, including azimuth , pitch angle and radius To effectively scan a specific area of the object, adjust the position of the scanner and shift it up or down or left or right at an angle. , while pointing to the center of the object; S340: According to the adjustable parameter azimuth , pitch angle ,radius and offset angle Composing action space.

5. The method according to claim 4, characterized in that S320 specifically: ； in, are the length, width and height of the object respectively, is the scanning radius of the scanner, is the near plane, is the far plane; The action space in S340 is specifically: 。 6. The method according to claim 5, characterized in that The mapping equation in S400 is specifically: ； in, is the unit direction vector of the z-axis, and Respectively represent the vertical and horizontal offset angles of the scanner orientation. is the scanner position vector pointing to the center of the blade and is expressed as: ； is the Rodriguez rotation formula, which is used to express the rotation along the vector Rotation The rotation formula of the vector, the rotation angle is , the formula is as follows: ； A six-degree-of-freedom robotic arm and a rotating stage are used to create a seven-degree-of-freedom measuring device. The rotating stage allows the blade to be adjusted to various postures, and the robotic arm can perform scanning movements within a plane. Scanner coordinates ( , , ) is converted into the coordinate system of the robot arm through coordinate transformation ( , , ), and calculate the angles of the robotic arm and the rotary table using the following equations: ； in, represents the joint angles of the robot arm, Indicates the rotation angle of the turntable; and Represent the lengths of the second and third sections of the robot arm, respectively.

7. The method according to claim 6, characterized in that The reward function in S500 includes coverage reward, overlap rate control reward and overlap rate threshold reward, including: The current state space contains a set of unoccupied voxels and a set of occupied voxels , after each scan, the set of unoccupied voxels in the result is , the set of occupied voxels is , the standard blade CAD model is re-meshed and subdivided to determine the number of occupied voxels in the model ; Coverage Rewards Calculated after each scan, the point cloud overlap rate of the scan is also calculated , which is defined as follows: ； ； in, represents the number of voxels; When the target ideal overlap ratio is set to When designing the overlap rate control reward as follows: ； In addition, in order to enhance the control of the overlap rate of the seven-degree-of-freedom measurement motion control system, an overlap rate threshold reward is designed. : ； in is the overlap rate deviation threshold set, is significantly larger than The reward value; In the initial step, the reward function encourages the 7DOF measurement motion control system to choose actions that maximize the model coverage. For subsequent steps, the 7DOF measurement motion control system minimizes the overlap deviation of the scanned point cloud while maximizing the model coverage.

8. The method according to claim 7, characterized in that S500 includes: S510: A soft actor-critic algorithm based on a maximum entropy framework is used to train a seven-degree-of-freedom measurement motion control system. The algorithm consists of an actor network and two independent critic networks. The actor network generates a viewpoint planning strategy and selects actions based on the current state. The two independent critic networks evaluate the Q-values of the actions and update the critic network parameters by minimizing the temporal difference error. S520: During training, the 7DOF measurement motion control system accumulates experience through continuous interaction with the environment and stores this experience in an experience replay pool. Experience includes state, action, reward, and next state. During training, data is randomly sampled from the experience replay pool to update the actor and critic networks. The actor network is optimized using a policy gradient method to maximize the accumulated reward and policy entropy. The critic network updates its parameters by minimizing the temporal difference error. To stabilize training, the SAC algorithm uses a soft update mechanism. S530: Through repeated iterative training, the seven-degree-of-freedom measurement motion control system gradually adjusts its viewpoint planning strategy and selects the best viewpoint to maximize the cumulative reward. When the coverage rate meets the set coverage rate or the number of actions reaches the maximum set number of actions, the training ends and the trained seven-degree-of-freedom measurement motion control system is obtained. It is applied to actual 3D measurement tasks to generate the optimal viewpoint planning to guide the robot to scan.

9. A viewpoint planning system for a robot 3D reconstruction device, characterized in that: include: The standard model acquisition module of the blade to be measured is used to obtain the standard three-dimensional CAD model of the special-shaped blade of the aircraft engine to be measured in the measurement system; The state space design module is used to design the voxel state space based on the standard 3D CAD model of the special-shaped blade of the aircraft engine to be tested, expand the voxel state space to accommodate the deviation, and update the voxel state of the state space by scanning the blade to be tested; The motion space design module is used to define the depth range of the scanner, calculate the scanner radius based on the size of the blade to be measured, use a spherical coordinate system to represent the position of the scanner, adjust the position of the scanner according to the offset angle, point to the center of the blade to be measured, and form the motion space based on adjustable parameters; A seven-degree-of-freedom measurement motion control system implementation module is used to convert the scanner position and orientation into the angles of the robotic arm and the rotation stage using the mapping equation and the Rodriguez rotation formula, thereby constructing a seven-degree-of-freedom measurement motion control system. The training and viewpoint planning module is used to design a reward function and train the 7-DOF measurement motion control system based on the current state and action using a reinforcement learning algorithm. During training, the 7-DOF measurement motion control system evaluates each action based on the feedback from the reward function and updates its decision-making strategy to maximize the cumulative reward and optimize viewpoint selection. When the training termination condition is reached, the trained 7-DOF measurement motion control system is obtained and applied to actual 3D measurement tasks to generate the optimal viewpoint plan to guide the robot for scanning. The specific reward function is defined by the following equation: ； in, and is the weight, For coverage bonus, To control the reward for the overlap rate, is the overlap rate threshold reward.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

Images