Robot three-dimensional reconstruction equipment viewpoint planning method and system and computer equipment

Abstract

The invention discloses a robot three-dimensional reconstruction equipment viewpoint planning method and system and computer equipment. The method comprises the steps that a standard three-dimensional CAD model of a to-be-tested aero-engine special-shaped blade is obtained; designing and expanding a voxel state space, and updating the voxel state of the state space by scanning a to-be-detected blade; defining the depth range of the scanner, calculating the radius of the scanner in combination with the size of the to-be-measured blade, representing the position of the scanner by adopting a spherical coordinate system, adjusting the position of the scanner according to the deviation angle, and forming an action space according to adjustable parameters; the position and orientation of the scanner are converted into the angles of a mechanical arm and a rotating table, and a seven-degree-of-freedom measurement motion control system is constructed; and designing a reward function, training the seven-degree-of-freedom measurement motion control system in combination with a reinforcement learning algorithm, obtaining the trained seven-degree-of-freedom measurement motion control system when a training termination condition is reached, applying the system to an actual 3D measurement task, and generating an optimal viewpoint plan to guide robot scanning. And efficient and accurate viewpoint planning is realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of automatic three-dimensional measurement, and particularly relates to a method, a system and a computer device for viewpoint planning of a robot three-dimensional reconstruction device. Background Art

[0002] Three-dimensional (3D) measurement technology is crucial in modern industrial manufacturing, especially in the inspection and quality control of industrial components with complex geometries. With the increasing demand for precision parts in various industries, the need for accurate and efficient 3D scanning methods has also grown significantly. Traditional 3D measurement techniques usually rely on a spherical coordinate system with a fixed radius. Although this method is simple, it cannot provide sufficient scanning coverage for complex objects, resulting in the accuracy and efficiency not meeting the requirements of modern industrial applications. Against this background, 3D measurement robots equipped with advanced sensors, especially structured light scanners, have gradually become the mainstream of the industry due to their high efficiency, flexibility and high precision.

[0003] However, a key challenge in automated 3D measurement is viewpoint planning, that is, selecting the best sensor positions to maximize surface coverage and minimize scanning time. Traditional viewpoint planning methods usually rely on manual intervention, guided by experienced technicians. Although effective in some cases, this method is usually less efficient and cannot always determine the best viewpoints, resulting in unsatisfactory scanning results. In addition, insufficient control of the point cloud overlap rate may lead to a decrease in alignment accuracy, and thus time-consuming and error-prone manual adjustment is required.

[0004] In recent years, deep reinforcement learning (DRL) has received increasing attention as a data-driven viewpoint planning solution. Methods such as DDQN, A3C and NBV-Net optimize the coverage metric and use neural networks to predict the next best viewpoints. However, the existing methods have three main limitations: (1) They usually operate in a discrete action space, which limits the ability to find the optimal viewpoints in a high-degree-of-freedom environment; (2) The actual constraints of the measurement device (such as depth range limitations and robot kinematics) are often not incorporated into the planning process, resulting in the generated viewpoints being theoretically optimal but not achievable in actual operation; (3) The problem of overlap rate control between consecutive scans has not been fully addressed, which affects the accuracy of 3D reconstruction.

[0005] Based on this, a method, a system and a computer device for viewpoint planning of a robot three-dimensional reconstruction device are proposed. Summary of the Invention

[0006] In view of the above technical problems, the present invention provides a method, a system and a computer device for viewpoint planning of a robot three-dimensional reconstruction device.

[0007] The technical solution adopted by the present invention to solve its technical problems is: Viewpoint planning method for a robot three-dimensional reconstruction device, the method comprising the following steps: S100: Obtain the standard three-dimensional CAD model of the special-shaped blade of the aero-engine to be measured in the measurement system; S200: Design a voxel state space based on the standard three-dimensional CAD model of the special-shaped blade of the aero-engine to be measured, expand the voxel state space to adapt to deviations, and update the voxel state of the state space by scanning the blade to be measured; S300: Define the depth range of the scanner, calculate the radius of the scanner in combination with the size of the blade to be measured, represent the position of the scanner in spherical coordinates, adjust the position of the scanner according to the offset angle, point to the center of the blade to be measured, and form an action space according to the adjustable parameters; S400: Use the mapping equation and the Rodriguez rotation formula to convert the position and orientation of the scanner into the angles of the robotic arm and the rotary table, and construct a seven-degree-of-freedom measurement motion control system; S500: Design a reward function, train the seven-degree-of-freedom measurement motion control system according to the current state and actions in combination with the reinforcement learning algorithm. During the training process, the seven-degree-of-freedom measurement motion control system evaluates each action according to the feedback of the reward function and updates its decision-making strategy to maximize the cumulative reward and optimize the viewpoint selection. When the training termination condition is reached, the trained seven-degree-of-freedom measurement motion control system is obtained and applied to the actual 3D measurement task to generate an optimal viewpoint plan to guide the robot to scan.

[0008] Preferably, S200 includes: S210: Design a state space based on the voxel set of the bounding box of the standard CAD model, with the voxel side length being L; S220: To adapt to the deviation between the blade to be measured and the CAD model, the state space is expanded outward based on the axis-aligned bounding box. Let the minimum vertex coordinates of the axis-aligned bounding box be , the length, width, and height of the bounding box are respectively , and the minimum vertex coordinates after expansion are , and the side length is ; S230: The state space includes unoccupied voxels, occupied voxels, and unknown voxels. After the scanner scans at the position of the blade to be measured , K voxels in the voxel space are occupied, and the state is 1. Project K rays from the origin of the sensor in the directions of these K occupied voxels. Update the voxel state through ray projection. The area between the scanner and the occupied voxels is unoccupied voxels, with the state being 0. The voxels after the occupied voxels are regarded as unknown voxels, with the state being 2. Update the voxel state of the state space after each scan.

[0009] Preferably, in S230, let the scanner be at After a scan is performed at a certain position, K voxels in the voxel space are occupied, and their central coordinates are , Let K rays be projected from the origin of the sensor in the directions of the K occupied voxels, denote the k-th ray, 1 ≤ k ≤ K, and the n-th voxel penetrated by the k-th ray is denoted by , 1 ≤ n ≤ N, and its voxel center coordinates are , then the voxel state is determined by the following formula: ; The area between the scanner and the occupied voxels are all empty voxels, and the voxels after the occupied voxels are unknown voxels.

[0010] Preferably, S300 includes: S310: Define the depth range of the scanner by a near plane and a far plane; S320: To ensure that the blade to be measured is kept within the depth range of the scanner, calculate the radius of the scanner according to the size of the blade to be measured in combination with the depth range of the scanner, where the scanner works around the center of gravity of the blade to be measured; S330: Represent the position of the scanner using spherical coordinates, including the azimuth angle , the pitch angle and the radius , to effectively scan a specific area of the object, adjust the position of the scanner, shift it up and down or left and right by an angle , and at the same time point to the center of the object; S340: According to the adjustable parameters azimuth angle , pitch angle , radius and the offset angle to form an action space.

[0011] Preferably, S320 is specifically: ; where, are the length, width and height of the object respectively, is the scanning radius of the scanner, is the near plane, is the far plane; The action space in S340 is specifically: .

[0012] Preferably, the mapping equation in S400 is specifically: ; where, is the unit direction vector of the z-axis, and respectively represent the vertical and horizontal offset angles of the scanner orientation, is the scanner position vector pointing to the center of the blade, expressed as: ; is the Rodriguez rotation formula, used to represent the rotation along the vector rotation vector rotation formula, with the rotation angle being , and its formula is as follows: ; Using a six-degree-of-freedom robotic arm and a rotary table, a seven-degree-of-freedom measuring device is created. The rotary table enables the blade to be adjusted in various postures, and the robotic arm can perform scanning movements within a plane. The scanner coordinates ( , , ) are transformed into the coordinate system of the robotic arm ( , , ) through coordinate transformation, and the angles of the robotic arm and the rotary table are calculated through the following equations: ; Among them, represents the joint angle of the robotic arm, represents the rotation angle of the rotary table; and respectively represent the lengths of the second and third segments of the robotic arm.

[0013] Preferably, the reward function in S500 includes coverage reward, overlap rate control reward, and overlap rate threshold reward, including: The current state space contains a set of unoccupied voxels and a set of occupied voxels . After each scan, the set of unoccupied voxels in the result is , and the set of occupied voxels is . After the standard blade CAD model undergoes remeshing and subdivision processing, it is used to determine the number of occupied voxels in the model ; Coverage reward is calculated after each scan, and at the same time, the point cloud overlap rate of this scan is also calculated , and its definition is as follows: ; ; Among them, represents the number of voxels.

[0014] When the target ideal overlap rate is set to , design the overlap rate control reward as follows: ; In addition, in order to enhance the control of the overlap rate by the seven-degree-of-freedom measurement motion control system, an overlap rate threshold reward is designed: ; where is the set overlap rate deviation threshold, is a reward value significantly greater than ; The specific reward function is defined by the following equation: ; where, and are weights; in the initial step, the reward function encourages the seven-degree-of-freedom measurement motion control system to select actions that can maximize the model coverage rate. For subsequent steps, the seven-degree-of-freedom measurement motion control system minimizes the overlap rate deviation of the scanned point cloud while maximizing the model coverage rate.

[0015] Preferably, S500 includes: S510: Adopt the Soft Actor-Critic algorithm based on the maximum entropy framework to train the seven-degree-of-freedom measurement motion control system. The algorithm includes an Actor network and two independent Critic networks. The Actor network is responsible for generating the viewpoint planning strategy and selecting actions according to the current state. The two independent Critic networks evaluate the Q value of the actions and update the Critic network parameters by minimizing the temporal difference error; S520: During the training process, the seven-degree-of-freedom measurement motion control system accumulates experience through continuous interaction with the environment and stores these experiences in the experience replay pool. The experiences include state, action, reward, and next state; during training, data is randomly sampled from the experience replay pool for updating the Actor and Critic networks. The Actor network is optimized by the policy gradient method to maximize the cumulative reward and policy entropy; the Critic network updates its parameters by minimizing the temporal difference error. To stabilize the training, the SAC algorithm adopts a soft update mechanism; S530: Through repeated iterative training, the seven-degree-of-freedom measurement motion control system gradually adjusts its viewpoint planning strategy to select the best viewpoint to maximize the cumulative reward. When the coverage rate meets the set coverage rate or the number of actions reaches the maximum set number of actions, the training ends, and the trained seven-degree-of-freedom measurement motion control system is applied to the actual 3D measurement task to generate the optimal viewpoint planning to guide the robot for scanning.

[0016] The viewpoint planning system of the robot three-dimensional reconstruction device includes: The standard model acquisition module for the blade to be measured is used to obtain the standard three-dimensional CAD model of the special-shaped blade of the aero-engine to be measured in the measurement system; The state space design module is used to design the voxel state space based on the standard three-dimensional CAD model of the special-shaped blade of the aero-engine to be measured, expand the voxel state space to adapt to the deviation, and update the voxel state of the state space by scanning the blade to be measured; The action space design module is used to define the depth range of the scanner, calculate the radius of the scanner in combination with the size of the blade to be measured, represent the position of the scanner in the spherical coordinate system, adjust the position of the scanner according to the offset angle, point to the center of the blade to be measured, and form the action space according to the adjustable parameters; The seven-degree-of-freedom measurement motion control system implementation module is used to convert the position and orientation of the scanner into the angles of the robotic arm and the rotary table by using the mapping equation and the Rodriguez rotation formula, and construct a seven-degree-of-freedom measurement motion control system; The training and viewpoint planning module is used to design the reward function, train the seven-degree-of-freedom measurement motion control system according to the current state and action in combination with the reinforcement learning algorithm. During the training process, the seven-degree-of-freedom measurement motion control system evaluates each action according to the feedback of the reward function and updates its decision-making strategy to maximize the cumulative reward and optimize the viewpoint selection. When the training termination condition is reached, the trained seven-degree-of-freedom measurement motion control system is obtained and applied to the actual 3D measurement task to generate the optimal viewpoint planning to guide the robot to scan.

[0017] A computer device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps of the viewpoint planning method of the robot three-dimensional reconstruction device are implemented.

[0018] The above-mentioned viewpoint planning method, system and computer device of the robot three-dimensional reconstruction device realize efficient and accurate viewpoint planning by introducing a multi-degree-of-freedom continuous action space and robot kinematic constraints, can process objects with complex geometric shapes, and provide measurement accuracy and efficiency superior to existing methods. In addition, precise overlap rate control and adaptive scanning strategies also make this method have important advantages in industrial applications. Brief Description of the Drawings

[0019] Figure 1 It is the flowchart of the viewpoint planning method of the robot three-dimensional reconstruction device in an embodiment of the present invention; Figure 2 It is the schematic diagram of the state update mode of the model during the training process in an embodiment of the present invention; Figure 3The orientation method of the scanner in an embodiment of the present invention; Figure 4 Map the position of the scanner to the joint angles of the robotic arm and the rotation angle of the turntable in an embodiment of the present invention; Figure 5 The reinforcement learning framework structure for blade viewpoint planning in an embodiment of the present invention; Figure 6 Measure the spatial voxel change process of the blade in an embodiment of the present invention; Figure 7 The experimental comparison results of different depth reinforcement methods in an embodiment of the present invention. Among them, (a) is the experimental comparison result diagram of 4 different depth reinforcement methods on blade model 1, (b) is the experimental comparison result diagram of 4 different depth reinforcement methods on blade model 2, (c) is the experimental comparison result diagram of 4 different depth reinforcement methods on blade model 3, and (d) is the experimental comparison result diagram of 4 different depth reinforcement methods on blade model 4. Detailed implementation manners

[0020] To enable those skilled in the art of this technology to better understand the technical solution of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings.

[0021] In one embodiment, as Figure 1 shown, a method for viewpoint planning of a robot three-dimensional reconstruction device, the method comprising the following steps: S100: Obtain the standard three-dimensional CAD model of the special-shaped blade of the aero-engine to be measured in the measurement system, and set the maximum coverage rate of the blade to be measured and the maximum number of actions of the scanning device; S200: Design a voxel state space based on the standard three-dimensional CAD model of the special-shaped blade of the aero-engine to be measured, expand the voxel state space to adapt to the deviation, and update the voxel state of the state space by scanning the blade to be measured; S300: Define the depth range of the scanner, calculate the radius of the scanner in combination with the size of the blade to be measured, represent the position of the scanner in a spherical coordinate system, adjust the position of the scanner according to the offset angle, point to the center of the blade to be measured, and form an action space according to the adjustable parameters; S400: Use the mapping equation and the Rodriguez rotation formula to convert the position and orientation of the scanner into the angles of the robotic arm and the turntable, and construct a seven-degree-of-freedom measurement motion control system; S500: Design a reward function. Based on the current state and actions, train a seven-degree-of-freedom measurement motion control system in combination with a reinforcement learning algorithm. During the training process, the seven-degree-of-freedom measurement motion control system evaluates each action according to the feedback of the reward function and updates its decision-making strategy to maximize the cumulative reward and optimize the viewpoint selection. When the training termination condition is reached, a trained seven-degree-of-freedom measurement motion control system is obtained and applied to an actual 3D measurement task to generate an optimal viewpoint plan to guide the robot to perform scanning.

[0022] In one embodiment, S200 includes: S210: Design a state space based on the bounding box voxel set of the standard CAD model, with the voxel side length being L; S220: To adapt to the deviation between the measured blade and the CAD model, the state space is expanded outward based on the axis-aligned bounding box. Let the minimum vertex coordinates of the axis-aligned bounding box be , and the length, width, and height of the bounding box are respectively , and the minimum vertex coordinates after expansion are , and the side length is ; S230: The state space includes unoccupied voxels, occupied voxels, and unknown voxels. After the scanner scans at the position of the measured blade , K voxels in the voxel space are occupied, and the state is 1. Project K rays from the sensor origin in the directions of these K occupied voxels. Update the voxel state through ray casting. The area between the scanner and the occupied voxels is unoccupied voxels, with the state being 0. The voxels after the occupied voxels are regarded as unknown voxels, with the state being 2. Update the voxel state of the state space after each scan.

[0023] Specifically, generally, the standard CAD model of the blade to be measured is known. Therefore, the observation space is defined as the bounding box voxel set of the CAD model, with the voxel side length being L, that is, the voxel is a cube with a size of L×L×L. Since there may be a certain deviation between the measured blade and its standard CAD model, in order to ensure that the measured blade is completely within the bounding box, the state space needs to be expanded outward by a certain amount based on its AABB bounding box. The state space is defined as a set of three spatial voxel states: unoccupied voxels, occupied voxels, and unknown voxels, represented by 0, 1, and 2 respectively.

[0024] In one embodiment, in S230, assume that after the scanner performs a scan at , K voxels in the voxel space are occupied, and their central coordinates are . Assume that K rays are projected from the sensor origin in the directions of the K occupied voxels. represents the kth ray, 1 ≤ k ≤ K. The nth voxel passed through by the kth ray is represented by , 1 ≤ n ≤ N, and its voxel central coordinate is , the voxel state is determined by the following formula: ; The areas between the scanner and the occupied voxels are all empty voxels, and the voxels after the occupied voxels are unknown voxels.

[0025] Specifically, the schematic diagram of the scanning and determination of the voxel state is as Figure 2 shown. The state space is updated using ray tracing. The state space is a set of unoccupied voxels, occupied voxels, and unknown voxels.

[0026] In one embodiment, S300 includes: S310: Defining the depth range of the scanner by a near plane and a far plane; S320: To ensure that the blade to be measured remains within the depth range of the scanner, calculate the radius of the scanner according to the size of the blade to be measured in combination with the depth range of the scanner, where the scanner works around the center of gravity of the blade to be measured; S330: Representing the position of the scanner using spherical coordinates, including the azimuth angle , the pitch angle and the radius . To effectively scan a specific area of the object, adjust the position of the scanner, offset it up and down or left and right by an angle , and at the same time point to the center of the object; S340: Composing an action space according to the adjustable parameters azimuth angle , pitch angle , radius and offset angle .

[0027] Specifically, in the device, the object to be measured must be located within a specific depth range of the scanner, called the depth of field (DOF). If the object exceeds this depth range, scanning cannot be performed, resulting in loss of relevant information. The depth range of the scanner is defined by two boundaries: the near plane and the far plane . It is set that the scanner works around the center of the object, and its radius is . To ensure that the measurement model remains within the depth range of the scanner, the radius must satisfy the following equation: ; where are the length, width, and height of the object respectively, is the scanning radius of the scanner, is the near plane, is the far plane;

[0028] The specific action space in S340 is: 。

[0029] In one embodiment, the mapping equation in S400 is specifically: ; where is the unit direction vector of the z-axis, and respectively represent the vertical and horizontal offset angles of the scanner's orientation, is the scanner position vector pointing to the blade center, expressed as: ; is the Rodriguez rotation formula, used to represent the rotation formula of the vector rotated along the vector by , and its formula is as follows: ; Using a six-degree-of-freedom robotic arm and a rotary table, a seven-degree-of-freedom measuring device is created. The rotary table enables the blade to be adjusted in various postures, and the robotic arm can perform scanning movements in a plane. The scanner coordinates ( , , ) are transformed into the coordinate system of the robotic arm ( , , ) through coordinate transformation, and the angles of the robotic arm and the rotary table are calculated through the following equations: ; where represents the joint angle of the robotic arm, represents the rotation angle of the rotary table; and respectively represent the lengths of the second and third segments of the robotic arm.

[0030] Specifically, as shown in Figure 3 , the orientation of the scanner can be adjusted vertically and horizontally to focus on a specific area around the object center. Mapping the position of the scanner to the joint of the robotic arm and the rotation angle of the turntable is shown in Figure 4 .

[0031] In one embodiment, the reward function in S500 includes a coverage reward, an overlap rate control reward, and an overlap rate threshold reward, including: The current state space contains a set of unoccupied voxels and a set of occupied voxels After each scan, the set of unoccupied voxels in the result is , and the set of occupied voxels is . After the standard blade CAD model undergoes remeshing and subdivision, it is used to determine the number of occupied voxels in the model ; Coverage reward Calculated after each scan, and the point cloud overlap rate of this scan is also calculated , which is defined as follows: ; ; where represents the number of voxels.

[0032] When the target ideal overlap rate is set to , the designed overlap rate control reward is as follows: ; In addition, to enhance the control of the overlap rate by the seven-degree-of-freedom measurement motion control system, an overlap rate threshold reward is designed: ; where is the set overlap rate deviation threshold, is a reward value significantly greater than ; The specific reward function is defined by the following equation: ; where and are weights; in the initial step, the reward function encourages the seven-degree-of-freedom measurement motion control system to select actions that can maximize the model coverage rate. For subsequent steps, the seven-degree-of-freedom measurement motion control system maximizes the model coverage rate while minimizing the overlap rate deviation of the scanned point cloud.

[0033] Specifically, the coverage reward : Reward is given according to the increase in the covered area of the model during the scanning process, encouraging the robot to select viewpoints that can maximize the coverage rate; the overlap rate control reward : Optimize the point cloud alignment accuracy by controlling the point cloud overlap rate between consecutive scans, preventing the overlap rate from being too high, resulting in data redundancy or alignment problems; the overlap rate threshold reward : If the overlap rate is controlled within the preset target range, an additional reward is given, thus promoting the seven-degree-of-freedom measurement motion control system to select viewpoints that can maintain a good overlap rate.

[0034] In one embodiment, S500 includes: S510: Use the Soft Actor-Critic algorithm based on the maximum entropy framework to train the seven-degree-of-freedom measurement motion control system. The SAC algorithm not only optimizes the cumulative reward but also maintains the exploration of the seven-degree-of-freedom measurement motion control system by maximizing the policy entropy, avoiding premature convergence to suboptimal policies. The algorithm includes an Actor network and two independent Critic networks. The Actor network is responsible for generating the viewpoint planning strategy, selecting actions according to the current state, and the two independent Critic networks evaluate the Q-values of the actions and update the Critic network parameters by minimizing the temporal difference error. S520: During the training process, the seven-degree-of-freedom measurement motion control system accumulates experience through continuous interaction with the environment and stores this experience in the experience replay pool. The experience includes state, action, reward, and next state. During training, data is randomly sampled from the experience replay pool for updating the Actor and Critic networks. The Actor network is optimized by the policy gradient method to maximize the cumulative reward and policy entropy. The Critic network updates its parameters by minimizing the temporal difference error. To stabilize the training, the SAC algorithm adopts a soft update mechanism, that is, the parameters of the target network are a slow moving average of the current network parameters, rather than directly copying. S530: Through repeated iterative training, the seven-degree-of-freedom measurement motion control system gradually adjusts its viewpoint planning strategy, selects the best viewpoints to maximize the cumulative reward. During the training process, the seven-degree-of-freedom measurement motion control system achieves a balance between exploration and exploitation, can efficiently cover the environment, and avoid repeatedly selecting suboptimal viewpoints. Finally, the seven-degree-of-freedom measurement motion control system can autonomously select the optimal viewpoints in a complex environment, complete the established tasks, and at the same time maintain high exploration and robustness. The entire solution realizes an efficient and stable reinforcement learning training process through technologies such as the SAC algorithm, experience replay, soft update, and double Q-learning. When the coverage rate meets the set coverage rate or the number of actions reaches the maximum set number of actions, the training ends, and the trained seven-degree-of-freedom measurement motion control system is obtained and applied to the actual 3D measurement task to generate the optimal viewpoint planning to guide the robot to scan.

[0035] Specifically, as Figure 5As shown in the figure, a seven-degree-of-freedom measurement motion control system is trained using a reinforcement learning algorithm. This system consists of two main parts: a perception body and an environment. The perception body consists of an action network and two evaluation networks. The action network determines the best action to take, while the evaluation network evaluates the action value to update the action network. On the other hand, the environment consists of a state space, an action space, and a reward function. Interact in the environment and select the best viewpoint. The present invention uses the SoftActor-Critic (SAC) algorithm, which is based on the maximum entropy framework and can maintain a high level of policy exploration while optimizing the reward. The training process gradually adjusts the policy by continuously interacting with the environment, maximizing the cumulative reward and optimizing the viewpoint selection. During the training process, the seven-degree-of-freedom measurement motion control system evaluates each action based on the feedback of the reward function and updates its decision-making policy. Apply the trained model to an actual 3D measurement task. Specifically, use the optimal viewpoint planning generated by this framework to guide the robot to scan. The robot automatically selects the scanning position according to the viewpoint planning strategy learned by the seven-degree-of-freedom measurement motion control system and performs precise scanning in the actual environment. This process greatly improves the measurement efficiency of industrial components (such as components with complex geometric shapes) and reduces the need for manual intervention.

[0036] The above-mentioned method for viewpoint planning of a robot three-dimensional reconstruction device has the following advantages compared with existing traditional methods and other reinforcement learning methods: 1. Multi-degree-of-freedom continuous action space: Existing methods usually rely on a discrete action space, such as a spherical coordinate system based on a fixed radius, which limits the flexibility and accuracy of viewpoint selection. In contrast, the multi-degree-of-freedom continuous action space proposed in the present invention can perform adaptive scanning in a high-dimensional space, allowing for more flexible viewpoint selection, thus better adapting to objects with complex geometric shapes and providing a better surface coverage rate. 2. Integration of actual robot kinematics: Traditional methods and some reinforcement learning methods usually ignore the actual constraints of the robot system, such as depth range and kinematic limitations, which can lead to some theoretically optimal viewpoints that cannot be actually executed. The method of the present invention effectively integrates these actual constraints (such as the depth range of the scanner and robot kinematics) into the viewpoint planning process, ensuring that the optimized viewpoints are feasible in actual applications, thereby improving the practical applicability and efficiency of the method. 3. Precise optimization of overlap rate control: In existing methods, the control of the overlap rate of the point cloud is often not precise enough, which may lead to low alignment accuracy of the scanned data and even require a large amount of manual intervention. In contrast, the dual reward mechanism proposed in the present invention can precisely control the overlap rate of the point cloud and improve the data alignment accuracy by controlling the overlap rate deviation, reducing manual intervention, thereby improving the accuracy and efficiency of 3D measurement.

[0037] In a detailed experiment, the present invention evaluates the performance of the viewpoint planning algorithm based on SAC reinforcement learning by using four different blades. Figure 6 The spatial voxel evolution of these blades is shown, where the dark color represents the covered area and the light color represents the unknown area. The coverage rate data of the model and the overlap rate data of each point cloud acquisition are marked in the figure. The set ideal overlap rate is 0.7. The experimental results show that the method of the present invention effectively maintains the overlap rate deviation of all test blades within 10%. In addition, considering the influence of the bottom area, the scanning coverage rate of the test blades can reach 100%.

[0038] As Figure 7 shown, the method SAC of the present invention is compared with several existing viewpoint planning methods based on reinforcement learning on 4 different blade models, including the viewpoint selection methods using DDQN, A3C, and DDPG algorithms. To ensure the fairness of the experiment, all methods are tested using the same simulation environment and dataset, and the goal of the scanning task is to complete the comprehensive scanning of the object with the least number of viewpoints. The experimental results show that the method of the present invention has a better reward value in each test scenario than other methods.

[0039] In one embodiment, a viewpoint planning system for a robot three-dimensional reconstruction device is provided, including: A standard model acquisition module for the blade to be measured, which is used to obtain the standard three-dimensional CAD model of the special-shaped blade of the aero-engine to be measured in the measurement system, and set the maximum coverage rate of the blade to be measured and the maximum number of actions of the scanning device; A state space design module, which is used to design the voxel state space based on the standard three-dimensional CAD model of the special-shaped blade of the aero-engine to be measured, expand the voxel state space to adapt to the deviation, and update the voxel state of the state space by scanning the blade to be measured; An action space design module, which is used to define the depth range of the scanner, calculate the radius of the scanner in combination with the size of the blade to be measured, represent the position of the scanner in the spherical coordinate system, adjust the position of the scanner according to the offset angle, point to the center of the blade to be measured, and form the action space according to the adjustable parameters; A seven-degree-of-freedom measurement motion control system implementation module, which is used to convert the position and orientation of the scanner into the angles of the robotic arm and the rotary table by using the mapping equation and the Rodriguez rotation formula, and construct a seven-degree-of-freedom measurement motion control system; A training and viewpoint planning module is used to design a reward function. According to the current state and actions, combined with a reinforcement learning algorithm, it trains a seven-degree-of-freedom measurement motion control system. During the training process, the seven-degree-of-freedom measurement motion control system evaluates each action based on the feedback of the reward function and updates its decision-making strategy to maximize the cumulative reward and optimize the viewpoint selection. When the training termination condition is reached, a trained seven-degree-of-freedom measurement motion control system is obtained and applied to an actual 3D measurement task to generate an optimal viewpoint plan to guide the robot to perform scanning.

[0040] For the specific limitations of the viewpoint planning device system of the robot 3D reconstruction device, reference can be made to the limitations of the viewpoint planning method of the robot 3D reconstruction device in the above text, which will not be elaborated here. Each module in the above-mentioned robot 3D reconstruction device viewpoint planning system can be implemented in whole or in part by software, hardware, and their combination. The above-mentioned modules can be embedded in the processor of the computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to the above-mentioned modules.

[0041] A computer device includes a memory and a processor. The memory stores a computer program. The feature is that when the processor executes the computer program, it implements the steps of the robot 3D reconstruction device viewpoint planning method.

[0042] Those of ordinary skill in the art can understand that all or part of the processes of implementing the methods in the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to the memory, storage, database, or other media used in the various embodiments provided in this application can include at least one of non-volatile and volatile memories. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0043] The above has introduced in detail the viewpoint planning device, method and system of the robot three-dimensional reconstruction device provided by the present invention. Specific examples are used in this article to elaborate on the principle and implementation manner of the present invention. The description of the above embodiments is only used to help understand the core idea of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and modifications can be made to the present invention, and these improvements and modifications also fall within the protection scope of the claims of the present invention.

Claims

1. Method for viewpoint planning of robot three-dimensional reconstruction equipment, characterized in that The method includes the following steps: S100: Obtain the standard three-dimensional CAD model of the special-shaped blade of the aero-engine to be measured in the measurement system; S200: Design a voxel state space based on the standard three-dimensional CAD model of the special-shaped blade of the aero-engine to be measured, expand the voxel state space to adapt to deviations, and update the voxel state of the state space by scanning the blade to be measured; S300: Define the depth range of the scanner, calculate the radius of the scanner in combination with the size of the blade to be measured, represent the position of the scanner in spherical coordinates, adjust the position of the scanner according to the offset angle, point to the center of the blade to be measured, and form an action space according to adjustable parameters; S400: Use the mapping equation and Rodriguez rotation formula to convert the position and orientation of the scanner into the angles of the robotic arm and the rotary table, and construct a seven-degree-of-freedom measurement motion control system; S500: Design a reward function, train the device according to the current state and action in combination with the reinforcement learning algorithm, and train the seven-degree-of-freedom measurement motion control system. During the training process, the seven-degree-of-freedom measurement motion control system evaluates each action according to the feedback of the reward function and updates its decision-making strategy to maximize the cumulative reward and optimize the viewpoint selection. When the training termination condition is reached, the trained seven-degree-of-freedom measurement motion control system is obtained and applied to the actual 3D measurement task to generate an optimal viewpoint plan to guide the robot to scan.

2. The method according to claim 1, wherein S200 includes: S210: Design a state space based on the voxel set of the bounding box of the standard CAD model, and the voxel side length is L; S220: To adapt to the deviation between the measured blade and the CAD model, the state space is expanded outward based on the axis-aligned bounding box. Let the minimum vertex coordinates of the axis-aligned bounding box be , the length, width, and height of the bounding box are respectively , the minimum vertex coordinates after expansion are , and the side length is ; S230: The state space includes unoccupied voxels, occupied voxels, and unknown voxels. After the scanner scans the blade under test at a certain position, K voxels in the voxel space are occupied and the state is 1. Project K rays from the sensor origin in the directions of these K occupied voxels. Update the voxel state through ray projection. The area between the scanner and the occupied voxels is the unoccupied voxels with a state of 0. The voxels after the occupied voxels are regarded as unknown voxels with a state of 2. Update the voxel state of the state space after each scan.

3. The method according to claim 2, characterized in that, In S230, after the scanner performs a scan at , K voxels in the voxel space are occupied, and their central coordinates are . Suppose K rays are projected from the origin of the sensor in the directions of the K occupied voxels. represents the k-th ray, where 1 ≤ k ≤ K. The n-th voxel passed through by the k-th ray is represented by , where 1 ≤ n ≤ N, and its voxel central coordinate is . Then the voxel state is determined by the following formula: ； The areas between the scanner and the occupied voxels are all empty voxels, and the voxels after the occupied voxels are unknown voxels.

4. The method according to claim 3, characterized in that, S300 includes: S310: Define the depth range of the scanner by the near plane and the far plane; S320: To ensure that the blade to be measured remains within the depth range of the scanner, calculate the radius of the scanner in combination with the depth range of the scanner according to the size of the blade to be measured, where the scanner works around the center of gravity of the blade to be measured; S330: Represent the position of the scanner using spherical coordinates, including the azimuth angle , the elevation angle and the radius , to effectively scan a specific area of the object, adjust the position of the scanner, offset it up and down or left and right by an angle , while pointing to the center of the object; S340: According to the adjustable parameters of azimuth , pitch angle , radius and offset angle to form an action space.

5. The method according to claim 4, wherein Specifically, S320 is: ； wherein, are the length, width and height of the object respectively, is the scanning radius of the scanner, is the near plane, is the far plane; Specifically, the action space in S340 is: 。 6. The method according to claim 5, wherein Specifically, the mapping equation in S400 is: ； Among them, is the unit direction vector of the z-axis, and respectively represent the offset angles in the vertical and horizontal directions of the scanner's orientation, is the scanner position vector pointing to the center of the blade, expressed as: ； is the Rodriguez rotation formula, used to represent a rotation along the vector by of the vector, with a rotation angle of . Its formula is as follows: ； Use a six-degree-of-freedom robotic arm and a rotary table to create a seven-degree-of-freedom measurement device. The rotary table enables the blade to be adjusted in various postures, and the robotic arm can perform scanning movements in a plane. Scanner coordinates ( , , ) are converted to the coordinate system of the robotic arm ( , , ) through coordinate transformation, and the angles of the robotic arm and the rotary table are calculated by the following equations: ； Among them, represents the joint angle of the robotic arm, represents the rotation angle of the turntable; and respectively represent the lengths of the second and third segments of the robotic arm.

7. The method according to claim 6, wherein The reward function in S500 includes coverage reward, overlap rate control reward, and overlap rate threshold reward, including: The current state space contains a set of unoccupied voxels and a set of occupied voxels , after each scan, the set of unoccupied voxels in the result is , and the set of occupied voxels is . After the standard blade CAD model is remeshed and subdivided, it is used to determine the number of occupied voxels in the model ; Coverage Reward Calculated after each scan, and at the same time, the point cloud overlap rate of this scan is also calculated , which is defined as follows: ； ； Among them, represents the number of voxels; When the target ideal overlap rate is set to , design the overlap rate control reward as follows: ； In addition, in order to enhance the control of the overlap rate by the seven-degree-of-freedom measurement motion control system, an overlap rate threshold reward is designed : ； wherein is the set overlap rate deviation threshold value, is a reward value significantly greater than ; The specific reward function is defined by the following equation: ； Among them, and are weights; in the initial step, the reward function encourages the seven-degree-of-freedom measurement motion control system to select actions that can maximize the model coverage rate. For subsequent steps, while maximizing the model coverage rate, the seven-degree-of-freedom measurement motion control system minimizes the overlap rate deviation of the scanned point cloud.

8. The method according to claim 7, wherein S500 includes: S510: Use the Soft Actor-Critic algorithm based on the maximum entropy framework to train the seven-degree-of-freedom measurement motion control system. The algorithm includes an Actor network and two independent Critic networks. The Actor network is responsible for generating a viewpoint planning strategy, selecting actions according to the current state, and the two independent Critic networks evaluate the Q value of the actions and update the Critic network parameters by minimizing the temporal difference error. S520: During the training process, the seven-degree-of-freedom measurement motion control system accumulates experience through continuous interaction with the environment and stores this experience in the experience replay pool. The experience includes state, action, reward, and next state. During training, data is randomly sampled from the experience replay pool to update the Actor and Critic networks. The Actor network is optimized using the policy gradient method to maximize the cumulative reward and policy entropy. The Critic network updates its parameters by minimizing the temporal difference error. To stabilize the training, the SAC algorithm adopts a soft update mechanism. S530: Through repeated iterative training, the seven-degree-of-freedom measurement motion control system gradually adjusts its viewpoint planning strategy to select the best viewpoint to maximize the cumulative reward. When the coverage rate meets the set coverage rate or the number of actions reaches the maximum set number of actions, the training ends, and a trained seven-degree-of-freedom measurement motion control system is obtained and applied to an actual 3D measurement task to generate an optimal viewpoint plan to guide the robot to scan.

9. The viewpoint planning system of the robot three-dimensional reconstruction device is characterized in that, Including: A standard model acquisition module for the blade to be measured, which is used to obtain the standard three-dimensional CAD model of the special-shaped blade of the aero-engine to be measured in the measurement system. A state space design module, which is used to design a voxel state space based on the standard three-dimensional CAD model of the special-shaped blade of the aero-engine to be measured, expand the voxel state space to adapt to deviations, and update the voxel state of the state space by scanning the blade to be measured. An action space design module, which is used to define the depth range of the scanner, calculate the scanner radius in combination with the size of the blade to be measured, represent the position of the scanner using a spherical coordinate system, adjust the position of the scanner according to the offset angle, point to the center of the blade to be measured, and form an action space according to adjustable parameters. A seven-degree-of-freedom measurement motion control system implementation module, which is used to convert the position and orientation of the scanner into the angles of the robotic arm and the rotary table using the mapping equation and the Rodriguez rotation formula to construct a seven-degree-of-freedom measurement motion control system. A training and viewpoint planning module, which is used to design a reward function, train the seven-degree-of-freedom measurement motion control system according to the current state and action in combination with the reinforcement learning algorithm. During the training process, the seven-degree-of-freedom measurement motion control system evaluates each action according to the feedback of the reward function and updates its decision-making strategy to maximize the cumulative reward and optimize the viewpoint selection. When the training termination condition is reached, a trained seven-degree-of-freedom measurement motion control system is obtained and applied to an actual 3D measurement task to generate an optimal viewpoint plan to guide the robot to scan.

10. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method described in any one of claims 1 to 8.

Images