Humanoid three-dimensional scanning motion control method and device based on point cloud

Abstract

The application provides a kind of point cloud-based human-like three-dimensional scanning motion control method and device, the point cloud-based human-like three-dimensional scanning motion control method includes: obtaining the camera viewpoint trajectory and the end effector trajectory of mechanical arm of object to be detected;Based on the camera viewpoint trajectory and the end effector trajectory of mechanical arm, trajectory surface and the deformation energy function of trajectory surface are obtained;Based on the deformation energy function and distance deviation function, the trajectory surface is optimized, and the motion planning trajectory curve of object to be detected is obtained;The motion planning trajectory curve of object to be detected is input to multi-object motion control model for solving, and the optimal solution of mechanical arm motion control corresponding to motion planning trajectory is obtained.The method of the application realizes the continuous scanning of multiple components or structures, improves the efficiency and accuracy of scanning, can meet the requirements of complex industrial environment and actual task, and is more suitable for multi-component discrete manufacturing scene, has wide industrial application prospect and objective economic benefit.

Description

Technical Field

[0001] This invention relates to the field of robot control technology, and in particular to a humanoid 3D scanning motion control method and device based on point clouds. Background Technology

[0002] Industrial surface quality inspection is an important part of the product manufacturing process. The high precision and flexibility of industrial robots enable them to replace humans in scanning products. In the discrete manufacturing process of multiple components, it is usually necessary to conduct comprehensive inspection of multiple components or different models of the product.

[0003] In related technologies, existing 3D scanning equipment scans a single component or structure of the object to be inspected. When scanning multiple components or structures, it is necessary to control the scanning equipment to perform multiple different scanning tasks. Furthermore, the parameter settings for scanning the equipment according to different trajectories of different components are complex, resulting in low detection efficiency. This cannot meet the high-efficiency and accurate scanning requirements for objects with complex structures. Summary of the Invention

[0004] This invention provides a point cloud-based humanoid 3D scanning motion control method and device to solve the shortcomings of existing 3D scanning equipment, such as the inability to continuously scan multiple parts or structures, complex parameter settings, and low detection efficiency when scanning multiple parts or structures individually. This invention improves the efficiency and accuracy of overall scanning of the object to be detected.

[0005] This invention provides a point cloud-based humanoid 3D scanning motion control method, comprising:

[0006] Acquire the camera viewpoint trajectory and the end effector trajectory of the object to be detected;

[0007] Based on the camera viewpoint trajectory and the trajectory of the robotic arm end effector, a trajectory surface and a deformation energy function of the trajectory surface are obtained. The deformation energy function is used to optimize the smoothness of the trajectory surface.

[0008] The trajectory surface is optimized based on the deformation energy function and the distance deviation function to obtain the motion planning trajectory curve of the object to be detected. The distance deviation function is used to reduce the degree of deviation of the trajectory surface during the optimization process.

[0009] The motion planning trajectory curve of the object to be detected is input into the multi-objective motion control model for solution, and the optimal solution of the robotic arm motion control corresponding to the motion planning trajectory is obtained. The multi-objective motion control model is obtained by mathematical modeling multiple optimization objectives of the object to be detected.

[0010] According to the point cloud-based humanoid 3D scanning motion control method provided by the present invention, the step of acquiring the camera viewpoint trajectory and the trajectory of the robotic arm end effector of the object to be detected includes:

[0011] An initial viewpoint is obtained based on the 3D point cloud of the object to be detected;

[0012] The initial viewpoint is fitted using the least squares asymptotic iterative approximation algorithm and NURBS curves to obtain the camera viewpoint trajectory and the trajectory of the robotic arm end effector.

[0013] According to the point cloud-based humanoid 3D scanning motion control method provided by the present invention, the step of optimizing the trajectory surface based on the deformation energy function and the distance deviation function to obtain the motion planning trajectory curve of the object to be detected includes:

[0014] The optimization function is obtained based on the deformation energy function and the distance deviation function;

[0015] The optimization function is iteratively optimized using the successive approximation method (SAM) to obtain the target optimization solution.

[0016] Multiple new control points from the NURBS curve are input into the target optimization function to obtain the optimized target function value. The motion planning trajectory curve is obtained when the difference between the original target function value and the optimized target function value is less than the iteration termination threshold or when the number of iterations exceeds the maximum number of iterations. The original target function value is calculated by inputting the control points into the optimization function.

[0017] According to the point cloud-based humanoid 3D scanning motion control method provided by the present invention, obtaining the trajectory surface and the deformation energy function of the trajectory surface includes:

[0018] The trajectory surface is input into the thin plate energy model to obtain the deformation energy function.

[0019] According to the point cloud-based humanoid 3D scanning motion control method provided by the present invention, the step of inputting the motion planning trajectory curve of the object to be detected into a multi-objective motion control model for solving, and obtaining the optimal solution for the motion control of the robotic arm corresponding to the motion planning trajectory, includes:

[0020] The trajectory tracking, attitude constraints, physical constraints, and obstacle avoidance constraints are summarized into a target optimization problem;

[0021] The target optimization problem is solved based on the metaheuristic recurrent neural network, the beetle horn search algorithm (BAS), and the motion planning trajectory curve of the object to be detected, so as to obtain the optimal solution for the motion control of the robotic arm corresponding to the motion planning trajectory.

[0022] According to the point cloud-based humanoid 3D scanning motion control method provided by the present invention, the attitude constraint is used to optimize the direction of the end effector of the robotic arm and the direction of the camera's field of view axis.

[0023] The obstacle avoidance constraint is achieved by solving the distance relationship between the center of the robotic arm and the center of the obstacle spheres using the artificial multi-sphere approximation method.

[0024] The physical constraints are used to constrain the joint angles and joint angular velocities of the robotic arm.

[0025] The present invention also provides a point cloud-based humanoid 3D scanning motion control device, comprising:

[0026] The acquisition module is used to acquire the camera viewpoint trajectory and the trajectory of the robotic arm end effector of the object to be detected;

[0027] The first processing module is used to obtain a trajectory surface and a deformation energy function of the trajectory surface based on the camera viewpoint trajectory and the trajectory of the robotic arm end effector. The deformation energy function is used to optimize the smoothness of the trajectory surface.

[0028] The second processing module is used to optimize the trajectory surface based on the deformation energy function and the distance deviation function to obtain the motion planning trajectory curve of the object to be detected. The distance deviation function is used to reduce the degree of deviation of the trajectory surface during the optimization process.

[0029] The third processing module is used to input the motion planning trajectory curve of the object to be detected into the multi-objective motion control model for solving, so as to obtain the optimal solution of the robotic arm motion control corresponding to the motion planning trajectory. The multi-objective motion control model is obtained by mathematical modeling multiple optimization objectives of the object to be detected.

[0030] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the point cloud-based humanoid 3D scanning motion control method and apparatus as described above.

[0031] The present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the point cloud-based humanoid 3D scanning motion control method and apparatus as described above.

[0032] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the point cloud-based humanoid 3D scanning motion control method and apparatus as described above.

[0033] The present invention provides a point cloud-based humanoid 3D scanning motion control method and device. It obtains the trajectory surface and its deformation energy function by using the camera viewpoint trajectory of the object to be detected and the trajectory of the robotic arm end effector. The trajectory surface is then optimized using the deformation energy function and a preset distance deviation function to obtain the motion planning trajectory curve. Finally, the motion planning trajectory curve of the object to be detected is input into a multi-objective motion control model for solution, yielding the optimal solution for robotic arm motion control corresponding to the motion planning trajectory. This enables continuous scanning of multiple components or structures, improving scanning efficiency and accuracy. It can meet the requirements of complex industrial environments and practical tasks, and is more suitable for discrete manufacturing scenarios with multiple components, possessing broad industrial application prospects and substantial economic benefits. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0035] Figure 1 This is a flowchart illustrating the point cloud-based humanoid 3D scanning motion control method provided by the present invention.

[0036] Figure 2 This is a schematic diagram of trajectory generation based on point clouds provided by the present invention;

[0037] Figure 3 This is a schematic diagram of the BAORNN algorithm provided by the present invention;

[0038] Figure 4 This is a schematic diagram of attitude constraints provided by the present invention;

[0039] Figure 5 This is a schematic diagram of obstacle modeling provided by the present invention;

[0040] Figure 6 This is a flowchart illustrating the point cloud-based humanoid 3D scanning motion control device provided by the present invention.

[0041] Figure 7 This is a schematic diagram of the physical structure of an electronic device provided by the present invention. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0043] The following is combined Figures 1-6 This invention describes a point cloud-based humanoid 3D scanning motion control method.

[0044] Figure 1 This is a flowchart illustrating the point cloud-based humanoid 3D scanning motion control method provided by the present invention, as shown below. Figure 1 As shown, the point cloud-based humanoid 3D scanning motion control method includes the following steps:

[0045] Step 110: Obtain the camera viewpoint trajectory and the trajectory of the robotic arm end effector of the object to be detected.

[0046] In this step, the object to be inspected can be a complex structure including multiple components. When the end effector of the terminal control robot arm performs a full scan of the object to be inspected, it can first use the 3D point cloud of the object to be inspected to generate an initial viewpoint, and then plan the camera viewpoint trajectory and the trajectory of the end effector of the robot arm based on the initial viewpoint.

[0047] In this embodiment, the camera can be an industrial camera or another camera used for scanning equipment.

[0048] In this embodiment, the initial viewpoint can be fitted using the Least-square Progressive and Iterative Approximation (LSPIA) algorithm and the Non-Uniform Rational B-Splines (NURBS) curve to obtain the aforementioned camera viewpoint trajectory and the trajectory of the robotic arm end effector.

[0049] Step 120: Based on the camera viewpoint trajectory and the trajectory of the robotic arm end effector, obtain the trajectory surface and the deformation energy function of the trajectory surface. The deformation energy function is used to optimize the smoothness of the trajectory surface.

[0050] In this step, after the camera viewpoint trajectory curve and the robotic arm end effector trajectory curve are synchronized with parameters, there is a one-to-one correspondence between the parameter points of the two curves.

[0051] In this step, the deformation energy function of the trajectory surface can be used to construct the trajectory surface through an energy model. The deformation energy function can handle the energy optimization problem of the surface model and ensure the smoothness of the trajectory surface.

[0052] In this embodiment, the energy model can be a thin-plate energy model.

[0053] In this embodiment, the camera viewpoint trajectory and the trajectory of the robotic arm end effector can be fitted using the surface module of the terminal to obtain the corresponding trajectory surface.

[0054] Step 130: Optimize the trajectory surface based on the deformation energy function and the distance deviation function to obtain the motion planning trajectory curve of the object to be detected. The distance deviation function is used to reduce the degree of deviation of the trajectory surface during the optimization process.

[0055] In this embodiment, energy optimization of the trajectory surface can improve the smoothness of the surface, but it also increases the distance between the points on the curve and the desired position. A distance deviation function can be set according to user needs to prevent the smoothed trajectory from deviating significantly from the initial path point. Specifically, a distance deviation optimization step is added to the optimization process of the trajectory surface to prevent the value from differing too much.

[0056] In this embodiment, to prevent the values ​​from differing too much, the joint function can be normalized to its maximum and minimum values ​​to obtain an objective function that can both improve the smoothness of the curve and ensure the distance deviation.

[0057] In this embodiment, under the premise that the scanning device covers 100% of the object to be detected, the distance error threshold can be appropriately adjusted. For example, a larger distance error threshold can be used during optimization, the expected distance error threshold used in fitting ensures a better fitting effect, and the maximum distance error threshold used in optimization maximizes the smoothness of the curve.

[0058] In this embodiment, the smoothness and deviation of the trajectory surface are optimized using an objective function to obtain the motion planning trajectory curve of the object to be detected.

[0059] Step 140: Input the motion planning trajectory curve of the object to be detected into the multi-objective motion control model for solving, and obtain the optimal solution of the robotic arm motion control corresponding to the motion planning trajectory. The multi-objective motion control model is obtained by mathematically modeling multiple optimization objectives of the object to be detected.

[0060] In this step, motion planning is performed in the joint space by tracking and controlling the trajectory of the robotic arm to enable the end effector to move along a specified trajectory.

[0061] It should be noted that the forward kinematics of the robotic arm realizes a nonlinear mapping from joint space to task space. By setting the joint angles of the robotic arm to satisfy the inverse kinematics, the tracking and control of its motion trajectory can be achieved. Since there are multiple solutions to the inverse kinematics of the redundant robotic arm, the robotic arm can be modeled as an optimization problem to be solved.

[0062] In this step, the multiple optimization objectives of the object to be detected can be trajectory tracking, attitude constraints, physical constraints, and obstacle avoidance constraints.

[0063] In this embodiment, attitude constraints enable humanoid multi-dimensional agile observation by constraining the actual end effector direction and the camera field of view axis direction; obstacle avoidance constraints are achieved by adopting a strategy that ensures the minimum distance between the robotic arm and the obstacle is greater than a distance threshold; physical constraints are imposed because the joint angles should be constrained due to the inherent characteristics of the robotic arm and the requirements of the working environment to ensure that the solution is within the mechanical limits of the joints, thus constraining the joint angles and joint angular velocities.

[0064] In this embodiment, after modeling the motion control optimization problem, trajectory tracking, attitude constraints, obstacle avoidance constraints, and physical constraints can be summarized into an optimization problem, and the trajectory tracking and attitude constraints in the objective function can be normalized by minimizing the maximum and minimum values.

[0065] In this embodiment, the optimization problem in the motion control model is calculated and iterated using the motion planning trajectory curve of the object to be detected. The optimal solution is obtained when the iteration conditions are met. The end effector of the terminal control robot arm scans and detects the object to be detected according to the planned route corresponding to the optimal solution.

[0066] The point cloud-based humanoid 3D scanning motion control method of this invention obtains the trajectory surface and its deformation energy function by using the camera viewpoint trajectory of the object to be detected and the trajectory of the end effector of the robotic arm. The trajectory surface is then optimized using the deformation energy function and a preset distance deviation function to obtain the motion planning trajectory curve. Finally, the motion planning trajectory curve of the object to be detected is input into a multi-objective motion control model for solution to obtain the optimal solution for robotic arm motion control corresponding to the motion planning trajectory. This method enables continuous scanning of multiple components or structures, improves scanning efficiency and accuracy, meets the requirements of complex industrial environments and actual tasks, and is more suitable for discrete manufacturing scenarios with multiple components. It has broad industrial application prospects and objective economic benefits.

[0067] In some embodiments, obtaining the camera viewpoint trajectory and the end effector trajectory of the object to be detected includes: obtaining an initial viewpoint based on the 3D point cloud of the object to be detected; and fitting the initial viewpoint based on the least squares asymptotic iterative approximation algorithm and the NURBS curve to obtain the camera viewpoint trajectory and the end effector trajectory of the robot arm.

[0068] In this embodiment, 3D point cloud data of the object to be detected can be acquired first to obtain an initial viewpoint. The initial viewpoint is then fitted using a NURBS curve, and the Least-square Progressive and Iterative Approximation (LSPIA) algorithm is used during the fitting process to obtain the trajectory equation C corresponding to the initial viewpoint. c (u) and its nearest neighbor set P N c (Weight set to 1), thus obtaining the position point of the robotic arm end effector corresponding to the initial viewpoint, and combining the NURBS curve and LSPIA algorithm to fit the trajectory equation C of the robotic arm end effector. e (u) and the nearest neighbor set Then, parameter synchronization is performed to establish a one-to-one correspondence between points on the two curves.

[0069] Figure 2 This is a schematic diagram of trajectory generation based on point clouds provided by the present invention. Figure 2 In the illustrated embodiment, the continuous motion path of the robotic arm is generated based on a series of initial path points and direction vectors. When fitting the trajectory of the robotic arm end effector using NURBS curves and the LSPIA algorithm, the LSPIA algorithm can optimize the number of control points on the NURBS curve to reduce the impact of sudden changes or noise at the input points, while meeting the requirements of the desired distance error threshold. By setting the weight coefficient of the camera viewpoint trajectory curve to 1, the equation of the camera viewpoint trajectory curve is simplified, and the nearest neighbor set is further obtained. Then, combined with the direction vector set corresponding to the robotic arm, the corresponding point set of the robotic arm end effector position is obtained, and the fitting curve of the robotic arm end effector and the nearest neighbor set are obtained.

[0070] In this embodiment, the node vectors of the camera viewpoint trajectory curve and the robotic arm end effector trajectory curve are different, and the point sets on the two curves do not satisfy a one-to-one correspondence. A continuous and smooth trajectory can be obtained by synchronizing the node vectors of these two curves.

[0071] In this embodiment, the NURBS curve combined with the LSPIA algorithm can suppress the influence of sudden changes or noise on the trajectory to a certain extent when optimizing the trajectory. Since adjusting the number of control points will cause changes in the entire curve, it is impossible to adjust the local area. In order to make the curve in the local area better, the smoothness of the curve is optimized. The surface equation is constructed by two NURBS curves, and the smoothness of the trajectory surface is calculated by the surface deformation energy function. The smaller the energy, the smaller the surface curvature, and the smoother the surface. When the energy is 0, the surface becomes a plane.

[0072] The point cloud-based humanoid 3D scanning motion control method of this invention obtains the camera viewpoint trajectory and the end effector trajectory of the robotic arm by fitting the initial viewpoint with the basis least squares asymptotic iterative approximation algorithm and NURBS curve, thereby reducing the influence of abrupt changes or noise on the NURBS curve and improving the smoothness of the curve.

[0073] In some embodiments, the trajectory surface is optimized based on the deformation energy function and the distance deviation function to obtain the motion planning trajectory curve of the object to be detected. This includes: obtaining an optimization function based on the deformation energy function and the distance deviation function; iteratively optimizing the optimization function using the successive approximation method (SAM) to obtain the target optimization solution; inputting multiple new control points from the NURBS curve into the target optimization function to obtain the optimized target function value; and obtaining the motion planning trajectory curve when the difference between the original target function value and the optimized target function value is less than the iteration termination threshold or the number of iterations exceeds the maximum number of iterations. The original target function value is calculated by inputting the control points into the optimization function.

[0074] In this embodiment, the smoothness of the NURBS curve is effectively improved through energy optimization, but the distance between the points on the curve and the desired position is also increased. In order to prevent the smoothed trajectory from deviating significantly from the initial path point, a distance deviation function is added, which is combined with the deformation energy function to obtain the optimization function.

[0075] In this embodiment, the Successive Approximation Method (SAM) can be used to iteratively solve the optimization function. That is, the optimization function is transformed into a quadratic programming problem using Taylor series. After solving the quadratic programming problem and obtaining the optimized solution, the new control points in the NURBS curve are substituted into the objective function to obtain the optimized objective function value.

[0076] In this embodiment, if the difference between the original objective function value and the optimized objective function value is less than the iteration termination threshold or the number of iterations is greater than the maximum number of iterations, then the iteration is stopped and the optimized solution is obtained.

[0077] In this embodiment, the objective function adaptively and dynamically adjusts the weight coefficients after each iteration. The adjustment method can be as follows: when the distance error is much smaller than the threshold, the proportion of the energy component is increased to maximize the smoothness of the curve while satisfying the distance error threshold; in addition, if optimization cannot be performed under the current weight coefficients, the proportion of the energy component will be reduced by shrinking the weight coefficients to ensure that the optimization is completed smoothly.

[0078] The point cloud-based humanoid 3D scanning motion control method of this invention obtains an optimization function by combining the deformation energy function and the distance deviation function, and then uses SAM to iteratively optimize the optimization function to obtain the target optimization function. Then, multiple new control points in the NURBS curve are input into the target optimization function to obtain the optimized target function value. Finally, when the iteration termination condition is met, the motion planning trajectory curve is obtained. This method can obtain the optimal solution to a multi-objective optimization problem and ensure the stability of the optimization process.

[0079] In some embodiments, obtaining the trajectory surface and the deformation energy function of the trajectory surface includes: inputting the trajectory surface into the thin plate energy model to obtain the deformation energy function.

[0080] In this embodiment, the trajectory surfaces corresponding to the camera viewpoint trajectory and the robotic arm end effector trajectory are input into the thin-plate energy model to construct the deformation energy function of the trajectory surface. The deformation energy function is then adjusted by modifying the C-axis of the NURBS curve. c and C e The coordinates of the control points can reduce the deformation energy of the surface and improve its smoothness.

[0081] In some embodiments, the motion planning trajectory curve of the object to be detected is input into a multi-objective motion control model for solution, and the optimal solution of the robotic arm motion control corresponding to the motion planning trajectory is obtained. This includes: summarizing trajectory tracking, attitude constraints, physical constraints and obstacle avoidance constraints into an objective optimization problem; solving the objective optimization problem based on a metaheuristic recurrent neural network, the beetle horn search algorithm (BAS) and the motion planning trajectory curve of the object to be detected, and obtaining the optimal solution of the robotic arm motion control corresponding to the motion planning trajectory.

[0082] It's important to note that the Beetle Antennae Search (BAS) algorithm mimics the behavior of longhorn beetles, using their pair of antennae to locate food in unknown environments based on scent. At each stage of the search, the beetle uses the scent from its two antennae to determine its next direction of exploration. During this process, the beetle stops each time to use its sense of smell to determine which direction is better before choosing the appropriate direction for its next move.

[0083] In this embodiment, using the BAS algorithm to solve the target optimization problem can reduce randomness to some extent; while the BAORNN algorithm, which combines the BAS algorithm and the metaheuristic recurrent neural network (RNN) to iteratively optimize the target optimization problem, can improve the algorithm's running efficiency while solving the optimization problem.

[0084] Figure 3This is a schematic diagram of the structure of the metaheuristic recurrent neural network and BAS algorithm combination provided by the present invention. Figure 3 In the illustrated embodiment, the process of solving the objective optimization problem using the BAS algorithm and a metaheuristic recurrent neural network is as follows: Assume that at time k, the robotic arm is located at joint space q k First, define a "beetle horn," generate a random direction vector b that follows a normal distribution, and normalize it. Then, calculate the positions of the left and right antennae, which are respectively located at the original q. k Add or subtract λ k b, λ k The objective function value is calculated using the above method, including the normalized influence of each joint of the robotic arm on the position of the end effector, which can be obtained from the pose transformation matrix. To prevent the end effector position from failing to meet the constraints, the position is projected into the constraint set to ensure that the joint angles, joint angular velocities, and the distance between the robotic arm and the obstacle meet the constraints. Then, the objective function value of the tentacle position is evaluated, and the updated position is calculated. The objective function value of the updated position is then calculated again, and it is determined whether it is better. If it is better, it is updated; otherwise, it remains unchanged. Finally, the robotic arm is controlled to move to q. k+1 If the threshold requirement is met, the iteration terminates.

[0085] The point cloud-based humanoid 3D scanning motion control method of this invention summarizes trajectory tracking, posture constraints, physical constraints, and obstacle avoidance constraints into a target optimization problem. The target optimization problem is solved based on a metaheuristic recurrent neural network, the beetle horn search algorithm (BAS), and the motion planning trajectory curve of the object to be detected. The optimal solution for the motion control of the robotic arm corresponding to the motion planning trajectory is obtained. Finally, the robotic arm is controlled to perform a full-coverage scan of the object to be detected according to the optimal solution, thereby improving scanning efficiency and accuracy.

[0086] In some embodiments, multiple optimization objectives include trajectory tracking, attitude constraints, physical constraints, and obstacle avoidance constraints. Attitude constraints are used to optimize the direction of the end effector of the robotic arm and the direction of the camera's field of view axis. Obstacle avoidance constraints are achieved by solving the distance relationship between the center of the robotic arm and the obstacle's spheres using the artificial multi-sphere approximation method. Physical constraints are used to constrain the joint angles and joint angular velocities of the robotic arm.

[0087] exist Figure 2In the illustrated embodiment, when setting up a humanoid 3D scanner to detect an object, it is necessary to control the robotic arm to move the camera along a specified trajectory for image capture. Therefore, trajectory tracking control is used to perform motion planning in joint space to achieve the movement of the end effector along the specified trajectory. The forward kinematics of the robotic arm realizes a nonlinear mapping from joint space to task space. To achieve trajectory tracking control, the joint angles need to be set to satisfy the inverse kinematics. However, for redundant robotic arms, there are multiple solutions to the inverse kinematics. Therefore, the motion trajectory tracking control of the robotic arm's end effector is modeled as an optimization problem to be solved.

[0088] Figure 4 This is a schematic diagram of attitude constraints provided by the present invention. Figure 4 In the embodiment shown, in order to achieve humanoid multi-target agile observation, the actual end effector direction and the camera field of view axis direction can be optimized by using attitude constraints. After the trajectory planning is completed, the desired end effector direction set is a vector direction composed of the points corresponding to the camera viewpoint trajectory curve and the end effector trajectory curve. The desired camera field of view axis direction can be specified manually or obtained in the viewpoint generation algorithm. The attitude constraint model of the robot arm motion trajectory is an optimization problem to be solved.

[0089] Figure 5 This is a schematic diagram of obstacle modeling provided by the present invention. Figure 5 In the illustrated embodiment, an artificial multi-sphere approximation method can be used to model obstacles on the robotic arm's motion path as multiple spheres to achieve complete envelopment of the obstacles, thereby simplifying the collision detection process and obtaining better approximation accuracy. During the envelopment process, the obstacle is first uniformly sliced ​​along the Z-axis to obtain a cross-section, and then spheres are used to envelop the obstacle along the cross-sectional curve until complete envelopment. By selecting appropriate radii and the number of spheres, complete envelopment and accurate modeling of the obstacle can be achieved. When modeling the robotic arm, cylinders are used for envelopment, and the positions of each joint during operation are obtained using initial joint angles and pose transformation matrices. The obstacle avoidance problem is then transformed into requiring the shortest distance between the center of the enveloping sphere and the robotic arm link to be greater than a distance threshold. Therefore, a vector method is used to find the shortest distance between the link and the center of the enveloping sphere.

[0090] In this embodiment, the physical constraint is due to the inherent characteristics of the robot and the requirements of the working environment. By physically constraining the joint angles of the robot arm, the solution is ensured to be within the mechanical limits of the joints.

[0091] The point cloud-based humanoid 3D scanning motion control method of this invention optimizes the direction of the end effector of the robotic arm and the direction of the camera's field of view axis by setting posture constraints, improves the robotic arm's obstacle avoidance performance by setting obstacle avoidance constraints, and improves the overall performance of the robotic arm when scanning complex structures by constraining the joint angles and joint angular velocities of the robotic arm through physical constraints, so as to meet the requirements of complex industrial environments and actual tasks.

[0092] The point cloud-based humanoid 3D scanning motion control device provided by the present invention will be described below. The point cloud-based humanoid 3D scanning motion control device described below can be referred to in correspondence with the point cloud-based humanoid 3D scanning motion control method described above.

[0093] Figure 6 This is a flowchart illustrating the point cloud-based humanoid 3D scanning motion control device provided by the present invention. The point cloud-based humanoid 3D scanning motion control device includes an acquisition module 610, a first processing module 620, a second processing module 630, and a third processing module 640.

[0094] The acquisition module 610 is used to acquire the camera viewpoint trajectory and the trajectory of the robotic arm end effector of the object to be detected.

[0095] The first processing module 620 is used to obtain the trajectory surface and the deformation energy function of the trajectory surface based on the camera viewpoint trajectory and the trajectory of the robotic arm end effector. The deformation energy function is used to optimize the smoothness of the trajectory surface.

[0096] The second processing module 630 is used to optimize the trajectory surface based on the deformation energy function and the distance deviation function to obtain the motion planning trajectory curve of the object to be detected. The distance deviation function is used to reduce the degree of deviation of the trajectory surface during the optimization process.

[0097] The third processing module 640 is used to input the motion planning trajectory curve of the object to be detected into the multi-objective motion control model for solving, so as to obtain the optimal solution of the robotic arm motion control corresponding to the motion planning trajectory. The multi-objective motion control model is obtained by mathematical modeling of multiple optimization objectives of the object to be detected.

[0098] The point cloud-based humanoid 3D scanning motion control device of this invention obtains the trajectory surface and its deformation energy function by using the camera viewpoint trajectory of the object to be detected and the trajectory of the robotic arm end effector. The trajectory surface is then optimized using the deformation energy function and a preset distance deviation function to obtain the motion planning trajectory curve. Finally, the motion planning trajectory curve of the object to be detected is input into a multi-objective motion control model for solution, obtaining the optimal solution for robotic arm motion control corresponding to the motion planning trajectory. This enables continuous scanning of multiple components or structures, improving scanning efficiency and accuracy. It can meet the requirements of complex industrial environments and actual tasks, and is more suitable for multi-component discrete manufacturing scenarios, possessing broad industrial application prospects and substantial economic benefits.

[0099] Figure 7 This is a schematic diagram of the physical structure of an electronic device provided by the present invention, such as... Figure 7 As shown, the electronic device may include: a processor 710, a communication interface 720, a memory 730, and a communication bus 740, wherein the processor 710, the communication interface 720, and the memory 730 communicate with each other through the communication bus 740. The processor 710 can call logic instructions in the memory 730 to execute a point cloud-based humanoid 3D scanning motion control method. This method includes: acquiring the camera viewpoint trajectory and the trajectory of the robotic arm end effector of the object to be detected; obtaining a trajectory surface and a deformation energy function of the trajectory surface based on the camera viewpoint trajectory and the robotic arm end effector trajectory, the deformation energy function being used to optimize the smoothness of the trajectory surface; optimizing the trajectory surface based on the deformation energy function and a distance deviation function to obtain the motion planning trajectory curve of the object to be detected, the distance deviation function being used to reduce the deviation of the trajectory surface during the optimization process; inputting the motion planning trajectory curve of the object to be detected into a multi-objective motion control model for solving, obtaining the optimal solution for robotic arm motion control corresponding to the motion planning trajectory, the multi-objective motion control model being obtained based on mathematical modeling of multiple optimization objectives of the object to be detected.

[0100] Furthermore, the logical instructions in the aforementioned memory 730 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0101] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the point cloud-based humanoid 3D scanning motion control method provided by the above methods. The method includes: acquiring the camera viewpoint trajectory and the trajectory of the robotic arm end effector of the object to be detected; obtaining the trajectory surface and the deformation energy function of the trajectory surface based on the camera viewpoint trajectory and the robotic arm end effector trajectory, wherein the deformation energy function is used to optimize the smoothness of the trajectory surface; optimizing the trajectory surface based on the deformation energy function and the distance deviation function to obtain the motion planning trajectory curve of the object to be detected, wherein the distance deviation function is used to reduce the deviation of the trajectory surface during the optimization process; inputting the motion planning trajectory curve of the object to be detected into a multi-objective motion control model for solving to obtain the optimal solution for robotic arm motion control corresponding to the motion planning trajectory, wherein the multi-objective motion control model is obtained by mathematically modeling multiple optimization objectives of the object to be detected.

[0102] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the point cloud-based humanoid 3D scanning motion control method provided by the above methods. The method includes: acquiring the camera viewpoint trajectory and the trajectory of the robotic arm end effector of the object to be detected; obtaining a trajectory surface and a deformation energy function of the trajectory surface based on the camera viewpoint trajectory and the robotic arm end effector trajectory, wherein the deformation energy function is used to optimize the smoothness of the trajectory surface; optimizing the trajectory surface based on the deformation energy function and the distance deviation function to obtain the motion planning trajectory curve of the object to be detected, wherein the distance deviation function is used to reduce the deviation of the trajectory surface during the optimization process; inputting the motion planning trajectory curve of the object to be detected into a multi-objective motion control model for solving to obtain the optimal solution for robotic arm motion control corresponding to the motion planning trajectory, wherein the multi-objective motion control model is obtained by mathematically modeling multiple optimization objectives of the object to be detected.

[0103] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0104] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0105] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A point cloud based anthropomorphic three-dimensional scanning motion control method, characterized in that, The method comprises the following steps: obtaining a camera viewpoint trajectory and a robot end effector trajectory of a to-be-detected object; based on the camera viewpoint trajectory and the robot end effector trajectory, obtaining a trajectory curve and a deformation energy function of the trajectory curve, the deformation energy function being used for optimizing the smoothness of the trajectory curve; based on the deformation energy function and a distance deviation function, optimizing the trajectory curve to obtain a motion planning trajectory curve of the to-be-detected object, the distance deviation function being used for reducing the deviation of the trajectory curve in the optimization process; inputting the motion planning trajectory curve of the to-be-detected object into a multi-objective motion control model to obtain a robot motion control optimal solution corresponding to the motion planning trajectory, the multi-objective motion control model being obtained based on mathematical modeling of multiple optimization objectives of the to-be-detected object.

2. The point cloud based anthropomorphic three-dimensional scan motion control method of claim 1, wherein, The method comprises the following steps: obtaining an initial viewpoint based on a three-dimensional point cloud of the to-be-detected object; based on a least squares progressive iterative approximation algorithm and a NURBS curve, fitting the initial viewpoint to obtain the camera viewpoint trajectory and the robot end effector trajectory.

3. The point cloud based anthropomorphic three-dimensional scan motion control method of claim 1, wherein, The method comprises the following steps: based on the deformation energy function and the distance deviation function, obtaining an optimization function; using a successive approximation method (SAM) to iteratively optimize the optimization function to obtain a target optimization solution; inputting multiple new control points in the NURBS curve into the target optimization function to obtain an optimized target function value, and when the difference between an original target function value and the optimized target function value is less than an iteration termination threshold or the number of iterations exceeds a maximum number of iterations, obtaining the motion planning trajectory curve, the original target function value being calculated by inputting the control points into the optimization function.

4. The point cloud based anthropomorphic three-dimensional scan motion control method of claim 1, wherein, The method comprises the following steps: inputting the trajectory curve into a thin plate energy model to obtain the deformation energy function.

5. The point cloud based anthropomorphic three-dimensional scan motion control method of claim 1, wherein, The multiple optimization objectives include trajectory tracking, pose constraint, physical constraint, and obstacle avoidance constraint, and the method comprises the following steps: summarizing the trajectory tracking, the pose constraint, the physical constraint, and the obstacle avoidance constraint into a target optimization problem; based on a meta-heuristic recursive neural network, a ball and stick algorithm (BAS), and the motion planning trajectory curve of the to-be-detected object, solving the target optimization problem to obtain the robot motion control optimal solution corresponding to the motion planning trajectory.

6. The point cloud based anthropomorphic three-dimensional scan motion control method of any one of claims 5, wherein, The pose constraint is used for optimizing the direction of the robot end effector and the direction of the camera field of view axis; The obstacle avoidance constraint is solved based on an artificial multi-sphere approximation method to obtain the sphere center distance relationship between the robot and the obstacle; The physical constraint is used for constraining the joint angle and the joint angular velocity of the robot.

7. A point cloud based anthropomorphic three-dimensional scanning motion control apparatus, characterized by, The method comprises the following steps: An acquisition module is configured to acquire a camera viewpoint trajectory and a robot end effector trajectory of a to-be-detected object; A first processing module is configured to obtain a trajectory surface and a deformation energy function of the trajectory surface based on the camera viewpoint trajectory and the robot end effector trajectory, and the deformation energy function is used to optimize the smoothness of the trajectory surface; A second processing module is configured to optimize the trajectory surface based on the deformation energy function and a distance deviation function to obtain a motion planning trajectory curve of the to-be-detected object, and the distance deviation function is used to reduce the deviation degree of the trajectory surface in the optimization process; A third processing module is configured to input the motion planning trajectory curve of the to-be-detected object into a multi-objective motion control model to obtain a robot motion control optimal solution corresponding to the motion planning trajectory, and the multi-objective motion control model is obtained based on mathematical modeling of multiple optimization objectives of the to-be-detected object.

8. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement the human-simulating three-dimensional scanning motion control method based on point cloud according to any one of claims 1 to 6. 9.A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the human-simulating three-dimensional scanning motion control method based on point cloud according to any one of claims 1 to 6.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the human-simulating three-dimensional scanning motion control method based on point cloud according to any one of claims 1 to 6.

Images