Layered reinforcement learning-based three-coordinate measuring machine detection path planning method

Abstract

The invention discloses a three-coordinate measuring machine detection path planning method based on hierarchical reinforcement learning. According to the hierarchical reinforcement learning model constructed by the invention, the optimal path generation considering the measurement angle change is realized, a bottom-layer agent adopts a weighted depth Q network to train a collision-free translation path planning strategy, and a top-layer agent adopts a group relative strategy optimization algorithm to train a measuring head rotation position decision strategy. On the basis, the learned reinforcement learning value function is used for estimating the measurement time between any two points, the measurement time is used for fitness calculation, and the optimal measurement sequence is solved through a heuristic algorithm. And finally, a complete detection path is generated in combination with the decision of the hierarchical agent and a post-processing algorithm. According to the method, the path value rather than the linear distance is used as an optimization target, efficient collaborative optimization of the measurement sequence, the obstacle avoidance path and the measurement head rotation is achieved, the total measurement time is remarkably shortened, and the problem that the efficiency is low due to planning disjunction in a traditional method is solved.

Description

Technical Field

[0001] This invention relates to the field of intelligent manufacturing and automated inspection technology, specifically to a method for planning the inspection path of a coordinate measuring machine based on hierarchical reinforcement learning. Background Technology

[0002] In modern industrial production, coordinate measuring machines (CMMs) are widely used for the inspection of the geometric dimensions and form and position tolerances of high-precision parts. To improve inspection efficiency, automated planning of inspection paths is crucial. Traditional path planning methods mainly involve two key technologies: first, optimization of the measurement point sequence (such as heuristic algorithms), and second, collision-free path generation based on collision detection.

[0003] However, existing technologies have significant drawbacks: the method of determining the measurement sequence based on the straight-line distance between measurement points and then planning obstacle avoidance for each path segment within that sequence ignores the impact of obstacle avoidance and probe rotation on path length, resulting in a suboptimal total measurement time. Conversely, the method of first planning obstacle avoidance between any two points and then optimizing the measurement sequence based on the obstacle avoidance path length requires explicitly determining a collision-free path, incurring enormous computational overhead and significantly reducing planning efficiency. Furthermore, the current use of simple Euclidean distance as the optimization objective fails to reflect the true measurement costs, including the complexity of obstacle avoidance paths and the planning of probe rotation positions. When measurement tasks require adjusting the probe angle, determining the optimal probe rotation position to minimize overall measurement time remains a complex and poorly resolved problem.

[0004] Therefore, there is an urgent need for an efficient and intelligent path planning method that can collaboratively optimize measurement sequence, collision-free path, and probe rotation decision-making. Summary of the Invention

[0005] The purpose of this invention is to overcome the above-mentioned defects of the prior art and provide a three-coordinate measuring machine detection path planning method based on hierarchical reinforcement learning. It aims to achieve efficient collaborative optimization of measurement sequence, collision-free path and probe rotation decision, significantly shorten the actual measurement time, and is especially suitable for workpiece detection scenarios with complex obstacles and multiple measurement angle requirements.

[0006] The technical solution of the present invention is as follows: I. A Path Planning Method for Coordinate Measuring Machine Detection Based on Hierarchical Reinforcement Learning (1) Construct a hierarchical reinforcement learning model that includes a bottom-level agent and a top-level agent, wherein the bottom-level agent is used to plan a collision-free translation path between two points with the same measurement angle, and the top-level agent is used to decide the optimal probe rotation position between two points with different measurement angles. (2) Train the hierarchical reinforcement learning model in a simulation environment so that the bottom agent learns the value function for planning a collision-free translation path and the top agent learns the strategy for deciding the optimal probe rotation position. (3) Determine the actual detection path planning task of the coordinate measuring machine and the corresponding optimization target, and then use the trained hierarchical reinforcement learning model to optimize the measurement sequence of the workpiece to be tested and the global detection path planning, so as to generate the final detection path.

[0007] In the underlying intelligent agent, the reward is calculated using the following formula:

[0008]

[0009]

[0010] in, Represents the reward that the underlying agent receives from the environment under the current state and action; it is a process reward. Rewards for successful arrival Penalties for collisions constitute; Representative coefficient factor, This represents the distance between the current center of the ball and the target point; This represents the reward for a successful plan. The threshold representing the success of the plan; This indicates rounding down a real number; This represents the penalty for a collision. This represents the threshold at which a collision occurs.

[0011] In the top-level intelligent agent, the state Satisfy the following formula:

[0012] in, This represents the state characteristics observed by the underlying agent from the environment at time t; The binary occupancy grid cell representing the current workpiece; The action is a possible rotation position within a preset spatial range. Its decision distribution follows the policy of the top-level agent; The reward consists of two translational motion rewards, calculated using the following formula:

[0013] Where p represents the starting position and q represents the target position. and This represents the position of the center of the probe ball before and after the probe rotates. The state encoding function representing the underlying intelligent agent; The optimal value function represents the underlying intelligent agent.

[0014] Specifically, (3) refers to: In the detection path planning task, any two adjacent measurement points form a measurement point pair. The trained hierarchical reinforcement learning model is used to evaluate the local path value of each measurement point pair of the workpiece under test, thereby obtaining the evaluation value corresponding to each measurement point pair. Combined with the optimization objective, the measurement order in the detection path planning task is optimized to obtain the optimal measurement order. Then, the hierarchical reinforcement learning model is used again to plan local paths for each measurement point pair corresponding to the optimal measurement order. Finally, the local paths corresponding to each measurement point pair are post-processed to obtain the final detection path.

[0015] The evaluation values ​​for each measurement point satisfy the following formula: in, This represents the evaluation value for each measurement point pair, where p represents the starting position of each measurement point pair, and q represents the target position of each measurement point pair. and These represent the measurement angles corresponding to the starting position p and the target position q, respectively. and This represents the position of the center of the probe ball before and after the probe rotates; The state encoding function representing the underlying intelligent agent; The optimal value function represents the underlying intelligent agent.

[0016] In the measurement sequence optimization process of the detection path planning task, a fitness value for the optimization method is generated based on the evaluation value of each measurement point. The formula is as follows:

[0017]

[0018] in, This represents the evaluation value for each pair of measurement points, where I represents the measurement sequence. This represents the (i+1)th measurement point, where i = 1, ..., N-1, and N represents the number of measurement points. This represents the initial position of the probe; Represents the natural logarithm; The post-processing includes state decoding and pruning compression.

[0019] II. A computer device The device includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the coordinate measuring machine detection path planning method based on hierarchical reinforcement learning.

[0020] III. A computer-readable storage medium The medium stores a computer program, which, when executed by a processor, implements the steps of the coordinate measuring machine detection path planning method based on hierarchical reinforcement learning.

[0021] IV. A computer program product The product includes a computer program / instruction that, when executed by a processor, implements the steps of the coordinate measuring machine detection path planning method based on hierarchical reinforcement learning.

[0022] The beneficial effects of this invention include: Significantly reduces measurement time: This invention ensures the global optimality of the measurement sequence by using the actual path cost (estimated by a value function) rather than the straight-line distance as the optimization objective.

[0023] Achieving efficient collaborative planning: This invention adopts a strategy of "first local path evaluation (value function estimation), then global sequential planning", which avoids the huge computational overhead of generating complete obstacle avoidance paths for each pair of measurement points and greatly improves planning efficiency.

[0024] Effectively solves the problem of multi-angle measurement: This invention uses a hierarchical reinforcement learning framework, in which the top-level agent can intelligently decide the optimal probe rotation position, realizing complete path planning for multi-angle measurement scenarios.

[0025] It has engineering application value: This invention can be effectively integrated into existing computer-aided planning and inspection systems (CAIP) to improve their automation, intelligence and overall planning efficiency. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of the overall process of the method of the present invention.

[0027] Figure 2 This is a schematic diagram of the probe's pose and the target measurement point.

[0028] Figure 3 This is a schematic diagram of the movable space of the probe.

[0029] Figure 4 This is a schematic diagram of the underlying value function network.

[0030] Figure 5 This is a schematic diagram of the training process.

[0031] Figure 6 This is a schematic diagram of the detection path planning process based on hierarchical reinforcement learning.

[0032] Figure 7 This is a schematic diagram of the initial path planned by the algorithm.

[0033] Figure 8 This is a schematic diagram of the final path after pruning.

[0034] Figure 9 Schematic diagram of detection path planning results for typical geometric features Figure 1 .

[0035] Figure 10 Schematic diagram of detection path planning results for typical geometric features Figure 2 .

[0036] Figure 11 Schematic diagram of detection path planning results for typical geometric features Figure 3 . Detailed Implementation

[0037] To better understand the technical solution of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific examples.

[0038] like Figure 1 As shown, the method for path planning of a coordinate measuring machine based on hierarchical reinforcement learning proposed in this invention specifically includes the following steps: (1) Construct a hierarchical reinforcement learning model containing a bottom-level agent and a top-level agent for local path planning. The bottom-level agent is used to plan a collision-free translation path between two points with the same measurement angle (i.e., measurement points). In other words, the bottom-level agent is responsible for planning a collision-free translation path from the current position to the target position while keeping the probe angle unchanged. The top-level agent is used to decide the optimal probe rotation position between two points with different measurement angles. In other words, the top-level agent is responsible for deciding the probe rotation position for the bottom-level agent when the probe angle needs to be adjusted. Then, the bottom-level agent plans two translation paths. At this time, the local path between the first measurement point and the second measurement point consists of two sub-paths. The underlying intelligent agent uses discrete state space and action space, with the underlying state... Using a partially observable Markov decision process design method, historical experience from k time steps backward from the current time t is taken and merged with the current observation into a single state. The details are as follows:

[0039] Where k represents the sequence length, This represents the state characteristics observed by the underlying agent from the environment at time t. This represents the state characteristics observed in the environment at time t-1. Represents the state characteristics observed in the environment at time t-(k-1); such as Figure 2 As shown, It represents the current position of the ball's center, consisting of the center point position and the probe angle; The target point is represented by the position of the approach point corresponding to the measurement point and the measurement angle. The approach point is the point obtained by moving a small distance along the plane normal to the location of the measurement point. Therefore, there is a one-to-one correspondence between the measurement point and the approach point. The relative expression compresses the state space, reduces the difficulty of model learning, and improves the model's final generalization performance. This represents the current distance of the probe system to the nearest obstacle; This represents the minimum range of motion of the probe system along the three coordinate axes, such as... Figure 3 As shown. Potential collisions between the probe and the workpiece are limited to the part and a certain area around it, which is represented by an extended bounding box EB, and relevant information is only obtained within EB.

[0040] Low-level actions The motion is selected from the discrete motion space, and the total motion space consists of translational motions along three coordinate axes, as shown in Table 1.

[0041] Table 1 Action Space of the Low-Level Agent

[0042] Where X, Y, and Z represent the three coordinate axes; 0.05 represents the relative translation step size of the normalization process, which normalizes the maximum axial bounding box of the part to (-0.5, -0.5, -0.5, 0.5, 0.5, 0.5), ensuring the universality of models of different sizes, simulating the measurement environment, and enhancing the generalization ability of the planning strategy.

[0043] The formula for calculating the reward is as follows:

[0044]

[0045]

[0046]

[0047] in, Represents the reward that the underlying agent receives from the environment under the current state and action; it is a process reward. Rewards for successful arrival Penalties for collisions constitute; This represents the coefficient factor, which here is set to 0.001; This represents the distance between the current center of the ball and the target point; This represents the reward for a successful plan, and is set to 1 here. The threshold representing the success of the plan, i.e. If the value is less than or equal to this value, it is considered that the target has been reached; here, the value is 0.05. This indicates rounding down a real number, when... When the integer result is less than 0, When the result of rounding is 0, the result of the max function that finds the maximum value is 1, thus giving the agent a reward for reaching the target. This represents the penalty for a collision, and here it is set to -0.5. This represents the threshold at which a collision occurs, the minimum distance from an obstacle. A collision is determined when the value is less than this threshold, and a value close to 0 is taken. Assuming that the time for each step is equal, maximizing the cumulative reward is approximately equivalent to minimizing the measurement time.

[0048] In the top-level intelligent agent, after selecting a probe rotation position, the measurement process is decomposed into three parts: translating from the current position to the rotation position, adjusting the probe angle at the rotation position, and translating from the rotated position to the target position. Ignoring the time differences in adjusting different probe angles, the theoretical optimization objective is as follows:

[0049] in, and This represents the position of the center of the probe ball before and after the probe rotates. represents the probe's movement time, and p and q represent the planned starting and target positions.

[0050] The selection of the probe rotation position is modeled as a Markov decision process with only one action: This means that when a decision fails, it stops making further decisions and uses a continuous action reinforcement learning model to solve the problem. and Represents the top-level state at the beginning and end. The probe rotation position represents the strategic decision. The reward received by the top-level intelligent agent.

[0051] The specific status, actions, and reward settings are as follows: state Satisfy the following formula

[0052] in, This represents the state characteristics observed by the underlying agent from the environment at time t; The binary occupancy mesh voxel representing the current workpiece is a fixed value that does not change over time in a single measurement task. If multiple workpieces are being measured, they are treated as a single unit. In actual measurement, voxelization is completed during the workpiece import phase to obtain the corresponding voxel mesh. Even with the same starting point and target, the optimal rotation position of different parts is very likely to differ. Increasing the observation information of the task scenario improves the accuracy of prediction, while the binary occupancy mesh balances spatial information representation and computational efficiency.

[0053] The action is a possible rotation position within a preset spatial range. Its decision distribution follows the policy of the top-level agent, specifically adopting a Gaussian distribution. ,Right now:

[0054] Among them, two distribution parameters and Predicted by strategy C, i.e. During training, The output will be sampled from the decision distribution, meaning it is not deterministic. However, during the path planning phase after training, a deterministic output will be used. .

[0055] The reward consists of two translational motion rewards, calculated using the following formula:

[0056] Where p represents the starting position and q represents the target position. and This represents the position of the center of the probe ball before and after the probe rotates. The state encoding function represents the underlying agent, and its output is the underlying state. ; The optimal value function representing the underlying intelligent agent is the translation path.

[0057] In a Markov decision process with only one action, and ignoring the influence of the final state on the cumulative reward, the reward function in the reinforcement learning model is equivalent to the action-state value function, i.e. Thus, the actual optimization objective for the reinforcement learning model is obtained:

[0058] in, These represent the network parameters of policy C in the top-level reinforcement learning model; Represents the initial top-level state. Top-level reward. Because the motion time of a local collision-free path from the current position p to the next position q is implicitly estimated by calculating the value function of the underlying agent, including rotational positions.

[0059] (2) A simulation environment for the coordinate measuring machine (CMM) is established. Then, the CMM's detection path planning task, the first optimization objective, and safety constraints are determined. A hierarchical reinforcement learning model is trained in the simulation environment, enabling the bottom-level agent to learn the value function for planning a collision-free translation path, and the top-level agent to learn the strategy for deciding the optimal probe rotation position. Specifically, the CMM's detection path planning task involves determining the optimal detection path for the probe to traverse all measurement points on a specified surface of the workpiece and its corresponding measurement angles, while satisfying the safety constraints. The safety constraints specifically ensure that no collisions occur between the CMM's motion axes, the probe system, the workpiece, and the measurement platform. The first optimization objective includes local paths between adjacent measurement points, thus minimizing the overall motion time consumed by the CMM when performing the detection path planning task.

[0060] The specific training methods include: (a) Construct a training scenario in the simulation platform and synchronize the constraints in the detection path planning task to the training environment; In one feasible implementation, when constructing the training scene, the workpiece model is processed using triangular meshes, and the workpiece size is normalized to a uniform scale; typical geometries such as cylinders and spheres are used to represent the probe system, and point cloud processing is performed. Constraint synchronization is achieved by using BVH trees and ray casting methods to sense the distance between the probe and the workpiece in space.

[0061] (b) For the planning of translation path, the underlying agent drives the probe to interact with the virtual environment to generate experience data, and performs several rounds of training until the cumulative reward tends to be maximized, and obtains the value function after training is completed; In one feasible implementation, the underlying agent value function uses an action state value function. The optimal result after training is The optimal value function ,in The action space represents the underlying intelligent agent; the specific training process is as follows: At each time step, the action state value function network of the underlying agent calculates the state obtained at time t. Calculate all actions Value, then select to let Biggest movement The probe is driven to complete the corresponding action; after the action is completed, the agent receives a reward. And the state in the next moment. Then the state of the current time step. ,action ,award and the state at the next moment Data is stored as a set of experience data in an experience pool; at regular intervals, data is extracted from the experience pool and the value function network parameters are trained using a Weighted Deep Q-Network (WDQN) method, specifically including: The cumulative reward for the target is calculated using a weighted average, as follows:

[0062]

[0063] in, This represents the cumulative reward in the n-step method. This is a special form when n=1; and Represent and The weight, This represents its weighted result.

[0064] The specific network structure uses a Double DQN network structure, which includes two value function networks that refer to each other to calculate the TD loss; it also includes two corresponding target networks, which are periodically updated copies of the main network. Their parameter update frequency is lower than that of the main network, and they are used to provide a stable target Q value reference when calculating the TD loss. However, unlike the soft update method in the original method, this method uses direct replacement to periodically replace the target network parameters with new network parameters, with an update frequency of 2500 interaction rounds. The sampling method employs a priority experience replay sampling technique, which dynamically updates sample weights after updating network parameters, thereby improving sample utilization. In this priority experience replay, the priority index α for key parameters is set to 0.6, the importance sampling correction intensity β is set to 0.5, and the minimum absolute priority ε is set to 1e. -5 .

[0065] (c) Regarding the planning of the probe rotation position, the top-level agent drives the probe to interact with the environment to generate experience data, and conducts several rounds of training until the cumulative reward tends to be maximized, thus obtaining the trained strategy.

[0066] In one feasible implementation, the top-level agent policy is trained using the Group Relative Policy Optimization (GRPO) method. The specific process is as follows: Based on the state at the initial moment predict From the probability distribution, G candidate actions are sampled. Then, the lower-level agent executes the movements from the current position to the rotated position and from the rotated position to the target position. After the movements are completed, the agent receives a reward. Then the state ,action and rewards The data is stored as a set of experience data in the experience pool. The above process is repeated for G candidate actions, and for each initial state, a total of G sets of data are collected. Then, the average reward of the G sets of data is calculated, and this is used as a baseline to calculate the advantage value of each set of data. After a certain number of samples are collected, the gradient is calculated and the parameters of the policy network are updated.

[0067] (3) Determine the actual detection path planning task of the coordinate measuring machine and the corresponding second optimization objective. Then, use the trained hierarchical reinforcement learning model to optimize the measurement sequence of the workpiece to be measured and plan the global detection path, thereby generating the final detection path. The second optimization objective includes the optimal measurement sequence. When combined with the first optimization objective, it can minimize the overall motion time consumed by the coordinate measuring machine in the detection path planning task.

[0068] (3) Specifically: In the detection path planning task, any two adjacent measurement points form a measurement point pair. A trained hierarchical reinforcement learning model is used to evaluate the local path value of each measurement point pair on the workpiece under test. The top-level agent decides the optimal probe rotation position corresponding to the measurement point pair composed of two points with different measurement angles, while the bottom-level agent estimates the value function to obtain the motion time corresponding to each measurement point pair, thus obtaining the evaluation value for each measurement point pair. Combined with the second optimization objective, the measurement order in the detection path planning task is optimized to obtain the optimal measurement order. Then, the hierarchical reinforcement learning model is used again to plan local paths for each measurement point pair corresponding to the optimal measurement order. Finally, after post-processing the local paths corresponding to each measurement point pair, the final detection path is obtained.

[0069] The cost (time) of a collision-free path between any two points is estimated using a hierarchical reinforcement learning model and used as an evaluation value. The evaluation value for each measurement point satisfies the following formula:

[0070] in, This represents the evaluation value for each measurement point pair, where p represents the starting position of each measurement point pair, and q represents the target position of each measurement point pair. and These represent the measurement angles (or probe angles) corresponding to the starting position p and the target position q, respectively. and This represents the position of the center of the probe ball before and after the probe rotates; The state encoding function representing the underlying intelligent agent; The optimal value function represents the underlying intelligent agent.

[0071] The above formula includes two scenarios: when the angle remains constant, the bottom-level agent directly estimates the translation time; when the angle changes, the top-level agent first decides the optimal rotation position when adjusting the probe angle, and then the bottom-level agent estimates the time of multiple motion segments.

[0072] In the process of optimizing the measurement sequence in the detection path planning task, the fitness value of the optimization method is generated based on the corresponding evaluation value of each measurement point. The formula is as follows:

[0073]

[0074] in, This represents the evaluation value for each pair of measurement points, where I represents the measurement sequence. This represents the (i+1)th measurement point, where i = 1, ..., N-1, and N represents the number of measurement points. This represents the initial position of the probe, which is generally kept fixed. Representing the natural logarithm, thus ensuring that the maximum fitness value is equivalent to the shortest measurement time; In one feasible implementation, the optimization method employs a heuristic algorithm, ultimately outputting a fitness value. The measurement sequence corresponding to the largest value is used as the optimal measurement order.

[0075] After determining the optimal measurement sequence, let For the optimal measurement sequence The hierarchical reinforcement learning model is used again to plan the path sequentially. Specifically, regarding the initial state Analyze whether the measured angles before and after are the same: if they are the same, call the underlying reinforcement learning model, use a greedy strategy to select the action that maximizes the Q value and drive the probe to move. After the movement is completed, obtain the state at the next moment, and repeat the process until the target position is reached. If they are different, the path is split into two translation sub-paths based on the rotation position decided by the top-level reinforcement learning model during sequential planning. Then, the bottom-level reinforcement learning model is called to repeat the translation path planning when the angle remains unchanged.

[0076] In one feasible implementation, post-processing includes state decoding and pruning compression. Specifically, since the detection path generated by the reinforcement learning model is expressed in the form of a state sequence and contains redundant points, it needs to be decoded into three-dimensional coordinate points and then optimized using the Ramer-Douglas-Peucker (RDP) pruning algorithm to obtain the final detection path.

[0077] This invention provides a path planning method for a coordinate measuring machine (CMM) based on hierarchical reinforcement learning, which consists of two phases: training and deployment. The specific implementation process is as follows: A Development Training The CAD models of the measuring machine, probe assembly, and parts were exported as STL files, and then XML files were written according to the import format requirements of the Mujoco simulation software. The Open3d library was used to build the simulation environment, including model import and processing, and distance observation between the probe and the environment. The PyTorch framework was used to develop the reinforcement learning model, where the underlying agent value function network uses a multilayer perceptron structure, combined with gating layers and residual blocks to improve its fitting ability. Figure 4 As shown, the gating layer input is processed by the weight matrix. W After dimensionality transformation, the sigmoid activation function is used. The residual block is designed as a feedforward neural network (FNN) that first increases and then decreases in dimensionality, and it uses the Dropout module, layer normalization, and ReLU activation function. The top-level agent policy network uses the classic 3D-ResNet18 network model, with convolution kernels, pooling, and other parameters remaining unchanged, but the residual stage is reduced from 4 to 3. Other key hyperparameters are shown in Table 2.

[0078] Table 2. Key hyperparameters for training a hierarchical reinforcement learning model for path planning in three-coordinate detection.

[0079] After developing the reinforcement learning model and setting up the training environment, the agent is then instructed to interact with the environment, collect samples, and undergo training. Figure 5As shown in Table 2, the bottom-level agent, upon receiving the bottom-level state, decides on the optimal bottom-level action. However, in the early stages of training, this action is randomly selected from the action space. As the number of iterations increases, it switches to a greedy strategy, selecting the action that maximizes the action state value. The top-level agent, upon receiving the top-level state, outputs the top-level action, i.e., the probability distribution of the rotation position. The actual action is sampled from this distribution. However, due to the GRPO training method, G actions need to be sampled for the same state. Therefore, a multi-process parallel training method is used, replicating multiple environments and models for sample collection. The storage order of the G sets of samples is not explicitly required. The data sampled by both the upper and lower-level models are stored in the experience pool, and the network parameters are updated based on the Adam optimizer. However, due to the different reinforcement learning models, the training modes and hyperparameters differ (Table 2). The bottom-level model uses a mode of training while interacting, while the top-level model uses a mode of training after interacting, which is the difference between "rounds" and "generations" in Table 2.

[0080] Performance evaluations are performed periodically during training, and the planning success rate is a core metric for evaluating the performance of reinforcement learning models. The criteria for determining whether a plan is successful are whether the center of the probe has moved within the target location's defined range and whether the current measuring angle of the probe matches the target angle.

[0081] B Deployment Verification The trained hierarchical reinforcement learning model is then... Figure 6 The flowchart is deployed into a coordinate measuring machine (CMM) path planning framework, where the heuristic algorithm uses Discrete Particle Swarm Optimization (DP) and the pruning algorithm uses Restricted Dynamic Programming (RDP). A typical CMM path planning system includes modules for feature extraction, measurement point and angle analysis, measurement sequence planning, and collision-free path planning between measurement points. A hierarchical reinforcement learning model is used to replace the collision-free path planning module in the traditional method, and the operational logic of measurement sequence planning and collision-free path planning in the traditional method is adjusted to achieve efficient unification of the two.

[0082] Figure 7 It demonstrates the initial path planned based on a reinforcement learning model. Figure 8 The path processed by the RDP algorithm is shown. Redundant points that do not meet the actual movement requirements of the device are removed from the result, making it the final path output for downstream tasks, such as measurement sequence programming. This serves as a generalization verification. Figure 9 , Figure 10 and Figure 11 Further path planning experiments for other geometric shapes, including box structures, bodies of revolution, and complex freeform surfaces, are provided, fully demonstrating the effectiveness of the method of the present invention.

[0083] The basic principles and main features of the present invention have been described in detail above with reference to the accompanying drawings. Using the above invention, the efficiency and quality of the detection path planning method for coordinate measuring machines can be effectively improved. Although the above embodiments only use WDQN and GRPO as hierarchical reinforcement learning model training methods and particle swarm optimization as heuristic algorithms to achieve the detection path planning task, the present invention can also implement detection path planning for other reinforcement learning algorithms and other heuristic optimization algorithms, and can ensure the effectiveness and efficiency of the planning. The scope of protection of the present invention is defined by the appended claims, and any modifications made based on the claims of the present invention are within the scope of protection of the present invention.

Claims

1. A path planning method for a coordinate measuring machine based on hierarchical reinforcement learning, characterized in that, Includes the following steps: (1) Construct a hierarchical reinforcement learning model that includes a bottom-level agent and a top-level agent, wherein the bottom-level agent is used to plan a collision-free translation path between two points with the same measurement angle, and the top-level agent is used to decide the optimal probe rotation position between two points with different measurement angles. (2) Train the hierarchical reinforcement learning model in a simulation environment so that the bottom agent learns the value function for planning a collision-free translation path and the top agent learns the strategy for deciding the optimal probe rotation position. (3) Determine the actual detection path planning task of the coordinate measuring machine and the corresponding optimization target, and then use the trained hierarchical reinforcement learning model to optimize the measurement sequence of the workpiece to be tested and the global detection path planning, so as to generate the final detection path.

2. The method for path planning of a coordinate measuring machine based on hierarchical reinforcement learning according to claim 1, characterized in that, In the underlying intelligent agent, the reward is calculated using the following formula: in, Represents the reward that the underlying agent receives from the environment under the current state and action; it is a process reward. Rewards for successful arrival Penalties for collisions constitute; Representative coefficient factor, This represents the distance between the current center of the ball and the target point; This represents the reward for a successful plan. The threshold representing the success of the plan; This indicates rounding down a real number; This represents the penalty for a collision. This represents the threshold at which a collision occurs.

3. The method for path planning in a coordinate measuring machine based on hierarchical reinforcement learning according to claim 1, characterized in that, In the top-level intelligent agent, the state Satisfy the following formula: in, This represents the state characteristics observed by the underlying agent from the environment at time t; The binary occupancy grid cell representing the current workpiece; The action is a possible rotation position within a preset spatial range. Its decision distribution follows the policy of the top-level agent; The reward consists of two translational motion rewards, calculated using the following formula: Where p represents the starting position and q represents the target position. and This represents the position of the center of the probe ball before and after the probe rotates. The state encoding function representing the underlying intelligent agent; The optimal value function represents the underlying intelligent agent.

4. The method for path planning of a coordinate measuring machine based on hierarchical reinforcement learning according to claim 1, characterized in that, Specifically, (3) refers to: In the detection path planning task, any two adjacent measurement points form a measurement point pair. The trained hierarchical reinforcement learning model is used to evaluate the local path value of each measurement point pair of the workpiece under test, thereby obtaining the evaluation value corresponding to each measurement point pair. Combined with the optimization objective, the measurement order in the detection path planning task is optimized to obtain the optimal measurement order. Then, the hierarchical reinforcement learning model is used again to plan local paths for each measurement point pair corresponding to the optimal measurement order. Finally, the local paths corresponding to each measurement point pair are post-processed to obtain the final detection path.

5. The method for path planning of a coordinate measuring machine based on hierarchical reinforcement learning according to claim 4, characterized in that, The evaluation values ​​for each measurement point satisfy the following formula: in, This represents the evaluation value for each measurement point pair, where p represents the starting position of each measurement point pair, and q represents the target position of each measurement point pair. and These represent the measurement angles corresponding to the starting position p and the target position q, respectively. and This represents the position of the center of the probe ball before and after the probe rotates; The state encoding function representing the underlying intelligent agent; The optimal value function represents the underlying intelligent agent.

6. The method for path planning of a coordinate measuring machine based on hierarchical reinforcement learning according to claim 4, characterized in that, In the measurement sequence optimization process of the detection path planning task, a fitness value for the optimization method is generated based on the evaluation value of each measurement point. The formula is as follows: in, This represents the evaluation value for each pair of measurement points, where I represents the measurement sequence. This represents the (i+1)th measurement point, where i = 1, ..., N-1, and N represents the number of measurement points. This represents the initial position of the probe; It represents the natural logarithm.

7. The method for path planning of a coordinate measuring machine based on hierarchical reinforcement learning according to claim 4, characterized in that, The post-processing includes state decoding and pruning compression.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the three-coordinate measuring machine detection path planning method based on hierarchical reinforcement learning as described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the three-coordinate measuring machine detection path planning method based on hierarchical reinforcement learning as described in any one of claims 1 to 6.

10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the three-coordinate measuring machine detection path planning method based on hierarchical reinforcement learning as described in any one of claims 1 to 6.

Images