A method for synchronous optimization of measurement viewpoint and measurement path based on reinforcement learning

Abstract

The present invention discloses a method for synchronous optimization of measurement viewpoints and measurement paths based on reinforcement learning. A dual convolutional neural network for synchronous optimization of measurement viewpoints and measurement paths is constructed based on DQN. An intelligent agent learns by interacting with the environment through states, actions and rewards. A robot posture set and coverage information are input as a state set. The robot moves to any viewpoint position as an action set. A reward function is designed according to the goals of meeting coverage requirements and minimizing the total motion cost. Strategic learning of synchronous optimization of measurement viewpoints and measurement paths is completed by training a neural network based on DQN. The initial state is input into the trained neural network for viewpoint selection and path optimization. A set of viewpoints that meet the coverage requirements and a path with the lowest cost for accessing the viewpoints are output, thereby obtaining the best measurement viewpoint and the best motion path, avoiding falling into a local optimum, and achieving better planning effect.

Description

Technical Field

[0001] The present invention belongs to the field of industrial robot path planning, and more specifically, relates to a method for synchronously optimizing a measurement viewpoint and a measurement path based on reinforcement learning. Background Art

[0002] With the development of surface structured light technology and industrial robot technology, current industrial measurement often combines the two for automated three-dimensional measurement. The industrial robot positions the three-dimensional measurement equipment, moves along the planned motion path to the pre-planned viewpoint position, scans and measures the target object from each viewpoint position, and then splices the single-piece data to obtain complete three-dimensional data of complex industrial parts.

[0003] Existing measurement planning methods mostly solve a set of viewpoints that cover the required surface area of ​​the target object first, and then find the optimal collision-free robot path that visits all viewpoints. The former is an NP-hard problem, and the latter is a traveling salesman problem. Solving both separately is computationally complex and easily falls into local optimality, affecting the overall optimization quality. Summary of the invention

[0004] In view of the above defects or improvement needs of the prior art, the present invention provides a method for synchronous optimization of measurement viewpoints and measurement paths based on reinforcement learning. By establishing a synchronous optimization model of viewpoint sampling and viewpoint access order, it can avoid the complexity of separate solution and fall into local optimality, obtain the best measurement viewpoint and the best motion path, and achieve better planning effect, which can effectively improve the efficiency of automated measurement.

[0005] To achieve the above object, according to a first aspect of the present invention, a method for synchronously optimizing a measurement viewpoint and a measurement path based on reinforcement learning is provided, which is characterized by comprising:

[0006] S1, randomly generating a viewpoint set in a preset sampling space, wherein the direction of each viewpoint in the viewpoint set points to the body center of the three-dimensional model of the workpiece to be measured; solving the posture of the industrial robot corresponding to each viewpoint based on the inverse kinematics principle and performing collision detection, and deleting unreachable viewpoints to obtain a candidate viewpoint set;

[0007] S2, calculate the local path and motion cost between any two candidate viewpoints and store them in the motion graph;

[0008] S3, determining whether each surface piece from each candidate viewpoint to the three-dimensional model of the workpiece to be measured is visible, so as to construct a visibility matrix;

[0009] S4, constructing a dual convolutional neural network for synchronously optimizing the measurement viewpoint and the measurement path based on DQN, iteratively training the dual convolutional neural network with the candidate viewpoint set, the motion map and the visibility matrix as training sets, and obtaining a trained synchronous optimization model of the measurement viewpoint and the measurement path;

[0010] S5, inputting the initial posture and initial coverage information of the industrial robot into the trained measurement viewpoint and measurement path synchronization optimization model to obtain the measurement viewpoint and measurement path of the workpiece to be measured.

[0011] According to a second aspect of the present invention, there is provided a system for synchronous optimization of measurement viewpoint and measurement path based on reinforcement learning, characterized in that it comprises: a computer-readable storage medium and a processor;

[0012] The computer-readable storage medium is used to store executable instructions;

[0013] The processor is used to read the executable instructions stored in the computer-readable storage medium and execute the method as described in the first aspect.

[0014] In general, the above technical solutions conceived by the present invention can achieve the following beneficial effects compared with the prior art:

[0015] 1. The reinforcement learning-based simultaneous optimization method of measurement viewpoint and measurement path provided by the present invention constructs a dual convolutional neural network based on DQN for simultaneous optimization of measurement viewpoint and measurement path. The intelligent agent learns by interacting with the environment through state, action and reward. The robot posture set and coverage information are input as the state set, and the robot moves to any viewpoint position as the action set. The reward function is designed according to the goals of meeting the coverage requirements and minimizing the total motion cost. The strategic learning of simultaneous optimization of measurement viewpoint and measurement path is completed by training the DQN-based neural network. The initial state is input into the trained neural network for viewpoint selection and path optimization, and a set of viewpoint sets that meet the coverage requirements and the path with the lowest cost to access the viewpoint set are output, so as to obtain the best measurement viewpoint and the best motion path, avoid falling into the local optimum, and achieve better planning effect.

[0016] 2. The reinforcement learning-based simultaneous optimization method for measurement viewpoints and measurement paths provided by the present invention defines a new set coverage reward function according to the data coverage increment and motion cost, and adds reflection constraints when calculating the data coverage increment to prevent image overexposure or underexposure and ensure the camera imaging quality, thereby obtaining complete three-dimensional data of complex industrial parts and achieving high-precision three-dimensional measurement.

[0017] 3. The reinforcement learning-based simultaneous optimization method of measurement viewpoint and measurement path provided by the present invention can automatically generate a measurement planning scheme according to the parameters of the surface structured light three-dimensional measurement system, the parameters of the industrial robot and the model of the workpiece to be measured, which can effectively improve the efficiency of automated measurement of complex industrial parts. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a flow chart of the main steps of a method for simultaneous optimization of measurement viewpoint and measurement path based on reinforcement learning;

[0019] FIG2(a) is a physical picture of the surface structured light 3D measurement system, and FIG2(b) is a schematic diagram of the measurable area of ​​the surface structured light 3D measurement system;

[0020] Figure 3 Schematic diagram of the DQN-based measurement viewpoint and measurement path simultaneous optimization model. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0022] At present, 3D measurement work mainly relies on manual experience. There are many factors that affect the measurement viewpoint, including: optical visibility of the workpiece surface, imaging quality, robot accessibility, workpiece surface material, exposure time, etc. For industrial parts with complex structural shapes, manual planning is very difficult and cannot guarantee the optimization of planning results, resulting in low measurement quality and efficiency.

[0023] The goal of the viewpoint planning problem is to achieve complete coverage of the target object surface with a minimum number of viewpoints. There are three main research directions: 1) Generate a re-detection model, which first generates candidate viewpoints, then evaluates the candidate viewpoints one by one according to the constraints of the measurement task, and finally selects the optimal viewpoint. This method is simple, can introduce multiple constraints, and has little dependence on the reference model, but lacks theoretical guidance when generating viewpoints. It can only increase the number of candidate points to improve the probability of generating the optimal viewpoint, resulting in low computational efficiency; 2) Synthesis mode, this mode analyzes the relationship between multiple constraints in the measurement task and generates a viewpoint that best meets the constraints, but the algorithm design of this mode is difficult, and the generation result depends on the parameter settings of each constraint, and the preprocessing time is long; 3) Expert mode, this mode is developed on the basis of the first two modes. In terms of candidate viewpoint generation, modeling is carried out based on the principles of visual imaging to optimize the quality of candidate viewpoints; in the screening process, the characteristic constraints of the target body are taken into account, and recommended values ​​of the parameters required to generate viewpoints can be given, and finally the viewpoints are generated by these recommended values. Although this mode is better than the first two modes, for the three-dimensional measurement of complex industrial parts, it is necessary to consider the influence of factors such as optical visibility, imaging quality, robot accessibility, workpiece surface material, and exposure time on the measurement viewpoint and path, so this aspect needs further research.

[0024] After generating the measurement viewpoints, it is necessary to find the motion path with the lowest motion cost in the task space of the industrial robot. In addition, collision detection is generally required for the industrial robot in industrial inspection to ensure that the robot can move safely in its workspace when visiting the generated viewpoints. From the perspective of graph theory, all viewpoints constitute a weighted undirected graph, so the measurement path planning problem is essentially a traveling salesman problem. Currently, the traveling salesman problem is mainly solved by greedy algorithms, genetic algorithms, ant colony algorithms, neural network algorithms, etc. The existing measurement planning methods are mostly to first solve a set of viewpoints covering the required surface area of ​​the target object, and then find the collision-free optimal robot path to visit all viewpoints. The former is an NP-hard problem, and the latter is a traveling salesman problem. Solving both separately is computationally complex and easy to fall into local optimality, affecting the overall optimization quality.

[0025] Based on this, an embodiment of the present invention provides a method for synchronously optimizing a measurement viewpoint and a measurement path based on reinforcement learning, such as Figure 1 As shown, including:

[0026] S1, randomly generate a viewpoint set in a preset sampling space, wherein the direction of each viewpoint in the viewpoint set points to the body center of the three-dimensional model of the workpiece to be measured; based on the inverse kinematics principle, solve the posture of the industrial robot corresponding to each viewpoint and perform collision detection, and delete the unreachable viewpoints to obtain a candidate viewpoint set.

[0027] Specifically, a viewpoint set including multiple viewpoints is randomly generated in the preset sampling space of the three-dimensional model of the workpiece to be measured (hereinafter referred to as the target model). The direction of each viewpoint in the viewpoint set points to the center of the target model. The robot postures corresponding to all candidate viewpoints are solved according to the inverse kinematics principle, and collision detection is performed. The unreachable viewpoints in the viewpoint set are deleted to obtain a candidate viewpoint set.

[0028] First, the candidate viewpoints that do not have inverse kinematics solutions are deleted. Then, collision detection is performed on the robot posture set solved by inverse kinematics to check whether the robot collides with the environment and the target model. The candidate viewpoints that collide are deleted. The candidate viewpoint set is represented as V = {v1,v2,…,v n}, the robot posture set is represented as P = {p v1 ,p v2 ,…,p vn}.

[0029] In step S1, in order to determine the sampling space and provide conditions for the screening of viewpoints, before planning, the three-dimensional measuring equipment is manually operated to reconstruct the triangular mesh model of the workpiece to be measured. The model describes the surface information of the three-dimensional model of the workpiece to be measured in units of triangular faces, including the coordinates of the vertices, the normal vectors and the normal vectors of the faces, as well as the topological relationships between points, edges, and faces.

[0030] Preferably, the preset sampling space is a spherical sampling space, the center of the sphere is the center of the target model, and the radius is the far point distance of the camera in the surface structured light 3D measurement system. Candidate viewpoints are randomly generated in the spherical sampling space, and the viewpoint direction points to the center of the sphere.

[0031] Specifically, as shown in FIG2(a), the surface structured light 3D measurement system used in the present invention includes left and right cameras and a projector, and the parameters of the left and right cameras are consistent. A spherical sampling space is determined according to the parameters of the surface structured light 3D measurement system and the target model, the radius is the far point distance D_far of the surface structured light 3D measurement system, and the center of the sphere coincides with the center of the target model. For example, 500 candidate viewpoints are randomly generated in the sampling space, and the viewpoint directions all point to the center of the sphere.

[0032] Far point distance:

[0033]

[0034] Wherein, F is the aperture value of the camera in the surface structured light 3D measurement system, f is the focal length, D is the focus distance, δ is the diameter of the confusion circle, and δ = diagonal degree of the amplitude / 1730.

[0035] The specific parameters of the surface structured light 3D measurement system are as follows:

[0036] Table 1 Parameters of surface structured light 3D measurement system

[0037] parameter index Camera Model BFLY-U3-23S6M-C Camera resolution 1920x1200 pixels Pixel size 5.86μm Target surface 11.2512x7.032mm Lens model ML-M1618HR Nominal focal length of lens 16mm Field of view 30°，22° Projector Model Tir-DLP4500SL02

[0038] S2, calculating the local path and motion cost between any two candidate viewpoints in the candidate viewpoint set and storing them in a motion graph.

[0039] Specifically, the RRT* algorithm is used to calculate the local path and motion cost between any two candidate viewpoints in the candidate viewpoint set, and the viewpoints, the local path and the motion cost are stored in a motion map.

[0040] Represent the motion graph using a weighted undirected graph:

[0041] G(V,E),

[0042] e(v i ,v j )∈E

[0043] Among them, node V is the candidate viewpoint set, boundary E is the local path between viewpoints, and v i ,v j are two candidate viewpoints, e(v i ,v j )∈E is the viewpoint v i and v j The local path between represents the viewpoint v i and v j The cost of movement between .

[0044] S3, determining whether each surface piece from each candidate viewpoint to the three-dimensional model of the workpiece to be measured is visible, so as to construct a visibility matrix.

[0045] Specifically, based on the parameters of the surface structured light 3D measurement system, viewing angle constraints, depth of field constraints, field angle constraints and occlusion constraints are imposed on the vertices and triangles on the triangular mesh model of the object to be measured, thereby realizing the simulation of the measurable area of ​​the surface structured light 3D measurement system under a specific viewing angle, as shown in Figure 2(b), where α1 and α2 are the horizontal field of view angles of the left and right cameras, α4 and α5 are the angles between the center lines of the left and right cameras and the center line of the projector, b is the distance between the camera and the projector, and the mesh area is the measurable area.

[0046] (1) Viewing angle constraints

[0047] Each vertex of the triangle must satisfy the viewing angle constraint

[0048] n·v<0

[0049] Among them, v is the direction of the candidate viewpoint, vector n is the normal vector of the triangle vertex, and when the angle between the two vectors is less than 75°, the vertex is visible to the candidate viewpoint.

[0050] (2) Depth of Field Constraint

[0051] The range in which a real image can be formed before and after the focusing object plane is called the depth of field. The approximate range of the depth of field is calculated based on the camera parameters:

[0052]

[0053]

[0054] Wherein, d1 is the foreground depth of field, d2 is the back depth of field, F is the aperture value, f is the focal length, D is the focus distance, and δ is the diameter of the circle of confusion.

[0055] According to the depth of field range, each vertex of the triangle patch must satisfy the measurement distance constraint:

[0056] D_near=Df-d1

[0057] D_far=D-f+d2

[0058] D_near<d<D_far

[0059] Among them, D_near is the distance from the front boundary of the depth of field to the near point of the lens center, D_far is the distance from the back boundary of the depth of field to the far point of the lens center, and d is the distance from the vertex of the triangle patch to the candidate viewpoint.

[0060] (3) Field of view constraints

[0061] Each vertex of the triangle must satisfy the field of view constraint:

[0062]

[0063] Among them, α is the horizontal field of view angle, β is the vertical field of view angle, v is the candidate viewpoint direction, and v p is the direction of the line connecting the candidate viewpoint and the vertex of the triangle patch, v px and v py v p The vector components in the x and y directions of the image coordinate system.

[0064] (4) Occlusion constraints

[0065] Assume that point P is the vertex of the triangle, O is the optical center of the camera, the three vertices of a triangle are a, b, c, and the straight line PO intersects the plane where the triangle is located at point o. If:

[0066] S Δaob +S Δboc +S Δcoa >S Δabc

[0067] There is no occlusion between the candidate viewpoint and the vertex P. If there is no occlusion between the candidate viewpoint and the three vertices of the triangle, it is considered that there is no occlusion between the candidate viewpoint and the triangle.

[0068] According to the measurable area and the target model, visibility evaluation and analysis are performed on each candidate viewpoint to determine whether each triangular surface patch from each candidate viewpoint to the target model is visible, thereby constructing a visibility matrix.

[0069] The visibility matrix is ​​a two-dimensional matrix and can be expressed as:

[0070]

[0071] Among them, n surf is the number of triangles in the target model, n v is the number of candidate viewpoints.

[0072] The two-dimensional visibility matrix is ​​a 0-1 matrix. When the triangle face i satisfies the visibility, depth of field, field of view, and occlusion constraints for the candidate viewpoint j, the corresponding matrix element V ij is 1. If any constraint is not satisfied, the corresponding matrix element is 0.

[0073] S4, constructing a dual convolutional neural network based on DQN for synchronously optimizing the measurement viewpoint and the measurement path, iteratively training the dual convolutional neural network with the candidate viewpoint set, motion map and visibility matrix as training sets, and obtaining a trained synchronous optimization model of the measurement viewpoint and the measurement path.

[0074] S5, inputting the initial posture and initial coverage information of the industrial robot into the trained measurement viewpoint and measurement path synchronization optimization model to obtain the measurement viewpoint and measurement path of the workpiece to be measured.

[0075] Among them, the initial posture of the robot is the preset robot posture origin, and the initial coverage information is that the target model surface is not covered.

[0076] The initial posture and initial coverage information of the industrial robot are input into the trained measurement viewpoint and measurement path synchronization optimization model to perform viewpoint selection and path optimization based on the trained model. The model outputs a set of viewpoints that meet the coverage requirements and a path with the lowest cost to access the viewpoints. Based on the output viewpoint set and measurement path, the automated measurement of complex industrial parts is achieved.

[0077] Preferably, the action of the dual convolutional neural network is to select a candidate viewpoint and move the industrial robot to the position of the selected candidate viewpoint;

[0078] The state includes current coverage information of the target model surface and a current posture of the industrial robot, wherein the current coverage information is obtained according to the visibility matrix, and the current posture is obtained according to the motion graph;

[0079] The reward function is Among them, T(s t ,s t+1 ) is the travel cost between two states, Δcov(s t ,a t ) is in state s t Take action t The incremental coverage of Among them, patch new The number of new patches covered by the action, patch ref For the triangular patch that does not satisfy the reflection constraint, patch all is the total number of faces in the target model.

[0080] Specifically, Figure 3 As shown in the DQN-based measurement viewpoint and measurement path synchronous optimization model (i.e., a dual convolutional neural network constructed based on DQN for synchronously optimizing the measurement viewpoint and the measurement path):

[0081] (1) Agent: Build a DQN convolutional neural network (including the prediction network Q eval and the target network Q target ), continuously predicting the next action to be taken according to the strategy to maximize the final reward.

[0082] (2) Status: coverage information of the target model surface And the robot posture p at this time vi , the state parameter can be recorded as Then the state set can be recorded as S = {s1, s2, ..., s n},i=1,2,…,n.

[0083] Among them, v i The generated viewpoint set V = {v1,v2,…,v n}, i=i,2,…,n, the i-th viewpoint, p vi is the viewpoint v i The corresponding robot posture, robot posture set P = {p v1 ,p v2 ,…,p vn}, i = 1, 2, ..., n, which can be obtained by querying the motion graph, At this time, the coverage information of the training target model surface (how many surface patches are covered) can be obtained by querying the visibility matrix. When the pre-set coverage requirement is reached, this state is the termination state.

[0084] (3) Action: Action set A = {a1, a2, …, a n}, i = 1, 2, ..., n is to arbitrarily select a viewpoint in the generated viewpoint set and move the robot to the viewpoint position.

[0085] (4) Environment: The environment consists of a target model, a set of candidate viewpoints, a visibility matrix, and a motion graph. Based on the agent’s actions, that is, choosing which viewpoint to jump to next, the environment calculates the reward for that step, updates the next state, and feeds back the reward and next state to the agent.

[0086] (5) Reward function: Construct a reward function based on data coverage increment and motion cost:

[0087]

[0088] Among them, T(s t ,s t+1 ) is the travel cost between two states, Δcov(s t ,a t ) is in state s t Take action t Incremental coverage.

[0089] According to the surface structured light 3D measurement system diagram, as shown in Figure 2(b), we can know the centerline directions of the projector and the camera. For the convenience of calculation, assume that the following vectors are all unit vectors, then the angle between the projector centerline and the triangle patch center normal θ1 = arccos(n cen .v), where v is the candidate viewpoint direction (i.e., the projector centerline direction), and vector n cen is the normal vector to the center of the triangle.

[0090] According to the law of reflection, the angle of incidence is equal to the angle of reflection, so the reflected light vector n ref Satisfy n ref .n cen =cos(π-θ1), the angle between the camera center line and the reflected light θ2 = arccos(n cam .n ref ),

[0091] The intensity of light reflected from the projector into the camera

[0092] I=αtβI p cosθ2

[0093] Among them, α is the sensitivity of the camera, t is the exposure time of the camera, β is the reflection coefficient of the object being measured, I p is the projector light intensity.

[0094] Considering that overexposure can easily lead to incomplete data, adding reflection constraints can improve the accuracy of 3D measurement and the integrity of data. Based on this, in order to ensure the imaging quality, the light intensity needs to meet 20<I<160, that is:

[0095]

[0096] If the reflection constraint is not satisfied, the triangle does not belong to the newly covered face.

[0097] The incremental coverage is specifically expressed as:

[0098]

[0099] Among them, patch new The number of new patches covered by the action, patch ref For the triangular patch that does not satisfy the reflection constraint, patch all is the total number of faces in the target model.

[0100] Preferably, the dual convolutional neural network includes a prediction network and a target network;

[0101] The prediction network is used to predict the Q value of each action in the current state, and the network parameters are updated in each iteration;

[0102] The target network parameters are updated by copying the prediction network parameters every preset number of iterations. When the target network parameters are fixed, the learning target of the prediction network is The prediction network is also fixed, and iterative updates are performed based on the fixed learning target, which can effectively prevent overfitting of the training and achieve convergence of the algorithm.

[0103] Preferably, a training set is used to train the measurement viewpoint and measurement path synchronization optimization model, and the training process includes:

[0104] (1) Initialize the memory pool size to N, initialize the greedy algorithm probability to ε, and initialize the prediction network and target network to have the same structure and parameters; the input is the current state, and the output is the Q value of each action.

[0105] Preferably, the prediction network Q is initialized with random weights eval and the target network Q target For the same structure and parameters, that is, θ eval =θ target .

[0106] (2) Randomly select a state s from the state set S t ; Among them, the initial state is that the robot is in an initial posture (for example, a preset posture origin) and the surface of the target model is not covered.

[0107] (3) According to the current state s t , use the ε-greedy algorithm to select the action with the maximum Q value in the current state or randomly select an action as the optimal action a t .

[0108] In order to avoid the strategy converging to the local optimum, according to the current state s t , use the ε-greedy greedy algorithm to perform a certain proportion of random exploration, select the action with the maximum Q value in the current state or randomly select an action as the optimal action a t , specifically expressed as:

[0109]

[0110] Among them, ε is the probability of the ε-greedy algorithm randomly selecting an action, which is a positive number less than 1. Indicates the current state s t The optimal action a with the maximum Q value t .

[0111] (4) Calculate the reward r for this step according to the reward function t , and according to action a t Get new state s t+1 ;

[0112] (5) Repeat steps (2)-(4) and transfer the state result [s t ,a t ,r t ,s t+1 ] is stored in the memory pool;

[0113] (6) When the amount of data stored in the memory pool exceeds N, the DQN network training begins and a piece of data [s i ,a i ,r i ,s i+1 ], i = 1, 2, ..., n, use the prediction network to calculate the predicted value of Q, and use the target network to calculate the target value of Q. The target value of Q is:

[0114]

[0115] Among them, r i For state s i The reward under , γ is the discount factor for future returns, is the current target network weight parameter, s i+1 For the next state, a i+1 For the next step.

[0116] (7) Calculate the loss function between the predicted value and the target value of Q, and use the loss function to optimize the prediction network. The loss function is expressed as:

[0117]

[0118] (8) Use the stochastic gradient descent method to iteratively update the prediction network parameters to obtain the optimal action The iteration formula is as follows:

[0119]

[0120] Among them, r i For state s i The reward under , γ is the discount factor for future returns, is the current target network weight parameter, s i+1 For the next state, a i+1 For the next step, is the current target network weight parameter, s i is the current state, a i The current step action.

[0121] (9) For each fixed number of iterations, the prediction network parameter θ is used. eval Update the target network parameters θ target .

[0122] (10) Repeat steps (6)-(9) and stop iterating when the DQN network converges or reaches a preset number of iterations to obtain a trained model.

[0123] The embodiment of the present invention provides a method for synchronously optimizing a measurement viewpoint and a measurement path based on reinforcement learning, including:

[0124] A computer readable storage medium and a processor;

[0125] The computer-readable storage medium is used to store executable instructions;

[0126] The processor is used to read the executable instructions stored in the computer-readable storage medium and execute the method as described in the first aspect.

[0127] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for synchronous optimization of measurement viewpoint and measurement path based on reinforcement learning, characterized in that: include: S1, randomly generating a viewpoint set in a preset sampling space, wherein the direction of each viewpoint in the viewpoint set points to the body center of the three-dimensional model of the workpiece to be measured; solving the posture of the industrial robot corresponding to each viewpoint based on the inverse kinematics principle and performing collision detection, and deleting unreachable viewpoints to obtain a candidate viewpoint set; S2, calculate the local path and motion cost between any two candidate viewpoints and store them in the motion graph; S3, determining whether each surface piece from each candidate viewpoint to the three-dimensional model of the workpiece to be measured is visible, so as to construct a visibility matrix; S4, constructing a dual convolutional neural network for synchronously optimizing the measurement viewpoint and the measurement path based on DQN, iteratively training the dual convolutional neural network with the candidate viewpoint set, the motion map and the visibility matrix as training sets, and obtaining a trained synchronous optimization model of the measurement viewpoint and the measurement path; S5, inputting the initial posture and initial coverage information of the industrial robot into the trained measurement viewpoint and measurement path synchronization optimization model to obtain the measurement viewpoint and measurement path of the workpiece to be measured.

2. The method according to claim 1, characterized in that The action of the dual convolutional neural network is to select a candidate viewpoint and move the industrial robot to the position of the selected candidate viewpoint; The state includes current coverage information of the surface of the three-dimensional model of the workpiece to be measured and the current posture of the industrial robot, wherein the current coverage information is obtained according to the visibility matrix, and the current posture is obtained according to the motion graph; The reward function is Among them, T(s t ,s t+1 ) is the travel cost between two states, Δcov(s t ,a t ) is in state s t Take action t The incremental coverage of Among them, patch new The number of new patches covered by the action, patch ref For the triangular patch that does not satisfy the reflection constraint, patch all is the total number of facets of the 3D model of the workpiece to be tested.

3. The method according to claim 2, characterized in that The reflection constraints are: Among them, α is the sensitivity of the camera, t is the exposure time of the camera, β is the reflection coefficient of the object being measured, I p is the projector light intensity, θ2 is the angle between the camera center line and the reflected light.

4. The method according to claim 2, characterized in that The dual convolutional neural network includes a prediction network and a target network; The prediction network is used to predict the Q value of each action in the current state, and the network parameters are updated in each iteration; The target network copies the network parameters of the prediction network every preset number of iterations to update. When the target network parameters are fixed, the learning target of the prediction network is is also fixed, and the prediction network is iteratively updated for a fixed learning target; where r i For state s i The reward under , γ is the discount factor for future returns, is the target network weight parameter, s i+1 For the next state, a i+1 For the next step.

5. The method according to any one of claims 1 to 4, characterized in that: The training process of the measurement viewpoint and measurement path synchronization optimization model includes: (1) Initialize the memory pool size to N, initialize the greedy algorithm probability to ε, and initialize the prediction network and target network to have the same structure and parameters; the input is the current state, and the output is the Q value of each action; (2) Randomly select an initial state s from the state set S t ; (3) According to the current state s t , use the ε-greedy algorithm to select the action with the maximum Q value in the current state or randomly select an action as the optimal action a t ; (4) Calculate the reward r for this step according to the reward function t , and according to action a t Get new state s t+1 ; (5) Repeat steps (2)-(4) and transfer the state result [s t ,a t ,r t ,s t+1 ] is stored in the memory pool; (6) When the amount of data stored in the memory pool exceeds N, the DQN network training begins and a piece of data [s i ,a i ,r i ,s i+1 ], i = 1, 2, ..., n, use the prediction network to calculate the predicted value of Q, and use the target network to calculate the target value of Q; (7) Calculate the loss function between the predicted value and the target value of Q; (8) Use stochastic gradient descent method to iteratively update the prediction network parameters; (9) every predetermined fixed number of iterations, the target network parameters are updated using the predicted network parameters; (10) Repeat steps (6)-(9) and stop iterating when the DQN network converges or reaches a preset number of iterations to obtain a trained model.

6. The method according to claim 5, characterized in that Use the ε-greedy algorithm to select the action with the maximum Q value in the current state or randomly select an action as the optimal action a t , specifically expressed as: Among them, ε is the probability of the ε-greedy algorithm randomly selecting an action, which is a positive number less than 1. Indicates the current state s t The optimal action a with the maximum Q value t .

7. The method according to claim 5, characterized in that The target value of Q is: Among them, r i For state s i The reward under , γ is the discount factor for future returns, is the target network weight parameter, s i+1 For the next state, a i+1 For the next step.

8. The method according to claim 5, characterized in that In step (8), the stochastic gradient descent method is used to iteratively update the target network parameters of the neural network, thereby obtaining the optimal action The iteration formula is: Among them, r i For state s i The reward under , γ is the discount factor for future returns, is the current target network weight parameter, s i+1 For the next state, a i+1 For the next step, is the current prediction network weight parameter, s i is the current state, a i The current step action.

9. The method according to claim 1, characterized in that The preset sampling space is a spherical sampling space, the center of the sphere is the body center of the three-dimensional model of the sample to be measured, and the radius is the far point distance of the camera in the surface structured light three-dimensional measurement system.

10. A measurement viewpoint and measurement path synchronization optimization system based on reinforcement learning, characterized in that: include: A computer readable storage medium and a processor; The computer-readable storage medium is used to store executable instructions; The processor is used to read the executable instructions stored in the computer-readable storage medium and execute the method according to any one of claims 1 to 9.

Images