Optimization Method for Camera Arrangement in Manipulator Pose Detection Based on Genetic Algorithm

Abstract

The present invention belongs to the field of visual pose recognition, and discloses an optimization method for camera arrangement in the pose detection of a manipulator based on a genetic algorithm, which includes step S1. Setting the working pose parameters of the manipulator, and discretizing the working space and common working states of the manipulator; step S2. Setting the camera parameters, including the number parameter of the cameras, the installable space of the cameras, and the optical parameters of the cameras such as the field of view angle, depth of field, focal length, and resolution, etc.; step S3. Establishing a visual recognition error model based on the working pose parameters of the manipulator and the camera parameters, and evaluating and scoring the camera arrangement scheme; step S4. Setting the hyperparameters of the genetic algorithm and initializing the population, the algorithm hyperparameters include the gene size, the number of population individuals, the maximum number of iterations, the crossover probability, and the mutation probability, etc., and initializing the population; step S5. Conducting genetic iterative optimization to obtain the best individual. The method of the present invention improves the camera arrangement efficiency and the pose recognition accuracy.

Description

Technical Field

[0001] The present invention belongs to the field of visual pose recognition, and particularly relates to an optimization method for camera arrangement for manipulator pose detection based on a genetic algorithm. Background Technique

[0002] With the advancement of intelligent manufacturing and the continuous improvement of manipulator technology, manipulators represented by industrial robots are widely used in industrial production, emergency rescue, and daily life. In scenarios such as high temperature, high humidity, and nuclear radiation, electronic devices installed on the joints of the manipulator are prone to failure, and manipulators with all-mechanical transmissions are often used in these scenarios. Due to the lack of electronic sensors on the joints of the robotic arm, it is difficult to accurately guarantee the positioning accuracy of the manipulator. Using a camera to identify feature points on the robotic arm and calculating the spatial pose of the manipulator based on the feature points can improve the motion control accuracy of the manipulator. To achieve the detection of a large motion range of the manipulator, a single camera is difficult to handle, and the spatial configuration of multiple cameras can better cover the working space of the manipulator. The selection of the number of cameras and the arrangement of the camera group have important impacts on the coverage of various states of the manipulator, the pose accuracy of the manipulator, and the usage efficiency of the cameras. Currently, the spatial pose of the camera group is mainly determined according to the experience of workers, or the equal-distance arrangement and symmetric arrangement methods are adopted. The equal-distance arrangement means installing a camera at a certain interval, and the symmetric arrangement means arranging a pair of cameras at symmetric positions.

[0003] Conventional camera arrangement methods have many disadvantages. First, relying on the experience of workers for arrangement is likely to have omissions, and the actual effect is difficult to control. At the same time, this method requires continuous disassembly, installation, and test adjustment, which is time-consuming and laborious. Second, although the methods of equal-distance arrangement and symmetric arrangement are simple, there are redundant situations, and the utilization efficiency of the cameras is necessarily not high, resulting in increased costs. Third, at present, these methods do not combine the actual task of manipulator pose detection, and do not comprehensively consider how to arrange the cameras to improve the accuracy of pose detection and the usage efficiency of the cameras. Summary of the Invention

[0004] The purpose of the present invention is to provide an optimization method for camera arrangement for manipulator pose detection based on a genetic algorithm to solve the above technical problems.

[0005] To solve the above technical problems, the specific technical solution of the optimization method for camera arrangement for manipulator pose detection based on a genetic algorithm of the present invention is as follows:

[0006] An optimization method for camera arrangement for manipulator pose detection based on a genetic algorithm includes the following steps:

[0007] Step S1. Set the working attitude parameters of the manipulator, and discretize the working space and common working states of the manipulator, that is, obtain the parameters of the target to be observed;

[0008] Step S2. Set the camera parameters, including the number parameter of the cameras, the installable space of the cameras, and the optical parameters of the cameras, that is, obtain the parameters of the observation cameras.

[0009] Step S3. Establish a visual recognition error model based on the manipulator parameters and the camera parameters, and evaluate and score the camera layout scheme, that is, establish the fitness function of the subsequent genetic algorithm.

[0010] Step S4. Set the hyperparameters of the genetic algorithm and initialize the population. The algorithm hyperparameters include the gene size, the number of population individuals, the maximum number of iterations, the crossover probability, and the mutation probability. Initializing the population means setting the original scheme of the camera layout.

[0011] Step S5. Perform genetic iterative optimization, and the best individual obtained is the best layout scheme of the cameras.

[0012] Furthermore, the step S1 includes the following specific steps:

[0013] Combined with the common tasks and working conditions of the manipulator, obtain the common working space of the manipulator, and discretize the set working space to obtain a large number of working states of the manipulator.

[0014] Use the position and pose of the end of the manipulator to represent the state of the manipulator, and use a 4×4 pose matrix to represent the pose of the end of the manipulator:

[0015]

[0016] In the formula, T is the pose matrix of the end of the manipulator, R represents the rotation matrix of the end, and P represents the translation vector of the end.

[0017] Furthermore, the step S2 includes the following specific steps:

[0018] Set the number n of the cameras, and perform manipulator pose recognition as a camera group. The optimization goal is to obtain the best layout scheme of the corresponding number of cameras based on the given number of cameras.

[0019] Set the installable space parameters of the cameras. When the installation freedom is large, the cameras can be installed in a large range of three-dimensional space and can be oriented arbitrarily; or more in line with the actual situation, the cameras can only be installed on the truss in the working space, and the orientation is restricted by the truss structure.

[0020] Set the optical parameters of the cameras, including the vertical field of view FOVV, the horizontal field of view FOVH, the depth of field DOF, the focal length f, and the resolution X×Y.

[0021] Only considering whether the target is visible, the visible range model of the camera is a quadrangular pyramid determined by the field of view angle. If considering whether the target can be clearly imaged, the visible range model of the camera is a frustum of a pyramid determined by the field of view angle and the depth of field;

[0022] The arrangement scheme of the camera group is represented by an n×6 matrix:

[0023]

[0024] In the formula, A represents the arrangement scheme of the entire camera group, and A n represents the installation state of the nth camera. The [X n Y n Z n in A is the three-dimensional space coordinates of the camera, representing the installation position of the camera. The [Vec_x n Vec_y n Vec_z n is the optical axis vector of the camera, representing the installation orientation of the camera. n is the optical axis vector of the camera, representing the installation orientation of the camera.

[0025] Furthermore, the step S3 includes the following specific steps:

[0026] The end pose detection is completed by extracting multiple feature points on a certain plane at the end. The theoretical calculation of the visual pose detection error is carried out. The feature points can either be the obvious point features on the robotic arm or the artificially pasted marker points. When the number of feature points is X and the feature points are located at the vertices of a regular polygon, the following theoretical errors of pose calculation are as follows:[[ID=3s]]

[0027] Z-direction position error:

[0028] X / Y-direction position error:

[0029] Z-direction attitude error:

[0030] X / Y-direction attitude error:

[0031] Among them, μ is the pixel size of the camera; f is the focal length of the camera; l o is the distance between feature points; is the feature point positioning accuracy; δ is the pixel resolution; a is a coefficient related to the number of feature points;

[0032] It is obtained that the pose detection accuracy depends on the quality of the camera itself, the distance between the camera and the feature point group, the distribution of the feature point group, and the feature point positioning accuracy;

[0033] According to the error analysis theory, starting from improving the pose detection accuracy, an evaluation scoring system for the camera arrangement scheme is established. First, the evaluation indicators for a single camera and a certain manipulator state include four items: camera field of view, target relative distance, target relative position, and target relative pose. According to the influence degree of each indicator on the pose accuracy, corresponding weight scores are set for these four evaluation indicators;

[0034] The camera field of view indicator reflects whether the end of the manipulator is within the camera's field of view and is not blocked by the manipulator body itself. This sub-item accounts for A% of the total score; the target relative distance indicator reflects whether the end of the manipulator is within the depth of field of the camera, and this sub-item accounts for B% of the total score; the target relative position indicator reflects whether the end of the manipulator is at the center of the camera's field of view, and this sub-item accounts for C% of the total score; the target relative pose indicator reflects the degree to which the normal of the end plane of the manipulator deviates from the normal of the camera, and this sub-item accounts for D% of the total score. Finally, the sum of A, B, C, and D is 100;

[0035] When a certain manipulator state and the camera placement are determined, the scores of each sub-item can be calculated. The cumulative sum of the scores of each sub-item reflects whether the placed camera can accurately perceive this manipulator state at this time, that is, the score reflects the quality of this camera placement.

[0036] Furthermore, the specific calculation methods for the scores of each sub-item in step S3 are as follows:

[0037] The camera field of view indicator considers two aspects. One is whether the end of the manipulator is within the field of view, which is determined by calculating whether the end of the manipulator is within the quadrangular pyramid determined by the field angle; the other is whether the end of the manipulator will be blocked by the manipulator body itself. It is judged whether there is a manipulator entity on the line connecting the end of the manipulator and the camera. The manipulator entity is represented by a simplified rule model. Judging the occlusion problem is transformed into judging the intersection problem between the line segment and the rule model. If the end of the manipulator is within the field of view and is not blocked by the manipulator body itself, this item can obtain all the scores of this sub-item;

[0038] The manipulator entity is simplified into a combination of rule models. First, the rule models are parameterized, and then the intersection condition is:

[0039]

[0040] In the formula, the left side is the parameter representation result of a point on line segment BC, and point B is the starting point of the line segment, That is is the direction vector of the line segment, θ4 is a parameter, and its value range is [0,1]; the right side is the parameter representation result of a point in the rule model space. A is a special point of the rule model, such as the vertex of a cuboid, the center of the bottom surface of a cylinder, and the center of a sphere, and is a set of orthogonal vectors. The lengths and directions of the orthogonal vectors are related to the model type. θ1, θ2, and θ3 are parameters, and the value ranges of the three parameters are related to the model type.

[0041] The above equation can be transformed into a non-homogeneous system of equations:

[0042]

[0043] The line segment having an intersection with the regular model is equivalent to the non-homogeneous system of equations having a solution that satisfies the corresponding constraint conditions;

[0044] The target relative distance index starts from the actual clear imaging of the camera. When the equivalent distance between the end of the manipulator and the camera is within the depth of field range, this item can obtain all the scores of this sub-item. If the equivalent distance is too short or too long, the imaging of the manipulator will be blurred and the pose accuracy will decrease, so the score will be correspondingly reduced;

[0045] The target relative position index starts from reducing the influence of camera distortion. When the end of the manipulator is located on the optical axis of the camera, the imaging of the end is at the center of the field of view and the distortion is the smallest. At this time, this item can obtain all the scores of this sub-item. If the end of the manipulator deviates from the optical axis, the score will be correspondingly reduced;

[0046] The target relative pose index starts from improving the positioning accuracy of feature points. When the plane at the end of the manipulator is perpendicular to the optical axis of the camera, the positioning accuracy of the feature points is the highest. At this time, this item can obtain all the scores of this sub-item. As the angle between the normal direction of the plane at the end of the manipulator and the optical axis of the camera increases, the score will be correspondingly reduced;

[0047] For a certain state of the manipulator, as long as one camera can achieve high-precision pose detection, the requirement is satisfied. Therefore, the evaluation score for the camera group and a certain end state of the manipulator is the highest score among all cameras;

[0048] The score of the final camera layout scheme is the average value of the scores of the camera group and all manipulator states.

[0049] Further, the step S4 includes the following specific steps:

[0050] Set the hyperparameters of the genetic algorithm, and the hyperparameters include the individual gene size, the number of individuals in the population, the maximum number of iterations, the crossover probability, and the mutation probability;

[0051] Initialize the population. Each individual in the population represents a camera layout scheme. Initializing the population is to set the initial camera layout scheme. Starting from achieving the best pose detection for a small number of manipulator states, the camera layout is initialized in a targeted manner.

[0052] Further, the step S5 includes the following specific steps:

[0053] Perform genetic iterative optimization. In the genetic algorithm, the individuals in the population are the camera layout schemes, and the genes of the individuals represent the installation conditions of each camera, including the installation position and the camera orientation. The individual fitness is the score of the camera layout scheme in the evaluation scoring system.

[0054] The genetic algorithm transforms the search for the best layout scheme of the camera group into the search for the individual with the highest fitness in the population. Initialize the population as the first-generation population, perform genetic and mutation operations, and continuously generate a better population of the new generation. The best individual in each generation of the population is the best layout scheme of the camera group.

[0055] The method for optimizing the layout of the manipulator pose detection camera based on the genetic algorithm of the present invention has the following advantages: The present invention incorporates the requirements of the manipulator pose detection task into the fitness function of the genetic algorithm, and uses the powerful global optimization ability of the genetic algorithm to determine the best scheme for camera layout. On the one hand, it can reduce the workload of a large number of installation, disassembly, and test adjustment. On the other hand, it can improve the layout efficiency of the pose detection camera and the accuracy of the pose recognition result. Brief Description of the Drawings

[0056] Figure 1 It is a schematic flow chart of the optimization method of the present invention;

[0057] Figure 2 It is a schematic structural diagram of the manipulator and the camera group implemented by the present invention;

[0058] Figure 3 It is a schematic diagram of the camera occlusion model in step S3 of the present invention;

[0059] Figure 4 It is a schematic flow chart of the genetic optimization in step S5 of the present invention. Detailed Embodiment

[0060] In order to better understand the purpose, structure and function of the present invention, the following further describes in detail a method for optimizing the layout of a manipulator pose detection camera based on the genetic algorithm of the present invention with reference to the drawings.

[0061] A method for optimizing the layout of a manipulator pose detection camera based on the genetic algorithm of the present invention can, under the condition of a given number of cameras, use the genetic algorithm to find the best camera layout scheme, under which the recognition accuracy of the manipulator pose is the highest. The structure of the manipulator and the dual cameras is as Figure 2 shown, and its overall optimization process is as Figure 1 shown, including the following steps:

[0062] Step S1. Set the manipulator working attitude parameters, and discretize the working space and common working states of the manipulator, that is, obtain the parameters of the target to be observed;

[0063] Combined with the common tasks and working conditions of the manipulator, obtain the common working space of the manipulator. Discretize the set working space to obtain a large number of working states of the manipulator.

[0064] The pose detection of the overall manipulator is equivalent to the pose detection of the end gripper of the manipulator. The working state of the manipulator mainly concerns the state of the end gripper of the manipulator. We use the position and orientation of the end to represent the state of the manipulator. Use a 4×4 pose matrix to represent the pose of the end:

[0065]

[0066] In the formula, T is the pose matrix of the end of the manipulator, R represents the rotation matrix of the end, and P represents the translation vector of the end.

[0067] To analyze the self-occlusion of the manipulator, the working state of the manipulator also needs to add the state of the entire entity of the manipulator. To reduce the calculation amount, regular objects such as cylinders and cuboids are used to approximately replace the manipulator entity.

[0068] In this embodiment, the work of the manipulator is simplified to a movement process of a section of the manipulator. The end of the manipulator moves from A[x1, y1, z1] to B[x2, y2, z2], and at the same time the normal of the end plane changes from to Use this movement process to represent the working space of the manipulator. Discretization means uniformly selecting m manipulator states in the movement process. The manipulator state including the end of the manipulator and the entire entity of the manipulator is the manipulator parameter in this embodiment.

[0069] Step S2. Set the camera parameters, including the number parameter of the cameras, the installable space of the cameras, and the optical parameters of the cameras such as the field of view angle, depth of field, focal length, and resolution, that is, obtain the parameters of the observation cameras;

[0070] Set the number n of the cameras. For the camera group for manipulator pose recognition, the number of cameras has a great influence on the result. The optimization goal is to obtain the best layout plan for the corresponding number of cameras based on the given number of cameras. In this embodiment, n is 1 to 4, that is, try to find the best layout plan when the number of cameras ranges from single to four.

[0071] Set the installable space parameters of the cameras. When the installation freedom is large, the cameras can be installed in a large range of three-dimensional space and can be oriented arbitrarily. It can also be more in line with the actual situation that the cameras can only be installed on the truss in the working space, and the orientation is also restricted by the truss structure. In this embodiment, to make full use of the global optimization ability of the genetic algorithm, the cameras are set with large freedom and can be arranged at any position and in any orientation in a cube space of a*a*a.

[0072] Set the optical parameters of the camera, including the camera's field of view angle, depth of field, focal length, resolution, etc. Include the vertical field of view angle FOV V , horizontal field of view angle FOV H , depth of field DOF, focal length f, and resolution X×Y.

[0073] These parameters reflect the camera's field of view and imaging quality. Only considering whether the target is visible, the visible range model of the camera is a quadrangular pyramid determined by the field of view angle. If considering whether the target can be clearly imaged, the visible range model of the camera is a frustum of a quadrangular pyramid determined by the field of view angle and the depth of field.

[0074] The layout scheme of the camera group is represented by an n×6 matrix:

[0075]

[0076] In the formula, A represents the layout scheme of the entire camera group, A n represents the installation state of the nth camera, and [X in A n n Y n Z n is the three-dimensional space coordinates of the camera, representing the installation position of the camera, and [Vec_x n Vec_y n Vec_z n is the optical axis vector of the camera, representing the installation orientation of the camera.

[0077] Step S3. Establish a visual recognition error model based on the manipulator parameters and camera parameters, and evaluate and score the camera layout scheme, that is, establish the fitness function of the subsequent genetic algorithm;

[0078] The end pose detection is completed by extracting multiple feature points on a certain plane at the end. The theoretical calculation of the visual pose detection error is carried out. The feature points can either be obvious point features on the robotic arm or artificially placed marker points. When the number of feature points is X and the feature points are located at the vertices of a regular polygon, the following are the theoretical errors of pose calculation:

[0079] Z-direction position error:

[0080] X / Y-direction position error:

[0081] Z-direction attitude error:

[0082] X / Y-direction attitude error:

[0083] Among them, μ is the camera pixel size; f is the camera focal length; l​​o is the distance between feature points; is the feature point positioning accuracy; δ is the pixel resolution; a is a coefficient related to the number of feature points. For example, when X is 4, a is 2; when X is 6, a is 4;

[0084] It can be seen that the pose detection accuracy mainly depends on aspects such as the quality of the camera itself, the distance between the camera and the feature point group, the distribution of the feature point group, and the feature point positioning accuracy.

[0085] According to the error analysis theory, starting from improving the pose detection accuracy, an evaluation scoring system for the camera layout scheme is established. First, the evaluation indicators for a single camera and a certain manipulator state include four items: camera field of view, target relative distance, target relative position, and target relative attitude. According to the influence degree of each indicator on the pose accuracy, corresponding weight scores are set for these four evaluation indicators.

[0086] The camera field of view indicator reflects whether the end of the manipulator is within the camera's field of view and is not blocked by the manipulator body itself. This sub-item accounts for A% of the total score; the target relative distance indicator reflects whether the end of the manipulator is within the depth of field of the camera. This sub-item accounts for B% of the total score; the target relative position indicator reflects whether the end of the manipulator is at the center of the camera's field of view. This sub-item accounts for C% of the total score; the target relative attitude indicator reflects the degree to which the normal of the end plane of the manipulator deviates from the normal of the camera. This sub-item accounts for D% of the total score. Finally, the sum of A, B, C, and D is 100.

[0087] When a certain manipulator state and the camera placement are determined, the scores of each sub-item can be calculated. The cumulative sum of the scores of each sub-item reflects whether the placed camera can accurately perceive this manipulator state at this time, that is, the score reflects the quality of this camera placement.

[0088] The specific calculation method for the scores of each sub-item is as follows:

[0089] The camera field of view indicator needs to consider two aspects. One is whether the end of the manipulator is within the field of view, which can be determined by calculating whether the end of the manipulator is within the quadrangular pyramid determined by the field angle; the other is whether the end of the manipulator will be blocked by the manipulator body itself, which requires judging whether there is a manipulator entity on the line connecting the end of the manipulator and the camera. The manipulator entity is represented by the aforementioned simplified rule model, and judging the occlusion problem is transformed into judging the intersection problem between the line segment and the rule model. If the end of the manipulator is within the field of view and is not blocked by the manipulator body itself, this item can obtain all the scores of this sub-item.

[0090] The manipulator entity is simplified into a combined body of regular models such as cuboids, cylinders, and spheres. First, the regular models are parameterized, as Figure 3 shown. Then the intersection condition is:

[0091]

[0092] In the formula, the left side is the parameter representation result of a point on the line segment BC, point B is the starting point of the line segment, Right now is the direction vector of the line segment, θ4 is a parameter with a value range of [0,1]; the right side is the parameter representation result of a point in the regular model space, A is a special point of the regular model, such as the vertex of the cuboid, the center of the bottom surface of the cylinder and the center of the sphere, and is a set of orthogonal vectors, the length and direction of the orthogonal vectors are related to the model type, θ1, θ2 and θ3 are parameters, and the value range of the three parameters is related to the model type.

[0093] The above formula can be transformed into a non-homogeneous system of equations:

[0094]

[0095] The intersection of the line segment and the regular model is equivalent to the above non-homogeneous equations having a solution that satisfies the corresponding constraints.

[0096] The target relative distance indicator is based on the actual clear imaging of the camera. When the equivalent distance between the end of the robot and the camera is within the depth of field range, this item can obtain the full score of the sub-item. If the equivalent distance is too short or too long, the robot imaging will be blurred and the posture accuracy will decrease, so the score will be reduced accordingly.

[0097] The target relative position indicator is based on reducing the impact of camera distortion. When the end of the robot is on the optical axis of the camera, the end image is in the center of the field of view and the distortion is minimal. At this time, this item can obtain the full score of the sub-item. If the end of the robot deviates from the optical axis, the score will be reduced accordingly.

[0098] The target relative posture indicator starts from improving the positioning accuracy of feature points. When the end plane of the manipulator is perpendicular to the optical axis of the camera, the positioning accuracy of the feature points is the highest. At this time, this item can obtain the full score of the sub-item. As the angle between the normal of the end plane of the manipulator and the optical axis of the camera increases, the score will decrease accordingly.

[0099] For a certain state of the robot, as long as there is a camera that can achieve high-precision posture detection, the requirements are met. Therefore, the evaluation score of the camera group and a certain robot end state is the highest score among all cameras.

[0100] The final camera placement score is the average of the scores of the camera group and all robot states. For example, in this embodiment, the robot can be discretized into m states. The camera group and each robot state are first evaluated and scored, and the final camera placement score is the average of the first m scores.

[0101] Step S4. Setting the genetic algorithm hyperparameters and initializing the population. The algorithm hyperparameters include gene size, number of individuals in the population, maximum number of iterations, crossover probability, and mutation probability. Initializing the population means setting the original plan for camera layout.

[0102] The hyperparameters of the genetic algorithm are set. The selection of hyperparameters has a significant impact on the optimization speed and the final result. Hyperparameters include individual gene size, number of individuals in the population, maximum number of iterations, crossover probability, and mutation probability. In this embodiment, each individual contains a gene segment, and each gene segment represents a camera arrangement. The camera arrangement includes six parameters: [X n Y n Z n ] indicates the installation position of the camera, [Vec_x n Vec_y n Vec_z n ] represents the installation direction of the camera, and each parameter is represented by b binary numbers, that is, the length of each gene segment is a*b, and the total gene length of each individual is a*b*n; the number of individuals in the population is set to N; the maximum number of iterations is set to Max_Iterations; the crossover probability is set to x; and the mutation probability is set to y.

[0103] Initialize the population, where each individual represents a camera placement scheme. Initializing the population essentially sets the initial camera placement scheme. Unlike random initialization, the camera placement can be initialized specifically to achieve optimal pose detection for a small number of the aforementioned manipulator states. In this embodiment, half of the individuals in the population are randomly initialized, while the other half are initialized specifically, with the cameras positioned on the normal to a specific manipulator end surface, at a distance within the depth of field, and with the camera orientation aligned with the surface normal. This is the optimal camera for that manipulator state.

[0104] Step S5: Perform genetic iterative optimization, and the best individual obtained is the optimal camera arrangement solution.

[0105] Genetic iterative optimization is performed. In the genetic algorithm, the individuals in the population are camera layout plans. The genes of the individuals represent the installation conditions of each camera, involving the installation position and camera orientation. The individual fitness is the score of the camera layout plan in the evaluation and scoring system.

[0106] The genetic algorithm transforms the search for the best camera group layout into the search for individuals with the highest fitness in the population. The population is initialized as the first generation population, and genetic and mutation operations are performed to continuously generate a new generation of better populations. The genetic optimization process is as follows: Figure 4 The best individuals in the population over the generations are the optimal camera group arrangement scheme.

[0107] It can be understood that the present invention is described by means of some embodiments. Those skilled in the art will know that various changes or equivalent substitutions can be made to these features and embodiments without departing from the spirit and scope of the present invention. Additionally, under the teaching of the present invention, these features and embodiments can be modified to adapt to specific situations and materials without departing from the spirit and scope of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application belong to the scope protected by the present invention.

Claims

1. A camera layout optimization method for manipulator posture detection based on genetic algorithm, characterized in that: The steps include: Step S1. Set the working posture parameters of the manipulator and discretize the working space and common working states of the manipulator, that is, obtain the parameters of the target to be observed; Step S2. Setting camera parameters, including the number of cameras, the installation space of the cameras, and the optical parameters of the cameras, that is, obtaining the parameters of the observation camera; Step S3. Establish a visual recognition error model based on the manipulator parameters and camera parameters, evaluate and score the camera layout scheme, that is, establish the fitness function of the subsequent genetic algorithm; Based on error analysis theory, and aiming to improve pose detection accuracy, an evaluation and scoring system for camera placement schemes was established. First, the evaluation indicators for the state of a single camera and a manipulator include the camera field of view, relative distance to the target, relative position of the target, and relative pose of the target. Based on the degree of influence of each indicator on pose accuracy, corresponding weight scores were set for these four evaluation indicators. The camera field of view indicator reflects whether the end of the manipulator is within the camera field of view and is not blocked by the manipulator body. This sub-item accounts for A% of the total score; The target relative distance indicator reflects whether the end of the manipulator is within the depth of field of the camera, and this sub-item accounts for B% of the total score; The target relative position indicator reflects whether the end of the manipulator is in the center of the camera's field of view, and this sub-item accounts for C% of the total score; The target relative posture index reflects the degree to which the normal of the manipulator end plane deviates from the camera normal. This sub-item accounts for D% of the total score. Finally, the sum of A, B, C and D is 100; When the state of a certain manipulator and the placement of the camera are determined, the scores of each item are calculated. The cumulative sum of the scores of each item reflects whether the camera placed at this time can perceive the state of the manipulator with high precision, that is, the score reflects the quality of the camera placement; Step S4. Setting the genetic algorithm hyperparameters and initializing the population. The algorithm hyperparameters include gene size, number of individuals in the population, maximum number of iterations, crossover probability, and mutation probability. Initializing the population means setting the original plan for camera arrangement. Step S5: Perform genetic iterative optimization, and the best individual obtained is the optimal camera arrangement solution.

2. The method for optimizing camera arrangement for manipulator posture detection based on genetic algorithm according to claim 1, characterized in that: The step S1 includes the following specific steps: Combined with the common tasks and working conditions of the manipulator, the common working space of the manipulator is obtained, the set working space is discretized, and a large number of manipulator working states are obtained; The position and posture of the end of the manipulator are used to represent the state of the manipulator, and the posture of the end of the manipulator is represented by a 4×4 posture matrix: Where T is the pose matrix of the end of the manipulator, R is the rotation matrix of the end, and P is the translation vector of the end.

3. The method for optimizing camera arrangement for manipulator posture detection based on genetic algorithm according to claim 1, characterized in that: The step S2 includes the following specific steps: Set the number of cameras n and perform manipulator pose recognition as a camera group. The optimization goal is to obtain the optimal arrangement of the corresponding number of cameras based on the given number of cameras. Set the camera's installation space parameters. When the installation freedom is large, the camera can be installed in a large range of three-dimensional space and can be oriented in any direction. Or, more practically, the camera can only be installed on the truss in the workspace, and the orientation is restricted by the truss structure. Set the camera's optical parameters, including vertical field of view FOVV, horizontal field of view FOVH, depth of field DOF, focal length f, and resolution X×Y; If only considering whether the target is visible, the camera's visual range model is a quadrangular pyramid determined by the field of view angle. If considering whether the target can be clearly imaged, the camera's visual range model is a quadrangular pyramid determined by the field of view angle and depth of field. The layout of the camera group is represented by an n×6 matrix: Where A represents the layout of the entire camera group, A n Indicates the installation status of the nth camera, A n [X n Y n Z n ] is the camera's three-dimensional space coordinate, indicating the camera's installation position, [Vec_x n Vec_y n Vec_z n ] is the camera optical axis vector, which indicates the installation direction of the camera.

4. The method for optimizing camera arrangement for manipulator posture detection based on genetic algorithm according to claim 3, characterized in that: The step S3 includes the following specific steps: The end-point pose detection is completed by extracting multiple feature points on a certain plane of the end-point. The visual pose detection error is theoretically calculated. The feature points can be either obvious point features on the robot arm or manually placed markers. When the number of feature points is X and the feature points are located at the vertices of a regular polygon, the theoretical errors of the pose calculation are as follows: Z-direction position error: X / Y direction position error: Z-direction attitude error: X / Y direction attitude error: Where μ is the camera pixel size; f is the camera focal length; l o is the distance between feature points; is the positioning accuracy of feature points; δ is the pixel resolution; a is a coefficient, which is related to the number of feature points; The accuracy of pose detection depends on the quality of the camera itself, the distance between the camera and the feature point group, the distribution of the feature point group and the accuracy of feature point positioning; 5. The method for optimizing camera arrangement for manipulator posture detection based on genetic algorithm according to claim 4, characterized in that: The specific calculation method of each sub-item score in step S3 is as follows: The camera field of view index considers two aspects: first, whether the end of the manipulator is in the field of view, which is determined by calculating whether the end of the manipulator is in the quadrangular pyramid determined by the field of view angle; second, whether the end of the manipulator is blocked by the manipulator body, which is determined by judging whether there is a manipulator entity on the line connecting the end of the manipulator and the camera. The manipulator entity is represented by a simplified rule model, and the occlusion problem is transformed into the problem of judging the intersection of the line segment and the rule model. If the manipulator end is in the field of view and is not blocked by the manipulator body, this item can obtain the full score of the sub-item; The manipulator entity is simplified into a combination of rule models. The rule models are first parameterized, and the intersection condition is: In the formula, the left side is the parameter representation result of a point on the line segment BC, point B is the starting point of the line segment, Right now is the direction vector of the line segment, θ4 is a parameter with a value range of [0,1]; the right side is the parameter representation result of a point in the regular model space, A is a special point of the regular model, such as the vertex of the cuboid, the center of the bottom surface of the cylinder and the center of the sphere, and is a set of orthogonal vectors. The length and direction of the orthogonal vectors are related to the model type. θ1, θ2 and θ3 are parameters. The range of the three parameters is related to the model type. The above formula is transformed into a non-homogeneous system of equations: The intersection of the line segment and the regular model is equivalent to the above non-homogeneous equations having a solution that satisfies the corresponding constraints; The target relative distance indicator is based on the actual clear imaging of the camera. When the equivalent distance between the end of the manipulator and the camera is within the depth of field range, this item can obtain the full score of the sub-item. If the equivalent distance is too short or too long, the manipulator image will be blurred, the posture accuracy will decrease, and the score will be reduced accordingly. The target relative position indicator is designed to reduce the impact of camera distortion. When the end of the manipulator is on the optical axis of the camera, the end image is in the center of the field of view, and the distortion is minimal. In this case, this item can obtain the full score of the item. If the end of the manipulator deviates from the optical axis, the score will be reduced accordingly. The target relative posture indicator is based on improving the positioning accuracy of feature points. When the plane of the manipulator end is perpendicular to the camera optical axis, the feature point positioning accuracy is the highest. At this time, this item can obtain the full score of the sub-item. As the angle between the normal of the manipulator end plane and the camera optical axis increases, the score will decrease accordingly. For a certain state of the robot, as long as there is a camera that can achieve high-precision posture detection, the requirement is met. Therefore, the evaluation score of the camera group and a certain robot end state is the highest score among all cameras. The final camera arrangement scheme is scored by taking the average of the camera group and all robot status scores.

6. The method for optimizing camera arrangement for manipulator posture detection based on genetic algorithm according to claim 1, characterized in that: The step S4 includes the following specific steps: Set the hyperparameters of the genetic algorithm, including individual gene size, number of individuals in the population, maximum number of iterations, crossover probability, and mutation probability; Initialize the population. Each individual in the population represents a camera layout scheme. Initializing the population means setting the initial camera layout scheme. Starting from achieving the optimal posture detection of a small number of manipulator states, the camera layout is initialized in a targeted manner.

7. The method for optimizing camera arrangement for manipulator posture detection based on genetic algorithm according to claim 1, characterized in that: The step S5 includes the following specific steps: Perform genetic iterative optimization. In the genetic algorithm, the individuals in the population are camera layout plans. The genes of the individuals represent the installation conditions of each camera, including the installation position and camera orientation. The individual fitness is the score of the camera layout plan in the evaluation and scoring system. The genetic algorithm transforms the search for the optimal camera group layout into the search for the individual with the highest fitness in the population. The population is initialized as the first generation population, and genetic and mutation operations are performed to continuously generate a new generation of better populations. The best individuals in the populations of all generations are the optimal camera group layout.

Images