Assembly line reflective low-texture part pose detection method based on light field inverse rendering

Abstract

The invention discloses an assembly line reflective low-texture part pose detection method based on light field inverse rendering. Comprising the following steps: S1, equipment calibration; s2, generating an ambient light field through light field calibration; s3, collecting an image of the metal part, obtaining an actual edge through edge detection, and generating an initial prediction pose through pose estimation; s4, in the environment light field, obtaining a rendered image under the predicted pose through rendering, obtaining the gradient of the three-dimensional model of the metal part to be detected in the rendered image, and performing normalization and exponentiation processing to obtain a rendered edge; calculating a loss function value and a pose gradient according to the rendering edge and the actual edge, and updating a predicted pose according to the pose gradient; and S5, repeating the step S4 until a preset termination condition is met, and obtaining an optimal predicted pose as a pose detection result. According to the invention, after primary light field calibration is carried out, pose detection tasks of all parts on the assembly line can be completed, secondary calibration is not needed, and the requirement of efficient detection in practical application can be met.

Description

Technical Field

[0001] This invention relates to the fields of computer graphics and intelligent manufacturing, specifically to a method for detecting the pose of reflective, low-texture parts in a pipeline based on inverse light field rendering. Background Technology

[0002] Part pose detection is crucial for the normal operation of intelligent production lines and is a vital part of robotic assembly. Typically, part pose calculations are performed by identifying features such as holes and corners, combined with the part's CAD information. However, the initial pose detection accuracy of this feature recognition method is insufficient, usually exhibiting an error of 5-10 mm or 5-10°. For intelligent production lines, especially in precision assembly applications with stringent requirements, such as hole-shaft mating, this level of accuracy is unacceptable for actual production needs. Therefore, high-precision pose detection strategies are required.

[0003] For industrial parts, the edge features of the part are typically used as a reference for pose detection. An optimization method is employed to match the edge features of the rendered simulated part with those of the actual captured image, iterating and optimizing until a precise pose is obtained. However, because industrial parts are usually metal, their high reflectivity and low texture cause interference from specular reflection and self-reflection when directly photographed in real-world scenes. This can lead to edge loss and misjudgment of shadow contours when identifying the edges of the captured image, resulting in incorrect matching between the captured and rendered edges and a decrease in the accuracy of iterative optimization. Summary of the Invention

[0004] To address the issue of poor pose detection accuracy for reflective, low-texture parts in a pipeline due to the mismatch between the actual part edges and the rendered edges, this invention proposes a pose detection method for reflective, low-texture parts based on inverse lighting rendering. The method first proposes a lighting field calibration method for a pipeline scene. Then, the geometry is placed within the lighting field, and the ambient light source is calculated through inverse rendering. Finally, the ambient light source is used to achieve pose estimation of the reflective, low-texture parts based on the pipeline lighting field information.

[0005] The technical solution of this invention is as follows:

[0006] The pose detection method includes the following steps:

[0007] S1. Obtain the transformation relationship between the 3D camera coordinate system and the 2D image coordinate system, and the transformation relationship between the 3D world coordinate system and the 3D camera coordinate system through calibration.

[0008] S2. Generate the ambient light field through light field calibration.

[0009] Step S2 includes the following steps:

[0010] S2.1. Prepare a geometric object and its corresponding 3D model and material information. Place the geometric object in front of the camera lens, obtain the pose of the geometric object, and take a calibration image with the lighting and shadow information of the geometric object from a fixed angle.

[0011] S2.2 Generate an initial virtual light field based on the calibration image captured in step S2.1;

[0012] S2.3 Using the Monte Carlo ray tracing method, under a virtual light field, the 3D model, pose, and material information of the geometry are used to render a calibration rendering image.

[0013] S2.4 Obtain the similarity between the calibration rendered image and the calibration image;

[0014] S2.5. After numerically differentiating the virtual light field based on the similarity, calculate the gradient of the virtual light field. Generate a new virtual light field based on the gradient of the virtual light field and return to step S2.3.

[0015] S2.7 Repeat steps S2.3 to S2.5 until the similarity meets the preset conditions to obtain the ambient light field.

[0016] S3. Acquire an image of the metal part to be detected, and obtain the actual edge of the metal part through edge detection; estimate the pose of the metal part based on the actual edge of the metal part to be detected, and generate an initial predicted pose.

[0017] Specifically, the pose consists of six degrees of freedom of the metal part in the three-dimensional world coordinate system. The six degrees of freedom are: three translational degrees of freedom along the X, Y, and Z directions, and three rotational degrees of freedom with X, Y, and Z as axes.

[0018] S4. Under the ambient light field constructed in step S2, a rendered image of the 3D model of the metal part to be detected in the predicted pose is obtained through rendering. The gradient of the surface of the 3D model of the metal part to be detected in the rendered image is obtained, and the gradient is normalized and exponentially processed to obtain the rendered edge in the predicted pose. The loss function value and pose gradient are calculated based on the rendered edge and the actual edge obtained in step S3, and the predicted pose is updated based on the pose gradient.

[0019] Step S4 includes the following steps:

[0020] S4.1 Under the ambient light field generated in step S2, the three-dimensional model of the metal part to be detected is rendered according to the pre-constructed model, the material information of the metal part to be detected, and the predicted pose, so as to obtain the rendered image of the three-dimensional model of the metal part to be detected under the predicted pose.

[0021] Preferably, in step S4.1, the rendering method employs differentiable rendering.

[0022] Specifically, in step S4.1, the rendered image I is represented by the following formula:

[0023] I = R(G(Pose), Light) out M)

[0024] In the formula, I represents the rendered image, R() represents the rendering process, G(Pose) represents the coordinates of the 3D model of the metal part to be detected in the camera coordinate system, and Light... out M represents the ambient light field, and M represents the material information of the metal part to be tested.

[0025] S4.2 Calculate the gradient at various points on the surface of the 3D model of the metal part to be detected in the rendered image to obtain the gradient image.

[0026] S4.3. Normalize and exponentiate the gradient image to obtain the rendering edge under the predicted pose.

[0027] Preferably, in step S4.3, the exponentiation is performed according to the following formula:

[0028]

[0029] In the formula, Indicates the rendering edge. This represents the normalized gradient image, where m represents the exponentiation order, and m is selected from 5 to 10.

[0030] S4.4 Input the rendered edge and the actual edge obtained in step S3 into the loss function to obtain the loss function value.

[0031] Preferably, in step S4.4, the loss function is set according to the following formula:

[0032]

[0033] In the formula, loss represents the loss function. represents the rendered edge, P represents the actual edge, c(·) represents the convolution operation on the image "·", and ‖‖ represents the norm.

[0034] S4.5 Calculate the pose gradient using the following formula:

[0035]

[0036] In the formula, Let represent the pose gradient, ... Indicates the rendering edge;

[0037] S4.6 Update the predicted pose based on the pose gradient and the pre-set learning rate to obtain the new predicted pose.

[0038] Preferably, in step S4.6, the predicted pose is updated according to the following formula:

[0039]

[0040] In the formula, Pose t+1 Pose represents the new predicted pose. t Indicates the current predicted pose. η represents the current pose gradient, and η represents the pre-set learning rate.

[0041] S5. Repeat step S4 until the preset iteration termination condition is reached, and finally obtain the best predicted pose as the pose detection result.

[0042] The beneficial effects of this invention are:

[0043] 1) This invention obtains the actual light field under the camera image view through anti-rendering, and renders the metal parts through the simulated light field. It simulates the interference caused by the high reflectivity and low texture of the metal parts under the light field. At this time, only two edges affected by equivalent light field interference need to be matched to complete the pose calculation, which has high efficiency.

[0044] 2) Compared with the traditional pose iteration optimization method, the present invention introduces the light field factor, which can directly render the actual image information of highly reflective and low-texture metal parts under the light field, making it easier to visualize the impact of high reflectivity and low texture of metal parts, and has higher pose detection accuracy and stronger robustness for such parts.

[0045] 3) This invention can effectively simulate feature defects (such as missing edges) caused by the high reflectivity and low texture of metal parts in production line scenarios, avoiding the mismatch between traditional simple rendering of part edge features and the actual scene captured part image edge features, thus improving the efficiency and accuracy of part pose detection. Furthermore, after one light field calibration, the pose detection task for all parts on the production line can be completed without secondary calibration, meeting the high-efficiency detection requirements in practical applications. Attached Figure Description

[0046] Figure 1 This is a flowchart of the method of the present invention;

[0047] Figure 2 Calibration plate used for camera calibration;

[0048] Figure 3 Example diagram of calibrating the light field for a standard geometry;

[0049] Figure 4 Example image taken;

[0050] Figure 5 Example image of actual edge;

[0051] Figure 6 A comparison of rendered edge examples with and without the addition of the light field calculated in this article;

[0052] Figure 7 This is an iterative process. Detailed Implementation

[0053] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0054] First, the meanings of some of the terms involved in this invention are explained as follows:

[0055] In this invention, "actual edge" refers to the edge obtained by edge extraction from part images taken in the actual environment.

[0056] In this invention, "rendered edge" refers to the edge extracted from the rendered image.

[0057] The process of adjusting the pose in this invention involves six degrees of freedom of the metal part in space: three translational degrees of freedom along the X, Y, and Z directions, and three rotational degrees of freedom with X, Y, and Z as axes.

[0058] This invention provides a method for detecting the pose of pipelined reflective low-texture parts based on inverse light field rendering.

[0059] The method of the present invention includes the following steps:

[0060] S1. Obtain the transformation relationship between the 3D camera coordinate system and the 2D image coordinate system, and the transformation relationship between the 3D world coordinate system and the 3D camera coordinate system through calibration.

[0061] Specifically, the calibration includes camera calibration and optional camera-robot hand-eye calibration.

[0062] S2. Generate the ambient light field around the image acquisition points of the industrial production line through light field calibration.

[0063] Step S2 includes the following steps:

[0064] S2.1. Prepare a geometric object and its corresponding 3D model and material information. Place the geometric object in front of the camera lens. Given a fixed pose of the geometric object, obtain its pose and record it as "Pose". s When arranging the geometry, ensure that the geometry can be accurately identified in the camera image, and take a calibration image with the light and shadow information of the geometry from a fixed angle for subsequent calibration.

[0065] The geometric shape can be a sphere or a cube, etc.

[0066] S2.2 Generate an initial virtual light field based on the calibration image captured in step S2.1;

[0067] S2.3 Using the Monte Carlo ray tracing method, under a virtual light field, the 3D model, pose, and material information of the geometry are used to render a calibration rendering image.

[0068] S2.4 Obtain the similarity between the calibration rendered image and the calibration image;

[0069] S2.5. After numerically differentiating the virtual light field based on the similarity, calculate the gradient of the virtual light field. Generate a new virtual light field based on the gradient of the virtual light field and return to step S2.3.

[0070] S2.7 Repeat steps S2.3 to S2.5 until the similarity meets the preset conditions to obtain the ambient light field.

[0071] S3. Acquire images of the metal parts to be inspected at image acquisition points on the industrial production line, and obtain the actual edges of the metal parts through edge detection. Then, estimate the pose of the metal parts based on the actual edges to generate an initial predicted pose.

[0072] The pose consists of six degrees of freedom of the metal part in the three-dimensional world coordinate system. The six degrees of freedom are: three translational degrees of freedom along the X, Y, and Z directions, and three rotational degrees of freedom with the X, Y, and Z axes as axes.

[0073] S4. Under the ambient light field constructed in step S2, a rendered image of the three-dimensional model of the metal part to be detected under the predicted pose is obtained by rendering. The gradients at various points on the surface of the three-dimensional model of the metal part to be detected in the rendered image are obtained, and the gradients at various points are normalized and exponentially processed to obtain the rendered edges under the predicted pose. The loss function value and pose gradient are calculated based on the rendered edges and the actual edges obtained in step S3, and the predicted pose is updated based on the pose gradient.

[0074] Step S4 includes the following steps:

[0075] S4.1 Under the ambient light field generated in step S2, the three-dimensional model of the metal part to be detected is rendered according to the pre-constructed model, the material information of the metal part to be detected, and the predicted pose, so as to obtain the rendered image of the three-dimensional model of the metal part to be detected under the predicted pose.

[0076] In step S4.1, the rendered image I is represented by the following formula:

[0077] I = R(G(Pose), Light)out M)

[0078] In the formula, I represents the rendered image, R() represents the rendering process, G(Pose) represents the coordinates of the 3D model of the metal part to be detected in the camera coordinate system, and Light... out M represents the ambient light field, and M represents the material information of the metal part to be tested.

[0079] In step S4.1, the rendering process employs Differentiable Rendering. This rendering method allows gradient propagation for gradient calculation and pose iteration.

[0080] Among them, the material information of metal parts refers to the rendering parameter information corresponding to the PBR (Physically-Based Rendering) material of the metal parts, which usually includes reflectivity, metallicity and roughness.

[0081] S4.2 Calculate the gradient at various points on the surface of the 3D model of the metal part to be detected in the rendered image to obtain the gradient image.

[0082] S4.3. Normalize and exponentiate the gradient image to obtain the rendered edge at the predicted pose. The exponentiation operation is to facilitate the calculation of the derivative later.

[0083] In step S4.3, the exponentiation is performed according to the following formula:

[0084]

[0085] In the formula, Indicates the rendering edge. This represents the normalized gradient image, where m represents the exponentiation order, and m is selected from 5 to 10.

[0086] S4.4 Input the rendered edge and the actual edge obtained in step S3 into the loss function to obtain the loss function value.

[0087] In step S4.4, the loss function is set according to the following formula:

[0088]

[0089] In the formula, loss represents the loss function. P represents the actual edge, c(·) represents the convolution operation on the image "·", which is used to transform the binary edge image into a continuous edge image. The closer to the edge, the higher the pixel value. ‖‖ represents the norm.

[0090] Specifically, the loss function represents the following operation: for rendering edges For each pixel, calculate the minimum distance from that pixel to the corresponding pixel on the actual edge, and then perform norm calculations (either Norm 1 or Norm 2).

[0091] S4.5 Calculate the pose gradient using the following formula:

[0092]

[0093] In the formula, Let represent the pose gradient, ... Indicates the rendering edge. This represents taking the partial derivative with respect to a certain parameter. A typical pose includes x, y, z, and r. x ,r y ,r z With six parameters, the partial derivative operation is actually the partial derivative of the loss with respect to each of the six parameters. At this time, the gradient of Pose is a six-dimensional array.

[0094] S4.6 Update the predicted pose based on the pose gradient and the pre-set learning rate to obtain the new predicted pose.

[0095] In step S4.6, the predicted pose is updated according to the following formula:

[0096]

[0097] In the formula, Pose t+1 Pose represents the new predicted pose. t Indicates the current predicted pose. η represents the current pose gradient, and η represents the pre-set learning rate.

[0098] S5. Repeat step S4 until the preset iteration termination condition is reached, and finally obtain the best predicted pose as the pose detection result.

[0099] In step S5, the iteration termination condition is that the loss function converges to its minimum value.

[0100] Specific embodiments of the present invention are as follows:

[0101] Example

[0102] In this embodiment, a standard sphere is used for light field calibration, such as... Figure 4 The industrial part shown is used as the target for pose detection, i.e., the metal part to be detected. The standard sphere has the same PBR material information as the industrial part.

[0103] like Figure 1 As shown, this embodiment specifically includes the following steps:

[0104] Step 1: Equipment Calibration. Equipment calibration aims to obtain the correspondence between 3D coordinates in the camera coordinate system and 2D image coordinates, as well as the transformation relationship between the 3D world coordinate system and the camera coordinate system.

[0105] The equipment calibration process is as follows:

[0106] Will Figure 2 The checkerboard calibration plate shown is fixed within the camera's shooting range. After capturing images from multiple angles, the camera is calibrated using the Zhang Zhengyou calibration method to obtain its intrinsic parameters. These intrinsic parameters include the image's horizontal and vertical focal lengths and the position of the principal point.

[0107] Import the captured images into MATLAB's Camera Calibration toolbox, input the checkerboard parameters, and obtain the following camera intrinsic parameter matrix:

[0108]

[0109] The meanings of each parameter in the formula are as follows:

[0110] f / dx: Describes the length of the focal length along the x-axis in pixels;

[0111] f / dy: Describes the length of the focal length along the y-axis in pixels;

[0112] u0, v0: The position of the origin of the camera coordinate system in the image coordinate system, in pixels.

[0113] At this point, the camera calibration is complete, which establishes a linear correspondence between spatial pose and image features.

[0114] Optionally, if the robot is subsequently used to complete the task, hand-eye calibration of the robot is required. The calibration process involves solving the transformation matrix between the camera coordinate system and the robot base coordinate system. The robot's end effector is fixed to a calibration plate, its pose is changed, images are captured using the camera, and the transformation matrix is ​​solved based on the T-Sai method to complete the hand-eye calibration.

[0115] Step 2: Light Field Calibration. Light field calibration aims to quantitatively analyze light field factors and reconstruct the light field in the actual scene through inverse rendering. This invention is geared towards industrial assembly line scenarios with relatively constant lighting conditions; therefore, the light fields involved are all static light fields, meaning that the light intensity, illumination direction, and other factors at various locations in the scene are fixed or change only slightly.

[0116] Step S2.1: Data Preparation. Prepare standard geometric objects and their CAD models. Arrange the geometric objects in different orientations, ensuring their positions are fixed and knowable, and that these geometric objects can be accurately identified in the camera images without interference between them.

[0117] Step 2.2: Initialize the light field. The light field can be described by the positions of each pixel on a two-dimensional image and its optical information, usually saved as an .hdr file. During actual rendering, it can be remapped back to the light in three-dimensional space. In this embodiment, the light field is represented as Light = l(x,y), where x and y are the coordinates of each pixel in the light field. Based on the image with standard geometric lighting information, an initial virtual light field Light0 is initially calculated using the method proposed by Marc-André Gardner (Learning to Predict Indoor Illumination from a Single Image. ACM Transactions on Graphics, Volume 36, Issue 6, 2017).

[0118] Step 2.3: Rendering. Refer to the PBR material information M′ of the standard geometry, input the CAD model G of the standard geometry (stored as triangular facets), and the pose of the standard geometry in the camera coordinate system. s And it is rendered under the virtual light field Light0.

[0119] In this embodiment, Monte Carlo ray tracing is used for rendering. The rendering process can be represented by the following function:

[0120] I′=R'(G(Pose s ),Light0,M)

[0121] In the formula, I′ represents the rendered image of the standard geometric model, R′() represents the rendering process based on the Monte Carlo ray tracing method, and G(Pose) represents the rendering process based on the Monte Carlo ray tracing method. s ) represents the CAD model coordinates of the standard geometry in the camera coordinate system, Light0 represents the virtual light field, and M represents the PBR material information of the standard geometry.

[0122] Step 2.4: Calculate the image similarity between the rendered image I′ and the actual image K. The similarity is represented by the Peak Signal-to-Noise Ratio (PSNR). The formula for calculating PSNR is:

[0123]

[0124] In the formula, the size of both the rendered image I and the actual image K is m×n pixels, the pixel coordinates are (i,j), MSE is the mean square error, and MaxValue is the maximum value that an image pixel can take. This invention uses an 8-bit image, so it is set to 255. The higher the peak signal-to-noise ratio (PSNR), the closer the two images are.

[0125] Step 2.5: De-rendering gradient calculation. The light field is numerically differentiated using PSNR to calculate the gradient, and a new virtual light field is generated based on the obtained virtual light field gradient.

[0126] Step 2.6: Continuously render the CAD model of the standard geometry based on the new virtual light field, and solve for PSNR and gradient. That is, repeat steps 2.3 to 2.5 until PSNR reaches the set threshold. At this point, the obtained light field is... out The light field is considered as part of the actual environment, i.e.:

[0127]

[0128] In the formula, K represents the actual captured image. This formula means that Light is the output light field when two images are most similar.

[0129] In this embodiment, the calibration results of the ambient light field are as follows: Figure 3 As shown.

[0130] Step 3: Pose Determination. After calibration, when the part pose detection task needs to be completed, this paper uses the ambient light field obtained in Step 2 to simulate the 3D model of the part. The simulation results are compared with the actual captured images, and the pose is iteratively optimized to finally obtain the best predicted pose as the detection result.

[0131] The specific process of step 3 is as follows:

[0132] Step 3.1: Image Capture and Edge Extraction. Use a camera to capture an image of the metal part to be detected. In this embodiment, the image of the metal part to be detected is as follows: Figure 4 As shown. Based on the Canny edge detection theory proposed by John F., the edges are extracted from the image of the metal part to be detected, and the actual edges are denoted as P. The actual edges obtained in this embodiment are as follows. Figure 5 As shown.

[0133] Step 3.2: Initialize pose information. Using the PVNet method (PVNet: Pixel-wise Voting Network for 6DoF Pose Estimation, CVPR, 2019), pose estimation is performed based on the actual edges (captured by the camera) to obtain the 6D predicted pose of the part relative to the camera:

[0134] Pose0=[2.85,1.8,1.67,-74,-9,246].

[0135] Step 3.3: Rendering parts and edges based on light field. In this embodiment, the metal part to be detected is a polished aluminum alloy surface. The corresponding PBR material information M is consulted, and the CAD model G, material information, and ambient light field Light obtained in step 2 are used to render the metal part. out The following process is rendered, and this process is represented by the following formula:

[0136] I = R(G(Pose0), Light) out M)

[0137] In the formula, I represents the rendered image, R() represents the rendering process, G(Pose0) represents the coordinates of the 3D model of the metal part to be detected in the camera coordinate system, and Light... out M represents the ambient light field, and M represents the material information of the metal part to be tested.

[0138] In this step, the rendering process R() employs differentiable rendering. This allows gradient propagation for gradient calculation and pose iteration.

[0139] Step 3.4: After rendering is complete, specifically, this step does not use the Canny edge detection mentioned in Step 3.1 for edge extraction. Instead, it calculates the image gradient, normalizes the gradient image, and performs an exponentiation operation to enhance gradient contrast, obtaining a rendered edge that can approximately represent the edges of the parts in the rendered image. Rendering edges This can be expressed by the following formula:

[0140]

[0141] In the formula, Indicates the rendering edge. The image represents the normalized gradient, and m represents the power of exponentiation, which is selected from 5 to 10. In this embodiment, m is set to 5.

[0142] The rendered edge obtained in this embodiment like Figure 6 As shown in Figure (b), it can be seen that using the light field-based rendering method of this invention is superior to direct rendering. Figure 6 (a) diagram is closer to Figure 5 The edges of the actual captured image are shown. The purpose of this step is to preserve the differentiable gradient from the rendered image to the edge image, satisfying the gradient propagation from the predicted pose to the rendered image and then to the rendered edges.

[0143] Step 3.5: Gradient Calculation and Pose Iterative Optimization. The rendered edge is compared with the actual edge, and the loss function is calculated. The loss is calculated as follows:

[0144]

[0145] The loss function represents the difference between the edge features obtained from the simulation rendering and the edge features of the actual captured image. Here, c() represents performing a convolution operation on the image to make the binarized edge image into a continuous edge image, where the pixel value is higher the closer it is to the edge.

[0146] The smaller the loss function, the smaller the difference between the rendered edge and the actual edge. Therefore, the optimization iteration process of the method of this invention can be represented by the following formula:

[0147]

[0148] Step 3.6: Calculate the pose gradient using the following formula:

[0149]

[0150] In the formula, Represents the pose gradient. denoted by , where represents the partial derivative, ... This indicates the rendering edge.

[0151] As can be seen from the above formula, each component of the gradient is calculated separately in the above steps, without the need for computationally intensive numerical gradient solving, thus saving computational costs.

[0152] Step 3.7: Update the predicted pose based on the pose gradient and the pre-set learning rate to obtain the new predicted pose. After the t-th iteration, the updated predicted pose is:

[0153]

[0154] In the formula, Pose t+1 Pose represents the new predicted pose. t Indicates the current predicted pose. η represents the current pose gradient, and η represents the pre-set learning rate.

[0155] Step 3.8: Process the updated predicted pose in the same way as in Steps 3.3 to 3.7 for processing the initial pose.

[0156] Step 3.9: Repeat step 3.8 until the loss function reaches the threshold.

[0157] Step 3.10: After completing the accurate pose detection, convert the coordinates of the part in the camera coordinate system to the coordinates in the world coordinate system to facilitate subsequent robot operations.

[0158] In this embodiment, the iterative process is as follows: Figure 7As shown in the diagram. Thick lines represent actual edges, and thin lines represent the edges of the rendered image under the predicted pose. After 200 iterations, the pose is obtained. out =[0,1.49,0.97,10.4,-20.5,246.8], which represents the final optimized precise pose, compared with the actual pose measured using highly manual marking. truth =[3,2,1.5,-80,-10,250] is close.

[0159] The above specific embodiments are used to explain and illustrate the present invention, but not to limit the present invention. Any modifications and changes made to the present invention within the spirit and scope of the claims shall fall within the protection scope of the present invention.

Claims

1. A method for detecting the pose of pipelined reflective low-texture parts based on inverse light field rendering, characterized in that, Includes the following steps: S1. Obtain the transformation relationship between the 3D camera coordinate system and the 2D image coordinate system, and the transformation relationship between the 3D world coordinate system and the 3D camera coordinate system through calibration; S2. Generate the ambient light field through light field calibration; S3. Acquire an image of the metal part to be detected, and obtain the actual edge of the metal part to be detected through edge detection; The pose of the metal part to be detected is estimated based on its actual edges to generate an initial predicted pose. S4. Under the ambient light field constructed in step S2, the rendered image of the three-dimensional model of the metal part to be detected under the predicted pose is obtained by rendering. The gradient of the surface of the three-dimensional model of the metal part to be detected in the rendered image is obtained, and the gradient is normalized and exponentially processed to obtain the rendered edge under the predicted pose. The loss function value and pose gradient are calculated based on the rendered edge and the actual edge obtained in step S3, and the predicted pose is updated based on the pose gradient. S5. Repeat step S4 until the preset iteration termination condition is reached, and finally obtain the best predicted pose as the pose detection result.

2. The method for detecting the pose of pipelined reflective low-texture parts based on inverse light field rendering according to claim 1, characterized in that: Step S4 includes the following steps: S4.1 Under the ambient light field generated in step S2, the three-dimensional model of the metal part to be detected is rendered according to the pre-constructed model, the material information of the metal part to be detected, and the predicted pose, so as to obtain the rendered image of the three-dimensional model of the metal part to be detected under the predicted pose. S4.2 Calculate the gradient at various points on the surface of the 3D model of the metal part to be detected in the rendered image to obtain the gradient image; S4.

3. Normalize and exponentiate the gradient image to obtain the rendering edge under the predicted pose; S4.4 Input the rendered edge and the actual edge obtained in step S3 into the loss function to obtain the loss function value; S4.5 Calculate the pose gradient using the following formula: In the formula, Let represent the pose gradient, ... Indicates the rendering edge; S4.6 Update the predicted pose based on the pose gradient and the pre-set learning rate to obtain the new predicted pose.

3. The method for detecting the pose of pipelined reflective low-texture parts based on inverse light field rendering according to claim 2, characterized in that: In step S4.1, the rendered image I is represented by the following formula: I=R(G(Pose),Light out ,M) In the formula, I represents the rendered image, R() represents the rendering process, G(Pose) represents the coordinates of the 3D model of the metal part to be detected in the camera coordinate system, and Light... out M represents the ambient light field, and M represents the material information of the metal part to be tested.

4. The method for detecting the pose of pipelined reflective low-texture parts based on inverse light field rendering according to claim 3, characterized in that: In step S4.1, the rendering method adopts differentiable rendering.

5. The method for detecting the pose of pipelined reflective low-texture parts based on inverse light field rendering according to claim 2, characterized in that: In step S4.3, the exponentiation is performed according to the following formula: In the formula, Indicates the rendering edge. This represents the normalized gradient image, where m represents the exponentiation order, and m is selected from 5 to 10.

6. The method for detecting the pose of pipelined reflective low-texture parts based on inverse light field rendering according to claim 2, characterized in that: In step S4.4, the loss function is set according to the following formula: In the formula, loss represents the loss function. represents the rendered edge, P represents the actual edge, c(·) represents the convolution operation on the image "·", and ‖‖ represents the norm.

7. The method for detecting the pose of pipelined reflective low-texture parts based on inverse light field rendering according to claim 2, characterized in that: In step S4.6, the predicted pose is updated according to the following formula: In the formula, Pose t+1 Pose represents the new predicted pose. t Indicates the current predicted pose. η represents the current pose gradient, and η represents the pre-set learning rate.

8. The method for detecting the pose of pipelined reflective low-texture parts based on inverse light field rendering according to claim 1, characterized in that: Step S2 includes the following steps: S2.

1. Prepare a geometric object and its corresponding 3D model and material information. Place the geometric object in front of the camera lens, obtain the pose of the geometric object, and take a calibration image with the lighting and shadow information of the geometric object from a fixed angle. S2.2 Generate an initial virtual light field based on the calibration image captured in step S2.1; S2.3 Using the Monte Carlo ray tracing method, under a virtual light field, the 3D model, pose, and material information of the geometry are used to render a calibration rendering image. S2.4 Obtain the similarity between the calibration rendered image and the calibration image; S2.

5. After numerically differentiating the virtual light field based on the similarity, calculate the gradient of the virtual light field. Generate a new virtual light field based on the gradient of the virtual light field and return to step S2.

3. S2.7 Repeat steps S2.3 to S2.5 until the similarity meets the preset conditions to obtain the ambient light field.

9. The method for detecting the pose of pipelined reflective low-texture parts based on inverse light field rendering according to claim 1, characterized in that: The pose consists of six degrees of freedom of the metal part in the three-dimensional world coordinate system. The six degrees of freedom are: three translational degrees of freedom along the X, Y, and Z directions, and three rotational degrees of freedom with X, Y, and Z as axes.

Images