Automatic measurement method for strong reflection light parts based on face structure light view point planning

Abstract

The application discloses a kind of strong light reflection parts automatic measurement method based on face structure light view point planning, belong to three-dimensional measurement field, this method is based on the generation view point of local area visibility, and the sampling probability is calculated by the visibility of each facet and its adjacent secondary facet;It can ensure that each facet has suitable opportunity to be sampled, thereby avoiding the problem that part of facet is oversampled or ignored, which provides effective guarantee for view point screening;According to left and right camera irradiance image, multiple exposure times are automatically generated, to ensure that the optimal measurement result is obtained at the view point.This strategy not only can more accurately identify the measurable area under the current view point, but also can more comprehensively and accurately capture the surface information of the object, significantly improve the effect and efficiency of view point planning, and provide a more reliable basis for automated three-dimensional measurement.

Description

Technical Field

[0001] This invention belongs to the field of three-dimensional measurement, and more specifically, relates to an automated measurement method for highly reflective parts based on surface structured light viewpoint planning. Background Technology

[0002] Highly reflective parts are widely used in major national equipment projects such as aviation, aerospace, shipbuilding, and energy. Their quality directly affects the quality of industrial products and the lifespan of machinery. Three-dimensional morphology measurement plays a crucial role in the quality inspection of these parts. Structured light methods offer advantages such as high measurement accuracy and speed, while industrial robots offer flexibility and ease of operation. Combining these two approaches to create an automated measurement method can ensure high-precision measurement, improve product quality, increase inspection efficiency, reduce costs, and thus enhance the product's market competitiveness. Accurate and efficient measurement viewpoint planning is key to achieving automated measurement of highly reflective parts.

[0003] Currently, the viewpoint planning for automated measurement of highly reflective parts still adopts the traditional method, which is to first generate sampling viewpoints around the CAD model of the workpiece being measured by completely random sampling, then directly generate the viewpoint positions based on the normal direction of the sampled surface patches, and then filter the viewpoints with constraints such as visibility and robot reachability to obtain the optimal set of viewpoints that meet the coverage requirements.

[0004] However, using completely random sampling to generate candidate viewpoints may lead to two problems: first, unstable sampling, meaning that when there are many facets, the completely random method may result in some facets not being sampled at all; second, unbalanced sampling, meaning that when some areas have a high facet density, those areas may have many sampling points, while other areas may have too few sampling points. This will result in incomplete and inaccurate viewpoint sampling, failing to include or reflect the complete information of the measured part. Summary of the Invention

[0005] To address the aforementioned deficiencies or improvement needs of existing technologies, this invention provides an automated measurement method for highly reflective parts based on viewpoint planning using surface structured light, thereby solving the technical problem that existing viewpoint planning methods are unable to achieve rapid and complete three-dimensional measurement of highly reflective parts.

[0006] To achieve the above objectives, according to a first aspect of the present invention, an automated measurement method for highly reflective parts based on viewpoint planning of structured light is provided, comprising:

[0007] S1. Based on the STL model of the highly reflective part to be tested, multiple measurement viewpoints of the three-dimensional probe are randomly generated; the number of triangular facets with a visibility count below a threshold under all random viewpoints in the local region where each triangular facet of the STL model is located is determined, and the number is normalized to obtain the sampling probability of each triangular facet; wherein, the region formed by each triangular facet and its adjacent and second-order adjacent triangular facets is defined as the local region where each triangular facet is located; the three-dimensional probe includes a binocular camera and a projector;

[0008] S2, randomly select a target triangular facet based on the sampling probability, determine whether the measurement viewpoint of the local area where the target triangular facet is located is reachable, if so, use it as a candidate viewpoint and add it to the candidate viewpoint set, and proceed to S3, otherwise randomly select a triangular facet again based on the sampling probability.

[0009] S3, determine the triangular facets visible in the STL model from the candidate viewpoint;

[0010] S4, input the exposure time, exposure image and fused HDR image of the left and right cameras at the current moment under the candidate viewpoint into the pre-trained multiple exposure time prediction network to obtain the corresponding multiple exposure time; obtain the HDR fused image of the next moment according to the multiple exposure time, and determine whether the intensity of each visible triangular facet in the HDR fused image of the next moment is within a preset range. If so, the visible triangular facet is measurable; otherwise, it is not measurable.

[0011] S5, repeat S2-S4 until the candidate viewpoints in the candidate viewpoint set achieve the required measurement coverage; select the viewpoint sequence that optimizes the measurement time cost from the candidate viewpoint set, and have the three-dimensional probe measure the highly reflective part under test according to the viewpoint order and viewpoint position in the viewpoint sequence at the corresponding multiple exposure time.

[0012] According to a second aspect of the present invention, an automated measurement system for highly reflective parts based on surface structured light viewpoint planning is provided, comprising: a computer-readable storage medium and a processor;

[0013] The computer-readable storage medium is used to store executable instructions;

[0014] The processor is configured to read executable instructions stored in the computer-readable storage medium and execute the method as described in the first aspect.

[0015] According to a third aspect of the invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to perform the method as described in the first aspect.

[0016] In summary, compared with the prior art, the above-described technical solutions conceived by this invention can achieve the following beneficial effects:

[0017] 1. Current viewpoint generation methods generate sampled viewpoints through completely random sampling, which can easily lead to some patches being left unsampled or some regions being undersampled. The method provided in this invention generates viewpoints based on the visibility of local regions. It calculates the sampling probability by considering the visibility of each patch and its adjacent secondary patches; the higher the sum of the visibility of a patch, the greater its probability of being sampled; conversely, the lower the sum of the visibility of a patch, the lower its probability of being sampled. This ensures that each patch has a suitable opportunity to be sampled, thus avoiding the problem of some patches being oversampled or ignored, and providing effective assurance for subsequent viewpoint selection.

[0018] 2. Existing viewpoint generation methods often directly generate the viewpoint position based on the normals of sampled surfaces, then calculate the viewpoint direction (Z-axis) based on potential field methods, and randomly generate the X and Y axes. This ignores the influence of adjacent surfaces and the actual measured field of view of the probe, potentially leading to poor quality of sampled viewpoints generated at model edges and inaccurate viewpoint pose calculations. This invention first considers the influence of local regions to generate a more suitable viewpoint position. This means that when determining the viewpoint position, not only the normals of the current surface but also the normals and positions of adjacent surfaces must be considered to ensure a more reasonable and accurate generated viewpoint position. Secondly, this invention uses the potential field method to calculate the viewpoint direction. Unlike traditional methods, this invention also introduces the principal axis analysis method (PAC method) to calculate the viewpoint coordinate system. This method can better consider the field of view of structured light, thus generating a viewpoint coordinate system that better matches the actual measured field of view.

[0019] 3. In existing viewpoint planning techniques, the conventional approach is to treat the target object as a solid with a diffuse reflective surface, or, while incorporating the object's actual reflective characteristics, still rely on a fixed exposure time to determine the measurable area. Such methods often fail to fully adapt to the diverse reflective properties of object surfaces, leading to deviations between the planning results and the actual situation, or inefficient measurement. To overcome these limitations, this invention automatically generates multiple exposure times based on the irradiance images from the left and right cameras, ensuring optimal measurement results are obtained at the viewpoint. This strategy not only more accurately identifies the measurable area at the current viewpoint but also captures the object's surface information more comprehensively and precisely, significantly improving the effectiveness and efficiency of viewpoint planning and providing a more reliable foundation for automated 3D measurement.

[0020] 4. Traditional multiple exposure time prediction methods often only consider a single camera image, neglecting the coordination and mutual influence between the two cameras in a binocular camera system. This can easily lead to a situation where one image reaches its optimal grayscale level while the other suffers from overexposure or other quality issues, thus affecting the overall measurement accuracy and reliability. This invention proposes a multiple exposure time prediction method based on binocular camera irradiance maps, comprehensively utilizing the irradiance image information of the binocular camera system to accurately calculate the multiple exposure time. It can accurately extract and analyze the intensity information in the left and right camera images, ensuring that both maintain optimal grayscale levels throughout the measurement process, avoiding measurement instability and errors caused by problems in one aspect. Through this method, we can not only achieve more stable and accurate measurement results but also significantly improve the measurement performance and reliability of binocular camera systems in complex environments.

[0021] In summary, the method provided by this invention fully considers the influence of local area visibility and structured light measurement field of view, generates more suitable candidate viewpoints, and adaptively generates multiple exposure times for highly reflective surfaces to ensure that the measurable area is maximized under a single viewpoint. Attached Figure Description

[0022] Figure 1 This is one of the flowcharts of an automated measurement method for highly reflective parts based on surface structured light viewpoint planning provided in an embodiment of the present invention.

[0023] Figure 2 This is the second schematic diagram of the automated measurement method for highly reflective parts based on surface structured light viewpoint planning provided in an embodiment of the present invention.

[0024] Figure 3 This is a schematic diagram of the measurement viewpoint coordinate system provided in an embodiment of the present invention;

[0025] Figure 4 A schematic diagram of the basic components of the measurement system provided in an embodiment of the present invention;

[0026] Figure 5 This is a schematic diagram of the measurable area of ​​the binocular structured light probe provided in an embodiment of the present invention;

[0027] Figure 6 This is a schematic diagram illustrating the problems of the traditional completely random sampling method provided in the embodiments of the present invention;

[0028] Figure 7 A schematic diagram illustrating the problems with the viewpoint position and coordinate system generation method provided in the embodiments of the present invention;

[0029] Figure 8 This is a basic framework diagram for multiple exposure time prediction provided in the embodiments of the present invention;

[0030] Figure 9A neural network diagram for predicting multiple exposure times provided in an embodiment of the present invention;

[0031] Figure 10 The camera predicted irradiance map and the actual map provided in the embodiments of the present invention;

[0032] Figure 11 The grayscale intensity weighting map provided for embodiments of the present invention. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0034] This invention provides an automated measurement method for highly reflective parts based on viewpoint planning using surface structured light, such as... Figure 1-2 As shown, it includes:

[0035] S1, based on the STL model of the highly reflective part to be tested, multiple measurement viewpoints of the three-dimensional probe are randomly generated; the number of triangular facets with a visibility count below a threshold under all random viewpoints in the local region where each triangular facet of the STL model is located is determined, and the number is normalized to obtain the sampling probability of each triangular facet; wherein, the region formed by each triangular facet and its adjacent and second-adjacent triangular facets is defined as the local region where each triangular facet is located; the three-dimensional probe includes a binocular camera and a projector.

[0036] Specifically, for the given highly reflective part to be tested and the 3D probe used, the STL model of the highly reflective part to be tested and the parameters of the 3D probe (including the parameters of the left and right cameras, such as resolution, field of view, intrinsic and extrinsic parameters, optimal measurement distance, measurement depth of field, etc., and the projector parameters, such as light intensity, decay coefficient, etc.) are obtained. To ensure the stability of the visibility matrix calculation, the first n sampling viewpoints are set to be completely randomly generated. Then, for each triangular facet, its adjacent secondary facets are calculated, and a visibility frequency threshold ε is set. The sum m of the number of adjacent facets of each triangular facet in the STL model with a visibility frequency less than ε is calculated. Based on m, the sampling probability of each triangular facet is calculated.

[0037] This invention employs a surface structured light three-dimensional measurement system including left and right cameras and a projector, such as... Figure 4 As shown.

[0038] The viewpoint is defined as the pose (position and orientation) of the coordinate system of the probe carried by the robot. In this invention, the highly reflective part to be measured is fixed, and the probe is located at the end of the industrial robot's flange. The industrial robot carries the probe to achieve automated measurement. Therefore, the position of the viewpoint is defined as the three-dimensional spatial coordinates of the robot probe, and the orientation of the viewpoint is defined as the orientation of the robot probe. Simultaneously, the coordinate system of the viewpoint is consistent with the coordinate system of the probe.

[0039] The coordinate system is based on the robot base coordinate system, and all other coordinate systems are relative to the robot base coordinate system.

[0040] The aforementioned random sampling to generate the viewpoint involves randomly selecting triangular facets from the STL model and then extending the optimal measurement distance d of the 3D probe according to the normal direction. opt Then, the measurement direction is randomly changed to generate a viewpoint.

[0041] The number of random sampling viewpoints, n, can be set as needed. Preferably, in this invention, n = 5.

[0042] Determine the adjacent faces and second-order adjacent faces of each triangular face. Traverse all adjacent faces of the current face and perform a depth search recursively to find the second-order adjacent faces of each face.

[0043] The visibility threshold ε can be set according to the requirements of the actual highly reflective parts being tested.

[0044] This invention uses the regions formed by each triangular facet of the STL model and its adjacent and secondarily adjacent triangular facets as the local regions where each triangular facet is located. The normalized result of the number of triangular facets in the local regions with a visibility count below a threshold under all random viewpoints is used as the sampling probability of each triangular facet. This can increase the sampling probability of triangular facets with a low visibility count under all random viewpoints, thereby increasing the probability of triangular facets with a low visibility count being sampled when randomly generating candidate viewpoints.

[0045] Furthermore, to provide a more effective guarantee for the selection of candidate viewpoints, triangular facets with a visibility count of 0 under all random viewpoints should be given a higher sampling probability than other triangular facets with visibility below the threshold.

[0046] Based on this, as a further preferred embodiment of the present invention, the sampling probability is calculated using the approximate calculation formula proposed in the present invention. That is, preferably, in step S1, the number of triangular facets in the local region where each triangular facet of the STL model is located, whose visibility count is lower than a threshold under all random viewpoints, is approximately calculated using the following formula:

[0047] ε * mlocal -sum+ε*n0;

[0048] Where ε is the threshold, m local ε represents the number of all triangular facets in the local region where each triangular facet is located; sum represents the number of times each triangular facet in the local region where each triangular facet is located is visible from all random viewpoints; if the number of times a certain triangular facet is visible is greater than ε, then its number of times visible is recorded as ε; n0 represents the number of triangular facets in the local region where each triangular facet is located whose number of times visible is 0 from all random viewpoints.

[0049] That is, according to the above formula, the sum of the number of times the visibility of each triangular facet's adjacent faces and its second-to-last adjacent faces is less than a specified number ε under all random viewpoints is calculated. Preferably, in this invention, ε = 3.

[0050] The value of m for each triangular facet is normalized to obtain the sampling probability of each triangular facet.

[0051] To avoid excessive differences in the sampling probabilities of each triangular facet, preferably, after obtaining the sampling probabilities of each triangular facet, the method further includes:

[0052] The sampling probability and random probability 1 / N of each triangular facet. CAD After addition, normalization is performed again to update the sampling probability of each triangular facet; N CAD This represents the total number of triangular faces in the STL model.

[0053] Specifically, each triangular facet is given an equal random probability of 1 / N. CAD Then, normalization is performed again to obtain the updated normalized sampling probability for each patch.

[0054] S2, randomly select a target triangular facet based on the sampling probability, determine whether the measurement viewpoint of the local area where the target triangular facet is located is reachable, if so, use it as a candidate viewpoint and add it to the candidate viewpoint set, proceed to S3, otherwise randomly select a triangular facet again based on the sampling probability.

[0055] Specifically, based on the sampling probability of each triangular facet in the STL model obtained in S1, a facet number N is randomly generated based on discrete distribution sampling, that is, a target triangular facet is randomly assigned.

[0056] For each generated patch number, the viewpoint position and orientation are calculated based on the local patch, and it is determined whether the viewpoint is reachable.

[0057] Existing viewpoint planning methods, which directly generate viewpoint positions based on the sampled surface normal directions, are prone to two problems, such as... Figure 7 As shown, firstly, when the sampling viewpoint is at the periphery of the model, the quality of the generated viewpoint is poor; secondly, the field of view for structured light measurement is rectangular, but currently, the process of generating the viewpoint direction often only considers the Z-axis direction, treating the field of view as a circle (although actual visibility analysis is still based on a rectangle), which does not conform to the actual situation of the structured light measurement system. This will lead to inaccurate pose calculation of the sampling viewpoint, affecting the quality of viewpoint planning. Based on this, preferably, the viewpoint coordinate system is determined based on principal component analysis (PCA), that is, in step S2, determining whether the measurement viewpoint of the local area is reachable includes:

[0058] The position of the measurement viewpoint of the local region is determined based on the average center and average normal direction of the local region.

[0059] The center point of each triangular facet in the local region is projected onto the plane defined by the average normal direction. The principal axis and secondary axis of the projection point are determined by principal component analysis. The coordinate system of the measurement viewpoint of the local region is established in combination with the average normal direction to obtain the pose of the measurement viewpoint.

[0060] Determine whether the measurement viewpoint is reachable based on its pose.

[0061] Specifically, for the generated patch number, the viewpoint position and orientation are calculated based on the local patch, and then the viewpoint coordinate system is determined based on the principal component analysis (PCA) method to determine whether the viewpoint is reachable;

[0062] Based on the center and normal vector of the surface patch and its surrounding adjacent secondary surfaces, the average center and weighted normal direction of the local region are calculated. The optimal measurement distance (determined by the optical parameters of the 3D optical measuring instrument) is extended along the normal direction to obtain the position of the viewpoint. The principal axis of the viewpoint direction is determined by the principal axis analysis method (PCA). The viewpoint direction is calculated by weighting the attraction of the potential field of the local region of the highly reflective part under test. Since the pose of the measurement viewpoint is known, the robot's inverse kinematics model is then used to determine whether the robot can safely and without collision reach the generated measurement viewpoint, and the quality of the generated viewpoint is preliminarily evaluated.

[0063] Principal Component Analysis (PCA) is used to determine the principal axes of the viewpoint space. First, the average position of the centers of all patches in the local area is calculated. The center of each surface is projected onto the plane defined by the average normal. Then, PCA is used to find the principal and secondary axes of the projection points. Combined with the normal direction, the coordinate system can be determined.

[0064] Furthermore, based on the potential field method to obtain the viewpoint orientation, firstly, a direction vector (0,0,0) is initialized, and then all facets are traversed to calculate the distance and angle between each facet and the given facet. When the distance and angle requirements are met, the current direction vector is the direction vector to be solved. After normalization and coordinate system transformation, the pose of the viewpoint relative to the robot's base coordinates can be obtained, which is the pose of the desired viewpoint.

[0065] Preferably, in this invention, all facets are traversed within the field of view, with the included angle required to be less than 90°.

[0066] Furthermore, solving the robot pose through the robot inverse kinematics model refers to solving the parameters of each joint given the robot's end-effector pose. In this invention, the inverse kinematics plugin (Inverse KinematicsFast, IKFast) of the ROS system is used to solve the robot pose corresponding to all candidate viewpoints.

[0067] Furthermore, robot collision detection checks whether the robot collides with its surrounding environment and the target model.

[0068] Furthermore, the robot reachability judgment is that if any viewpoint does not have an inverse kinematics solution or the collision detection fails, it indicates that it cannot meet the reachability verification, and therefore the viewpoint cannot be added to the candidate viewpoint set.

[0069] S3, determine the triangular facets visible in the STL model from the candidate viewpoint.

[0070] Specifically, after generating candidate viewpoints, the visibility matrix of the current viewpoint is first calculated based on constraints such as viewing angle and depth of field. Then, based on the depth buffer, the irradiance map and ID map of the model under the left and right cameras are calculated to achieve rapid determination of measurement angle constraints and occlusion. This is because the calculation of determining measurement angle constraints and occlusion is relatively complex, and invisible triangular faces must not meet the measurement angle constraints and occlusion constraints. Therefore, the visibility matrix of the current viewpoint is determined first, and then it is determined whether these visible triangular faces meet the measurement angle constraints and occlusion constraints to speed up the calculation efficiency.

[0071] The calculation of the preliminary visibility matrix for the current viewpoint based on constraints such as viewing angle and depth of field follows the same steps for both cameras and projectors. Therefore, only the judgment criteria for one of them are described. The overall visibility matrix is ​​equivalent to the intersection of the three. The viewpoint coordinate system is as follows: Figure 3 As shown. Taking the viewpoint (projector) coordinate system as an example, the visibility matrix determination mainly consists of the following three aspects:

[0072] 1) Traverse all triangular faces and determine if they are within the measurement field of view of the left and right cameras and projectors. First, calculate the vector ray from the given viewpoint to each vertex of the triangular face and normalize these rays in the base coordinate system. Next, transform the vector rays in the base coordinate system to the viewpoint coordinate system. This step is achieved by applying the transformation matrix between the viewpoint coordinate system and the base coordinate system. Then, in the viewpoint coordinate system, calculate two important included angles: the angle between the ray and the XZ plane, and the angle between the ray and the YZ plane. These two included angles are calculated because the actual field of view angles of the camera, projector, etc., are different in the horizontal and vertical directions, so they need to be calculated separately. The included angles are calculated by calculating the dot product between the ray and the unit vector of the corresponding plane, and then using the inverse cosine function to calculate the angle. Finally, based on the preset field of view width and height parameters, determine whether the calculated included angles meet the field of view conditions. Generally, there are two types of judgments: one is that all vertices must be within the field of view, and the other is that as long as one vertex is within the field of view, it is acceptable. Preferably, the present invention uses a second judgment method to avoid judgment problems caused by uneven division of certain facets, such as facets being too long. The field of view is as follows: Figure 5 As shown.

[0073] 2) Traverse the triangular facets within the field of view from the previous step, determining whether they are within the depth-of-field measurement range of the left and right cameras and the projector. First, obtain the direction vector of the given viewpoint; this vector will be used for subsequent distance calculations. Then, iterate over each vertex of the triangular facet, calculating the distance from each vertex to the viewpoint. This distance is obtained by calculating the difference between the vertex position vector and the viewpoint position vector, and then calculating the dot product of this difference vector and the direction vector. Finally, based on preset minimum and maximum viewing distance parameters, determine whether the calculated distance meets the viewing distance range conditions. There are two types of judgments: one is that all vertices must be within the viewing distance range, and the other is that as long as one vertex is within the viewing distance range, it is acceptable. Similarly, this invention uses the second judgment method.

[0074] In a size of N CAD In a visibility matrix of ×1, the visibility of a triangular facet that satisfies both 1) and 2) is 1, otherwise it is 0.

[0075] The irradiance map and ID map of the left and right camera models calculated based on depth buffering are mainly simulations of the actual irradiance map and ID map of the image based on image rendering principles and light reflection models. The irradiance map is similar to the actual grayscale image captured by the camera, but the value stored in each pixel is the actual irradiance rather than grayscale. The ID map is also similar to the actual grayscale image captured by the camera, but the value stored in each pixel is the ID number of its corresponding triangle. The basic calculation process is as follows:

[0076] 1) Initialize data: First, initialize the rendered irradiance image, depth map and triangle ID matrix according to the camera parameters, and set them all to zero.

[0077] 2) Model conversion: Convert the 3D model from the base coordinate system to the camera coordinate system for subsequent projection and rendering.

[0078] 3) Calculate model reflection and depth: Calculate the Blinn-Phong reflection intensity and depth information of the model, which will be used for subsequent pixel rendering.

[0079] 4) Image rendering: Traverse each triangle surface in the initial visibility matrix and perform the following steps for each surface: First, project each vertex of the triangle from 3D space onto the 2D image plane and use camera parameters to calculate the pixel coordinates of each vertex; then, fill the pixels within the triangle using the scanline algorithm and update the irradiance and ID of the pixels according to the depth buffer; check if the pixels of the triangle exceed the boundary of the image. If they do, the triangle is unmeasurable, and the visibility matrix is ​​updated accordingly (this takes into account that some areas of some triangles may be outside the field of view, and after rendering, the pixels of the triangles will exceed the boundary of the image, so their visibility is updated to 0).

[0080] Furthermore, the rapid determination of measurement angle constraints and occlusion is mainly based on the pixel area occupied by the triangle ID in the ID image. Specifically:

[0081] 1) Pixel Traversal and ID Frequency Statistics: Traverse all pixels in the ID map to count the number of times each triangle ID appears in the ID map. This step is to obtain the pixel occupancy of each triangle facet in the rendered image (i.e., the number of pixels occupied by a triangle facet with a certain ID), thus providing a data basis for subsequent angle and occlusion determination.

[0082] 2) Measurement Angle Constraint Judgment: First, the system compares the occurrence count of each ID with the actual area of ​​its corresponding triangular facet. Specifically, it calculates the ratio of occurrence count to actual area and checks if this ratio is within a preset threshold range. If the ratio is within the threshold range, it indicates that the measurement angle of the current triangular facet is acceptable, and further occlusion judgment can be performed. Conversely, if the ratio exceeds the threshold range, it indicates that the measurement angle between the triangular facet and the camera is too large, thus excluding the facet from further consideration.

[0083] 3) Occlusion Detection: When performing occlusion and partial overshoot detection, the first step is to determine whether a triangle is occluded by other objects by comparing the frequency of occurrence of each ID with the theoretical projected area of ​​its corresponding triangle (determined using a scanline algorithm). Specifically, we calculate the ratio of the frequency of occurrence to the theoretical projected area and determine the measurability of the triangle based on whether this ratio falls within a preset threshold range. If the ratio is within the threshold range, it indicates that the current triangle is not occluded by other objects and is measurable. Furthermore, this threshold setting can also handle cases where a triangle partially exceeds the actual image range of the camera. If only a small portion of a triangle (e.g., a vertex) exceeds the image range, we still consider the triangle measurable because it still contains sufficient information for measurement and analysis. This method not only ensures measurement accuracy but also avoids missing important measurement information due to partial overshoot, thereby improving overall measurement efficiency and accuracy.

[0084] S4, set the exposure time t of the left and right cameras at the current time from the candidate viewpoint. c Left and right cameras at t c The exposed image and the fused HDR image from the left and right cameras are input into a pre-trained multiple exposure time prediction network to obtain the corresponding multiple exposure time. Based on the multiple exposure time, the HDR fused image at the next moment is obtained. It is determined whether the intensity of each visible triangular facet in the HDR fused image at the next moment is within a preset range. If so, the visible triangular facet is measurable; otherwise, it is not measurable.

[0085] Existing viewpoint planning methods often treat the probe as a whole to analyze its reflection characteristics and calculate visibility, neglecting the influence of different reflection intensities in the same area between binocular cameras. Therefore, when measuring highly reflective parts, the complex reflection characteristics of the part's surface often lead to overexposure or underexposure of the binocular camera images, resulting in missing measurement data and severely impacting viewpoint planning efficiency. Furthermore, while existing viewpoint planning methods consider the workpiece's reflection characteristics, they often use fixed parameters for measurement. When measuring highly reflective parts, this fixed-parameter method means that a single viewpoint can only measure a portion of the area, requiring multiple additional sampling viewpoints to obtain the missing areas, easily leading to a large number of planned viewpoints and low efficiency. Based on this, the method provided in this invention uses a pre-trained multiple exposure time prediction network to predict multiple exposure times.

[0086] The process of training a multi-exposure time prediction network will be introduced below.

[0087] In the training phase, firstly, a dataset for a multi-exposure time prediction network based on binocular camera irradiance maps is built, and a reinforcement learning-based training framework is proposed to construct a neural network for predicting multi-exposure times.

[0088] Specifically, viewpoints are first generated by random sampling based on the model, and then the irradiance map of the left camera and the matching irradiance map of the right camera corresponding to the viewpoint are calculated and stored as a dataset. Then, a multi-exposure time prediction framework based on deep reinforcement learning is built, the core of which is an intelligent agent, that is, a neural network that predicts the multi-exposure time.

[0089] Furthermore, a dataset for predicting multiple exposure times based on binocular camera irradiance maps is constructed. The main process is as follows: First, multiple viewpoints are generated using a random sampling method. These viewpoints serve as the basic observation units in the dataset, each with a specific spatial location and orientation, capable of capturing different scenes and lighting conditions. Then, for each generated viewpoint, a depth-buffered irradiance map rendering technique is used to generate a binocular camera irradiance map (including the left camera irradiance map and the right camera matched irradiance map) for the current viewpoint. These images not only contain the geometric information of the scene but also incorporate the influence of factors such as illumination, reflection, and occlusion, thus more realistically simulating the performance of the binocular camera in real-world environments. Specifically, the right camera matched irradiance map is actually still a left camera image, except that the value of each pixel corresponds to the irradiance on the right camera. Finally, these irradiance maps are stored to form a structured dataset, which will serve as the basis for training the neural network model to predict multiple exposure times, thereby achieving more accurate and efficient viewpoint planning.

[0090] Furthermore, the left camera irradiance map in the binocular camera irradiance map is mainly a simulated image irradiance map based on image rendering principles and a light reflection model. The irradiance map is similar to a grayscale image actually captured by the camera, but the value stored in each pixel is the actual irradiance rather than grayscale. The basic calculation process is as follows: 1) Data initialization: First, initialize the rendering irradiance image and depth map matrix according to camera parameters. 2) Model transformation: Transform the 3D model from the basic coordinate system to the camera coordinate system for subsequent projection and rendering. 3) Calculate model reflection and depth: Calculate the Blinn-Phong reflection intensity and depth information of the model, which will be used for subsequent pixel rendering. 4) Image rendering: Traverse each triangular surface in the initial visibility matrix, performing the following steps for each surface: First, project each vertex of the triangle from 3D space to the 2D image plane, using camera parameters to calculate the pixel coordinates of each vertex; then, fill the pixels within the triangle using a scanline algorithm, and update the pixel irradiance according to the depth buffer. A comparison of the calculated irradiance map with the real image is shown below. Figure 10 As shown.

[0091] Specifically, this invention emphasizes the method for constructing the right camera matching irradiance map. It is actually based on the left camera image, but the value of each pixel is determined by the irradiance at the corresponding position on the right camera. In this way, the matching relationship between the left and right cameras can be directly stored, providing a more accurate and efficient means for predicting exposure time for binocular cameras. This ensures optimal image quality under various environmental conditions, improving the performance and reliability of the binocular camera system. The construction method of the right camera matching irradiance map is as follows: Based on the left camera ID map, traverse all pixels and find the irradiance of the corresponding ID on the right camera as the value of that pixel; thus, the right camera irradiance map can be obtained.

[0092] Similarly, the irradiance map from the right camera and the matching irradiance map from the left camera can also be used.

[0093] That is, the dataset used to train the multi-exposure time prediction network includes: the irradiance map of the strongly reflective sample from the left camera and the matching irradiance map of the right camera from a random viewpoint; or, the irradiance map of the strongly reflective sample from the right camera and the matching irradiance map of the left camera from a random viewpoint.

[0094] The right camera matched irradiance map is used to characterize the irradiance of each pixel in the left camera exposure image in the right camera; the left camera matched irradiance map is used to characterize the irradiance of each pixel in the right camera exposure image in the left camera.

[0095] Furthermore, the structure and details of the reinforcement learning-based training framework are as follows: Figure 8 As shown in the diagram. The core of this framework is an agent composed of a multi-exposure time prediction network, responsible for selecting the optimal action based on the current state. In this framework, the state is a tuple composed of the current intensity map from the stereo camera (intensity is the product of exposure time and irradiance), the current exposure time, and the fused HDR map from the stereo camera, providing the agent with rich state information to help make more accurate decisions. An action is defined here as a multiple of the exposure time, indicating how many times the next exposure time is greater than the current exposure time. To ensure the accuracy and flexibility of actions, we define the action space as 1 to 10, with each action spaced 0.1 apart, ensuring the agent can select the optimal exposure time within a fine-grained range. The environment consists of the stereo camera irradiance image and a reward function, interacting with the agent to update the state and return a reward. The reward function is a multi-factor function, referencing multiple factors to calculate the reward value; details can be found in section S2. With such an environment and reward function, the agent can update its state based on the actions it takes and the feedback from the environment, thereby continuously learning and optimizing its strategy during training to achieve more efficient and accurate multiple exposure time prediction.

[0096] Furthermore, the neural network for predicting multiple exposure times has the following basic components: Figure 9 As shown. The input to the multi-exposure time prediction neural network is the state, specifically represented by the current stereo camera image, the current exposure time, and the fused HDR image from the current stereo camera. The basic structure and process of the network are as follows:

[0097] In the initial stage, the current images from the left and right cameras are respectively sent to the preprocessing stage, where they undergo histogram normalization to form a two-dimensional histogram layer. This layer not only maintains the intensity information of the original image, but also performs depth feature extraction through several fully connected layers to capture the subtle details and structural information of the image.

[0098] In parallel, the current binocular camera image, the current exposure time, and the fused HDR image are integrated into a high-dimensional layer, which is then refined through four residual modules to extract local feature information, and then expanded for further processing.

[0099] Based on this, the outputs of the histogram layer and the high-dimensional layer are fused to form a comprehensive feature layer. This layer gathers information from multiple sources, providing rich contextual information for subsequent decision-making.

[0100] In the final stage, the Dueling DQN strategy is employed to refine this comprehensive feature layer. This strategy optimizes the network's decision-making process by decoupling state values ​​and actions, thereby more accurately evaluating the expected reward of each potential action. Through this approach, the network can identify and select the most valuable actions, achieving more accurate and efficient multi-exposure time prediction.

[0101] Then, during the training phase, the neural network is iteratively trained by constructing a reward function that couples measurement efficiency and measurement quality.

[0102] Specifically, a reward function that couples measurement efficiency and measurement quality is constructed, and based on this, the system is iteratively trained under a deep reinforcement learning framework to obtain the optimal value function.

[0103] The measurement efficiency is mainly divided into two parts: (1) total exposure time T s It directly affects the data acquisition speed. (2) Number of exposures N s This not only results in an additional set of exposures but also directly impacts the computational cost and time of multi-exposure image fusion. Specifically, each exposure introduces an equal amount of computation. Therefore, measurement efficiency can be defined as…

[0104] M e =-w t *T s -N s *b, (1.1)

[0105] Where b is a fixed value used to represent the effect of the number of exposures, w t This represents the weight assigned to exposure time. Correspondingly, the decrease in measurement efficiency caused by each exposure can be expressed as...

[0106] m e =-w t *tb. (1.2)

[0107] Furthermore, the measurement quality is mainly divided into two parts: measurement coverage (C) and measurement accuracy (A). s Coverage rate refers to the ratio of the actual measurable area to the maximum measurable area.

[0108] Commonly used pixel-measurable evaluation methods are mainly based on pixel intensity, which can be expressed as:

[0109]

[0110] Where i represents pixel intensity, g m This indicates the measurable intensity range [30, 250].

[0111] Many factors influence measurement accuracy, such as system calibration, hardware equipment, and image intensity. However, in the context of multiple exposure fusion technology, the only variable to consider is image intensity. More specifically, the optimal image intensity corresponds to the highest measurement accuracy. Preferably, this invention uses the following grayscale-accuracy relationship, such as... Figure 11 As shown, specifically

[0112]

[0113] As image intensity increases, accuracy initially improves significantly, then increases slowly, and finally decreases. This is mainly because when the intensity is low, nonlinear response and random noise have a strong impact, resulting in low accuracy. As the intensity increases, the influence of both factors weakens simultaneously, leading to a significant improvement in accuracy. When the intensity is moderate, the impact of nonlinear response is small, so accuracy increases slowly with increasing intensity. When the intensity is too high, the impact of random noise is small, while the impact of nonlinear response is large, thus accuracy begins to decrease.

[0114] Therefore, measurement quality can be defined as

[0115] M q =C s +A s =C(H)+A(H), (1.5)

[0116]

[0117]

[0118] Where H l H r These represent the HDR images from the left and right cameras, respectively; the ∩ symbol indicates finding the intersection of corresponding pixels in the left and right camera images, meaning that a pixel is considered measurable only when both pixels meet a specified intensity range, i.e., they intersect; N represents the total number of pixels in the image; c max H represents the maximum measurable coverage area of ​​an image, determined by the minimum and maximum exposure times. max c max The corresponding maximum measurable matrix. H max In this model, measurable elements are assigned a value of 1, while all others are assigned a value of 0. Based on the above measurement quality assessment, the measurement coverage and optimal intensity of the binocular camera can be effectively considered.

[0119] These are the measurability matrices of the left and right camera HDR images at the current time, respectively. The calculation formula is given in Formula 1.3. These are the precision matrices of the left and right camera HDR images at the current moment, respectively. The calculation formula is shown in Formula 1.4.

[0120] Furthermore, the reward function that couples factors such as measurement quality, exposure time, and number of exposures is specifically divided into several parts. In the context of optimizing the training strategy, the reward function can be categorized into three distinct types: single-step reward, sub-objective reward, and main objective reward.

[0121] Training typically focuses on using a single-step reward function, meaning the reward function includes a single-step reward function because it concisely represents the effect at each step.

[0122] However, this reward method is sparse and cannot fully represent the final result, posing a challenge in ensuring training speed and overall optimization. Therefore, preferably, the above three reward methods are combined to achieve more refined and comprehensive training optimization:

[0123] (1) Single-step reward function: The single-step reward is mainly used to represent the improvement in measurement quality and the decrease in measurement efficiency brought about by a single exposure time, specifically as follows:

[0124] R = w q *R q +w e *R e (1.8)

[0125] Among them, R q R represents the reward corresponding to the measured quality. e w represents the reward for measurement efficiency. q ,w e R respectivelyq ,R e The corresponding weights. For R e It can be directly represented as

[0126] R e =m e =-w t *tb, (1.9)

[0127] For R q The reward should be for improvements in the measured quality, not for the quality itself. Therefore, it can be expressed as...

[0128]

[0129] in, The increment of the measured mass is expressed as...

[0130]

[0131] This indicates the total loss of image quality due to improper exposure timing. For example, when an action results in a long subsequent exposure time, the image quality may improve significantly, but some areas may be overexposed and fail to achieve the desired quality, which subsequent actions cannot recover. Therefore, It can be represented as

[0132]

[0133] Among them, O(H c ),O(H p This represents the overexposure matrix of the HDR image at the current moment and the HDR image at the previous moment, with overexposure elements being 1 and other elements being 0.

[0134] (2) Sub-objective reward function: Sub-objective reward refers to decomposing the objective into sub-objectives and giving them rewards to promote faster convergence of the agent and improve performance. This invention defines two sub-objectives: coverage requirement and measurement accuracy requirement, expressed as follows:

[0135]

[0136] Where G1 and G2 represent the sub-targets of measurement area coverage and measurement accuracy, respectively. m ,Thr q These represent the set threshold values ​​for G1 and G2, respectively. Preferably, in this invention, Thr is set... m =0.98,Thr q =0.75. A reward will be given upon achieving any sub-goal after each action is completed.

[0137] (3) Main storyline objective rewards

[0138] The principal objective reward represents the efficiency and quality of the final result obtained from a series of actions given by the agent. Therefore, it can be represented as...

[0139]

[0140] Where w Q ,w E M respectively q M e The corresponding weights. F1 indicates that the specified goal has been achieved. F2 indicates that the maximum number of exposures or the longest exposure time has been reached, but the goal has not been achieved. The reason for the difference in weight between the final reward and the single-step reward is that the final reward does not need to consider the losses caused by each step.

[0141] Furthermore, the iterative training within the deep reinforcement learning framework specifically involves training using the Adam optimizer. Preferably, during training, images are sampled to 960*600 pixels to improve training efficiency, and data augmentation methods such as rotation and cropping are used to expand the dataset. The Adam optimizer is used for 60,000 training rounds. The learning rate for the first 30,000 rounds is 10. -4 The learning rate for the next 30,000 rounds is 10. -5 The training parameters are as follows: learning interval steps are 20, batch size is 16, memory size is 2000, and neural network replacement interval is 200. Here, "one round" refers to the predicted exposure sequence meeting specified conditions and the state being reinitialized.

[0142] In step S4, the multiple exposure time is automatically generated based on the irradiance maps of the left and right cameras to accurately calculate the measurability matrix of the current viewpoint.

[0143] First, an initial state is constructed based on the irradiance maps of the left and right cameras. This state is then input into a multi-exposure time prediction network to generate actions. The next state is updated based on these actions, and this process is iterated until the set requirements are met. Finally, the measurability matrix of the current viewpoint is accurately calculated based on the multi-exposure time.

[0144] The initial state is constructed based on the irradiance maps of the left and right cameras, including three items: 1) the current HDR images of the left and right cameras; 2) the current exposed images of the left and right cameras; and 3) the current exposure time. The current exposure time is set to the minimum exposure time, the current exposed images of the left and right cameras are the images corresponding to the minimum exposure time, and the current HDR images of the left and right cameras are empty matrices, i.e., all values ​​are 0. Preferably, the minimum exposure time is set to 10ms in this invention.

[0145] Based on the multi-exposure time prediction network, an action is generated. Specifically, as in the training process, the state is input into the multi-exposure time prediction network to obtain the exposure multiplier.

[0146] The next state is updated based on the action, specifically by generating the next exposure time multiplier based on the action and calculating the next exposure time; then, the exposure images of the left and right cameras for the next exposure time are calculated accordingly, and a fused HDR image of the left and right cameras is generated. This is specifically represented as follows:

[0147] (H lc H rc )=F(H lp ,I l H rp ,I r ), (1.17)

[0148] Where H lc H rc H lp H rp I represents the left and right camera HDR images at the current moment and the left and right camera HDR images at the previous moment. l ,I r This represents the camera exposure images at the current moment, and F represents the fusion function. Specifically...

[0149]

[0150] A pixel is replaced only if the sum of the precision of the pixels corresponding to the left and right cameras in the next exposure is greater than the sum of the precision of the pixels corresponding to the left and right cameras in the current exposure.

[0151] The method of accurately calculating the measurability matrix of the current viewpoint based on the multiple exposure time mainly involves generating HDR fusion images of the left and right cameras based on the multiple exposure time according to Formula 1.18 to obtain the intensity of each triangular facet. Then, iterate through all visible triangular facets in the visibility matrix and determine whether the intensity corresponding to the triangular facet is within the set threshold range. If it is, it is determined to be measurable.

[0152] S5, repeat S2-S4 until the candidate viewpoints in the candidate viewpoint set achieve the required measurement coverage; select the viewpoint sequence that optimizes the measurement time cost from the candidate viewpoint set, and have the three-dimensional probe measure the highly reflective part under test according to the viewpoint order and position in the viewpoint sequence at the multiple exposure time corresponding to each viewpoint.

[0153] Specifically, steps S2-S4 are executed repeatedly until the set requirements are met; then the above viewpoints are filtered and optimized to select the best viewpoint sequence that meets the measurement requirements, and the robot and other hardware are controlled to execute the automated 3D measurement process.

[0154] Specifically, the above steps are repeated to generate viewpoints until the set requirements are met (mainly referring to the coverage rate meeting the requirements. Preferably, the coverage rate is set to 99%). Then, the Monte Carlo tree search method is used to filter the viewpoints and generate a viewpoint sequence with globally optimal measurement cost. Finally, based on the generated viewpoint sequence, the robot and other hardware are controlled to execute an automated 3D measurement process.

[0155] Among them, the viewpoint sequence that optimizes the measurement time cost is selected from the candidate viewpoint set. The selection method can be Monte Carlo tree search, random genetic algorithm, greedy algorithm or ant colony algorithm, etc.

[0156] Taking the Monte Carlo tree search method as an example to filter viewpoints and generate a viewpoint sequence with globally optimal measurement cost:

[0157] First, a search tree is initialized, where the root node represents the current viewpoint state. Then, starting from the root node, the most promising child node is recursively selected using a strategy (such as the UCB strategy) until a leaf node is reached. At the leaf node, new child nodes are expanded based on possible actions or decisions; these nodes represent different viewpoints or viewpoint sequences. Next, simulations are performed on the newly expanded child nodes, estimating the potential value of each node through a series of random actions. Based on the simulation results, the statistics of all nodes on the path from the leaf node to the root node are updated, such as the number of visits and value estimates. This process is repeated until a predetermined termination condition is met, such as reaching the maximum number of iterations or a time limit. Finally, the optimal viewpoint sequence is extracted from the root node, corresponding to the path with the highest value estimate.

[0158] Furthermore, the value of a node in the Monte Carlo tree search method, more specifically referred to as cost in this invention, indicates the measurement time cost required from that node to the end of the measurement. The measurement time cost from the current viewpoint to the next viewpoint is defined as...

[0159] T = t move +t exp +N exp (1.19)

[0160] Where t move t represents the motion time from the current viewpoint to the next viewpoint. exp N represents the total exposure time for the next viewpoint. exp This indicates the total number of exposures for the next viewpoint.

[0161] Furthermore, the control robot and other hardware execute an automated 3D measurement process, mainly referring to the control robot traversing all viewpoints. At each viewpoint, the control probe begins measurement according to the planned multiple exposure time, then moves to the next viewpoint, iterating repeatedly until the viewpoint traversal is complete. A specific hardware diagram is shown below. Figure 4 As shown.

[0162] This invention provides an automated measurement system for highly reflective parts based on surface structured light viewpoint planning, comprising: a computer-readable storage medium and a processor;

[0163] The computer-readable storage medium is used to store executable instructions;

[0164] The processor is configured to read executable instructions stored in the computer-readable storage medium and execute the method as described in any of the above embodiments.

[0165] This invention provides a computer-readable storage medium storing computer instructions that cause a processor to perform the method described in any of the embodiments.

[0166] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An automated measurement method for highly reflective parts based on viewpoint planning of structured light, characterized in that, include: S1, randomly generate multiple measurement viewpoints of the three-dimensional probe based on the STL model of the highly reflective part to be tested; The number of triangular facets in the local region where each triangular facet of the STL model is located is determined, and the number of times the triangular facets are visible under all random viewpoints is lower than a threshold. The number of triangular facets is then normalized to obtain the sampling probability of each triangular facet. The region consisting of each triangular facet and its adjacent and second-order adjacent triangular facets is defined as the local region where each triangular facet is located. The 3D probe includes a binocular camera and a projector. S2, randomly select a target triangular facet based on the sampling probability, determine whether the measurement viewpoint of the local area where the target triangular facet is located is reachable, if so, use it as a candidate viewpoint and add it to the candidate viewpoint set, and proceed to S3, otherwise randomly select a triangular facet again based on the sampling probability. S3, determine the triangular facets visible in the STL model from the candidate viewpoint; S4, input the exposure time, exposure image and fused HDR image of the left and right cameras at the current moment under the candidate viewpoint into the pre-trained multiple exposure time prediction network to obtain the corresponding multiple exposure time; obtain the HDR fused image of the next moment according to the multiple exposure time, and determine whether the intensity of each visible triangular facet in the HDR fused image of the next moment is within a preset range. If so, the visible triangular facet is measurable; otherwise, it is not measurable. S5, repeat S2-S4 until the candidate viewpoints in the candidate viewpoint set achieve the required measurement coverage; select the viewpoint sequence that optimizes the measurement time cost from the candidate viewpoint set, and make the three-dimensional probe measure the strong reflective part under test according to the viewpoint order and viewpoint position in the viewpoint sequence at the corresponding multiple exposure time. In step S1, the number of triangular faces in the local region where each triangular facet of the STL model is located, whose visibility count is lower than a threshold under all random viewpoints, is approximately calculated using the following formula: ； in, For the threshold, This refers to the number of triangular facets in the local region where each triangular facet is located. Let be the number of times each triangular facet is visible in the local region it belongs to, across all random viewpoints. If the number of visibilitys of any triangular facet is greater than 1, then 1 is considered a triangular facet. Then, its visibility count is denoted as ; This refers to the number of triangular facets in the local region where each triangular facet is located, whose visibility count is 0 across all random viewpoints.

2. The method as described in claim 1, characterized in that, In step S2, determining whether the measurement viewpoint of the local area is reachable includes: The position of the measurement viewpoint of the local region is determined based on the average center and average normal direction of the local region. The center point of each triangular facet in the local region is projected onto the plane defined by the average normal direction. The positions of all projected points are statistically analyzed using principal component analysis to determine the principal and secondary axes of the viewpoint coordinate system. The coordinate system of the measurement viewpoint in the local region is then established in conjunction with the average normal direction to obtain the pose of the measurement viewpoint. Determine whether the measurement viewpoint is reachable based on its pose.

3. The method as described in claim 1, characterized in that, In step S1, after obtaining the sampling probability of each triangular facet, the method further includes: The sampling probability and random probability 1 / for each triangular facet N CAD After addition, normalization is performed again to update the sampling probability of each triangular facet; N CAD This represents the total number of triangular faces in the STL model.

4. The method as described in claim 1, characterized in that, The exposure images of the left and right cameras under the candidate viewpoint are obtained based on the exposure time of the left and right cameras under the candidate viewpoint and the irradiance map of the left camera and the right camera under the candidate viewpoint. Alternatively, the exposure images of the left and right cameras under the candidate viewpoint are obtained based on the exposure time of the left and right cameras under the candidate viewpoint and the irradiance map of the right camera and the matching irradiance map of the left camera under the candidate viewpoint. The right camera matched irradiance map is used to characterize the irradiance of each pixel in the left camera exposure image in the right camera; the left camera matched irradiance map is used to characterize the irradiance of each pixel in the right camera exposure image in the left camera.

5. The method as described in claim 1, characterized in that, The multi-exposure time prediction network is built based on DQN. Actions are multiples of the exposure time, and states include the exposure times of the left and right cameras at the current moment, the exposed image, and the fused HDR image. The reward function includes: Single-step reward function: ; in, As a reward for measuring quality, , , , , , These are the HDR images of the left and right cameras at the current moment. These are the left and right camera HDR images from the previous moment, respectively; It is the intersection of measurable pixels in the images exposed by the left and right cameras; The total number of pixels in the HDR image fused from the left and right cameras; This indicates the maximum measurable coverage area of ​​the fused HDR image from the left and right cameras. for The corresponding maximum measurable matrix; In this context, measurable elements are represented by 1, and all other elements by 0. For the maximum measurable coverage, , , For the measurable intensity range, , They are respectively Overexposure matrix, , They are respectively Overexposure matrix; Incentives for measuring efficiency , For fixed values, for t The weighting coefficients, t The exposure time at the current moment; , They are respectively , Weighting coefficients; The training set of the multiple exposure time prediction network includes: the exposure time of the highly reflective sample under random viewpoints, the exposure images, and the fused HDR images of the left and right cameras; The exposure images of the highly reflective sample under the left and right cameras at a random viewpoint are obtained based on the exposure time of the left and right cameras under the random viewpoint and the irradiance map of the left camera and the matching irradiance map of the right camera under the random viewpoint. Alternatively, the exposure images of the highly reflective sample from the left and right cameras at a random viewpoint are obtained based on the exposure time of the left and right cameras at the random viewpoint and the irradiance map of the right camera and the matching irradiance map of the left camera at the random viewpoint. The right camera matched irradiance map is used to characterize the irradiance of each pixel in the left camera exposure image in the right camera; the left camera matched irradiance map is used to characterize the irradiance of each pixel in the right camera exposure image in the left camera.

6. The method as described in claim 5, characterized in that, The reward function also includes: Sub-goal reward function: ; in, These are the measurement coverage sub-target and the measurement accuracy sub-target after each action is performed. They are respectively The threshold; Main objective reward function: ; in, F 1 indicates that the main objective is achieved after performing all actions. The main objective is to simultaneously satisfy G1 and G2. F 2 indicates that the maximum number of exposures or the longest exposure time has been reached after all actions have been performed, but the main objective has not been achieved; They are respectively Weighting coefficients; , Total exposure time Number of exposures; , , , These represent HDR images from the left and right cameras, respectively.

7. The method as described in claim 5 or 6, characterized in that, In the initial state, The status includes the current exposure time of the left and right cameras being the set minimum exposure time, the current exposure image of the left and right cameras being the exposure image under the minimum exposure time, and the intensity of each pixel in the current fused HDR image of the left and right cameras being 0.

8. An automated measurement system for highly reflective parts based on viewpoint planning using structured surface lighting, characterized in that, include: Computer-readable storage media and processors; The computer-readable storage medium is used to store executable instructions; The processor is configured to read executable instructions stored in the computer-readable storage medium and execute the method as described in any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a processor to perform the method as described in any one of claims 1-7.

Images