A method for automatically measuring a three-dimensional shape of a high-reflectivity object based on reinforcement learning
By constructing a dual convolutional neural network based on reinforcement learning, the problem of automating the setting of multiple exposure time series in the 3D measurement of highly reflective complex parts was solved. The global optimization of exposure time, number of exposures and measurable area of the image was achieved, improving measurement speed and accuracy.
Patent Information
- Application Number
- CN202211091678.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-09-07
- Publication Date
- 2025-11-21
- Estimated Expiration
- 2042-09-07
AI Technical Summary
In the 3D measurement of highly reflective and complex parts, existing technologies struggle to achieve global optimization of exposure time, number of exposures, and measurable area in the automated setting of multiple exposure time series, resulting in low measurement efficiency and low accuracy.
A reinforcement learning-based approach is adopted to construct a dual convolutional neural network. Through training and application phases, it predicts multiple exposure time series. By combining the reward function of exposure time, number of exposures and measurable area of the image, stable and fast multiple exposure time series calculation is achieved.
It improves the speed and accuracy of three-dimensional measurement of highly reflective parts, achieves global optimization of exposure time, number of exposures and measurable area of image, and enhances measurement efficiency and stability.
Smart Images
Figure CN116309765B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application belongs to the field of three-dimensional measurement, and more particularly relates to an automatic three-dimensional measurement method for high-reflective objects based on reinforcement learning. BACKGROUND
[0002] The three-dimensional measurement technology based on structured light has good stability and measurement speed, and is widely used in many aspects of industrial production, such as defect detection, waste sorting and processing parameter optimization. However, due to the limitation of the dynamic range of camera imaging, when measuring high-reflective complex parts such as leaves and automobile coverings, the image is prone to over-saturation or low signal-to-noise ratio, resulting in measurement failure or large noise, which seriously affects the measurement accuracy and stability.
[0003] To solve this problem, traditional methods mainly include contact measurement using three-coordinate machines and other equipment, spraying developing solution, multi-view measurement, etc., which can to some extent fill the difficult-to-measure areas, but the operation is cumbersome and may affect the measurement accuracy, making it difficult to meet the actual application requirements. In view of this, researchers have proposed methods including multi-exposure image fusion, photometric stereo, deep learning, adaptive projection, etc., which can to some extent improve the measurement speed and automation level, but still cannot meet the demand of real-time measurement in industrial field automation. Among these methods, the multi-exposure fusion method has the advantages of simple operation and good stability, and is one of the most commonly used methods in industrial production measurement. Its main idea is to select the best pixels of the same position in the multi-exposure time sequence according to the maximum and unsaturated standard to realize high dynamic range imaging, so as to overcome the problem of over-saturation and low signal-to-noise ratio in the image, and the accuracy of the final result depends on the setting of the multi-exposure time.
[0004] However, at present, how to automatically set the appropriate multi-exposure time is still a difficult problem. The existing technology usually projects multiple images first, calculates the reflectance coefficient (equivalent to irradiance) of the image, and then divides the image according to histogram partitioning, similar region searching and other methods, and calculates the appropriate exposure time of each region. This method can to some extent realize the automatic setting of exposure time, but there are two problems: 1) it often needs to project multiple images to calculate the reflectance coefficient, affecting the measurement efficiency; 2) the exposure time sequence calculated based on region segmentation only considers the overexposure of the fused image, and does not consider the pixel quality of the image, making it difficult to achieve the global optimization of exposure time, exposure frequency and image measurable area (overexposure and high signal-to-noise ratio area).
[0005] Therefore, there is an urgent need for a fast and automatic multi-exposure time sequence calculation method to realize the stable and automatic generation of multi-exposure time sequence, achieve the global optimization of exposure time, exposure frequency and image measurable area, and improve the efficiency and accuracy of three-dimensional measurement of high-reflective objects. SUMMARY
[0006] In view of the above defects or improvement needs of the prior art, the present application provides an automatic high-reflective object three-dimensional measurement method based on reinforcement learning, thereby solving the problem that the existing multiple-exposure-time sequence prediction method takes a long time and the exposure time, number of times, and image measurable area are difficult to optimize.
[0007] To achieve the above-mentioned purpose, according to the first aspect of the present application, an automatic high-reflective object three-dimensional measurement method based on reinforcement learning is provided, comprising:
[0008] Training phase:
[0009] A double convolutional neural network for predicting a multiple-exposure-time sequence is constructed based on DQN; based on a preset multiple-exposure-time sequence t1, t2, …, t i Multiple-exposure-projection-image sequence Wherein, is the exposure projection image of the sample j at the pose k obtained at the i th exposure time, i = 1, 2, …, I, j = 1, 2, …, J, k = 1, 2, …, K, I, J, K are all positive integers greater than 1;
[0010] t1, t2, …, t i and The double convolutional neural network is iteratively trained as a training set, and a trained multiple-exposure-time sequence prediction model is obtained; the action of the double convolutional neural network is exposure time increment, and the reward function is R = w c *c + w t *t-b, t represents the exposure time, w t represents the exposure time weight, b is a fixed value, used to represent the influence of the number of exposures, w c represents the weight of the coverage increment, and c represents the coverage increment;
[0011] Application phase:
[0012] S1, a uniform white image is projected onto the surface of the workpiece to be measured by a projector, and the left and right cameras synchronously acquire the projection image of the workpiece to be measured at a preset low exposure time; the preset low exposure time and the projection image acquired by the left camera or the right camera are input into the trained multiple-exposure-time sequence prediction model, and a multiple-exposure-time sequence is obtained.
[0013] S2, according to the multiple-exposure-time sequence, a multiple-exposure-raster-fringe image sequence is projected and synchronously acquired, and multiple-exposure fusion is performed to obtain an HDR raster image.
[0014] S3, according to the HDR raster image, a phase map is calculated, matching and reconstruction of the three-dimensional point cloud of the workpiece to be measured is performed.
[0015] According to a second aspect of the present application, there is provided an automatic high-reflective object three-dimensional measurement system based on reinforcement learning, comprising: a computer readable storage medium and a processor;
[0016] The computer readable storage medium is used to store executable instructions;
[0017] The processor is used to read the executable instructions stored in the computer readable storage medium, and execute the method according to the first aspect.
[0018] Overall, compared with the prior art, the above technical solutions conceived by the present application can achieve the following beneficial effects:
[0019] 1. The traditional method calculates multiple exposure time sequences by dividing multiple different exposure regions through region segmentation, and then calculates the exposure time of each exposure region, only considering whether the fused image is overexposed, and not comprehensively weighing the relationship between exposure time, exposure times, and image measurable regions. The automatic high-reflective object three-dimensional measurement method based on reinforcement learning provided by the present application constructs a multiple exposure time sequence prediction model based on reinforcement learning, first constructs a simulation environment based on irradiance and camera response function, then constructs a reward function according to the requirements of image quality and time in structured light three-dimensional reconstruction, and iteratively trains it. The reward function comprehensively weighs the relationship between exposure time, exposure times, and image measurable regions. Thus, based on a single image, multiple exposure time sequences are stably and quickly predicted, and the global optimum of exposure time, exposure times, and image measurable regions (regions that are not overexposed and have high signal-to-noise ratio) is achieved, which can improve the measurement speed and the measurement effect of high-reflective parts.
[0020] 2. The existing multiple exposure automatic sequence automatic calculation method based on structured light often needs to take multiple images to calculate irradiance to determine the exposure time sequence, which takes a long time. The automatic high-reflective object three-dimensional measurement method based on reinforcement learning provided by the present application builds a convolutional neural network and a large-scale data set, and constructs an image quality evaluation standard based on camera response function, image exposure, and other factors, obtains the true value of irradiance for training, so as to realize high-precision irradiance map prediction from a single image, which can improve the speed and stability of multiple exposure time sequence calculation. BRIEF DESCRIPTION OF DRAWINGS
[0021] Figure 1 The prediction process schematic diagram of the multiple exposure time sequence prediction model provided by the present application in the application stage;
[0022] Figure 2 The structured light imaging schematic diagram provided by the present application;
[0023] Figure 3 (a), (b), (c) in FIG. 1 are respectively a low-exposure uniform illumination image of the workpiece to be measured acquired by the left camera without projecting the grating, an actual irradiance image obtained by fusing multiple-exposure images, and an irradiance image predicted by the irradiance neural network;
[0024] Figure 4 (a), (b), (c) in FIG. 2 are respectively a low-exposure image of the workpiece to be measured acquired by the left camera with projecting the grating, an actual irradiance image obtained by fusing multiple-exposure images, and an irradiance image predicted by the irradiance neural network;
[0025] Figure 5 (a) in FIG. 3 is an actual multiple-exposure illumination image of the workpiece to be measured under uniform illumination acquired by the left camera when the exposure time is 150 ms, 300 ms, and 500 ms; Figure 5 (b) in FIG. 3 is a multiple-exposure image under uniform illumination predicted based on the irradiance and the camera response function when the exposure time is 150 ms, 300 ms, and 500 ms;
[0026] Figure 6 (a) in FIG. 4 is an actual multiple-exposure illumination image of the workpiece to be measured under the projected grating acquired by the left camera when the exposure time is 150 ms, 300 ms, and 500 ms; Figure 6 (b) in FIG. 4 is a multiple-exposure image under the projected grating predicted based on the irradiance and the camera response function when the exposure time is 150 ms, 300 ms, and 500 ms;
[0027] Figure 7 (a), (b), (c) in FIG. 5 are respectively a high-reflective workpiece image under uniform illumination, a 20-time multiple-exposure reconstruction result set manually, and a 5-time multiple-exposure reconstruction result automatically generated by using the method provided by the present application;
[0028] Figure 8 FIG. 6 is a network structure schematic diagram of a prediction network and a target network in a multiple-exposure prediction model provided by an embodiment of the present application;
[0029] Figure 9 FIG. 7 is a network structure schematic diagram of a fused irradiance prediction convolutional neural network provided by an embodiment of the present application. DETAILED DESCRIPTION
[0030] In order to make the objectives, technical solutions, and advantages of the present application clearer, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application. In addition, the technical features involved in each embodiment of the present application described below can be combined with each other as long as they do not conflict with each other.
[0031] The embodiment of the application provides a kind of based on reinforcement learning's automated high-reflective object three-dimensional measurement method, comprising:
[0032] Training phase:
[0033] Based on DQN, double convolutional neural network for predicting multiple exposure time series is constructed; based on preset multiple exposure time series t1,t2,…,t i Obtain multiple exposure projection image sequence Wherein, It is the exposure projection image of sample piece j in pose k obtained at the i th exposure time, i=1,2,…,I, j=1,2,…,J, k=1,2,…,K, I, J, K are all positive integers greater than 1;
[0034] Specifically, based on preset multiple exposure time series t1,t2,…,t i Obtain multiple exposure projection images of each sample piece in the J sample pieces in the k th pose in turn
[0035] With t1,t2,…,t i And As training set, the double convolutional neural network is iteratively trained, and a trained multiple exposure time series prediction model is obtained.
[0036] Specifically, the type of sample piece can be selected according to actual demand, for example: sheet metal piece.
[0037] Wherein, the multiple exposure projection image sequence is obtained synchronously by left and right cameras, that is: a uniform white image is projected to the surface of sample piece by projector, and multiple exposure projection image sequence of sample piece under preset multiple exposure time sequence is obtained synchronously by left and right cameras; the multiple exposure projection image sequence obtained by preset multiple exposure time sequence, left camera and / or right camera is iteratively trained as training set to the double convolutional neural network, and a trained multiple exposure time series prediction model is obtained.
[0038] For example, when J=5, K=13, then the exposure images of 5 sample pieces under 13 poses (i.e. 5*13*i images are obtained by left camera and right camera respectively) are taken as training set. i
[0039] For the convenience of collection, as shown in Figure 1 The multiple exposure projection image is preferably a grating fringe map.
[0040] The action of the double convolutional neural network is exposure time increment, and the reward function is R=w c *c+w t t-b, t represents exposure time, w t represents exposure time weight, b is a fixed value, used to represent the influence of exposure times, w c represents the weight of the coverage increment, c represents the coverage increment;
[0041] Application stage:
[0042] S1, a uniform white image is projected onto the surface of the workpiece to be measured by a projector, and the left and right cameras synchronously acquire the projection image of the workpiece to be measured under a preset low exposure time; the preset low exposure time and the projection image acquired by the left camera or the right camera are input into the trained multiple exposure time sequence prediction model to obtain a multiple exposure time sequence;
[0043] S2, a multiple-exposure grating fringe image sequence is projected onto the workpiece to be measured according to the multiple-exposure time sequence and is synchronously acquired, and multiple-exposure fusion is performed to obtain an HDR grating image.
[0044] Specifically, a series of low-to-high exposure fringe image sequences are first acquired according to the multiple-exposure time sequence, and then an HDR grating image is generated by selecting pixels based on the principle that all pixels occupying the same position in the same frequency sequence fringe image are unsaturated and maximum.
[0045] For example, for each group of images, three fringe images with a phase shift of 2π / 3 are acquired under the same exposure, that is, the unsaturated points in each group of fringe images can be used to independently reconstruct a three-dimensional graph point. Assuming that the brightness of the fringe image set decreases with the increase of the number of exposures, that is: Then the final fringe image for three-dimensional reconstruction is: Wherein, But At least one of them is less than or equal to 255, where m = min(n) is the minimum value of the function n, that is, each pixel in the reconstructed fringe image is generated by selecting the highest intensity unsaturated corresponding pixel from a group of fringe images, that is: Through the multiple-exposure technology, the well-exposed part of multiple fringe images with different exposure times as parameters can be fused to reconstruct a high dynamic range fringe image.
[0046] S3, calculating a phase map according to the HDR grating image, performing matching and reconstructing a three-dimensional point cloud of the workpiece to be measured.
[0047] Specifically, phase calculation is performed according to the left and right camera HDR grating images, and then distortion correction and polar correction are performed to make the corresponding matching points of the left and right cameras in the same row, then phase matching is performed to find the corresponding points on each row, and finally a three-dimensional point cloud is reconstructed according to the principle of triangulation.
[0048] Further, the action of the dual convolutional neural network is exposure time increment, and the reward function is R=w c *c+w t *t-b, t represents exposure time, w t represents exposure time weight, b is a fixed value, and w c represents the weight of coverage increment, and c represents coverage increment; the state includes updated exposure time and updated HDR image; the updated exposure time is the sum of the current exposure time and the exposure time increment;
[0049] In the training stage, t1 is taken as the initial exposure time, and (i.e. the exposure projection image of the jth sample at the kth pose when the exposure time is t1) is taken as the initial HDR image, the state is updated, and the updated HDR image is the image obtained by fusing the image calculated according to the updated exposure time, the fusion irradiance and the camera response function with the previous HDR image;
[0050] In the application stage, the preset low exposure time is taken as the initial exposure time, and the projection image obtained by the left camera or the right camera is taken as the initial HDR image, the state is updated, and the updated HDR image is the image obtained by fusing the image calculated according to the updated exposure time,
[0051] Preferably, the preset low exposure time (also referred to as fixed low exposure time) is the minimum value in the preset multiple exposure time sequence t1, t2, …, t i .
[0052] Specifically, the preset low exposure time can be the minimum value in the preset multiple exposure time sequence t1, t2, …, t i , or can be set according to actual requirements.
[0053] Preferably, limited by the current projector projection picture speed, the fixed low exposure time is 33.33 ms.
[0054] Preferably, the dual convolutional neural network includes a prediction network and a target network;
[0055] The prediction network is used to predict the reward function of each action in the current state;
[0056] The target network is used to predict the reward function of each action in the next state s next , so as to obtain the exposure time increment sequence to realize the maximum global reward.
[0057] Preferably, the training process of the multiple exposure time sequence prediction model comprises:
[0058] (1) initialize the prediction network and the target network to have the same structure and parameters, input the current state, and output the Q value of each action;
[0059] (2) randomly extract t1 and t2 from the training set; and set t1 and t2 as the initial state;
[0060] (3) based on the current state, calculate the expected reward of each action in the action sequence through the prediction network, select the action according to the greedy strategy, and pass it to the environment model;
[0061] (4) after receiving the action a, the environment model returns the reward R and the state s to the double convolutional neural network;
[0062] (5) after receiving the state and reward, the double convolutional neural network stores s, a, R, s next , judges whether the current state meets the learning condition, updates the state if it meets the condition, and continues to the next step for learning; if it does not meet the condition, updates the state and returns to step (2); wherein the learning condition is a specified maximum training step and a maximum exposure time;
[0063] (6) randomly extract a predetermined number of s, a, R, s next from the storage space for training, and optimize the prediction network according to the loss calculated by the fixed target network;
[0064] (7) judge whether the network parameter update setting is met, if it is met, update the network, assign the parameters of the prediction network to the target network, return to step (2) to start a new round, and stop training until the preset training number is reached; if it is not met, do not update, return to step (2) to start a new round, and stop training until the preset training number is reached; wherein the network parameter update setting is a preset training step. That is, update every preset training step, for example, if the pre-training step is 100, update at training steps 100, 200, 300, ….
[0065] Preferably, the prediction network and the target network have the same network structure, and are both multi-input convolutional neural networks. As shown in Figure 8 , first, the exposure image and the exposure time are taken as inputs; after preliminary convolution of the exposure image, it is divided into two sub-processes, one of which divides the exposure image into a histogram and extracts features according to a fully connected layer, and the other combines the exposure image and the exposure time and directly extracts features through convolution; then the two sub-processes are combined again to input a fully connected layer, and output different action feature values; finally, select the appropriate action as the output according to the size of the feature value.
[0066] Specifically, the method provided by the application comprises construction, training and application of a multiple-exposure-time sequence prediction model.
[0067] (1) Construction of a data set
[0068] A large-scale data set is constructed by keeping the projection intensity unchanged and acquiring multiple-exposure-time projection images according to a fixed multiple-exposure-time sequence (i.e. a preset multiple-exposure-time sequence), including t1, t2, …, t i and
[0069] (2) Construction of a multiple-exposure-time sequence prediction model, including a simulation environment, an agent, a reward function, a state and an action.
[0070] a) Agent: a double convolutional neural network (prediction network, target network) is constructed based on DQN to predict the next action, the prediction network is used to predict the reward function of each action under the current state, and the target network is used to predict the reward function of each action under the next state, and the ultimate goal is to obtain a series of exposure time increments to achieve the maximum global reward, which is specifically represented as an optimized value function
[0071]
[0072] wherein v π (s) represents the maximum value expected to be obtained according to the policy π under the current state, the application uses a convolutional neural network to represent the policy; R k+1 represents the reward of the k+1 step; and s represents the state.
[0073] That is, the agent predicts the expected reward of each action (exposure time increment) under the current state (HDR image) based on a convolutional neural network.
[0074] b) State: updated exposure time (current exposure time + action), updated HDR image (the updated exposure time image is calculated according to the updated exposure time and irradiance, and then fused with the previous HDR image), and the updated HDR image is represented as
[0075] H new =Fusion(H,I)
[0076] wherein H new represents the updated HDR image, H represents the previous HDR image, I represents the image under the updated exposure time, and Fusion represents a fusion function, which is specifically
[0077]
[0078] c) reward function: based on exposure time, exposure times, image measurable area to build reward function, specifically
[0079] R = w c * w t * t - b
[0080] Where t represents the exposure time, w t represents the exposure time weight; b is a fixed value, which represents the influence of exposure times, w c represents the weight of coverage increment, c represents the coverage increment
[0081]
[0082] Where num represents the number of pixels that meet the measurable requirements in H new , num all represents the total number of pixels
[0083] d) Environment: based on the irradiance map and camera response function to build simulation environment (Environment), when receiving the action (exposure increment), according to the current state to calculate the reward of this step action and the next state, and return to the agent.
[0084] e) Action: define the fixed exposure time (i.e. preset low exposure time) increment sequence as the action space, each action is an exposure time increment;
[0085] (3) Train multiple exposure time sequence prediction model
[0086] a) Initialize the prediction network and target network of the agent to the same structure and parameters, input is the current state, and output is the Q value of each action;
[0087] b) Randomly extract fixed low exposure time images and their corresponding irradiance maps from the constructed multiple exposure projection image dataset, and set the images obtained under fixed low exposure time as the initial state,
[0088] c) Based on the current state, calculate the expected reward of each action in the action sequence through the prediction network, select the action according to the greedy strategy, and pass it to the environment model;
[0089] d) After receiving the action, the environment model returns the reward and state to the agent;
[0090] e) After receiving the state and reward, the agent stores s, a, R, s next , judge whether the current state meets the learning conditions (specified maximum training steps and maximum exposure time), if it meets, update the state and continue to the next step for learning; If not, update the state and return to step 2;
[0091] f) randomly extract a certain number of s, a, R, s from the storage space next , calculate the loss of the prediction network according to the fixed target network, optimize, and then perform the next step, specifically
[0092] loss = huber (q target , Q eval (s, a)) (17)
[0093]
[0094] Wherein, loss represents the loss; huber represents the huber loss function; Q eval (s, a) represents the calculation result of the prediction network, which represents the expected Q value of state (s, a); q target represents the expected maximum Q value of the fixed target network at (s next , a) plus the reward of performing a action at state s; γ represents the reward decay coefficient;
[0095] g) judge whether the network parameter update setting (preset training step number) is met, if met, update the network, assign the parameters of the prediction network to the target network, and return to the second step to start a new round; if not met, do not update, and return to the second step to start a new round.
[0096] (4) According to the fixed low exposure time (i.e. the preset low exposure time) and the obtained image, the multiple exposure time sequence is predicted, as shown in Figure 2 , the binocular camera measurement system as shown in Figure 2 is adopted, specifically as follows:
[0097] a) set the fixed low exposure time (i.e. the minimum exposure time in Figure 2 , here taking the minimum value in the preset multiple exposure time sequence as the fixed low exposure time) and the image obtained under the fixed low exposure time as the initial state;
[0098] b) based on the current state, calculate the current action according to the agent, and deliver it to the environment model;
[0099] c) after receiving the action, the environment model returns the reward and state to the agent;
[0100] d) after receiving the state and reward, the agent judges whether the current state meets the set condition, if met, stops iteration, and performs the next step; otherwise, returns to the second step;
[0101] e) output the action (exposure time increment) sequence.
[0102] The existing automatic calculation method of the multi-exposure automation sequence based on structured light often needs to shoot multiple images to calculate irradiance to determine the exposure time sequence, which takes a long time. In view of this, the present application builds a convolutional neural network and a large-scale data set, and constructs an image quality evaluation standard based on factors such as camera response function and image exposure, obtains the true value of irradiance for training, so as to realize high-precision irradiance map prediction according to a single image, improve the speed of multi-exposure time sequence calculation and ensure stability. That is, the fusion irradiance of the projection image is obtained by the following method:
[0103] inputting the projection image into the trained irradiance prediction convolutional neural network to obtain the fusion irradiance thereof;
[0104] wherein the training process of the irradiance prediction convolutional neural network is: constructing an irradiance prediction convolutional neural network, taking t1, t2, …, t i and the fusion irradiance thereof as a training set to train the irradiance prediction convolutional neural network, and obtaining the trained irradiance prediction convolutional neural network;
[0105] wherein the calculation method of the fusion irradiance of t1, t2, …, t is:
[0106] calculating the irradiance of t1, t2, …, t according to the camera response function; calculating the weight w i of the i th exposure time according to t1, t2, …, t ; performing weighted fusion calculation based on the irradiance and the weight to obtain the fusion irradiance of t1, t2, …, t ;
[0107] wherein z is a gray value; l represents a local gray gradient; m represents a first derivative of g(z); w z , w l , and w m are corresponding weight coefficients respectively.
[0108] Specifically, first, a multi-exposure data set of a high-reflective workpiece is built, a camera response function is calculated, an irradiance map obtained by multi-exposure fusion is taken as a true value to construct an irradiance prediction convolutional neural network, and training is performed, including:
[0109] (1) keeping the projection intensity unchanged, obtaining multi-exposure projection images according to a preset fixed multi-exposure time sequence to construct a large-scale data set t1, t2, …, t i and the fusion irradiance thereof;
[0110] (2) calculating a camera response function based on the multi-exposure images and the exposure time;
[0111] z = f(Et)
[0112] wherein Z is a gray value, E is an irradiance, t is an exposure time, and f represents a function. For convenience of calculation, it can also be written as
[0113] g(z) = log(f(z)) = log(Et) = log(E) + log(t) -1
[0114] (3) Construct an image quality evaluation standard based on the camera response function, image exposure, gray gradient and other factors, and calculate the weight of each exposure time in the preset multiple exposure time sequence t1, t2, …, t i ;
[0115]
[0116] wherein z is a gray value, used to measure the exposure degree; l represents a local gray gradient, used to represent the local exposure sharpness; m represents the first derivative of g(z), used to measure the nonlinear response of the camera; w z ,w l ,w m respectively represent the corresponding weight coefficients.
[0117] (4) Based on the evaluation standard, the fusion irradiance is calculated according to the multiple exposure image fusion, that is, based on the irradiance and the weight of each exposure time in the preset multiple exposure time sequence t1, t2, …, t i , weighted fusion calculation is performed to obtain the fusion irradiance of , specifically
[0118]
[0119] E i = exp(g(z i )-log(t i ))
[0120] wherein i represents the i-th exposure time in the fixed multiple exposure time sequence, E f represents the calculated fusion irradiance, E i represents the irradiance calculated by the i-th exposure time, w i represents the weight of the i-th exposure time.
[0121] (5) Construct a fusion irradiance prediction convolutional neural network, taking an image obtained at a preset low exposure time (i.e., a single exposure image: an image of the j-th sample at the k-th pose obtained at a preset low exposure time) as input, and the fusion irradiance obtained by multiple exposure fusion (i.e., The training is performed by taking the fusion irradiance as the ground truth (GroundTruth) to predict the actual irradiance (i.e., the fusion irradiance).
[0122] Preferably, the fusion irradiance prediction convolutional neural network is a dilated convolutional neural network, and the network structure is as shown in Figure 9 As shown in the figure, the irradiance of the single-exposure image is taken as the input; after inputting the neural network, the exposure image is sequentially convolved, and then the output results of each convolution layer are finally connected together to ensure multiple receptive fields, and then convolved; and finally the predicted irradiance map is output.
[0123] The fusion irradiance prediction convolutional neural network provided by the application can realize stable and rapid prediction of the irradiance map based on a single image
[0124] To further illustrate the advantages of the rapid automatic high-reflective object three-dimensional measurement method based on reinforcement learning provided by the application, the method provided by the application is used to measure the high-reflective workpiece.
[0125] Figure 3 (a), (b), and (c) in FIG. 1 are respectively a low-exposure uniform light image of the workpiece acquired by the left camera without projecting the grating, an actual irradiance map obtained by fusing multiple-exposure images, and an irradiance map predicted by the irradiance neural network. Figure 4 (a), (b), and (c) in FIG. 2 are respectively a low-exposure image of the workpiece acquired by the left camera with projecting the grating, an actual irradiance map obtained by fusing multiple-exposure images, and an irradiance map predicted by the irradiance neural network. Figure 3 (a), (b), and (c) in FIG. 3 are respectively a low-exposure image of the workpiece acquired by the left camera with projecting the grating, an actual irradiance map obtained by fusing multiple-exposure images, and an irradiance map predicted by the irradiance neural network. Figure 4 As shown in (a), (b), and (c) in FIG. 4, the method provided by the application can effectively predict the irradiance map under any exposure time.
[0126] Figure 5 (a) and (b) in FIG. 5 are respectively an actual multiple-exposure light image of the workpiece acquired by the left camera under uniform light, and a multiple-exposure image predicted based on the irradiance and the camera response function under uniform light. Figure 6 (a) and (b) in FIG. 6 are respectively an actual multiple-exposure light image of the workpiece acquired by the left camera under the projected grating, and a multiple-exposure image predicted based on the irradiance and the camera response function under the projected grating. Figure 5 (a), (b), and (c) in FIG. 7 are respectively a low-exposure image of the workpiece acquired by the left camera under the projected grating, an actual irradiance map obtained by fusing multiple-exposure images, and an irradiance map predicted by the irradiance neural network. Figure 6 As shown in (a) and (b) in FIG. 8, the method provided by the application can effectively predict the image under any exposure time.
[0127] Figure 7 The measurement result of the high-reflective workpiece, wherein Figure 7(a), (b), (c) in Fig. 1 are respectively high-reflective workpiece images under uniform illumination, 20-time multiple exposure reconstruction results set manually, and 5-time multiple exposure reconstruction results generated automatically by using the method provided by the present application, as shown in Figure 7 As shown in (a), (b), (c) in Fig. 1, the 5-time multiple exposure reconstruction results generated by using the method provided by the present application are better in integrity than the 20-time multiple exposure reconstruction results set manually. Meanwhile, better results obtained under less exposure times can ensure higher measurement efficiency and less data processing amount, effectively verifying the effectiveness of the present method.
[0128] The embodiment of the present application provides a kind of automatic high-reflective object three-dimensional measurement system based on reinforcement learning, comprising: computer readable storage medium and processor;
[0129] The computer readable storage medium is used to store executable instructions;
[0130] The processor is used to read the executable instructions stored in the computer readable storage medium, and the method as described in any of the above embodiments is executed.
[0131] Those skilled in the art can easily understand that the above description is only a preferred embodiment of the present application, and is not intended to limit the present application, and any modification, equivalent replacement and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims
1. A method for automated three-dimensional measurement of highly reflective objects based on reinforcement learning, characterized in that, The application relates to a method for predicting a multi-exposure time sequence based on a double convolutional neural network (DQN) and a computer readable storage medium and a processor. The training phase comprises the following steps: a double convolutional neural network (DQN) is constructed for predicting a multi-exposure time sequence; based on a preset multiple-exposure time sequence t1, t2, …, t i acquiring a multiple-exposure projection image sequence wherein, is an exposure projection image of a sample j at a pose k acquired at an i-th exposure time, i = 1, 2, …, I, j = 1, 2, …, J, k = 1, 2, …, K, I, J, K are all positive integers greater than 1; t1, t2, …, t i and iteratively train the double convolutional neural network as a training set, to obtain a trained multi-exposure time sequence prediction model; the reward function of the double convolutional neural network is R = w c *c + w t *t-b, t represents an exposure time, w t is an exposure time weight, b is a fixed value, used to represent the influence of the number of exposures, w c is the weight of the coverage increment, and c is the coverage increment; The application phase comprises the following steps: S1. A uniform white image is projected onto the surface of a workpiece to be measured by a projector, and a projection image of the workpiece to be measured under a preset low exposure time is synchronously acquired by left and right cameras; the preset low exposure time and the projection image acquired by the left or right camera are input into the trained multi-exposure time sequence prediction model to obtain a multi-exposure time sequence; S2. A multi-exposure grating fringe image sequence is projected onto the workpiece to be measured according to the multi-exposure time sequence and is synchronously acquired, and multi-exposure fusion is carried out to obtain an HDR grating image; S3. A phase image is calculated according to the HDR grating image, and matching and reconstruction of a three-dimensional point cloud of the workpiece to be measured are carried out.
2. The method of claim 1, wherein, The action of the double convolutional neural network is exposure time increment, and the state comprises an updated exposure time and an updated HDR image; The updated exposure time is the sum of the current exposure time and the exposure time increment; In the training phase, with t1 as the initial exposure time, with as the initial HDR image, the state is updated, and the updated HDR image is an image obtained by fusing the previous HDR image with an image calculated according to the fusion irradiance and the camera response function of the updated exposure time. In the application phase, the preset low exposure time is used as the initial exposure time, and the projection image acquired by the left or right camera is used as the initial HDR image, the state is updated, and the updated HDR image is obtained by fusing the image calculated according to the updated exposure time, the fusion irradiance of the projection image and the camera response function with the previous HDR image.
3. The method of claim 2, wherein, The fusion irradiance of the projection image is obtained in the following way: the projection image is input into the trained irradiance prediction convolutional neural network to obtain the fusion irradiance thereof; The training process of the irradiance prediction convolutional neural network is as follows: constructing the irradiance prediction convolutional neural network, taking the fusion irradiance of t1, t2, …, t i and as a training set to train the irradiance prediction convolutional neural network, and obtaining the trained irradiance prediction convolutional neural network. The fusion irradiance is calculated as follows: The fusion irradiance is calculated as follows: According to the camera response function, calculate the irradiance of the i-th exposure time According to , calculate the weight w of the i-th exposure time i ; based on the irradiance and the weight, perform a weighted fusion calculation to obtain the fusion irradiance of the i-th exposure time ; wherein z is a gray value; I represents a local gray gradient; m represents a first derivative of g(z); w z ,w l ,w m are corresponding weighting coefficients, respectively.
4. The method of claim 1 or 2, wherein, The double convolutional neural network comprises a prediction network and a target network; The prediction network is used to predict the reward function of each action under the current state; The target network is used to predict the next state s next The reward function of each action to achieve the global reward maximum with the exposure time increment sequence.
5. The method of claim 4, wherein, The training process of the multi-exposure time sequence prediction model comprises the following steps: (1) the prediction network and the target network are initialized to have the same structure and parameters, the input is the current state, and the output is the Q value of each action; (2) randomly sampling t1 and and setting t1 and as initial states; (3) the expected reward of each action in the action sequence is calculated based on the current state by the prediction network, an action is selected according to the greedy strategy, and the action is transmitted to the environment model; (4) after receiving the action a, the environment model returns the reward R and the state s to the double convolutional neural network; (5) After receiving the state and reward, the double convolutional neural network outputs s, a, R, s next The storage is performed, and it is judged whether the current state satisfies the learning condition. If yes, the state is updated, and the next step of learning is continued. If not, the state is updated, and the step (2) is returned. The learning condition is the specified maximum training step number and maximum exposure time. (6) randomly draw a preset number of s, a, R, s from the storage space next The training is performed, and the loss of the prediction network is calculated according to the fixed target network to optimize the prediction network. (7) whether the network parameter update setting is met is judged, if yes, the network is updated, the parameters of the prediction network are assigned to the target network, a new round is started in step (2), and the training is stopped until a preset training number is reached; if not, the network is not updated, a new round is started in step (2), and the training is stopped until a preset training number is reached; wherein the network parameter update setting is a preset training step number.
6. The method of claim 1, wherein, The preset low exposure time is the minimum value in the preset multiple exposure time sequence t1, t2, …, t i .
7. A system for automated 3D measurement of highly reflective objects based on reinforcement learning, characterized in that, The application further relates to a computer readable storage medium and a processor. The computer readable storage medium is used for storing executable instructions. The processor is used for reading the executable instructions stored in the computer readable storage medium and executing the method according to any one of claims 1-6.