Structured light three-dimensional measurement method and device based on local adaptive stripe brightness
By acquiring camera images and using neural networks to predict projected grayscale values, an adaptive brightness-coded stripe image is generated, solving the problem of point cloud loss on the surface of highly reflective objects in structured light 3D reconstruction and achieving higher-precision 3D reconstruction.
Patent Information
- Application Number
- CN202410828428.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-25
- Publication Date
- 2024-11-12
AI Technical Summary
Existing structured light 3D reconstruction technology is prone to point cloud loss on surfaces of objects with high reflectivity or large reflectivity variations, resulting in inaccurate reconstruction.
By acquiring uniform brightness modulation images and coded stripe modulation images captured by the camera, a neural network model is used to predict the optimal projection grayscale value of the pixel saturation region, and the image is mapped onto the projector plane based on the homography relationship to generate an adaptive brightness coded stripe image, ultimately obtaining the three-dimensional point cloud of the object surface.
It effectively solves the problem of large-area loss of point cloud on highly reflective object surfaces, and improves the accuracy of 3D reconstruction, especially for highly reflective or highly variable reflectivity object surfaces.
Smart Images

Figure CN118941704A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a structured light three-dimensional measurement method and device based on local adaptive stripe brightness. Background Art
[0002] The non-contact measurement method in 3D reconstruction technology has the advantages of fast measurement speed, dense reconstruction point cloud, good accuracy, etc., which effectively overcomes the shortcomings of contact measurement methods represented by coordinate measuring machines (CMM), such as damage to the surface of the measured object, slow measurement, and limited measurement points. Among the many non-contact measurement methods, active 3D measurement technology is a common method in the field of optical measurement. It uses a certain projection device to actively project various light signals in the form of points, lines or surfaces to the measured object, effectively avoiding the problem of difficulty in matching corresponding point features on the surface of an object in images taken by multiple cameras in passive measurement methods such as stereo vision, and overcomes the problem of sparse information in reconstructing 3D point clouds caused by the lack of rich texture information on the surface of the measured object.
[0003] As one of the most widely used active measurement technologies in engineering, structured light 3D reconstruction technology has the advantages of high resolution and high precision. It has been widely used in industrial inspection, reverse engineering, cultural relics protection, criminal investigation and other fields. However, there are still some problems with structured light 3D reconstruction technology. One of the problems that needs to be solved is how to accurately measure objects with highly reflective surfaces or a large range of surface reflectivity. For example, for a surface area of a measured object with high reflectivity, when the actively projected structured light beam passes through this area and is reflected to the camera sensor, the image pixel value corresponding to this area is easily saturated, resulting in the complete loss of the highly modulated structured light information on the surface of the object. Therefore, the reconstruction result of this high reflectivity area will have serious point cloud missing. Summary of the invention
[0004] The present invention provides a structured light three-dimensional measurement method and device based on local adaptive stripe brightness, which are used to solve the technical problem of inaccurate three-dimensional reconstruction of the surface of a highly reflective object to be measured in the prior art.
[0005] In a first aspect, the present invention provides a structured light three-dimensional measurement method based on local adaptive stripe brightness, comprising: acquiring an image modulated by the surface of an object; the image modulated by the surface of the object comprises a first uniform brightness modulated image and a first coded stripe modulated image captured by a camera; Obtaining the optimal projection grayscale value of the projector pixel corresponding to the pixel saturation area in the first uniform brightness modulated image, and obtaining the homography relationship between each pixel saturation area in the first uniform brightness modulated image and the corresponding area of the projector projection plane based on the first coded fringe modulated image; Based on the homography relationship, the optimal projection grayscale value is mapped to a corresponding position on the projection plane of the projector to obtain a regional adaptive brightness coded fringe image; A three-dimensional point cloud of the object surface is acquired based on the regional adaptive brightness coding fringe image.
[0006] In some embodiments, obtaining the optimal projection grayscale value of the projector pixel corresponding to the pixel saturation area in the first uniform brightness modulated image includes: Acquire pixel saturation region information of the first uniform brightness modulated image; the pixel saturation region information includes position information of each pixel saturation region of the first uniform brightness modulated image and pixel information of edges of each pixel saturation region; Based on the pixel saturation area information and using the trained optimal projection brightness prediction model, the optimal projection grayscale value of the projector pixel corresponding to the pixel saturation area in the first uniform brightness modulated image is obtained.
[0007] In some embodiments, the obtaining of the pixel saturation region information of the first uniform brightness modulated image includes: extracting the positions of the saturated pixels in the first uniform brightness modulated image by binarizing the first uniform brightness modulated image to obtain a first saturation region image; Filling the holes inside the first saturated area image to obtain a second saturated area image; The second saturation region image is subjected to median filtering, region dilation and region numbering to obtain position information of saturation regions of each pixel of the first uniform brightness modulated image.
[0008] In some embodiments, the acquiring of pixel saturation area information of the first uniform brightness modulated image further includes: Extracting clockwise single-pixel edges of each pixel saturation region in the first uniform brightness modulated image; Performing chain code encoding on the clockwise single pixel edge; Invalid chain code encoding points on saturated pixel points in the first saturated area image are eliminated to obtain pixel information of edges of each pixel saturated area of the first uniform brightness modulated image.
[0009] In some embodiments, the optimal projection brightness prediction model is trained by the following steps: Acquire a second uniform brightness modulated image, and calculate the optimal projection brightness true value corresponding to each pixel in the second uniform brightness modulated image; Inputting the pixel brightness of the second uniform brightness modulated image and the projection brightness combination vector into the optimal projection brightness prediction model to obtain the optimal projection intensity; the optimal projection brightness prediction model is a fully connected layer neural network structure; A loss function is calculated based on the optimal projection intensity and the optimal projection brightness true value, and the optimal projection brightness prediction model is optimized based on the loss function.
[0010] In some embodiments, the step of acquiring the homography between each pixel saturation area in the first uniform brightness modulated image and a corresponding area on the projection plane of the projector based on the first coded fringe modulated image includes: Based on the three-dimensional spatial coordinates of the edge pixels of the pixel saturation area of the first coded fringe modulation image and the first uniform brightness modulated image, obtaining the correspondence between the pixel position of each pixel saturation area in the first uniform brightness modulated image and the pixel position of the projection plane of the projector; A homography matrix is obtained using a robust estimation algorithm and based on the correspondence between the pixel positions of each pixel-saturated area in the first uniform brightness modulated image and the pixel positions of the projector projection plane; the homography matrix is used to characterize the mapping relationship from the pixel-saturated area in the first uniform brightness modulated image to the projector projection plane.
[0011] In some embodiments, the step of acquiring a three-dimensional point cloud of an object surface based on the region-adaptive brightness coded fringe image comprises: Acquire a second coded fringe modulated image modulated by the surface of the object by projecting the adaptive brightness coded fringe image; The second coded fringe modulated image is decoded and triangulated to obtain a complete three-dimensional point cloud of the object surface.
[0012] In a second aspect, the present invention provides a structured light three-dimensional measurement device based on local adaptive stripe brightness, comprising: a first acquisition module, used to acquire an image modulated by the surface of an object; the image modulated by the surface of the object comprises a first uniform brightness modulated image and a first coded stripe modulated image taken by a camera; A second acquisition module is used to obtain the optimal projection grayscale value of the projector pixel corresponding to the pixel saturation area in the first uniform brightness modulated image, and obtain the homography relationship between each pixel saturation area in the first uniform brightness modulated image and the corresponding area of the projector projection plane based on the first coded fringe modulated image; A third acquisition module is used to map the optimal projection grayscale value to a corresponding position on a projection plane of a projector based on the homography relationship to obtain a regional adaptive brightness coded fringe image; The fourth acquisition module is used to acquire a three-dimensional point cloud of the object surface based on the regional adaptive brightness coding fringe image.
[0013] In a third aspect, the present invention provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the structured light three-dimensional measurement method based on local adaptive stripe brightness as described in the first aspect above is implemented.
[0014] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the structured light three-dimensional measurement method based on local adaptive stripe brightness as described in the first aspect above.
[0015] In a fifth aspect, the present invention further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the structured light three-dimensional measurement method based on local adaptive stripe brightness as described in the first aspect above.
[0016] The structured light three-dimensional measurement method and device based on local adaptive stripe brightness provided by the present invention obtain the best projection grayscale value of the projector pixel corresponding to the pixel saturation area in the first uniform brightness modulated image after obtaining the first uniform brightness modulated image and the first coded stripe modulated image taken by the camera, and obtain the homography relationship between each pixel saturated area in the first uniform brightness modulated image and the corresponding area of the projector projection plane based on the first coded stripe modulated image, and then map the obtained best projection grayscale value to the corresponding position of the projector based on the homography relationship to obtain a regional adaptive brightness coded stripe image, and finally obtain the three-dimensional point cloud of the object surface based on the regional adaptive brightness coded stripe image, which effectively solves the problem of large-area point cloud loss that is easy to occur when the structured light system measures the surface of a highly reflective object to be measured, and improves the accuracy of three-dimensional reconstruction of the surface of an object with high reflectivity or a large reflectivity variation range. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0018] Figure 1This is one of the flow charts of the structured light three-dimensional measurement method based on local adaptive stripe brightness provided by the present invention.
[0019] Figure 2 This is the second flow chart of the structured light three-dimensional measurement method based on local adaptive stripe brightness provided by the present invention.
[0020] Figure 3 It is a structural schematic diagram of the optimal projection brightness prediction model provided by the present invention.
[0021] Figure 4 It is a physical modeling and system schematic diagram for solving the optimal projection grayscale true value provided by the present invention.
[0022] Figure 5 It is a schematic diagram of the structure of a structured light three-dimensional measurement device based on local adaptive stripe brightness provided by the present invention.
[0023] Figure 6 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION
[0024] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0025] Figure 1 FIG. 1 is one of the flow charts of the structured light three-dimensional measurement method based on local adaptive stripe brightness provided by the present invention. Figure 1 As shown, the present invention provides a structured light three-dimensional measurement method based on local adaptive stripe brightness. The method includes the following steps 101 and 102.
[0026] Step 101, obtaining an image modulated by the surface of an object; the image modulated by the surface of the object includes a first uniform brightness modulated image and a first coded fringe modulated image captured by a camera.
[0027] Specifically, a surface structured light sensor device is used for image generation and acquisition, and the surface structured light sensor device includes a digital light processing (DLP) projector and an industrial camera. The DLP projector is used to project a coded stripe image, and the industrial camera is used to acquire a modulated coded stripe image.
[0028] Optionally, the surface structured light sensor device is connected to a computer via a High Definition Multimedia Interface (HDMI) and a Gigabit Ethernet (GigE) interface. When building the system, the camera and the projector are placed side by side, and the angle between the two is adjusted so that the projection format is exactly located at the center of the camera imaging and occupies most of the camera's field of view, and they are calibrated to obtain the intrinsic and extrinsic parameters of the two.
[0029] For example, when the system starts working, the computer transmits the pre-generated coded images to the projector via HDMI in a certain sequence. Each time the projector projects an image, it triggers the camera to start collecting the surface information of the current object being measured. Finally, the camera transmits the collected image sequence back to the computer via the GigE interface, completing a closed-loop process of obtaining information about the object being measured. On the computer side, after image processing, coded image decoding, 3D reconstruction and other algorithm calculations, a dense 3D reconstructed point cloud of the surface of the object being measured can be obtained to complete the measurement.
[0030] Optionally, the coded fringe modulated image may be a Gray code pattern, a Gray code combined with a line shift code pattern, a phase shift code pattern, or the like.
[0031] Step 102: obtain the optimal projection grayscale value of the projector pixel corresponding to the pixel saturation area in the first uniform brightness modulated image, and obtain the homography relationship between each pixel saturation area in the first uniform brightness modulated image and the corresponding area of the projector projection plane based on the first coded stripe modulated image.
[0032] Specifically, after obtaining the first uniform brightness modulated image, the optimal projection grayscale value of the projector pixel corresponding to the pixel saturation area in the first uniform brightness modulated image is obtained.
[0033] For example, Figure 2 FIG. 2 is a flow chart of the structured light three-dimensional measurement method based on local adaptive stripe brightness provided by the present invention. Figure 2 As shown, first, a pixel saturation area in a first uniform brightness modulated image is extracted, and then a pre-trained neural network model is used to obtain the optimal projection grayscale value of the projector pixel corresponding to each pixel in the pixel saturated area, wherein the neural network model is trained based on a sample uniform brightness modulated image and the optimal projection brightness true value corresponding to the pixel in the sample uniform brightness modulated image.
[0034] After obtaining the first uniform brightness modulated image and the first coded fringe modulated image, the homography relationship between each pixel saturated area in the first uniform brightness modulated image and the corresponding area of the projector projection plane is obtained based on the first coded fringe modulated image.
[0035] For example, the first coded fringe modulated image, which is a single-direction coded fringe image, is used to solve the pixel point (x c ,y c ) and the pixel point (x p ,y p ) and use the robust estimation algorithm to obtain accurate and robust homography relationship.
[0036] Step 103: Map the optimal projection grayscale value to a corresponding position on a projection plane of a projector based on the homography relationship to obtain a regional adaptive brightness coded fringe image.
[0037] Specifically, based on the homography relationship between each pixel saturation area in the obtained first uniform brightness modulated image and the corresponding area of the projector projection plane, the obtained optimal projection grayscale value is mapped to the corresponding position of the projector plane to obtain a regional adaptive brightness coded stripe image.
[0038] For example, the best projection intensity prediction of the first uniform brightness modulation image is performed using the trained best projection brightness neural network model to obtain the best projection grayscale value. Combined with the homography between the saturated areas of each camera image and the corresponding areas of the DLP projector projection plane, the projection intensity of the projector plane pixel position corresponding to the saturated area of the first uniform brightness modulated image is set to The pixel values of the projector projection plane corresponding to the remaining unsaturated areas are the original pixel values of the stripe coded image, so as to generate an adaptive brightness coded stripe image.
[0039] Step 104: Acquire a three-dimensional point cloud of the object surface based on the regional adaptive brightness coded fringe image.
[0040] Specifically, after obtaining the regional adaptive brightness coded stripe image, the image is projected, decoded and triangulated to obtain a three-dimensional point cloud of the object surface.
[0041] In some embodiments, Figure 2 As shown, obtaining a three-dimensional point cloud of the object surface based on the regional adaptive brightness coding stripe image includes: Acquire a second coded fringe modulated image modulated by the surface of the object by projecting the adaptive brightness coded fringe image; The second coded fringe modulated image is decoded and triangulated to obtain a complete three-dimensional point cloud of the object surface.
[0042] In an embodiment of the present application, the object to be three-dimensionally reconstructed may be a highly reflective object, such as a metal object, etc. The embodiment of the present application is particularly capable of dealing with objects with highly reflective surfaces and obtaining a complete three-dimensional point cloud of their surface.
[0043] For example, a metal workpiece with a length and width of 70×130 mm is selected. This workpiece contains many typical surface features, including planes, concave and convex surfaces, curved surfaces, spherical surfaces and milling patterns. These features will show different and very complex surface reflection conditions, which brings difficulties to measurement; the reconstruction results of the local adaptive projection brightness algorithm proposed in the embodiment of the present application have complete and excellent reconstruction performance on all types of surfaces of the workpiece, and the point cloud reconstruction density is high.
[0044] The structured light three-dimensional measurement method based on local adaptive stripe brightness provided in the embodiment of the present application obtains the best projection grayscale value of the projector pixel corresponding to the pixel saturation area in the first uniform brightness modulated image after obtaining the first uniform brightness modulated image and the first coded stripe modulated image taken by the camera, and obtains the homography relationship between each pixel saturated area in the first uniform brightness modulated image and the corresponding area of the projector projection plane based on the first coded stripe modulated image, and then maps the obtained best projection grayscale value to the corresponding position of the projector based on the homography relationship to obtain a regional adaptive brightness coded stripe image, and finally obtains the three-dimensional point cloud of the object surface based on the regional adaptive brightness coded stripe image, which effectively solves the problem of large-area point cloud loss that is easy to occur when the structured light system measures the surface of a highly reflective object to be measured, and improves the accuracy of three-dimensional reconstruction of the surface of an object with high reflectivity or a large reflectivity variation range.
[0045] In some embodiments, obtaining the optimal projection grayscale value of the projector pixel corresponding to the pixel saturation area in the first uniform brightness modulated image includes: Obtain pixel saturation area information of the first uniform brightness modulated image; the pixel saturation area information includes position information of each pixel saturation area of the first uniform brightness modulated image and pixel information of the edge of each pixel saturation area; based on the pixel saturation area information and using the trained optimal projection brightness prediction model, obtain the optimal projection grayscale value of the projector pixel corresponding to the pixel saturation area in the first uniform brightness modulated image.
[0046] Specifically, Figure 2As shown, to obtain the best projection grayscale value of the projector pixel corresponding to the pixel saturation area in the first uniform brightness modulated image, it is first necessary to obtain the pixel saturation area information of the first uniform brightness modulated image, and the pixel saturation area information includes the position information of each pixel saturation area of the first uniform brightness modulated image and the pixel information of the edge of each pixel saturation area. Then, based on the pixel saturation area information, the trained best projection brightness prediction model is used to obtain the best projection grayscale value of the projector pixel corresponding to the pixel saturation area in the first uniform brightness modulated image.
[0047] In some embodiments, the obtaining of the pixel saturation region information of the first uniform brightness modulated image includes: extracting the positions of the saturated pixels in the first uniform brightness modulated image by binarizing the first uniform brightness modulated image to obtain a first saturation region image; Filling the holes inside the first saturated area image to obtain a second saturated area image; The second saturation region image is subjected to median filtering, region dilation and region numbering to obtain position information of saturation regions of each pixel of the first uniform brightness modulated image.
[0048] Specifically, the steps for obtaining the position information of the saturated areas of each pixel of the first uniform brightness modulated image are as follows: Step 1, threshold binarization processing, extracting the positions of the saturated pixels (pixel value is 255) of the first uniform brightness image sequence, and obtaining the first saturated area image.
[0049] Step 2: Using the connectivity principle, fill the internal holes of the saturated region of the first saturated region image to make the saturated region complete and continuous, thereby obtaining a second saturated region image.
[0050] Step 3: perform an erosion operation and a median filter on the second saturated area image to remove small areas and isolated saturated point positions.
[0051] Step 4: dilate the filtered second saturated region image multiple times to expand the saturated region area to the level before corrosion, and fill the holes inside the dilated saturated region at the same time.
[0052] Step 5: Find the 8-neighborhood saturated pixel connected regions and number each region to complete the robust saturated region extraction and obtain the final pixel saturated region position information.
[0053] In some embodiments, the acquiring of pixel saturation area information of the first uniform brightness modulated image further includes: Extracting clockwise single-pixel edges of each pixel saturation region in the first uniform brightness modulated image; Performing chain code encoding on the clockwise single pixel edge; Invalid chain code encoding points on saturated pixel points in the first saturated area image are eliminated to obtain pixel information of edges of each pixel saturated area of the first uniform brightness modulated image.
[0054] Specifically, the steps for acquiring pixel information of the edges of each pixel saturation region of the first uniform brightness modulated image are as follows: Step 1: extract the clockwise single-pixel edges of each complete saturation region by using image morphological processing, and add number labels.
[0055] Step 2: Traverse each area and perform chain code encoding (8-direction chain code) for edge pixels. The encoding strategy is: track the edge in clockwise order. If two points on the edge are on the same line, the chain code values of the two points are the same; otherwise, the code value is increased by 1.
[0056] Step 3: Using the obtained first saturation area image, remove the invalid edge information on the saturation point (ie, invalid chain code encoding point), and complete the extraction of the edge pixel information of the pixel saturation area of the first uniform brightness modulated image.
[0057] The structured light three-dimensional measurement method based on local adaptive stripe brightness provided in the embodiment of the present application performs a series of preprocessing on the image to more effectively and accurately extract the local saturated area and its edge information in the image, thereby improving the accuracy of subsequent model prediction and enhancing the accuracy of three-dimensional reconstruction of the object.
[0058] In some embodiments, the optimal projection brightness prediction model is trained by the following steps: Acquire a second uniform brightness modulated image, and calculate the optimal projection brightness true value corresponding to each pixel in the second uniform brightness modulated image; Inputting the pixel brightness of the second uniform brightness modulated image and the projection brightness combination vector into the optimal projection brightness prediction model to obtain the optimal projection intensity; the optimal projection brightness prediction model is a fully connected layer neural network structure; A loss function is calculated based on the optimal projection intensity and the optimal projection brightness true value, and the optimal projection brightness prediction model is optimized based on the loss function.
[0059] Specifically, before the optimal projection brightness prediction model performs online calculation on the collected first uniform brightness modulated image, it is necessary to train the optimal projection brightness prediction model. First, obtain the second uniform brightness modulated image for model training, and calculate the optimal projection brightness true value corresponding to each pixel in the second uniform brightness modulated image, and use the second uniform brightness modulated image and its corresponding optimal projection brightness true value as sample data. Then input the pixel brightness and projection brightness combination vector of the second uniform brightness modulated image into the optimal projection brightness prediction model to obtain the optimal projection intensity, and calculate the loss function based on the optimal projection intensity and the optimal projection brightness true value, and use the loss function to optimize the optimal projection brightness prediction model.
[0060] Optionally, the structure of the optimal projection brightness prediction model is a fully connected layer neural network structure, including a multi-layer structure, each layer of the structure contains multiple neurons, and each neuron includes a nonlinear activation function. Figure 3 Schematic diagram of the structure of the optimal projection brightness prediction model provided by the present invention. Figure 3 As shown in the figure, the neural network model contains: (from left to right) an input layer, three hidden layers and an output layer, where each hidden layer contains 14 neurons. The input vector is input into the input layer, and the data passes through each hidden layer in turn, and the best projected grayscale value can be obtained in the output layer.
[0061] Optionally, the nonlinear activation function contained in the neurons in the optimal projection brightness prediction model may be ReLu, tanh or sigmoid, etc.
[0062] Optionally, before starting training, it is necessary to determine the uniform brightness grayscale value sequence of the projection; since the projection brightness of ordinary DLP projectors is affected by the Gamma effect, the linearity of the projection grayscale value between 40 and 220 is better, for example, the grayscale value can be projected There are seven uniform brightness light pattern sequences of 40, 70, 100, 130, 160, 190, and 220, and the pixel value I corresponding to each pixel position of the camera under these 7 brightnesses is obtained. i For model training and testing; at this time, the projection brightness combination vector input by the optimal projection brightness neural network model is Therefore, the input layer contains 14 neurons at this time.
[0063] Optionally, before starting training, the neuron parameters included in the neural network need to be initialized. For example, the initialization method may be: all-zero or equal value initialization, normal initialization, random uniform initialization, etc.
[0064] Specifically, the optimal projection brightness true value corresponding to each pixel in the second uniform brightness modulated image is calculated.
[0065] For example, Figure 4 The physical modeling and system diagram for solving the optimal projection grayscale true value provided by the present invention are as follows: Figure 4 As shown, it is assumed that the intensity of the projector light received at a certain surface measurement point is I p The ambient light intensity is β1, and the light intensity from other surfaces of the object being measured that is reflected to this point is I i , they are all reflected from a point on the surface of the object and imaged onto a certain pixel in the camera. The reflectivity of the surface of the object can be set to α(x c ,y c ), where (x c ,y c ) is the camera pixel coordinate. The value of this pixel point is also affected by the ambient light intensity β2 directly received by this pixel position and the sensor noise I n After the above analysis, the total light intensity received by a pixel point on the final imaging plane of the camera can be expressed as: I(x c ,y c )=α(x c ,y c )[β1(x c ,y c )+I p (x c ,y c )+I i (x c ,y c )]+β2(x c ,y c )+I n (x c ,y c )
[0066] More generally, based on the above formula, considering the camera parameters that play a major role in the imaging results during the acquisition process, the imaging results can be expressed as: I=k·t·α(β1+I p +I i )+k·t·β2+I n
[0067] where k is the camera gain and t is the exposure time. Note that, strictly speaking, α is the reflectivity corresponding to the Lambertian component of surface reflection, which depends on the surface normal according to the Lambert cosine theorem; at the same time, the sensor noise I n Generally obeys Gaussian distribution, that is, I n ~N(0,σ 2). The expression of the imaging result is rewritten as: I=ktα·I p +kt·[α(β1+I i )+β2]+I n
[0068] Let x1 = kt·I p , x2=kt,β=α(β1+I i )+β2, the above formula can be written as: I=αx1+βx2+I n
[0069] Assume that by projecting the above 7 uniform light pattern sequences with different brightness, 7 sets of data values (I i , x i1 , xi2 ), i = 1, ..., 7, x i1 and x i2 As an independent variable, I i Assuming is the dependent variable, we can estimate the values of parameters α and β. To minimize Q, we can set Rearranging the equation yields:
[0070] make The above formula can be expressed as: T XB=X T I, if X T If X is invertible, then the estimated value of parameter B for:
[0071] The estimated value of the imaging result can be written as:
[0072] The estimated grayscale value of the pixel is determined by the camera parameters k and t, the projector brightness I p , and the estimated value of parameter B If the pixel value is fixed To an ideal unsaturated value I exp , while fixing the camera parameters k and t to k exp ,t exp , then the projection intensity I p It is expressed as:
[0073] Therefore, after selecting the desired pixel intensity I exp After, for example, I expIt can be taken as 248, which can be estimated by the reflectivity α Estimated values of other irrelevant light intensity factors And camera parameters to calculate the optimal projection intensity of the projector corresponding to this camera pixel This calculation result is used as the true value of the optimal projection brightness.
[0074] Specifically, the pixel brightness of the second uniform brightness modulated image and the projection brightness combination vector are input into the optimal projection brightness prediction model to obtain the optimal projection intensity.
[0075] For example, input the second uniform brightness modulated image pixel brightness and projection brightness combination vector To the optimal projection brightness prediction model, the output is the calculated optimal projection intensity
[0076] Optionally, the loss function used in training may be: mean square error loss function, mean absolute error loss function or Huber loss function, etc. The optimizer used in training may be: stochastic gradient descent optimizer, root mean square support optimizer or adaptive moment estimation optimizer, etc.
[0077] The structured light three-dimensional measurement method based on local adaptive stripe brightness provided in the embodiment of the present application predicts the optimal projection grayscale value of the DLP projector pixel at the corresponding position of the image saturation area through a pre-trained neural network model, thereby improving the accuracy of the prediction of the optimal projection brightness, thereby ensuring the accuracy of the three-dimensional reconstruction of highly reflective objects.
[0078] In some embodiments, the step of acquiring the homography between each pixel saturation area in the first uniform brightness modulated image and a corresponding area on the projection plane of the projector based on the first coded fringe modulated image includes: Based on the three-dimensional spatial coordinates of the edge pixels of the pixel saturation area of the first coded fringe modulation image and the first uniform brightness modulated image, obtaining the correspondence between the pixel position of each pixel saturation area in the first uniform brightness modulated image and the pixel position of the projection plane of the projector; A homography matrix is obtained using a robust estimation algorithm and based on the correspondence between the pixel positions of each pixel-saturated area in the first uniform brightness modulated image and the pixel positions of the projector projection plane; the homography matrix is used to characterize the mapping relationship from the pixel-saturated area in the first uniform brightness modulated image to the projector projection plane.
[0079] Specifically, only a single-direction coded stripe image (i.e., the first coded stripe modulation image) is used, and the three-dimensional coordinates of the edge pixels of the saturated area of the first uniform brightness modulation image are used as calculation intermediate values during the calculation process to obtain a one-to-one correspondence between the edge pixel positions of the saturated area of the camera image and the pixel positions of the projection plane of the DLP projector.
[0080] For example, if the projector is regarded as an inverse camera model, for the pinhole imaging model, a point in space (X w , Y w , Z w ) at a point (x) in the camera c ,y c ) imaging satisfies the following formula (recorded as the first formula): s c (x c ,y c , 1) T =K c [R c |t c ](X w , Y w , Z w ) T
[0081] Among them, K c is the camera internal parameter, [R c |t c ] is the camera external parameter, s c is the zoom factor corresponding to the camera. w , Y w , Z w ) at a point (x) in the projector p ,y p ) imaging satisfies the following formula (recorded as the second formula): s p (x p ,y p , 1) T =K p [R p |t p ](X w , Y w , Z w ) T
[0082] Among them, K p is the internal parameter of the projector, [R p |t p ] is the external parameter of the projector, s p is the scaling factor corresponding to the projector. If the vertical stripes (x p The direction encoding value is known, that is, x pKnown), then for a point (x c ,y c ) can be obtained by decoding the code value and x p Correspondingly, the goal is to simultaneously find the y corresponding to this pixel point in the projector plane p , to form a point-to-point relationship; assuming M c 、M p for:
[0083] (X w , Y w , Z w ) as the intermediate value of the calculation process, combined with M c 、M p The expression of can be solved by the first and second equations above:
[0084] At this time, for the second formula, s p (x p ,y p , 1) T =K p [R p |t p ](X w , Y w , Z w ) T Currently, only s p ,y p Unknown, but we can directly construct two equations to solve these two unknowns separately:
[0085] That is, using (X w , Y w , Z w ) is an intermediate value, and the reverse solution successfully obtains (x c ,y c ) and (x p ,y p )’s correspondence.
[0086] Specifically, after obtaining the correspondence between the pixel positions of each pixel saturated area in the first uniform brightness modulated image and the pixel positions of the projector projection plane, the accurate homography relationship between each camera pixel saturated area and the corresponding area of the DLP projector projection plane is calculated using a robust estimation method.
[0087] For example, assuming that the homography relationship is represented by H, the principle of using the robust estimation algorithm to obtain a robust homography relationship is: after the coordinate correspondence relationship of the saturated region is established, the homography is calculated for each region. The homography relationship is represented by: (x p ,y p , 1) T =H·(x c ,y c , 1) T
[0088] The correct calculation of the local homography matrix H requires at least four sets of corresponding points (any three points are not collinear), so the sampling method without replacement is used to randomly select two points from the points with the same chain code value (i.e., the same straight line) as candidate points to participate in the calculation of the matrix H, avoiding the problem of singular solution caused by the selected points being collinear. At the same time, robust estimation algorithms such as RANSAC (Random Sample Consensus) or MSAC (M-estimator Sample Consensus) are used for all candidate points in each region to find the correct data (Inliers) for the calculation of the homography matrix, so as to obtain an accurate and robust local mapping relationship from the saturated area of each first uniform brightness modulated image to the projector plane.
[0089] The structured light three-dimensional measurement method based on local adaptive stripe brightness provided in the embodiment of the present application only uses a single-direction coded stripe image to solve the correspondence between the saturated area of the first uniform brightness modulated image and the plane pixels, and uses a robust estimation algorithm to obtain an accurate and robust homography relationship, thereby achieving efficient acquisition of the homography relationship and enhancing the robustness of the method application, thereby improving the efficiency and accuracy of three-dimensional reconstruction of objects.
[0090] The embodiments of the present application are particularly capable of handling objects with strong surface reflection or a large range of reflectivity variation, such as metal objects, and obtaining a complete three-dimensional point cloud of their surface.
[0091] For example, a metal workpiece with a length and width of 70×130 mm is selected. This workpiece contains many typical surface features, including planes, concave and convex surfaces, curved surfaces, spherical surfaces and milling patterns. These features will show different and very complex surface reflection conditions, which brings difficulties to measurement; the reconstruction results of the local adaptive projection brightness algorithm proposed in the embodiment of the present application have complete and excellent reconstruction performance on all types of surfaces of the workpiece, and the point cloud reconstruction density is high.
[0092] The structured light 3D measurement method based on local adaptive fringe brightness provided in the embodiment of the present application proposes a complete set of solutions from hardware to local adaptive structured light algorithm, which effectively solves the problem of large-area point cloud loss that is easy to occur when the structured light system measures the surface of a highly reflective object to be measured, and improves the accuracy of 3D reconstruction of the surface of an object with high reflectivity or a large reflectivity variation range; in addition, the present invention also has the following advantages: The hardware system adopted by the present invention is composed of only a projector and a camera without any additional hardware equipment, thus saving hardware expenses.
[0093] The present invention is highly efficient: it uses a neural network to accelerate the calculation process, adopts the most basic image processing algorithm to extract local saturation areas and their edge information, and cleverly constructs an algorithm that only needs to project a directional stripe to calculate the homography corresponding to each area.
[0094] The present invention is also robust and universal: the robustness is reflected in the use of a robust estimation method to obtain the homography relationship, and the result is not prone to errors; the universality is reflected in the fact that the algorithm can be integrated into most structured light grating fringe projection three-dimensional reconstruction technologies to solve the problems caused by the surface of highly reflective objects to the measurement.
[0095] Figure 5 is a schematic diagram of the structure of a structured light three-dimensional measurement device based on local adaptive fringe brightness provided by the present invention, such as Figure 5 As shown, the present invention provides a structured light three-dimensional measurement device based on local adaptive stripe brightness, including a first acquisition module 501 , a second acquisition module 502 , a third acquisition module 503 and a fourth acquisition module 504 .
[0096] The first acquisition module 501 is used to acquire an image modulated by the surface of an object; the image modulated by the surface of an object includes a first uniform brightness modulated image and a first coded fringe modulated image captured by a camera; The second acquisition module 502 is used to obtain the optimal projection grayscale value of the projector pixel corresponding to the pixel saturation area in the first uniform brightness modulated image, and obtain the homography relationship between each pixel saturation area in the first uniform brightness modulated image and the corresponding area of the projector projection plane based on the first coded fringe modulated image; The third acquisition module 503 is used to map the optimal projection grayscale value to the corresponding position of the projection plane of the projector based on the homography relationship to obtain a regional adaptive brightness coded fringe image; The fourth acquisition module 504 is used to acquire a three-dimensional point cloud of the object surface based on the regional adaptive brightness coded fringe image.
[0097] In some embodiments, the second acquisition module includes: A saturation region information acquisition unit, configured to acquire pixel saturation region information of the first uniform brightness modulated image; the pixel saturation region information includes position information of each pixel saturation region of the first uniform brightness modulated image and pixel information of edges of each pixel saturation region; The optimal projection grayscale value acquisition unit is used to acquire the optimal projection grayscale value of the projector pixel corresponding to the pixel saturation area in the first uniform brightness modulated image based on the pixel saturation area information and using the trained optimal projection brightness prediction model.
[0098] In some embodiments, the saturation region information acquisition unit includes: A binarization processing subunit is used to extract the positions of saturated pixels in the first uniform brightness modulated image by performing binarization processing on the first uniform brightness modulated image to obtain a first saturated area image; a hole filling subunit, configured to fill the holes inside the first saturated region image to obtain a second saturated region image; The saturation region position information acquisition subunit is used to perform median filtering, region dilation and region numbering on the second saturation region image to obtain the saturation region position information of each pixel of the first uniform brightness modulated image.
[0099] In some embodiments, the saturation region information acquisition unit further includes: An extraction subunit, used to extract the clockwise single-pixel edge of each pixel saturation area in the first uniform brightness modulated image; a chain code encoding subunit, used to perform chain code encoding on the clockwise single-pixel edge; The saturation area edge pixel information acquisition subunit is used to remove invalid chain code encoding points on the saturated pixel points in the first saturation area image to obtain pixel information of the edges of the saturation areas of each pixel in the first uniform brightness modulated image.
[0100] In some embodiments, the optimal projection brightness prediction model is trained by the following steps: Acquire a second uniform brightness modulated image, and calculate the optimal projection brightness true value corresponding to each pixel in the second uniform brightness modulated image; Inputting the pixel brightness of the second uniform brightness modulated image and the projection brightness combination vector into the optimal projection brightness prediction model to obtain the optimal projection intensity; the optimal projection brightness prediction model is a fully connected layer neural network structure; A loss function is calculated based on the optimal projection intensity and the optimal projection brightness true value, and the optimal projection brightness prediction model is optimized based on the loss function.
[0101] In some embodiments, the second acquisition module includes: a pixel position correspondence relationship acquisition unit, configured to acquire a correspondence relationship between a pixel position of each pixel saturation region in the first uniform brightness modulated image and a pixel position of a projection plane of a projector based on the three-dimensional spatial coordinates of the edge pixels of the pixel saturation region of the first coded fringe modulation image and the first uniform brightness modulated image; A homography matrix acquisition unit is used to obtain a homography matrix using a robust estimation algorithm and based on the correspondence between the pixel positions of each pixel saturated area in the first uniform brightness modulated image and the pixel positions of the projector projection plane; the homography matrix is used to characterize the mapping relationship from the pixel saturated area in the first uniform brightness modulated image to the projector projection plane.
[0102] In some embodiments, the fourth acquisition module includes: A projection unit, used for projecting the adaptive brightness coded fringe image and acquiring a second coded fringe modulated image modulated by the surface of the object; The decoding and triangulation unit is used to decode and triangulate the second coded fringe modulated image to obtain a complete three-dimensional point cloud of the object surface.
[0103] Specifically, the above-mentioned structured light three-dimensional measurement device based on local adaptive stripe brightness provided by the present invention can implement all the method steps implemented by the above-mentioned structured light three-dimensional measurement method embodiment based on local adaptive stripe brightness, and can achieve the same technical effect. The parts and beneficial effects of this embodiment that are the same as the method embodiment will not be described in detail here.
[0104] It should be noted that the division of units / modules in the above-mentioned embodiments of the present invention is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, the functional units in the various embodiments of the present application may be integrated into a processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware or in the form of software functional units.
[0105] Figure 6 is a schematic diagram of the structure of the electronic device provided by the present invention, such as Figure 6 As shown, the electronic device may include: a processor 601, a communication interface 602, a memory 603 and a communication bus 604, wherein the processor 601, the communication interface 602 and the memory 603 communicate with each other through the communication bus 604. The processor 601 may call the logic instructions in the memory 603 to execute the structured light three-dimensional measurement method based on local adaptive stripe brightness, the method comprising: Acquire an image modulated by the surface of the object; the image modulated by the surface of the object comprises a first uniform brightness modulated image and a first coded fringe modulated image captured by a camera; Obtaining the optimal projection grayscale value of the projector pixel corresponding to the pixel saturation area in the first uniform brightness modulated image, and obtaining the homography relationship between each pixel saturation area in the first uniform brightness modulated image and the corresponding area of the projector projection plane based on the first coded fringe modulated image; Based on the homography relationship, the optimal projection grayscale value is mapped to a corresponding position on the projection plane of the projector to obtain a regional adaptive brightness coded fringe image; A three-dimensional point cloud of the object surface is acquired based on the regional adaptive brightness coding fringe image.
[0106] Specifically, the processor 601 can be a central processing unit (CPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or a complex programmable logic device (CPLD), and the processor can also adopt a multi-core architecture.
[0107] The logic instructions in the memory 603 can be implemented in the form of software functional units and can be stored in a processor-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program codes.
[0108] In some embodiments, a computer program product is further provided, the computer program product comprising a computer program, the computer program may be stored on a non-transitory computer-readable storage medium, and when the computer program is executed by a processor, the computer can execute the structured light 3D measurement method based on local adaptive stripe brightness provided by the above-mentioned method embodiments, the method comprising: Acquire an image modulated by the surface of the object; the image modulated by the surface of the object comprises a first uniform brightness modulated image and a first coded fringe modulated image captured by a camera; Obtaining the optimal projection grayscale value of the projector pixel corresponding to the pixel saturation area in the first uniform brightness modulated image, and obtaining the homography relationship between each pixel saturation area in the first uniform brightness modulated image and the corresponding area of the projector projection plane based on the first coded fringe modulated image; Based on the homography relationship, the optimal projection grayscale value is mapped to a corresponding position on the projection plane of the projector to obtain a regional adaptive brightness coded fringe image; A three-dimensional point cloud of the object surface is acquired based on the regional adaptive brightness coding fringe image.
[0109] Specifically, the above-mentioned computer program product provided in the embodiment of the present application can implement all the method steps implemented by the above-mentioned method embodiments, and can achieve the same technical effect. The parts and beneficial effects of this embodiment that are the same as the method embodiment will not be described in detail here.
[0110] In some embodiments, a computer-readable storage medium is further provided, wherein the computer-readable storage medium stores a computer program, wherein the computer program is used to enable a computer to execute the structured light three-dimensional measurement method based on local adaptive fringe brightness provided by the above-mentioned method embodiments, the method comprising: Acquire an image modulated by the surface of the object; the image modulated by the surface of the object comprises a first uniform brightness modulated image and a first coded fringe modulated image captured by a camera; Obtaining the optimal projection grayscale value of the projector pixel corresponding to the pixel saturation area in the first uniform brightness modulated image, and obtaining the homography relationship between each pixel saturation area in the first uniform brightness modulated image and the corresponding area of the projector projection plane based on the first coded fringe modulated image; Based on the homography relationship, the optimal projection grayscale value is mapped to a corresponding position on the projection plane of the projector to obtain a regional adaptive brightness coded fringe image; A three-dimensional point cloud of the object surface is acquired based on the regional adaptive brightness coding fringe image.
[0111] Specifically, the above-mentioned computer-readable storage medium provided by the present invention can implement all the method steps implemented by the above-mentioned method embodiments, and can achieve the same technical effects. The parts and beneficial effects that are the same as the method embodiments in this embodiment will not be described in detail here.
[0112] It should be noted that the computer-readable storage medium can be any available medium or data storage device that can be accessed by the processor, including but not limited to magnetic storage (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO), etc.), optical storage (such as CD, DVD, BD, HVD, etc.), and semiconductor storage (such as ROM, EPROM, EEPROM, non-volatile memory (NAND FLASH), solid-state drive (SSD)), etc.
[0113] It should also be noted that the terms "first", "second", etc. in the present invention are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present invention can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first" and "second" are usually of the same type, and the number of objects is not limited. For example, the first object can be one or more.
[0114] In the present invention, the term "plurality" refers to two or more than two, and other quantifiers are similar to them.
[0115] In the present invention, "determine B based on A" means that the factor A should be considered when determining B. It is not limited to "B can be determined based on A alone", but should also include: "determine B based on A and C", "determine B based on A, C and E", "determine C based on A, and further determine B based on C", etc. In addition, it can also include taking A as a condition for determining B, for example, "when A meets the first condition, use the first method to determine B"; for another example, "when A meets the second condition, determine B", etc.; for another example, "when A meets the third condition, determine B based on the first parameter", etc. Of course, it can also be a condition that takes A as a factor for determining B, for example, "when A meets the first condition, use the first method to determine C, and further determine B based on C", etc.
[0116] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) that contain computer-usable program code.
[0117] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer executable instructions. These computer executable instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0118] These processor executable instructions may also be stored in a processor readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the processor readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0119] These processor-executable instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0120] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.
Claims
1. A structured light three-dimensional measurement method based on local adaptive fringe brightness, characterized in that: include: Acquire an image modulated by the surface of the object; The image modulated by the surface of the object includes a first uniform brightness modulated image and a first coded fringe modulated image captured by a camera; Obtaining the optimal projection grayscale value of the projector pixel corresponding to the pixel saturation area in the first uniform brightness modulated image, and obtaining the homography relationship between each pixel saturation area in the first uniform brightness modulated image and the corresponding area of the projector projection plane based on the first coded fringe modulated image; Based on the homography relationship, the optimal projection grayscale value is mapped to a corresponding position on the projection plane of the projector to obtain a regional adaptive brightness coded fringe image; A three-dimensional point cloud of the object surface is acquired based on the regional adaptive brightness coding fringe image.
2. The structured light three-dimensional measurement method based on local adaptive fringe brightness according to claim 1, characterized in that: The obtaining of the optimal projection grayscale value of the projector pixel corresponding to the pixel saturation area in the first uniform brightness modulated image comprises: Acquire pixel saturation region information of the first uniform brightness modulated image; the pixel saturation region information includes position information of each pixel saturation region of the first uniform brightness modulated image and pixel information of edges of each pixel saturation region; Based on the pixel saturation area information and using the trained optimal projection brightness prediction model, the optimal projection grayscale value of the projector pixel corresponding to the pixel saturation area in the first uniform brightness modulated image is obtained.
3. The structured light three-dimensional measurement method based on local adaptive fringe brightness according to claim 2, characterized in that: The acquiring pixel saturation area information of the first uniform brightness modulated image includes: By performing binarization processing on the first uniform brightness modulated image, positions of saturated pixels in the first uniform brightness modulated image are extracted to obtain a first saturated area image; Filling the holes inside the first saturated area image to obtain a second saturated area image; The second saturation region image is subjected to median filtering, region dilation and region numbering to obtain position information of saturation regions of each pixel of the first uniform brightness modulated image.
4. The structured light three-dimensional measurement method based on local adaptive fringe brightness according to claim 3, characterized in that: The step of acquiring pixel saturation area information of the first uniform brightness modulated image further includes: Extracting clockwise single-pixel edges of each pixel saturation region in the first uniform brightness modulated image; Performing chain code encoding on the clockwise single pixel edge; Invalid chain code encoding points on saturated pixel points in the first saturated area image are eliminated to obtain pixel information of edges of each pixel saturated area of the first uniform brightness modulated image.
5. The structured light three-dimensional measurement method based on local adaptive fringe brightness according to claim 2, characterized in that: The optimal projection brightness prediction model is trained by the following steps: Acquire a second uniform brightness modulated image, and calculate the optimal projection brightness true value corresponding to each pixel in the second uniform brightness modulated image; Inputting the pixel brightness of the second uniform brightness modulated image and the projection brightness combination vector into the optimal projection brightness prediction model to obtain the optimal projection intensity; the optimal projection brightness prediction model is a fully connected layer neural network structure; A loss function is calculated based on the optimal projection intensity and the optimal projection brightness true value, and the optimal projection brightness prediction model is optimized based on the loss function.
6. The structured light three-dimensional measurement method based on local adaptive fringe brightness according to claim 1, characterized in that: The step of acquiring the homography relationship between each pixel saturation area in the first uniform brightness modulated image and the corresponding area of the projection plane of the projector based on the first coded fringe modulated image includes: Based on the three-dimensional spatial coordinates of the edge pixels of the pixel saturation area of the first coded fringe modulation image and the first uniform brightness modulation image, obtaining the correspondence between the pixel position of each pixel saturation area in the first uniform brightness modulation image and the pixel position of the projection plane of the projector; A robust estimation algorithm is used to obtain a homography matrix based on the correspondence between the pixel positions of each pixel-saturated area in the first uniform brightness modulated image and the pixel positions of the projector projection plane; the homography matrix is used to characterize the mapping relationship from the pixel-saturated area in the first uniform brightness modulated image to the projector projection plane.
7. The structured light three-dimensional measurement method based on local adaptive fringe brightness according to claim 1, characterized in that: The method of acquiring a three-dimensional point cloud of an object surface based on the regional adaptive brightness coding fringe image comprises: Acquire a second coded fringe modulated image modulated by the surface of the object by projecting the adaptive brightness coded fringe image; The second coded fringe modulated image is decoded and triangulated to obtain a complete three-dimensional point cloud of the object surface.
8. A structured light three-dimensional measurement device based on local adaptive fringe brightness, characterized in that: include: A first acquisition module, used for acquiring an image modulated by the surface of an object; The image modulated by the surface of the object includes a first uniform brightness modulated image and a first coded fringe modulated image captured by a camera; A second acquisition module is used to obtain the optimal projection grayscale value of the projector pixel corresponding to the pixel saturation area in the first uniform brightness modulated image, and obtain the homography relationship between each pixel saturation area in the first uniform brightness modulated image and the corresponding area of the projector projection plane based on the first coded fringe modulated image; A third acquisition module is used to map the optimal projection grayscale value to a corresponding position on a projection plane of a projector based on the homography relationship to obtain a regional adaptive brightness coded fringe image; The fourth acquisition module is used to acquire a three-dimensional point cloud of the object surface based on the regional adaptive brightness coding fringe image.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the structured light three-dimensional measurement method based on local adaptive stripe brightness as claimed in any one of claims 1 to 7 is implemented.
10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the structured light three-dimensional measurement method based on local adaptive stripe brightness as claimed in any one of claims 1 to 7 is implemented.
Citation Information
Cited By
Image gray scale estimation method and system, computer equipment and storage medium
CN119941832A
Self-adaptive fringe projection method oriented to difference characteristics
CN120991743A
Partitioned structured light reconstruction method, computer device and medium
CN122737384A
Partitioned structured light reconstruction method, computer device and medium
CN122737384B
