Color three-dimensional reconstruction method, system, terminal and medium for high reflectivity surface
By correcting the grayscale variance and peak evaluation function combined with Gaussian curve fitting, high dynamic range color pictures are generated and color information is fused, which solves the problem that automatic zoom technology cannot measure high reflectivity surfaces, and achieves high-precision color three-dimensional reconstruction, overcoming The problem of missing edge contour and color information.
Patent Information
- Application Number
- CN202111438091.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-11-30
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2041-11-30
AI Technical Summary
Existing automatic zoom technology is difficult to measure high reflectivity surface microstructures, and cannot handle edge profiles and ignore color information, resulting in missing or incorrect measurement results.
Using a focus evaluation function and a peak evaluation function based on the corrected grayscale variance, combined with Gaussian curve fitting, a high dynamic range color picture is generated by multi-exposure method, color information is separated and fused with three-dimensional point clouds, and the color three-dimensional reconstruction of high reflectivity surface color is realized. .
The noise problem of high reflectivity surface measurement is overcome, the accuracy of measurement data is improved, the color information is retained and restored, and the measurement of the entire area of the high reflectivity surface is achieved, reducing hardware requirements.
Smart Images

Figure CN114387387B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of surface microstructure measurement, and in particular to a method, system, terminal and medium for color three-dimensional reconstruction of a high-reflectivity surface based on an automatic zoom technology. Background Art
[0002] In recent years, with the rapid development of emerging manufacturing industries, the requirements for surface microstructure measurement have been continuously improved. In the production and processing process, measuring surface microstructure is of great significance to improving product quality and improving equipment operation stability. Surface microstructure measurement is divided into two categories: contact measurement and non-contact measurement. Among them, non-contact measurement based on optical measurement has been widely used due to its high precision, high speed and low damage. Automatic zoom technology is an optical three-dimensional structure measurement method with image processing technology as the core. This technology originated from shape from focus (SFF) and is mainly used in microscopy to measure surface microstructure. This technology can reconstruct the complete three-dimensional morphology of the object to be measured through continuous vertical scanning in the z direction and advanced image processing technology, and realize the measurement of geometric parameters such as roughness and inclination of the object.
[0003] After searching, we found:
[0004] The Chinese invention patent application numbered 202011247624.5, "An automatic zoom three-dimensional shape measurement system and method", includes: a light source illumination system, a first tube lens, a CCD acquisition system, a second tube lens, a spectroscope, a microscope lens and piezoelectric ceramics. The measurement method specifically scans the object longitudinally first, then collects a set of pictures carrying the height information of the object, and then uses the focus evaluation function based on the Laplace operator to analyze the picture to obtain the focus evaluation function value curve of the pixel point, and finally combines the Gaussian curve fitting algorithm to obtain the accurate focus position of the pixel point, thereby restoring the three-dimensional shape of the object. This technology cannot be applied to high reflectivity surfaces and only measures position information.
[0005] The Chinese invention patent application numbered 201910972508.0, "A structured light micro-nano 3D shape measurement method based on a focus evaluation algorithm", projects the pattern generated by the DMD onto the surface of the object, and scans the object longitudinally to obtain the image to be processed. For a pixel point, it is a process from blur to clear and then blur again in the image sequence. The focus evaluation algorithm is used to locate the image position where the pixel point has the clearest imaging, and then the curve fitting algorithm is combined to extract the precise focus position. The 3D shape of the object can be obtained by traversing all the pixels. This technology uses structured light projection to deal with smooth surface measurement, which has high hardware requirements; it only measures position information.
[0006] In summary, the above-retrieved prior arts all disclose an automatic zoom technology and provide the basic structure of a measurement system using the technology, such as Figure 3 As shown, it includes: light source 1, light source modulator 2, first lens 3, CCD camera 4, second lens 5, beam splitter 6, microscope lens 7, piezoelectric ceramic 8. Among them, the first lens is matched at the light output side of the light source illumination system, the beam splitter is matched at the light output side of the first lens, the microscope lens is matched at the light output side of the beam splitter, the piezoelectric ceramic is matched at the light output side of the microscope lens, the second lens is matched at the reflected light output side of the beam splitter, and the CCD acquisition system is located at the focal plane position of the second lens. During measurement, the light beam emitted by the white light source illumination system passes through the first lens, the beam splitter and the microscope lens in sequence and then irradiates the surface of the object to be measured. After being reflected by the surface of the object, the light path passes through the second lens, and the CCD acquisition system collects the imaging picture carrying the height information of the object, and then the piezoelectric ceramic performs longitudinal scanning on the object and synchronously collects the image. After longitudinally scanning the object to collect a group of pictures carrying the height information of the object, the focusing evaluation function based on the Laplace operator is used to analyze the picture to obtain the focusing evaluation function value curve of the pixel point, and finally the Gaussian curve fitting algorithm is combined to obtain the accurate focusing position of the pixel point, thereby restoring the three-dimensional morphology of the object.
[0007] However, the above prior art still has the following deficiencies:
[0008] 1. The above technology has certain limitations on the reflectivity of the surface of the object to be measured. When the reflectivity of the surface of the object to be measured changes significantly (for example, the horizontal part of the hemispherical metal surface will have a greater reflectivity than the inclined part; some crystal structures on the diamond surface will have a very high reflectivity, etc.), the dynamic range of the scene will be significantly improved (dynamic range refers to the ratio of the maximum light intensity to the minimum light intensity, often expressed as a logarithm with a base of 10). When measuring the microstructure of a smooth metal surface or a diamond surface, the scene dynamic range can reach more than 60dB, while the dynamic range of an industrial CCD camera is generally 48dB. When the scene dynamic range exceeds the CCD dynamic range, no matter how the light intensity or exposure time is changed, there will always be overexposed (too bright to exceed the upper limit of the CCD acquisition value) or underexposed (too dark to be lower than the minimum acquisition value of the CCD) parts on the image collected by the CCD. Since the automatic zoom technology is a measurement technology based on image processing, the data processing of the overexposed and underexposed parts of the image will greatly affect the results of the data processing, resulting in the loss of three-dimensional data results at these locations. Therefore, the above technical solution is difficult to detect microstructures on high reflectivity surfaces.
[0009] 2. The above technology cannot cope with situations with edge contours. In the three-dimensional detection of tool microstructures, contour morphology is an important measurement part. When measuring edge contours, all parts outside the contour are outside the focus range. All points outside the contour do not have a height value that can be focused. After applying the focus evaluation function to these points, what is obtained is no longer a Gaussian distribution curve, but pure noise. Therefore, Gaussian fitting cannot be used for points outside the edge contour, otherwise incorrect values will be obtained. In measurements with contours such as tool detection, the parts outside the contour should be ignored and not involved in the Gaussian fitting calculation.
[0010] 3. The above technology omits the original color information of the object surface. Color information plays an important role in judging the damage type and property change of the processed surface. The above technology only obtains the three-dimensional position data information of the surface microstructure and ignores the color information.
[0011] In summary, the existing surface microstructure measurement technology cannot meet the measurement needs. Currently, no description or report of similar technology to the present invention has been found, and similar information at home and abroad has not been collected. Summary of the invention
[0012] In view of the above-mentioned deficiencies in the prior art, the present invention provides a method, system, terminal and medium for color three-dimensional reconstruction of a high-reflectivity surface based on automatic zoom technology.
[0013] According to one aspect of the present invention, a method for color 3D reconstruction of a high reflectivity surface is provided, comprising:
[0014] A white light source is projected onto the surface of the object to be measured, a plurality of initial images at a plurality of set height positions are collected, and the plurality of initial images at the same height position are synthesized into a high dynamic range color image to obtain a plurality of high dynamic range color images;
[0015] Performing color separation on each of the high dynamic range color images to obtain image color information and a high dynamic range grayscale image;
[0016] The high dynamic range grayscale image is analyzed by using a focus evaluation function based on a corrected grayscale variance, and the focus evaluation function value of each pixel in each grayscale image is calculated to obtain a focus evaluation function curve; a peak evaluation function is used to remove the non-peak points of the focus evaluation function curve, and the focus position of the pixel is obtained by combining Gaussian curve fitting, and all pixel points are traversed to obtain a three-dimensional point cloud of the object to be measured;
[0017] According to the focus position of the pixel point, the image color information of the pixel point at the focus position is retrieved, and the corresponding color is obtained through the color mapping function; the color is fused with the three-dimensional point cloud of the object to be measured to obtain the colored three-dimensional point cloud of the object to be measured, and the colored three-dimensional reconstruction of the high reflectivity surface is completed.
[0018] Preferably, the step of projecting a white light source onto the surface of the object to be measured and collecting a plurality of initial images at a plurality of set height positions includes:
[0019] Projecting a white light source onto the surface of the object to be measured, and controlling the motor step change to vertically scan the object to be measured;
[0020] By changing the exposure time and / or the light intensity of the white light source, a plurality of initial images at a plurality of set height positions are collected by the color CCD;
[0021] Repeat the above steps until you get pictures at all set height positions.
[0022] Preferably, the step of synthesizing a plurality of initial images at the same height position into a high dynamic range color image comprises:
[0023] The synthesis formula of the high dynamic range color picture is:
[0024]
[0025] Among them, (x, y) represents the pixel point at a certain position on the initial image, E represents the pixel value of the high dynamic range color image, I represents the grayscale value received by the CCD, W represents the measurable range of the CCD, and p represents the change parameter of the white light source.
[0026] Preferably, performing color separation on each of the high dynamic range color pictures to obtain picture color information includes:
[0027] The following calculations are performed on the RGB channels of the CCD to obtain:
[0028]
[0029]
[0030]
[0031] Among them, R, G, and B represent the intensity values of red, green, and blue light respectively, and the color information of the picture is obtained.
[0032] Preferably, the use of a focus evaluation function based on a corrected grayscale variance to analyze the high dynamic range grayscale image, calculating a focus evaluation function value of each pixel in each grayscale image, and obtaining a focus evaluation function curve includes:
[0033] The focusing evaluation function of the corrected grayscale variance is:
[0034]
[0035] Among them, F represents the high dynamic range grayscale image data matrix, M represents the mean filter, and F′ represents the evaluation data matrix;
[0036] The value of any pixel in the high dynamic range grayscale image in each evaluation data matrix is the focus evaluation function value;
[0037] A focusing evaluation function curve is obtained by using the focusing evaluation function value.
[0038] Preferably, the peak evaluation function is used to remove the non-peak points of the focus evaluation function curve, and the focus position of the pixel point is obtained by combining Gaussian curve fitting, and all the pixel points are traversed to obtain the three-dimensional point cloud of the object to be measured, including:
[0039] Substitute the focusing evaluation function curve data into the peak evaluation function, wherein the peak evaluation function is:
[0040]
[0041] Wherein, f(x) represents the focusing evaluation function curve, K represents the number of numerical points of the focusing evaluation function curve, and t represents the focusing evaluation function value;
[0042] By setting a threshold, the corresponding pixel points whose focus evaluation function value t is less than the threshold are deleted, that is, the points without peaks in the focus evaluation function curve are removed;
[0043] Perform Gaussian curve fitting on the pixels that have not been deleted to obtain the focus position of the pixels;
[0044] Traverse all the undeleted pixels and perform Gaussian curve fitting to obtain the three-dimensional point cloud of the object to be measured.
[0045] Preferably, the color mapping function is:
[0046]
[0047] Among them, W represents the mapping value of the red, green and blue color channels, E z Indicates the stored light intensity value of the pixel point (x, y) at the focus position, E max and E min It represents the maximum and minimum scanning intensity of the pixel (x, y) on the image at the focus position. k is the intensity contrast coefficient. The larger the k value, the higher the contrast.
[0048] According to another aspect of the present invention, there is provided a high reflectivity surface color 3D reconstruction system, comprising:
[0049] A high dynamic range color image generation module projects a white light source onto the surface of the object to be measured, collects multiple initial images at multiple set height positions, and synthesizes the multiple initial images at the same height position into a high dynamic range color image to obtain multiple high dynamic range color images;
[0050] A module for acquiring image color information and grayscale images, which performs color separation on each high dynamic range color image to obtain image color information and a high dynamic range grayscale image;
[0051] A three-dimensional point cloud acquisition module, which uses a focus evaluation function based on a corrected grayscale variance to analyze the high dynamic range grayscale image, calculates the focus evaluation function value of each pixel in each grayscale image, and obtains a focus evaluation function curve; uses a peak evaluation function to remove the non-peak points of the focus evaluation function curve, combines Gaussian curve fitting to obtain the focus position of the pixel point, and traverses all pixel points to obtain a three-dimensional point cloud of the object to be measured;
[0052] A color 3D reconstruction module for high reflectivity surfaces retrieves the image color information of the pixel at the focus position according to the focus position of the pixel point, obtains the corresponding color through the color mapping function; fuses the color with the 3D point cloud of the object to be measured to obtain the color 3D point cloud of the object to be measured, and completes the color 3D reconstruction of the high reflectivity surface.
[0053] According to a third aspect of the present invention, there is provided a terminal comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor can be used to execute any of the above-described methods or to run the above-described system when executing the program.
[0054] According to a fourth aspect of the present invention, there is provided a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can be used to execute any of the above-mentioned methods, or to run the above-mentioned system.
[0055] Due to the adoption of the above technical solution, the present invention has at least one of the following beneficial effects compared with the prior art:
[0056] The method, system, terminal and medium for color three-dimensional reconstruction of high-reflectivity surface provided by the present invention realize three-dimensional measurement of high-reflectivity surface based on automatic zoom technology. For surface structures with significant reflectivity changes, a high dynamic range image is obtained by a multi-exposure method. Focus evaluation calculation is performed by replacing the image directly obtained by the CCD camera with the high dynamic range image, so as to realize measurement of all areas on the high-reflectivity surface.
[0057] The method, system, terminal and medium for color 3D reconstruction of high reflectivity surface provided by the present invention use the peak evaluation function to remove non-peak noise points. For the part that does not reach the focusing height during the measurement process, the focusing evaluation function will not obtain the Gaussian peak value, and the Gaussian fitting will fail. Adding the peak evaluation function between the focusing evaluation function and the Gaussian fitting step can remove the unfocused points during the measurement process.
[0058] The high reflectivity surface color 3D reconstruction method, system, terminal and medium provided by the present invention realize color fusion. The original color information of the surface is obtained by color CCD, and after high dynamic range correction, it is fused with the measured 3D information to obtain color 3D data.
[0059] The method, system, terminal and medium for color three-dimensional reconstruction of a high-reflectivity surface provided by the present invention overcome the problem that the automatic zoom technology cannot measure the high-reflectivity surface.
[0060] The high-reflectivity surface color three-dimensional reconstruction method, system, terminal and medium provided by the present invention effectively shield unfocused points and improve the accuracy of three-dimensional measurement data.
[0061] The high-reflectivity surface color three-dimensional reconstruction method, system, terminal and medium provided by the present invention retain and restore the color information of the object to be measured.
[0062] The method, system, terminal and medium for color three-dimensional reconstruction of a high-reflectivity surface provided by the present invention adopt a peak evaluation function to reduce noise points.
[0063] The high reflectivity surface color three-dimensional reconstruction method, system, terminal and medium provided by the present invention measure color information in addition to position information.
[0064] The high-reflectivity surface color three-dimensional reconstruction method, system, terminal and medium provided by the present invention adopt a multi-exposure method, thereby reducing the requirements on hardware. BRIEF DESCRIPTION OF THE DRAWINGS
[0065] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:
[0066] Figure 1 This is a flow chart of a method for color 3D reconstruction of a high reflectivity surface in one embodiment of the present invention.
[0067] Figure 2 This is a flow chart of a method for color 3D reconstruction of a high reflectivity surface in a preferred embodiment of the present invention.
[0068] Figure 3The figure is a structural diagram of an automatic zoom measurement device in a specific application example of the present invention.
[0069] Figure 4 The CCD captures a high dynamic range diamond tool image in a specific application example of the present invention.
[0070] Figure 5 This is a high dynamic range grayscale image of the tool surface in a specific application example of the present invention.
[0071] Figure 6 The figure is a schematic diagram of a colored three-dimensional point cloud of a diamond tool surface measured in a specific application example of the present invention.
[0072] Figure 7 Schematic diagram of the components of a high reflectivity surface color 3D reconstruction system in one embodiment of the present invention. DETAILED DESCRIPTION
[0073] The following is a detailed description of the embodiments of the present invention: This embodiment is implemented on the premise of the technical solution of the present invention, and a detailed implementation method and a specific operation process are given. It should be pointed out that for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention.
[0074] Figure 1 A flow chart of a method for color 3D reconstruction of a high-reflectivity surface provided by an embodiment of the present invention.
[0075] like Figure 1 As shown, the high reflectivity surface color 3D reconstruction method provided in this embodiment may include the following steps:
[0076] S100, projecting a white light source onto the surface of the object to be measured, collecting multiple initial images at multiple set height positions, and synthesizing the multiple initial images at the same height position into a high dynamic range color image, thereby obtaining multiple high dynamic range color images;
[0077] S200, performing color separation on each high dynamic range color image to obtain image color information and a high dynamic range grayscale image;
[0078] S300, using a focus evaluation function based on corrected grayscale variance to analyze high dynamic range grayscale images, calculate the focus evaluation function value of each pixel in each grayscale image, and obtain a focus evaluation function curve; using a peak evaluation function to remove the non-peak points of the focus evaluation function curve, combining Gaussian curve fitting to obtain the focus position of the pixel, traversing all pixel points to obtain a three-dimensional point cloud of the object to be measured;
[0079] S400, according to the focus position of the pixel point, the image color information of the pixel point at the focus position is retrieved, and the corresponding color is obtained through the color mapping function; the color is merged with the three-dimensional point cloud of the object to be measured to obtain the color three-dimensional point cloud of the object to be measured, and the color three-dimensional reconstruction of the high reflectivity surface is completed.
[0080] In S100 of this embodiment, as a preferred embodiment, projecting a white light source onto the surface of the object to be measured and collecting a plurality of initial images at a plurality of set height positions may include the following steps:
[0081] S101, projecting a white light source onto the surface of the object to be measured, and controlling the motor step change to vertically scan the object to be measured;
[0082] S102, by changing the exposure time and / or light intensity of the white light source, a plurality of initial images at a plurality of set height positions are collected by the color CCD;
[0083] S103, repeat the above steps until all pictures at the set height positions are obtained.
[0084] In S100 of this embodiment, as a preferred embodiment, combining multiple initial images at the same height position into a high dynamic range color image may include the following steps:
[0085] The synthesis formula for high dynamic range color images is:
[0086]
[0087] Among them, (x, y) represents the pixel point at a certain position on the initial image, E represents the pixel value of the high dynamic range color image, that is, a numerical pixel intensity information, I represents the grayscale value received by the CCD, W represents the measurable range of the CCD, and p represents the change parameter of the white light source.
[0088] In this embodiment, the dynamic range refers to the ratio of the maximum and minimum signal light intensity. For a high reflectivity surface, the dynamic range of its reflected light is high, exceeding the range of a traditional RGB image, so a high dynamic range image is required to be substituted into subsequent image processing.
[0089] In S200 of this embodiment, as a preferred embodiment, color separation is performed on each high dynamic range color picture to obtain picture color information, which may include the following steps:
[0090] The following calculations are performed on the RGB channels of the CCD to obtain:
[0091]
[0092]
[0093]
[0094] Among them, R, G, and B represent the intensity values of red, green, and blue light respectively, and the color information of the picture is obtained.
[0095] In S300 of this embodiment, as a preferred embodiment, a focus evaluation function based on a corrected grayscale variance is used to analyze a high dynamic range grayscale image, and a focus evaluation function value of each pixel in each grayscale image is calculated to obtain a focus evaluation function curve, which may include the following steps:
[0096] S3011, the focus evaluation function for correcting grayscale variance can be:
[0097]
[0098] Among them, F represents the high dynamic range grayscale image data matrix, M represents the mean filter, and F′ represents the evaluation data matrix;
[0099] S3012, the value of any pixel in the high dynamic range grayscale image in each evaluation data matrix is the focus evaluation function value;
[0100] S3013, using the focusing evaluation function value to obtain a focusing evaluation function curve.
[0101] In S300 of this embodiment, as a preferred embodiment, using the peak evaluation function to remove the non-peak points of the focusing evaluation function curve may include the following steps:
[0102] S3021, substituting the focusing evaluation function curve data into the peak evaluation function, wherein the peak evaluation function is:
[0103]
[0104] Wherein, f(x) represents the focusing evaluation function curve, K represents the number of numerical points of the focusing evaluation function curve, and t represents the focusing evaluation function value;
[0105] S3022, by setting a threshold, the corresponding pixel points whose focus evaluation function value t is less than the threshold are deleted, that is, the points without peaks on the focus evaluation function curve are removed.
[0106] In S300 of this embodiment, as a preferred embodiment, the focus position of the pixel point is obtained by combining Gaussian curve fitting, and all the pixel points are traversed to obtain the three-dimensional point cloud of the object to be measured, which may include the following steps:
[0107] S3031, performing Gaussian curve fitting on the pixels that have not been deleted to obtain the focus position of the pixels;
[0108] S3032, traverse all the undeleted pixels to perform Gaussian curve fitting, so as to obtain the three-dimensional point cloud of the object to be measured.
[0109] In S400 of this embodiment, as a preferred embodiment, the color mapping function may be:
[0110]
[0111] Among them, W represents the mapping value of the red, green and blue color channels, E z Indicates the stored light intensity value of the retrieved pixel point (x, y) at the focus position, E max and E min It represents the maximum and minimum scanning intensity of the retrieved pixel point (x, y) on the image at the focus position. k is the intensity contrast coefficient. The larger the k value, the higher the contrast.
[0112] Figure 2 A flow chart of a method for color 3D reconstruction of a high-reflectivity surface provided in a preferred embodiment of the present invention.
[0113] like Figure 2 As shown, the method for color 3D reconstruction of a high reflectivity surface provided in this embodiment may include the following steps:
[0114] Step 1: Project a white light source onto the surface of the object to be measured, and collect multiple initial images of the height position by the color CCD by changing the exposure time or the light intensity of the light source; control the motor step change to vertically scan the object to be measured, repeat the above steps for each height position, and store the obtained data in the computer;
[0115] Step 2: Combine these initial images into a high dynamic range color image, separate the color information and store it in a computer;
[0116] Step 3: Use the focus evaluation function based on the corrected grayscale variance to analyze the saved high dynamic range grayscale image in the computer, calculate the focus evaluation function value of each pixel in each grayscale image, and obtain the focus evaluation function curve. Use the peak evaluation function to remove the non-peak points, combine the Gaussian curve fitting to obtain the focus position of the pixel point, and traverse all the pixel points to obtain the three-dimensional point cloud of the object to be measured;
[0117] Step 4: According to the focus position of the pixel point, the color information of the point at the focus position is retrieved, and the color that can be displayed on the computer is obtained through the color mapping function. The color is merged with the obtained three-dimensional point cloud to obtain the color three-dimensional point cloud of the object to be measured.
[0118] As a preferred embodiment, the color mapping function is:
[0119]
[0120] Where W represents the mapping value of the red, green, and blue color channels, E z Indicates the stored light intensity value of the pixel point (x, y) at the focus position, E max and E min It represents the maximum and minimum scanning light intensity of the pixel (x, y) on the image at the focus position. k is the light intensity contrast coefficient. The larger the k value, the higher the contrast.
[0121] As a preferred embodiment, in step 1:
[0122] Multiple exposure initial images are used to synthesize a high dynamic range color image. The high dynamic range color image synthesis formula is:
[0123]
[0124] Wherein, (x, y) represents a pixel point at a certain position on the initial image, E represents the pixel value of the high dynamic range color image, I represents the grayscale value received by the CCD, W represents the measurable range of the CCD, and p represents the change parameter of the white light source. The change parameter can be the exposure time and / or the light intensity of the light source;
[0125] As a preferred embodiment, in step 2:
[0126] The synthesis formula in step 1 is used for the three channels of RGB of CCD, namely:
[0127]
[0128]
[0129]
[0130] In the formula, R, G, and B represent the intensity values of red, green, and blue light respectively. The color information of the picture is obtained and the values are stored in the computer.
[0131] As a preferred embodiment, step 3 includes the following steps:
[0132] Step 3.1: Use the modified graylevel variance focus evaluation function to obtain a focus evaluation curve.
[0133] The focusing evaluation function of correcting grayscale variance is:
[0134]
[0135] Where F represents the high dynamic range grayscale image data matrix, M represents the mean filter, and F′ represents the evaluation data matrix.
[0136] The value of any pixel point in the high dynamic range grayscale image in each evaluation data matrix is the focus evaluation function value, and the focus evaluation function value is used to make a focus evaluation function curve;
[0137] Step 3.2: Substitute the focusing evaluation function curve data into the peak evaluation function calculation, where the peak evaluation function is:
[0138]
[0139] Wherein, f(x) represents the focusing evaluation function curve, K represents the number of numerical points of the focusing evaluation function curve, and t represents the focusing evaluation function value.
[0140] By setting a threshold, the corresponding pixel points whose value t is less than the set threshold are deleted;
[0141] Step 3.3: Perform Gaussian fitting on the undeleted pixels to obtain the focus position. Traverse the undeleted pixels to obtain a three-dimensional point cloud.
[0142] The technical solution in the above embodiment of the present invention is further described in detail below in conjunction with the accompanying drawings and a specific application implementation.
[0143] This specific application example is to measure the three-dimensional morphology of the surface of a highly reflective diamond tool. The automatic zoom measurement device used is Figure 3 As shown, the reconstruction process is as follows Figure 1 or Figure 2 shown.
[0144] Diamond is a highly reflective material. The dynamic range of the surface of a diamond tool can reach more than 60 dB. Take the method of the present invention for example to measure the three-dimensional morphology of the diamond surface. Figure 4 The image of the surface of a diamond tool captured by a CCD is shown. The bright spots on the image indicate that the area has high reflectivity. When measuring these areas with traditional autofocus technology, calculation errors and data loss will occur. The high reflectivity grayscale image obtained by the color 3D reconstruction method of the reflective surface provided in the above embodiment of the present invention is shown in FIG. Figure 5 As shown, correct calculation can be performed after obtaining the high-reflection grayscale image.
[0145] The depth information obtained from the high dynamic range image after the focus evaluation, peak evaluation and Gaussian fitting steps provided in the above embodiment of the present invention is combined with the color fusion algorithm to obtain the color three-dimensional point cloud of the diamond surface, such as Figure 6 The unfocused data points outside the tool contour are deleted by the peak evaluation function.
[0146] Through the reconstruction results of this specific application example, it can be seen that the high-reflectivity surface color 3D reconstruction method provided by the above embodiment of the present invention overcomes the problem that the automatic zoom technology cannot measure the high-reflectivity surface, effectively shields the unfocused points, and improves the accuracy of the 3D measurement data.
[0147] Figure 7 A schematic diagram of the components of a high-reflectivity surface color 3D reconstruction system provided by an embodiment of the present invention.
[0148] like Figure 7 As shown, the high reflectivity surface color 3D reconstruction system provided by this embodiment may include: a high dynamic range color image generation module, an image color information and grayscale image acquisition module, a 3D point cloud acquisition module and a high reflectivity surface color 3D reconstruction module. Among them:
[0149] A high dynamic range color image generation module projects a white light source onto the surface of the object to be measured, collects multiple initial images at multiple set height positions, and synthesizes the multiple initial images at the same height position into a high dynamic range color image to obtain multiple high dynamic range color images;
[0150] Image color information and grayscale image acquisition module, which performs color separation on each high dynamic range color image to obtain image color information and high dynamic range grayscale image;
[0151] 3D point cloud acquisition module, which uses a focus evaluation function based on corrected grayscale variance to analyze high dynamic range grayscale images, calculates the focus evaluation function value of each pixel in each grayscale image, and obtains a focus evaluation function curve; uses a peak evaluation function to remove the non-peak points of the focus evaluation function curve, combines Gaussian curve fitting to obtain the focus position of the pixel, and traverses all pixel points to obtain a 3D point cloud of the object to be measured;
[0152] A color 3D reconstruction module for high-reflectivity surfaces retrieves the image color information of the pixel at the focus position according to the focus position of the pixel, and obtains the corresponding color through the color mapping function; the color is merged with the 3D point cloud of the object to be tested to obtain the color 3D point cloud of the object to be tested, thus completing the color 3D reconstruction of the high-reflectivity surface.
[0153] An embodiment of the present invention provides a terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it can be used to execute any one of the methods in the above embodiments of the present invention, or to run the system in the above embodiments of the present invention.
[0154] Optionally, the memory is used to store programs; the memory may include volatile memory (English: volatile memory), such as random-access memory (English: random-access memory, abbreviated: RAM), such as static random-access memory (English: static random-access memory, abbreviated: SRAM), double data rate synchronous dynamic random access memory (English: Double Data Rate Synchronous Dynamic Random Access Memory, abbreviated: DDR SDRAM), etc.; the memory may also include non-volatile memory (English: non-volatile memory), such as flash memory (English: flash memory). The memory is used to store computer programs (such as applications, functional modules, etc. that implement the above method), computer instructions, etc., and the above computer programs, computer instructions, etc. can be partitioned and stored in one or more memories. And the above computer programs, computer instructions, data, etc. can be called by the processor.
[0155] The above-mentioned computer programs, computer instructions, etc. may be stored in one or more memories in partitions, and the above-mentioned computer programs, computer instructions, data, etc. may be called by a processor.
[0156] The processor is used to execute the computer program stored in the memory to implement the various steps in the method involved in the above embodiment. For details, please refer to the relevant description in the above method embodiment.
[0157] The processor and the memory may be independent structures or integrated structures. When the processor and the memory are independent structures, the memory and the processor may be coupled and connected via a bus.
[0158] According to a fourth aspect of the present invention, there is provided a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can be used to execute any one of the methods in the above-mentioned embodiments of the present invention, or to run the system in the above-mentioned embodiments of the present invention.
[0159] The method, system, terminal and medium for color three-dimensional reconstruction of high-reflectivity surface provided by the present invention realize three-dimensional measurement of high-reflectivity surface based on automatic zoom technology. For surface structures with significant reflectivity changes, a high dynamic range image is obtained by a multi-exposure method. Focus evaluation calculation is performed by replacing the image directly obtained by the CCD camera with the high dynamic range image, so as to realize measurement of all areas on the high-reflectivity surface.
[0160] The method, system, terminal and medium for color three-dimensional reconstruction of a high-reflectivity surface provided by the present invention overcome the problem that the automatic zoom technology cannot measure the high-reflectivity surface, effectively shield the unfocused points, and improve the accuracy of the three-dimensional measurement data; retain and restore the color information of the object to be measured; adopt the peak evaluation function to reduce the noise points; in addition to measuring the position information, also measure the color information; adopt the multi-exposure method to reduce the requirements for hardware.
[0161] It should be noted that the steps in the method provided by the present invention can be implemented by using corresponding modules, devices, units, etc. in the system. Those skilled in the art can refer to the technical solution of the method to realize the composition of the system, that is, the embodiments in the method can be understood as preferred examples for constructing the system, which will not be elaborated here.
[0162] Those skilled in the art know that, in addition to implementing the system and its various devices provided by the present invention in a purely computer-readable program code, the system and its various devices provided by the present invention can be made to implement the same functions in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system and its various devices provided by the present invention can be considered as a hardware component, and the devices for implementing various functions included therein can also be considered as structures within the hardware component; the devices for implementing various functions can also be considered as both software modules for implementing the method and structures within the hardware component.
[0163] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art may make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention.
Claims
1. A method for color 3D reconstruction of a high reflectivity surface, characterized in that: include: A white light source is projected onto the surface of the object to be measured, a plurality of initial images at a plurality of set height positions are collected, and the plurality of initial images at the same height position are synthesized into a high dynamic range color image to obtain a plurality of high dynamic range color images; Performing color separation on each of the high dynamic range color images to obtain image color information and a high dynamic range grayscale image; The high dynamic range grayscale image is analyzed by using a focus evaluation function based on a corrected grayscale variance, and the focus evaluation function value of each pixel in each grayscale image is calculated to obtain a focus evaluation function curve; a peak evaluation function is used to remove the non-peak points of the focus evaluation function curve, and the focus position of the pixel is obtained by combining Gaussian curve fitting, and all pixel points are traversed to obtain a three-dimensional point cloud of the object to be measured; According to the focus position of the pixel point, the image color information of the pixel point at the focus position is retrieved, and the corresponding color is obtained through the color mapping function; the color is fused with the three-dimensional point cloud of the object to be measured to obtain the colored three-dimensional point cloud of the object to be measured, and the colored three-dimensional reconstruction of the high reflectivity surface is completed.
2. The method for color 3D reconstruction of a high reflectivity surface according to claim 1, characterized in that: The method of projecting a white light source onto the surface of the object to be measured and collecting a plurality of initial images at a plurality of set height positions includes: Projecting a white light source onto the surface of the object to be measured, and controlling the motor step change to vertically scan the object to be measured; By changing the exposure time and / or the light intensity of the white light source for multiple times, a plurality of initial images at multiple set height positions are collected by the color CCD; Repeat the above steps until you get pictures at all set height positions.
3. The method for color 3D reconstruction of a high reflectivity surface according to claim 1, characterized in that: The step of synthesizing a plurality of initial images at the same height position into a high dynamic range color image comprises: The synthesis formula of the high dynamic range color picture is: Among them, (x, y) represents the pixel point at a certain position on the initial image, E represents the pixel value of the high dynamic range color image, I represents the grayscale value received by the CCD, W represents the measurable range of the CCD, and p represents the change parameter of the white light source.
4. The method for color 3D reconstruction of a high reflectivity surface according to claim 1, characterized in that: The step of performing color separation on each high dynamic range color image to obtain RGB three-channel color information includes: The following calculations are performed on the RGB channels of the CCD to obtain: Among them, R, G, and B represent the intensity values of red, green, and blue light respectively, and the color information of the picture is obtained.
5. The method for color 3D reconstruction of a high reflectivity surface according to claim 1, characterized in that: The method of using a focus evaluation function based on a corrected grayscale variance to analyze the high dynamic range grayscale image, calculating a focus evaluation function value of each pixel in each grayscale image, and obtaining a focus evaluation function curve includes: The focusing evaluation function of the corrected grayscale variance is: Among them, F represents the high dynamic range grayscale image data matrix, M represents the mean filter, and F′ represents the evaluation data matrix; The value of any pixel in the high dynamic range grayscale image in each evaluation data matrix is the focus evaluation function value; A focusing evaluation function curve is obtained by using the focusing evaluation function value.
6. The method for color 3D reconstruction of a high reflectivity surface according to claim 1, characterized in that: The peak evaluation function is used to remove the non-peak points of the focus evaluation function curve, and the focus position of the pixel point is obtained by combining Gaussian curve fitting, and all the pixel points are traversed to obtain the three-dimensional point cloud of the object to be measured, including: Substitute the focusing evaluation function curve data into the peak evaluation function, wherein the peak evaluation function is: Wherein, f(x) represents the focusing evaluation function curve, K represents the number of numerical points of the focusing evaluation function curve, and t represents the focusing evaluation function value; By setting a threshold, the corresponding pixel points whose focus evaluation function value t is less than the threshold are deleted, that is, the points without peaks in the focus evaluation function curve are removed; Perform Gaussian curve fitting on the pixels that have not been deleted to obtain the focus position of the pixels; Traverse all the undeleted pixels and perform Gaussian curve fitting to obtain the three-dimensional point cloud of the object to be measured.
7. The method for color 3D reconstruction of a high reflectivity surface according to claim 1, characterized in that: The color mapping function is: Among them, W represents the mapping value of the red, green and blue color channels, E z Indicates the stored light intensity value of the pixel point (x, y) at the focus position, E max and E min It represents the maximum and minimum scanning intensity of the pixel (x, y) on the image at the focus position. k is the intensity contrast coefficient. The larger the k value, the higher the contrast.
8. A high reflectivity surface color 3D reconstruction system, characterized in that: include: A high dynamic range color image generation module projects a white light source onto the surface of the object to be measured, collects multiple initial images at multiple set height positions, and synthesizes the multiple initial images at the same height position into a high dynamic range color image to obtain multiple high dynamic range color images; A module for acquiring image color information and grayscale images, which performs color separation on each high dynamic range color image to obtain image color information and a high dynamic range grayscale image; A three-dimensional point cloud acquisition module, which uses a focus evaluation function based on a corrected grayscale variance to analyze the high dynamic range grayscale image, calculates the focus evaluation function value of each pixel in each grayscale image, and obtains a focus evaluation function curve; uses a peak evaluation function to remove the non-peak points of the focus evaluation function curve, combines Gaussian curve fitting to obtain the focus position of the pixel point, and traverses all pixel points to obtain a three-dimensional point cloud of the object to be measured; A color 3D reconstruction module for high reflectivity surfaces retrieves the image color information of the pixel at the focus position according to the focus position of the pixel point, obtains the corresponding color through the color mapping function; fuses the color with the 3D point cloud of the object to be measured to obtain the color 3D point cloud of the object to be measured, and completes the color 3D reconstruction of the high reflectivity surface.
9. A terminal comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, it can be used to execute the method described in any one of claims 1 to 7, or to run the system described in claim 8.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, it can be used to execute the method described in any one of claims 1 to 7, or to run the system described in claim 8.
