Roughness measurement method fusing 2D normal map and 3D replacement map

Abstract

The invention discloses a roughness measurement method fusing a 2D normal map and a 3D replacement map. The method comprises the following steps: acquiring an RGB image and a depth image of a cable joint; the RGB image is preprocessed; calculating to obtain a corresponding 2D normal map and a corresponding 3D replacement map; carrying out surface leveling operation on the 2D normal map, and obtaining the 2D roughness of the cable joint through a two-dimensional surface roughness calculation formula on the leveled image; fitting the 3D replacement map to the three-dimensional point cloud corresponding to the depth image, counting the height value of each point in the point cloud, and calculating the 3D roughness of the cable joint according to a three-dimensional surface roughness calculation formula; and carrying out weighted average on the 2D roughness and the 3D roughness to obtain a cable joint roughness measurement result. The method is suitable for being transplanted to a mobile terminal and an embedded system platform, and has the characteristic that the surface roughness of the cable joint can be quickly, conveniently, accurately and safely measured.

Description

A roughness measurement method that integrates 2D normal mapping and 3D displacement mapping Technical Field

[0001] This disclosure belongs to the fields of computer vision and image recognition technology, and specifically relates to a roughness measurement method that integrates 2D normal mapping and 3D displacement mapping. Background Technology

[0002] Surface roughness measurement methods can be broadly categorized into contact and non-contact methods. Contact methods primarily involve the stylus method, where a stylus is mounted on the top of a detector and tracks the sample surface, moving up and down for electronic detection. The electronic signal is then amplified and digitally converted before being recorded. Non-contact methods mainly include optical methods such as optical interferometry and confocal methods. These methods overcome the drawbacks of contact methods, such as the potential for damage to the sample and poor real-time performance. However, these methods require specific ambient light sources and lighting conditions, and the equipment is very expensive.

[0003] Traditional surface roughness testing instruments have specific requirements for the measurement environment and the object being measured, making them unsuitable for on-site installation of cable joints. Current methods primarily rely on the installer's experience, using touch and side lighting to determine if the cable insulation surface roughness meets installation requirements. However, there is virtually no literature on cable insulation surface roughness testing methods, both domestically and internationally. This is partly due to insufficient emphasis on cable surface roughness and partly because using specialized testing methods increases time and economic costs. In actual installation, specialized instruments are rarely used to measure cable surface roughness. Therefore, it is impossible to qualitatively determine the reliability of cable installation after completion. Summary of the Invention

[0004] In view of the above analysis, in order to solve the problems of accuracy and portability in measuring the roughness of cable surface joints, this disclosure provides a roughness measurement method that integrates 2D normal mapping and 3D displacement mapping, which includes the following steps:

[0005] S100: Acquires RGB and depth images of the cable connector;

[0006] S200: Preprocess the RGB image;

[0007] S300: Calculates the corresponding 2D normal map based on the preprocessed RGB image; and calculates the corresponding 3D displacement map based on the depth image and the preprocessed RGB image.

[0008] S400: Perform a surface leveling operation on the 2D normal map, and obtain the 2D roughness of the cable connector from the leveled image using a two-dimensional surface roughness calculation formula.

[0009] S500: Fit the 3D displacement map onto the three-dimensional point cloud corresponding to the depth image, count the height value of each point in the point cloud, and calculate the 3D roughness of the cable connector according to the three-dimensional surface roughness calculation formula.

[0010] S600: The 2D roughness and the 3D roughness are weighted and averaged to obtain the roughness measurement result of the cable joint.

[0011] Preferably, in step S100, a structured light depth camera is used to acquire RGB images and depth images.

[0012] Preferably, in step S200, the preprocessing includes three aspects: image contrast, image depth, and image detail.

[0013] Preferably, in step S300, the method for calculating the 2D normal map includes the following steps:

[0014] S301: Convert the preprocessed RGB image into a grayscale image, and then use Gaussian blur to eliminate high-frequency noise to obtain a grayscale image after eliminating high-frequency noise.

[0015] S302: Calculate the brightness gradient for each pixel using the Sobel operator;

[0016] S303: Derive the original normal vector by the cross product of tangent vectors and obtain the standardized normal vector;

[0017] S304: Convert the normalized normal vector into RGB channel data through linear mapping.

[0018] Preferably, in step S300, the method for calculating the 3D displacement map includes converting the preprocessed RGB image into a grayscale image, setting the light source direction, solving for the slope, correcting the slope compatibility, and integrating to calculate the height to generate the 3D displacement map.

[0019] Preferably, in step S300, the method for calculating the 3D displacement map includes converting the preprocessed RGB image into a grayscale image, setting the light source direction, solving for the slope, correcting the slope compatibility, and integrating to calculate the height to generate the 3D displacement map.

[0020] Preferably, the formula for calculating the two-dimensional surface roughness is:

[0021]

[0022] in, For 2D roughness, N is the total number of pixels. This represents the corrected pixel value at point i. This represents the total number of pixels multiplied by the actual area represented by a single pixel.

[0023] Preferably, the formula for calculating the three-dimensional surface roughness is:

[0024]

[0025] in, Points on the surface Height deviation from the reference plane The effective area of ​​the measured three-dimensional surface as an orthographic projection onto its reference plane; For 3D roughness.

[0026] Preferably, in step S500, the 3D displacement map fitting is used to ensure that the height information of the 3D displacement map is accurately matched with the three-dimensional structure of the point cloud.

[0027] Furthermore, this disclosure also reveals a roughness measurement system that integrates 2D normal maps and 3D displacement maps, including:

[0028] A device for acquiring RGB and depth images of cable connectors;

[0029] A device for preprocessing the RGB image;

[0030] Apparatus for calculating a corresponding 2D normal map based on a preprocessed RGB image; and for calculating a corresponding 3D displacement map based on a depth image and a preprocessed RGB image;

[0031] A device for performing surface leveling operations on the 2D normal map, and obtaining the 2D roughness of the cable connector from the leveled image using a two-dimensional surface roughness calculation formula.

[0032] An apparatus for fitting the 3D displacement map onto the three-dimensional point cloud corresponding to the depth image, calculating the height value of each point in the point cloud, and calculating the 3D roughness of the cable connector according to the three-dimensional surface roughness calculation formula.

[0033] An apparatus for performing a weighted average of the 2D roughness and the 3D roughness to obtain a roughness measurement result for a cable joint.

[0034] Furthermore, this disclosure also discloses a computer storage medium comprising computer instructions that, when executed on a computer, cause the computer to perform the method described thereon.

[0035] Furthermore, this disclosure also discloses an electronic device, wherein the electronic device includes:

[0036] Memory, processor, and computer program stored in memory and executable on the processor, wherein:

[0037] The processor implements the method when executing the program.

[0038] The beneficial effects of this disclosure are as follows: By using 2D normal maps and 3D displacement maps of cable connector-related images, and then performing surface leveling operations on the 2D normal maps to calculate the roughness values ​​in a two-dimensional sense, the roughness values ​​in a three-dimensional sense are calculated by fitting the 3D displacement maps onto the point cloud model corresponding to the cable connector surface. By weighted averaging of the 2D and 3D roughness values, the surface roughness measurement results of the cable connector can be obtained quickly. This disclosure features a fast, efficient, stable, and reliable roughness measurement process, making it suitable for applications in mobile devices or embedded objects for surface roughness measurement. Attached Figure Description

[0039] Figure 1 is a flowchart of a roughness measurement method that integrates 2D normal maps and 3D displacement maps according to an embodiment of this disclosure;

[0040] Figure 2 is a schematic diagram of the 2D normal map calculation process provided in one embodiment of this disclosure;

[0041] Figure 3 is a schematic diagram of the 3D displacement mapping calculation process provided in one embodiment of this disclosure;

[0042] Figure 4 is a schematic diagram of the linear regression model calculation provided in one embodiment of this disclosure;

[0043] Figure 5 is a schematic diagram of the 2D normal map and 3D displacement map calculated in one embodiment of this disclosure. Detailed Implementation

[0044] As shown in Figure 1, in one embodiment, this disclosure provides a roughness measurement method that integrates 2D normal maps and 3D displacement maps, which includes the following steps:

[0045] S100: Acquires RGB and depth images of the cable connector;

[0046] S200: Preprocess the RGB image;

[0047] S300: Calculates the corresponding 2D normal map based on the preprocessed RGB image; and calculates the corresponding 3D displacement map based on the depth image and the preprocessed RGB image.

[0048] S400: Perform a surface leveling operation on the 2D normal map, and obtain the 2D roughness of the cable connector from the leveled image using a two-dimensional surface roughness calculation formula.

[0049] S500: Fit the 3D displacement map onto the three-dimensional point cloud corresponding to the depth image, count the height value of each point in the point cloud, and calculate the 3D roughness of the cable connector according to the three-dimensional surface roughness calculation formula.

[0050] S600: The 2D roughness and the 3D roughness are weighted and averaged to obtain the roughness measurement result of the cable joint.

[0051] In this embodiment, the present disclosure calculates the surface roughness of the cable joint from two dimensions, two-dimensional space and three-dimensional space, and then weights and fuses the results. This fusion strategy avoids the problem of incomplete characterization of complex surfaces by a single two-dimensional method and makes up for the limitations of pure three-dimensional methods in terms of efficiency and cost. It takes into account the advantages of characterizing surface features under different observation dimensions and improves the comprehensiveness and accuracy of roughness assessment.

[0052] This disclosure can accurately measure the surface roughness of cable joints in high-voltage lines. It is portable to embedded system platforms or mobile devices, offering excellent reliability and convenience. The surface roughness calculation process for high-voltage cable joints in this disclosure is fast, efficient, safe, and reliable. The speed is due to the highly efficient calculation of both 2D normal mapping and 3D displacement mapping, while the accuracy is due to the full consideration of the characteristic differences in different spatial dimensions.

[0053] In another embodiment, in step S100, an RGB image and a depth image are acquired using a structured light depth camera.

[0054] In this embodiment, the structured light depth camera itself falls within the scope of existing technology; however, this disclosure creatively uses it for roughness measurement, which is not reported in the prior art. The structured light depth camera mainly includes: a projector, an RGB camera, an IR camera, and a depth calculation module. The projector in the structured light depth camera is an infrared laser speckle projector. Its workflow mainly includes: first, the infrared laser speckle projector projects a dense infrared laser beam outwards. After coherent interference and diffuse reflection from the object surface, the laser beam forms a coded pattern with a specific pattern. This coded pattern is a speckle pattern composed of randomly distributed speckles. This speckle pattern is fixed, only shifting horizontally or vertically with distance. The IR camera is responsible for acquiring and receiving this speckle pattern. The depth calculation module uses the acquired speckle pattern and an internally stored reference speckle pattern at a known distance as left and right binocular disparity maps for block matching disparity estimation, generating a disparity vector map. Finally, based on the depth calculation method, the depth image (i.e., the depth image) of the projection space or target object is obtained.

[0055] In another embodiment, the preprocessing in step S200 includes preprocessing of three aspects: image contrast, image depth, and image detail.

[0056] In this embodiment, the RGB image is processed to achieve uniform illumination, which involves processing the image contrast, image depth, and image detail.

[0057] In terms of image contrast, by increasing the image contrast, the dynamic range of the image is expanded. For a common 8-bit image representing each pixel, the dynamic range fills the entire 0-255 gray levels, which is significantly greater than using only local gray levels. The dynamic range stretching methods include (1) linear mapping, where the parameter setting may cause gray level loss due to saturation truncation when stretching the dynamic range proportionally; (2) nonlinear Gamma transformation mapping, where the mapping curve can be determined as needed to expand the dynamic range of high gray levels and shrink the dynamic range of low gray levels; and (3) using improved Gamma transformation to stretch from the middle gray level of the dynamic range to both ends.

[0058] As for the image's sense of depth, an adaptive histogram equalization algorithm with limited contrast is used. By dividing the image into blocks and calculating the histograms for mapping, the effect of enhancing the sense of depth in the local area is improved.

[0059] For image detail enhancement, an image sharpening algorithm is used to first separate the high-frequency information in the image, multiply it by an enhancement coefficient, and then superimpose it onto the original image. Sharpening can be applied to either 4-neighborhood or 8-neighborhood areas. Extending the sharpening algorithm, the separation of high and low frequencies can be achieved by varying the size and shape of the low-pass or high-pass filter window, resulting in different detail enhancement effects.

[0060] As shown in Figure 2, in another embodiment, the method for calculating the 2D normal map in step S300 includes the following steps:

[0061] S301: Convert the preprocessed RGB image into a grayscale image, and then use Gaussian blur to eliminate high-frequency noise to obtain a grayscale image after eliminating high-frequency noise.

[0062] S302: Calculate the brightness gradient for each pixel using the Sobel operator;

[0063] S303: Derive the original normal vector by the cross product of tangent vectors and obtain the standardized normal vector;

[0064] S304: Convert the normalized normal vector into RGB channel data through linear mapping.

[0065] In this embodiment, the preprocessed RGB image is converted to a grayscale image, and then Gaussian blur is used to eliminate high-frequency noise and avoid jagged edges in the normals. Subsequently, the Sobel operator is used to calculate the brightness gradient of each pixel. The Sobel operator is used to approximate the surface gradient by detecting changes in the image brightness of the grayscale image after high-frequency noise removal. Exemplarily, in this disclosure, the Sobel operator is a 3×3 convolution kernel, and gradient calculation is achieved by convolving with the image. Specifically, the convolution kernel is:

[0066]

[0067] For each pixel in the grayscale image , respectively nuclear and Multiply the kernel by its 3×3 neighboring pixels and sum them to obtain... and With gradients The normal vector can be derived using the cross product of tangent vectors, based on advanced mathematical knowledge: the horizontal tangent vector. ( Moving one unit in one direction results in a change in height. ); perpendicular tangent vector ( Moving one unit in direction changes the height. ); normal vector Therefore, the original normal vector is denoted as:

[0068]

[0069] Another vector length L is as follows:

[0070]

[0071] Therefore, the standardized normal vector is as follows:

[0072]

[0073] in, The original normal vector, The length of the vector. This is the standardized normal vector. The three-axis components are all [-1, 1], which cannot be directly stored as an image. They need to be converted to RGB channel data through linear mapping.

[0074]

[0075]

[0076]

[0077] This enables the mapping of standardized normal vectors to RGB channel data.

[0078] As shown in Figure 3, in another embodiment, the method for calculating the 3D displacement map in step S300 includes using the absolute scale provided by the depth image as a constraint, based on the grayscale image after eliminating high-frequency noise, further setting the light source direction, solving for the slope, correcting the slope compatibility, integrating to obtain the height sum to generate the 3D displacement map.

[0079] In this embodiment, using the absolute scale provided by the depth image as a constraint, the light source direction is further set based on the grayscale image after high-frequency noise is eliminated, the slope is solved, the slope compatibility is corrected, the height is calculated by integration, and a 3D displacement map is generated. The use of the absolute scale provided by the depth image as a constraint greatly avoids ambiguity in surface roughness when generating a 3D displacement map purely based on a two-dimensional grayscale image, thus facilitating accurate roughness measurement in this disclosure.

[0080] When setting the direction of the light source, it is assumed that the light source is a top light source, that is... Let the height function of the object's surface be... Then the surface normal vector Based on advanced mathematics, the gradient of the height function can be calculated:

[0081]

[0082]

[0083] Let the pixels in the image grayscale value According to the Lambert model:

[0084]

[0085] Combined with the direction of the light source Grayscale values ​​can be obtained. With slope Relationship:

[0086]

[0087] Since the height function is the integral of the slope, the slope The existence of the integral must be guaranteed by satisfying the compatibility condition, i.e., the mixed partial derivatives must be equal:

[0088]

[0089] If this condition is not met, slope compatibility needs to be corrected through smoothing constraints, for example, by minimizing... .

[0090] The Poisson integral is used to calculate height. In discrete images, the height function can be solved by accumulating the slope. Let the top left corner of the image be the origin (0,0), and the height be... (Refer to the benchmark), then along Directional accumulation ,along Directional accumulation :

[0091]

[0092]

[0093] in, The pixel spacing is usually set to 1. After obtaining the height value of each pixel, the height value is normalized to [0, 255] to generate the final 3D displacement map. The gray value 255 (note: 255 is the gray value of white) corresponds to the highest point, and the gray value 0 (note: 0 is the gray value of black) corresponds to the lowest point.

[0094] In another embodiment, the surface leveling operation in step S400 specifically includes: performing data statistics, calculating the mean, calculating the sum of deviation products, least squares estimation, and applying a regression model such as a linear regression model.

[0095] It should be noted that the purpose of surface leveling is to eliminate the overall tilt / bending trend of the image while preserving local undulation features for measuring roughness.

[0096] For example, data statistics require iterating through every pixel in the image and counting the pixel coordinates. and ) and the corresponding grayscale value ( ), calculate the sum, sum of squares, and cross product of these values. Specifically, calculate ( (sum) ( (sum) ( (sum) ( (sum of squares) ( (sum of squares) ( and (cross product sum) ( and (the sum of cross products) and ( and (The sum of cross products).

[0097] To calculate the mean, you need to calculate based on the sum obtained from statistics. , and mean , and Based on the mean, calculate the sum of the products of deviations:

[0098]

[0099]

[0100]

[0101]

[0102]

[0103] in, They represent and The sum of the products of deviations and The sum of the products of deviations and The sum of the products of deviations and The sum of the products of deviations and The sum of the products of deviations, This represents the number of pixels in the image. As shown in Figure 4, the coefficients of the linear regression model are calculated using the least squares method with the sum of deviations. , and The linear regression model is obtained. Applying a linear regression model to the i-th pixel, by analyzing the original pixel value y... i Subtract the predicted value y from the regression model i ', thus obtaining the corrected pixel value after correction at point i: This refers to the local fluctuation value after eliminating the overall trend; the corrected pixel value approximately represents the surface height change.

[0104] Treating the corrected pixel values ​​as analogous to height deviation, image area The 2D roughness is calculated by multiplying the total number of pixels by the actual area (or normalized area) represented by a single pixel, using the following formula for 2D surface roughness:

[0105]

[0106] in, For 2D roughness, N is the total number of pixels. This represents the corrected pixel value at point i.

[0107] In another embodiment, the formula for calculating the three-dimensional surface roughness is:

[0108]

[0109] in, Points on the surface Height deviation from the reference plane The effective area of ​​the measured three-dimensional surface as an orthographic projection onto its reference plane; For 3D roughness.

[0110] In another embodiment, a weighted average of the calculated 2D roughness and 3D roughness is performed to obtain the final roughness measurement result:

[0111] .

[0112] For example, the weighting coefficients are designed as follows: This means that the final measurement result is more biased towards the numerical value of 3D roughness.

[0113] Compared with using two-dimensional roughness values ​​alone, the fusion result takes into account the three-dimensional features of the cable joint surface, and more comprehensively reflects the true roughness of the cable joint surface. Compared with the value obtained by using three-dimensional roughness evaluation alone, the fusion result combines the advantages of two-dimensional roughness in reflecting local contour features, and reduces the information redundancy and errors that may be brought about by three-dimensional evaluation.

[0114] It can be observed that the above 2D roughness and 3D roughness represent, respectively, the arithmetic mean of the absolute values ​​of simulated height deviation per unit area and the arithmetic mean of the absolute values ​​of height deviation per unit area. Therefore, the final roughness measurement result obtained by weighted averaging of the two still conforms to the physical meaning of roughness. The above two roughness calculation formulas are defined according to the physical meaning of roughness, combined with the pixels of this disclosure. The above final roughness weighted formula is a further innovation of this disclosure.

[0115] In another embodiment, the 2D normal map and 3D displacement map calculated from the surface image of the cable connector are shown in Figure 5. Figure 5 also includes a schematic diagram of the image after the normal map has been further leveled. It can be clearly seen that the image features are more prominent and the bump information is more obvious after leveling.

[0116] In another embodiment, the 3D displacement map fitting in step S500 is used to ensure that the height information of the 3D displacement map is accurately matched with the three-dimensional structure of the point cloud.

[0117] For example, a method for fitting a 3D displacement map to a point cloud model using Blender software is specifically involved, which mainly includes the following steps: converting the point cloud into a mesh model, UV unwrapping of the mesh model, mapping and parameter adjustment of the 3D displacement map, and optimization of fitting accuracy.

[0118] The conversion of point clouds into mesh models primarily utilizes surface reconstruction-based methods, such as Poisson reconstruction, Delaunay triangulation, and greedy projection triangulation. The specific implementation mainly relies on the PCL library and the Open3D library.

[0119] When UV unwrapping a mesh, first enter UV editing mode: select the base mesh, press the "Tab" key to enter "Edit Mode," and press the "A" key to select all mesh faces; switch to the "UV Editing" workspace, with the UV editor on the left and the 3D view on the right. Then perform smart UV projection: in the 3D view, press the "U" key, select "Smart UV Project," and set the angle limit (30°) and island margin (0.02) parameters. Finally, optimize the UV layout: in the UV editor, press the "A" key to select all UV points, and execute "UV > Pack Islands" to automatically and compactly arrange UV blocks to fully utilize the texture space; for locally stretched areas (such as narrow UVs), select the corresponding mesh face in the 3D view, press the "U" key, and select "Unwrap" to re-unwrap, ensuring the UV coordinates are consistent with the mesh face ratio.

[0120] When mapping a 3D displacement map, you first need to create a displacement material: Switch to the "Shader Editor" workspace, select the base mesh, and click "New" to create a new material; delete the default connection between "Principled BSDF" and "Material Output," press "Shift+A" to add an "Image Texture" node, and click "Open" to import the 3D displacement map calculated by S300. Then, connect the nodes: Add a "Displacement" node, connect the "Color" output of "Image Texture" to the "Height" input of "Displacement," connect the "Displacement" output of "Displacement" to the "Displacement" input of "Material Output," and enable the Cycles rendering engine (which supports high-precision displacement calculations). Finally, adjust the height matching: In the "Displacement" node, adjust the displacement height using the "Scale" parameter. If the mesh bulge is lower than the point cloud, increase the "Scale" value; otherwise, decrease the "Scale" value.

[0121] For areas with a fitting deviation exceeding 0.1mm, modify the 3D displacement map using the brush tool in the "Image Editor" (e.g., deepen the grayscale values ​​of concave areas and brighten the grayscale values ​​of convex areas), and automatically update the model after saving; or add a "ColorRamp" node in the "ShaderEditor" and enhance local height details by adjusting the color level curve. After optimizing the fitting accuracy, the final fitted model can be exported. Based on each point of the final model... By analyzing the component values, the roughness value can be calculated.

[0122] In another embodiment, a roughness measurement device that integrates 2D normal mapping and 3D displacement mapping includes:

[0123] A device for acquiring RGB and depth images of cable connectors;

[0124] A device for preprocessing the RGB image;

[0125] Apparatus for calculating a corresponding 2D normal map based on a preprocessed RGB image; and for calculating a corresponding 3D displacement map based on a depth image and a preprocessed RGB image;

[0126] A device for performing surface leveling operations on the 2D normal map, and obtaining the 2D roughness of the cable connector from the leveled image using a two-dimensional surface roughness calculation formula.

[0127] An apparatus for fitting the 3D displacement map onto the three-dimensional point cloud corresponding to the depth image, calculating the height value of each point in the point cloud, and calculating the 3D roughness of the cable connector according to the three-dimensional surface roughness calculation formula.

[0128] An apparatus for performing a weighted average of the 2D roughness and the 3D roughness to obtain a roughness measurement result for a cable joint.

[0129] In this embodiment, the device is suitable for porting to mobile and embedded system platforms, and features fast, convenient, accurate and safe measurement of cable connector surface roughness.

[0130] Furthermore, this disclosure also discloses a computer storage medium comprising computer instructions that, when executed on a computer, cause the computer to perform any of the methods described above.

[0131] Furthermore, this disclosure also discloses an electronic device, wherein the electronic device includes:

[0132] Memory, processor, and computer program stored in memory and executable on the processor, wherein:

[0133] When the processor executes the program, it implements any of the methods described above.

[0134] Furthermore, the technical effects of the key technical means disclosed herein are summarized as follows:

[0135] Using a 3D depth camera to simultaneously acquire RGB and depth images of cable connectors enables non-contact measurement, protecting the measured object and avoiding potential physical damage to the cable insulation layer caused by traditional stylus methods. The depth image directly provides the three-dimensional coordinates (XYZ point cloud) of the object's surface, laying the foundation for subsequent three-dimensional roughness calculations. Compared to optical methods such as optical interferometry and confocal microscopy, which require extremely high lighting conditions, structured light depth cameras can operate under normal lighting conditions, making them more suitable for complex field environments. Compared to professional high-precision optical equipment, 3D depth cameras are lower in cost, smaller in size, and easily integrated into mobile or handheld devices, enabling portable field measurements.

[0136] Image preprocessing eliminates the effects of uneven lighting. Through contrast enhancement and histogram equalization, it effectively solves the problem of image brightness differences caused by uneven lighting during on-site shooting, ensuring the accuracy of subsequent calculations of normals and heights based on image grayscale values. The sharpening algorithm enhances the high-frequency information (edges and details) of the image, making minute surface bumps and depressions clearer, providing high-quality input data for accurate extraction of surface normals and slopes. The preprocessed RGB image is of higher quality, making subsequent 2D normal mapping and 3D displacement mapping calculations more stable and reducing calculation errors caused by poor image quality.

[0137] 2D normal mapping can quickly and efficiently capture minute local changes in surface normals, is highly sensitive to subtle surface textures and contours, and accurately reflects local roughness. Image gradient-based calculation methods are computationally inefficient, making them suitable for real-time operation on mobile or embedded systems. Through a "surface leveling" operation, the overall height trend caused by shooting angle or object tilt is effectively eliminated, ensuring the accuracy of roughness calculations and focusing only on the surface's microscopic unevenness.

[0138] 3D displacement maps contain absolute height information of the surface. Fitting them to a real 3D point cloud can completely and realistically reconstruct the 3D geometry of the cable connector surface. The calculated final roughness considers the real 3D coordinate system, and the result is more in line with physical reality, avoiding the insufficient representation of complex surfaces by pure 2D methods. In other words, this disclosure combines the geometric accuracy of the depth camera and the texture details of the image to provide high-precision 3D surface data: weighted fusion of 2D and 3D roughness improves comprehensiveness and accuracy. It perfectly combines the efficiency and local detail sensitivity of 2D methods with the realism and global spatial accuracy of 3D methods; it not only avoids the shortcomings of pure 2D methods in representing complex 3D surfaces, but also makes up for the detail loss problem that pure 3D methods (relying only on depth cameras) may cause due to resolution or noise; through weighted fusion, the final result considers both the macroscopic 3D topography and retains the microscopic texture features, making the roughness assessment more comprehensive, objective, and closer to reality.

[0139] In step S300, a pre-trained lightweight neural network can be used to directly generate normal maps and 3D displacement maps end-to-end from the pre-processed RGB image. This further improves computational efficiency; the neural network has extremely fast inference speed, making it particularly suitable for deployment on mobile or embedded systems to meet real-time requirements. A neural network trained on a large amount of data may learn complex lighting and material relationships better than traditional gradient-based or Lambertian model-based algorithms, generating more accurate maps, especially under non-ideal lighting conditions. Encapsulating the complex mathematical derivation process within the neural network simplifies system implementation.

[0140] Furthermore, this disclosure is analyzed in more detail:

[0141] The fusion of 2D normal mapping and 3D displacement mapping significantly reduces the dependence on the accuracy of a single sensor. In pure 3D methods, the resolution and accuracy of the depth camera directly determine the lower limit of roughness measurement. In this disclosure, 2D normal mapping can extract subpixel-level details from high-resolution RGB images, which can be "added" to the 3D point cloud, thus effectively improving the system's equivalent measurement resolution without increasing hardware costs. It also enhances robustness to complex lighting and materials. The 3D point cloud provides realistic geometric constraints that can be used to correct 2D normal / displacement map calculation errors caused by non-Lambertian reflections, specular highlights, or shadows. Conversely, the high-frequency details of the 2D normal map can also be used to "fix" incomplete 3D point clouds caused by depth camera noise or holes; the two act as mutual "error correction codes," resulting in overall anti-interference capabilities far exceeding any single-modal method.

[0142] A "digital twin" foundation for long-term tracking and comparison is generated by fitting a 3D displacement map onto a 3D point cloud. The fitted 3D model is not only a carrier of roughness but also a complete digital record of the surface condition of the cable connector. This allows for multiple scans of the same connector in the future, enabling precise comparison of the evolution of surface wear, aging, or corrosion, and achieving predictive maintenance. Additionally, the high-precision 3D model generated during the implementation of this disclosure can be directly imported into VR / AR systems, allowing engineers to "touch" and inspect the cable connector in a virtual environment for remote collaboration or training—an experience that traditional measurement methods cannot provide.

[0143] The use of lightweight neural networks for end-to-end generation of normal / displacement maps significantly shortens the deployment cycle for new scenarios. For a new cable type or installation environment, there's no need to re-derive complex physical models or adjust numerous algorithm parameters; simply fine-tuning the neural network with a small amount of new data allows for rapid adaptation, resulting in extremely high system generalization ability and deployment efficiency. It also potentially enables joint identification of "defect types." A well-designed neural network can output not only normal / displacement maps but also defect segmentation maps or classification results simultaneously. This means the system can automatically identify scratches, cracks, oxidation, and other defects while measuring roughness, achieving multi-functional integrated detection.

[0144] Although the embodiments of this disclosure have been described above in conjunction with the accompanying drawings, this disclosure is not limited to the specific embodiments and application fields described above. The specific embodiments described above are merely illustrative and instructive, and not restrictive. Those skilled in the art can make many other forms based on the guidance of this specification and without departing from the scope of protection of the claims of this disclosure, and all of these are within the scope of protection of this disclosure.

Claims

1. A roughness measurement method integrating 2D normal mapping and 3D displacement mapping, characterized in that, The process includes the following steps: S100: Acquire RGB and depth images of the cable connector; S200: Preprocess the RGB images; S3 00: Based on the preprocessed RGB image, the corresponding 2D normal map is calculated; Based on the depth image and the preprocessed RGB image, a corresponding 3D displacement map is calculated; S400: The 2D normal map is horizontalized, and the 2D roughness of the cable connector is obtained by using the two-dimensional surface roughness calculation formula; S500: The 3D displacement map is fitted onto the three-dimensional point cloud corresponding to the depth image, the height value of each point in the point cloud is counted, and the 3D roughness of the cable connector is calculated according to the three-dimensional surface roughness calculation formula; S600: The 2D roughness and the 3D roughness are weighted and averaged to obtain the roughness measurement result of the cable connector.

2. The method according to claim 1, characterized in that, In step S100, an RGB image and a depth image are acquired using a structured light depth camera.

3. The method according to claim 1, characterized in that, In step S200, the preprocessing includes three aspects: image contrast, image depth, and image detail.

4. The method according to claim 1, characterized in that, In step S300, the calculation method for the 2D normal map includes the following steps: S301: Convert the preprocessed RGB image to a grayscale image, and then use Gaussian blur to eliminate high-frequency noise to obtain a grayscale image after eliminating high-frequency noise; S302: Calculate the brightness gradient of each pixel using the Sobel operator; S303: Derive the original normal vector through the cross product of tangent vectors and obtain the normalized normal vector; S304: Convert the normalized normal vector into RGB channel data through linear mapping.

5. The method according to claim 1, characterized in that, In step S300, the method for calculating the 3D displacement map includes converting the preprocessed RGB image into a grayscale image, setting the light source direction, solving for the slope, correcting the slope compatibility, and integrating to calculate the height to generate the 3D displacement map.

6. The method according to claim 1, characterized in that, The formula for calculating the two-dimensional surface roughness is as follows: ,in, For 2D roughness, N is the total number of pixels. This represents the corrected pixel value at point i. This represents the total number of pixels multiplied by the actual area represented by a single pixel.

7. The method according to claim 1, characterized in that, The formula for calculating the three-dimensional surface roughness is as follows: ,in, Points on the surface Height deviation from the reference plane The effective area of ​​the measured three-dimensional surface as an orthographic projection onto its reference plane; For 3D roughness.

8. The method according to claim 1, characterized in that, In step S500, the 3D displacement map fitting is used to ensure that the height information of the 3D displacement map is accurately matched with the three-dimensional structure of the point cloud.

Images