Inspection robot adaptive visual detection method and system for strong light reflection environment

Abstract

This invention belongs to the field of computer vision, specifically an adaptive vision detection method and system for inspection robots in highly reflective environments. The method includes: acquiring images of highly reflective metal surfaces captured by the inspection robot; preprocessing each color channel using piecewise functions: linear pixel stretching at low thresholds, logarithmic pixel transformation between high and low thresholds, and power-law compression of pixels at high thresholds, to obtain a preprocessed image; convolving each color channel with a structure tensor-guided anisotropic Gaussian kernel at multiple scales, determining the kernel morphology direction based on the local gradient covariance eigenvector, and estimating multi-scale illumination components; solving for single-scale reflection components in the logarithmic domain, calculating pixel neighborhood information entropy to generate a multi-scale, multi-channel entropy weighted map, and weighted fusion to obtain the reflection components of each channel; constructing a color restoration function to correct the inverse transformation results of the reflection components, restoring color constancy, and obtaining a non-reflective enhanced image; and identifying and locating micro-defects in the metal using a target detection network.

Description

Technical Field

[0001] This application belongs to the field of computer vision, and in particular to an adaptive vision inspection method and system for inspection robots in highly reflective environments. Background Technology

[0002] In industrial production and maintenance, inspection robots equipped with machine vision systems are used for surface quality inspection of metal components. Metal surfaces are prone to specular reflection under specific lighting conditions, forming highlight areas. Strong reflections can saturate the image acquisition sensor, leading to localized overexposure and low-contrast dark areas in adjacent regions. Pixel information in overexposed areas is completely lost, making it impossible to identify minute defects hidden beneath the highlights. Image enhancement algorithms based on Retinex theory are techniques to solve the problems of uneven lighting and highlights. This theory decomposes the image observed by the human eye into an illumination component representing the intensity of light and a reflection component reflecting the inherent properties of the object's surface. By estimating and removing the influence of the illumination component, the true details and colors of the object can be recovered. From single-scale Retinex to multi-scale Retinex, and then to multi-scale Retinex with color restoration, i.e., MSRCR, the MSRCR algorithm and its variants employ an isotropic Gaussian filter to smooth the image when estimating the illumination component in the context of strong reflections on metal surfaces. While smoothing highlight areas, filters indiscriminately blur edge and texture details in images, easily producing halo artifacts that interfere with the judgment of minor defects. In multi-scale fusion, merging reflection component images at different scales using linear averaging or fixed weights fails to consider the differences in information content across different scales and image regions, potentially leading to insufficient detail enhancement in some areas and excessive noise amplification in others. The color restoration step in the MSRCR algorithm is global, and its correction capability for localized, severe color distortion caused by strong reflections is limited, failing to guarantee local color constancy in the enhanced image. Therefore, how to develop an image enhancement method that can suppress strong reflections, preserve edge details, and fuse multi-scale information to restore metal micro-defects is a pressing problem that current visual inspection technology for inspection robots needs to solve. Summary of the Invention

[0003] This invention proposes an adaptive vision inspection method for inspection robots in highly reflective environments. It addresses the lack of an image enhancement method in existing technologies that can suppress strong reflections, preserve edge details, and fuse multi-scale information to reconstruct micro-defects in metals. The method includes: A strong reflective image of the metal surface to be inspected is acquired by the inspection robot; the range of each color channel of the strong reflective image is adjusted by a piecewise function, pixels below a preset low threshold are linearly stretched, pixels between the low threshold and the high threshold are logarithmically transformed, and pixels above the preset high threshold are power-law compressed to obtain a preprocessed image. At multiple scales, anisotropic Gaussian kernels guided by structure tensor are used to convolve each color channel of the preprocessed image to obtain the illumination component estimates of each color channel at each scale. The shape and orientation of the anisotropic Gaussian kernels are determined by the eigenvectors of the local gradient covariance matrix of the image, so that the convolution is smooth along the image edge direction and suppressed in the vertical edge direction. Based on the preprocessed image and the illumination component estimation at each scale, multiple single-scale reflectance component images of each color channel are calculated in the logarithmic domain; for each color channel at each scale, pixel-level neighborhood information entropy is calculated based on the single-scale reflectance component images to generate a multi-scale multi-channel entropy weight map; the multiple single-scale reflectance component images of each color channel are weighted and fused using the multi-scale multi-channel entropy weight map to obtain the reflectance component images of each channel. The color restoration function is used to perform color correction on the results of the inverse exponential transformation of the reflection component images of each channel to restore the local color constancy of the image and obtain a non-reflective enhanced image; on the non-reflective enhanced image, a target detection network model is used to identify and locate metal micro-defects.

[0004] Furthermore, this invention also relates to an adaptive vision inspection system for inspection robots in highly reflective environments, comprising the following modules: The compression module is used to acquire a highly reflective image of the metal surface to be inspected captured by the inspection robot; the color channels of the highly reflective image are adjusted using a piecewise function, pixels below a preset low threshold are linearly stretched, pixels between the low threshold and the high threshold are logarithmically transformed, and pixels above the preset high threshold are power-law compressed to obtain a preprocessed image. The determination module is used to convolve each color channel of the preprocessed image at multiple scales using an anisotropic Gaussian kernel guided by the structure tensor, to obtain the illumination component estimate of each color channel at each scale. The shape and orientation of the anisotropic Gaussian kernel are determined by the eigenvector of the local gradient covariance matrix of the image, so that the convolution is smooth along the image edge direction and suppressed in the vertical edge direction. The fusion module is used to calculate multiple single-scale reflectance component images of each color channel in the logarithmic domain based on the preprocessed image and the illumination component estimation at each scale; for each color channel at each scale, it calculates pixel-level neighborhood information entropy based on the single-scale reflectance component images to generate a multi-scale multi-channel entropy weight map; and uses the multi-scale multi-channel entropy weight map to perform weighted fusion of the multiple single-scale reflectance component images of each color channel to obtain the reflectance component images of each channel. The detection module is used to perform color correction on the result of the inverse exponential transformation of the reflection component images of each channel using the color restoration function, to restore the local color constancy of the image and obtain a non-reflective enhanced image; on the non-reflective enhanced image, a target detection network model is used to identify and locate metal micro-defects.

[0005] This invention preserves detail information in both highlight and shadow areas of strongly reflective images by segmenting pixels across different brightness ranges. During reflection removal, anisotropic Gaussian kernels guided by structure tensors are used to estimate illumination components, smoothing the illumination while maintaining the image's edge structure and avoiding "halo" phenomena at edges, thus separating the reflection component from image details. Utilizing an information entropy-based weighting mechanism, when fusing multi-scale information, the invention highlights scale components containing more information, enhancing the texture features of minute defects and suppressing noise. A color restoration step corrects for potential color distortions during image processing, ensuring color fidelity in the output image. This generates clear, reflective, and color-accurate high-quality images, improving detection accuracy. Attached Figure Description

[0006] Figure 1 This is a schematic diagram illustrating image preprocessing. Figure 2 This is a schematic diagram for calculating the entropy weight graph. Detailed Implementation

[0007] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0008] This invention proposes an adaptive vision detection method for inspection robots in highly reflective environments, see reference. Figure 1 As shown, it includes the following steps: S1, acquire a highly reflective image of the metal surface to be inspected taken by the inspection robot.

[0009] like Figure 1As shown, by controlling the camera mounted on the inspection robot, a connection is established with the camera using the VideoCapture interface of the OpenCV computer vision library, and the read method is called to detect a high-resolution digital image. The image is stored in the form of a NumPy array with an RGB color space.

[0010] S2, the color channels of the strongly reflective image are adjusted using a piecewise function, pixels below a preset low threshold are linearly stretched, pixels between the low and high thresholds are logarithmically transformed, and pixels above the preset high threshold are power-law compressed to obtain a preprocessed image.

[0011] The RGB image is decomposed into three single-channel grayscale images (B, G, and R) using the `split` function from the OpenCV library. For each channel, the 1st and 99th percentiles of the pixel intensity histogram of that channel are calculated using the `percentile` function from the NumPy library and used as low thresholds. and high threshold Create an all-zero floating-point array with the same size as the single-channel image as the output channel. Iterate through each pixel in this channel; if the pixel value is less than... Then the linear stretching formula is applied, meaning the output value equals the input value multiplied by a stretching factor greater than 1. If the pixel value is between and Between these values, the logarithmic transformation formula is applied, where the output value equals the natural logarithm of the sum of the constant c and 1, where the constant c is used to expand the result to the desired display range. If the pixel value is higher than... Then, a power-law compression transform, i.e., γ correction, is applied, and the output value is equal to the constant c multiplied by the input value raised to the power of γ, where γ is less than 1, for example, 0.5. After all pixels have been processed, the three processed single-channel arrays are merged into a single RGB preprocessed image using the merge function of the OpenCV library; In one possible implementation, the step of adjusting the range of each color channel of the highly reflective image using a piecewise function includes: For each color channel: For pixel values ​​below the low threshold The pixels are processed using a linear stretching function, and the output pixel value = input pixel value × 1.5; For pixel values ​​between the low threshold and the high threshold The pixels between these points are processed using a logarithmic transformation function to output pixel values. ; For pixels with values ​​higher than a high threshold, a power-law compression transform function is used for processing, and the output pixel value is... .

[0012] The range adjustment step processes the input 8-bit RGB highly reflective image, handling the red (R), green (G), and blue (B) color channels by setting two key thresholds: a low threshold and a high threshold. and high threshold The pixel range of 0 to 255 is divided into three intervals. For dark areas, i.e., pixel values... The pixels, applied with a linear stretching function This can increase the brightness of darker pixels in the original image by 50%; for example, a pixel with a value of 30 will be mapped to 45, thus enhancing subtle defect features. For midtone areas, i.e. The pixels are expanded using a logarithmic transformation function. A constant of 75 ensures a smooth transition with the processing results in dark areas at the threshold point of 50, while the logarithmic transformation stretches the contrast in this range, highlighting the texture and potential defects of the metallic surface. For the highlight areas, i.e. The pixels are processed using power-law compression transformation, also known as gamma correction, with a power exponent of 0.6. This compresses high brightness values ​​above 200, weakens the intensity of reflective areas, and restores details lost due to overexposure. Simultaneously, the constant 200 and the coefficients ensure the continuity of the function at the threshold point 200 and the reasonableness of the output range. The entire process generates a pre-processed image with optimized ranges for each channel.

[0013] S3, at multiple scales, anisotropic Gaussian kernels guided by structure tensor are used to convolve each color channel of the preprocessed image to obtain the illumination component estimates of each color channel at each scale. The shape and orientation of the anisotropic Gaussian kernel are determined by the eigenvectors of the local gradient covariance matrix of the image, so that the convolution is smooth along the image edge direction and suppressed in the vertical edge direction.

[0014] Set three or more Gaussian kernel scale parameters, for example, σ values ​​of 15, 80, and 200. For each color channel of the preprocessed image, at each scale, perform the following: compute the first-order gradient of the image in the x and y directions using the Sobel operator from the OpenCV library. and For each pixel in the image, within a neighborhood of that pixel, for example a 5×5 window, calculate the three components of the structure tensor matrix J, i.e. Sum of squared neighborhoods The sum of the square's neighborhood and and The product's neighborhood sum is calculated. The eigenvalues ​​and eigenvectors of the structure's tensor matrix are computed using the `eigh` function from the `linalg` module of the NumPy library. The principal direction (edge ​​direction) and its perpendicular secondary direction are determined based on the eigenvectors. A two-dimensional anisotropic Gaussian kernel is constructed such that the principal axis is along the edge direction with a large standard deviation equal to the current scale parameter σ, and the secondary axis is perpendicular to the edge direction with a small standard deviation, thus achieving strong smoothing along the edge and weak smoothing across the edge. Since the kernel's shape varies with pixel position, pixel-wise filtering or guided filtering algorithms such as `GuidedFilter` are used to approximate this spatially varying convolution process, thereby obtaining the illumination component estimation image for this channel at this scale.

[0015] In one possible implementation, the step of convolving each color channel of the preprocessed image using a structure tensor-guided anisotropic Gaussian kernel at multiple scales includes: Three scales are set, corresponding to the basic standard deviation of the Gaussian kernel as 15 for the small scale, 80 for the medium scale, and 200 for the large scale. At each scale, for each pixel in the image, the gradient covariance matrix, i.e. the structure tensor, of the pixel in a dynamic neighborhood that is proportional to the base standard deviation of the current scale is calculated. The structural tensor is decomposed into eigenvalues ​​to obtain the maximum eigenvalue, the minimum eigenvalue, and the corresponding first eigenvector and second eigenvector. The maximum eigenvalue and the minimum eigenvalue are then normalized to the interval between 0 and 1. An anisotropic Gaussian kernel is constructed based on the eigenvalues ​​and eigenvectors, and the anisotropic Gaussian kernel has a first principal axis and a second principal axis; Wherein, the direction of the first principal axis is determined by the second eigenvector corresponding to the minimum eigenvalue, and the standard deviation of the first principal axis is equal to the base standard deviation of the current scale; The direction of the second principal axis is determined by the first eigenvector corresponding to the largest eigenvalue, and the standard deviation of the second principal axis is equal to the base standard deviation divided by (1 + the absolute value of the difference between the normalized largest and smallest eigenvalues).

[0016] An edge-preserving smoothing filter technique is employed, defining three scales and using the fundamental standard deviation of a Gaussian kernel. Manifestation: Small scale =15 is used to detect local detail lighting, mesoscale. =80 is used to detect regional changes in illumination, on a large scale. =200 is used to estimate the overall, slowly varying lighting background. For each color channel of the preprocessed image, at each scale... Next, iterate through each pixel (x, y) in the image. Using (x, y) as the center, take a base standard deviation relative to the current scale. A proportional dynamic neighborhood window is used to calculate the gradient of the image in the x-direction within the window. gradient in the y-direction Based on the gradient, a 2×2 structure tensor matrix is ​​constructed. The summation is performed within the dynamic neighborhood window. Eigenvalue decomposition is performed on the structure tensor J to obtain two eigenvalues. and the corresponding orthogonal eigenvectors and . Pointing in the direction of the fastest gradient change, which is perpendicular to the edge. The direction pointing to the slowest gradient change is parallel to the edge. Normalize the eigenvalues, for example, by... and This yields an index representing the consistency of the local structure.

[0017] A custom anisotropic Gaussian kernel is constructed for the current pixel (x, y), the kernel having a first principal axis and a second principal axis. The direction of the principal axes and the smoothness, i.e., the standard deviation, are determined by the eigenvectors, eigenvalues, and the fundamental standard deviation. Jointly determined: The second eigenvector corresponding to the smallest eigenvalue. The direction of the first principal axis is taken as the direction of the edge direction, and the standard deviation of the first principal axis is set as... This achieves sufficient smoothness along the edges; the first eigenvector corresponding to the largest eigenvalue is... The direction of the second principal axis is taken as the direction of the edge direction, and the standard deviation of the second principal axis is set as... When the edge protrudes, that is... Approaching 1, This reduces the size of the image, thereby suppressing blurring across edges. An anisotropic Gaussian kernel that varies with spatial location is used to convolve the entire channel image, generating illumination component estimation images at three scales (15, 80, and 200) for each color channel.

[0018] S4. Based on the preprocessed image and the illumination component estimation at each scale, multiple single-scale reflectance component images of each color channel are calculated in the logarithmic domain.

[0019] For each color channel and the estimated illumination component at each scale of the preprocessed image, the log1p function from the NumPy library is used to calculate 1 plus the natural logarithm of the input value, transforming it to the logarithmic domain. In the logarithmic domain, the logarithmic result of the illumination component estimation at the corresponding scale is subtracted from the logarithmic result of the preprocessed image channel to obtain the single-scale logarithmic reflectance component image at that scale.

[0020] S5, for each color channel at each scale, calculates the pixel-level neighborhood information entropy based on the single-scale reflectance component image to generate a multi-scale, multi-channel entropy weight map.

[0021] For the reflectance component image at each scale and for each channel, the `entropy` function of the `rank` submodule under the `filters` module in the scikit-image image processing library is used. This function calculates the Shannon entropy for each pixel in the image within a neighborhood window of a given size, such as 9×9. The entropy is calculated based on the histogram distribution of pixel values ​​within the neighborhood; a high entropy value indicates rich texture details and a large amount of information within the neighborhood. This process is performed on the reflectance component images at all scales and for all channels, resulting in a series of entropy weight maps of the same size as the original image, such as... Figure 2 As shown.

[0022] To assign pixel-level weights to the image based on the amount of information contained in each scale's reflectance component image, in one possible implementation, the step involves calculating pixel-level neighborhood information entropy for each color channel at each scale based on the single-scale reflectance component image to generate a multi-scale, multi-channel entropy weight map, including: For each single-scale reflectance component image of each color channel, the image pixel values ​​are linearly mapped to the integer range of 0 to 255, and a 9×9 neighborhood is taken with each pixel as the center. The probability distribution of pixels at 256 gray levels within the 9×9 neighborhood is statistically analyzed. The information entropy value of the center pixel is calculated according to the Shannon entropy formula, and the information entropy value is multiplied by a reflection suppression coefficient. The reflection suppression coefficient is inversely proportional to the brightness value of the corresponding pixel in the preprocessed image. All pixels are traversed to generate an entropy map of the corresponding scale and channel. Normalize the entropy maps at each scale for each pixel location to obtain the weight maps at each scale. The pixel value of each weight map is equal to the pixel value of the scale entropy map divided by the sum of the pixel values ​​of the small-scale, medium-scale, and large-scale entropy maps at that pixel location.

[0023] Obtain the single-scale reflectance component image of each color channel c at each scale k. It is calculated in the logarithmic field, that is To calculate information entropy, floating-point numbers need to be converted... The image is processed using min-max normalization, mapping the pixel value range to an 8-bit integer interval [0, 255], to obtain... .right For each pixel (x, y) in the image, a 9×9 neighborhood window centered on that pixel is defined. The frequency of each of the 256 gray levels (0 to 255) within this neighborhood of 81 pixels is counted, and their probabilities are calculated. It equals the number of occurrences of gray level i divided by 81. According to the Shannon entropy formula... Calculate the initial information entropy of the neighborhood. Obtain the brightness value of the preprocessed image at pixel location (x, y). Calculate the reflection suppression coefficient Where η is an adjustment constant and ε is a minimum value to prevent the denominator from being zero, making the suppression coefficient inversely proportional to the brightness value of the preprocessed image. The initial information entropy H is multiplied by the reflection suppression coefficient α(x,y), and this corrected entropy value is assigned to the output entropy map. The pixel at position (x,y) is used. By introducing a suppression mechanism inversely proportional to brightness, the high entropy value of normal, non-reflective areas, representing rich true texture details, is preserved and given high weight. Meanwhile, the falsely high entropy value caused by glaring highlight abrupt changes at strongly reflective boundaries is forcibly suppressed by a very small α(x,y), effectively preventing reflective artifacts from being incorrectly amplified. After traversing all pixels, an entropy map with reflection suppression is generated for each channel and each scale. To achieve fusion, the weights at different scales need to be normalized at each pixel location (x, y). Specifically: At each pixel, the sum of the weights of the three scales is 1, forming a multi-scale, multi-channel entropy weight map for weighted fusion.

[0024] S6, the multi-scale multi-channel entropy weighting map is used to perform weighted fusion of multiple single-scale reflectance component images of each color channel to obtain reflectance component images of each channel.

[0025] For each color channel, the single-scale logarithmic reflectance component image obtained at multiple scales is multiplied pixel-by-pixel with the corresponding entropy weight image. The results of multiplication at all scales are summed pixel-by-pixel to obtain a weighted sum image. Then, the entropy weight images at all scales are summed pixel-by-pixel to obtain a weighted sum image. The weighted sum image is divided pixel-by-pixel by the weighted sum image to achieve normalized weighted average fusion, resulting in a single-channel logarithmic reflectance component image. The fusion operation is performed on all three channels: B, G, and R.

[0026] To integrate the advantages of different scales and generate a high-quality reflectance component image, in one possible implementation, the multi-scale multi-channel entropy weighting map is used to weighted fuse multiple single-scale reflectance component images of each color channel to obtain reflectance component images of each channel, including: For each color channel (red, green, and blue), the three scale weight maps of the channel are multiplied at the pixel level with the corresponding single-scale reflectance component image. The results of the multiplication are added pixel by pixel to obtain the reflection component image of the channel. The value is equal to the sum of the products of the weight maps at the small scale, medium scale and large scale and the corresponding single-scale reflection component images.

[0027] For any one of the three color channels c (red, green, and blue), this process utilizes the entropy weight map generated in the previous step. and the calculated single-scale reflection component image Processing is required. The specific fusion formula is as follows: The calculation is performed pixel-by-pixel. For each coordinate point (x, y) in the image, the weight values ​​of that point on the small, medium, and large scale weight maps, and its pixel values ​​on the three single-scale reflectance component images (i.e., the logarithmic domain), are extracted, multiplied accordingly, and then summed. For example, in a textured region of the image, the small-scale entropy value is higher, therefore... Larger entropy values ​​in flat regions will result in a fusion result that retains detailed information from the smaller-scale reflectance component image. Conversely, in flat regions, larger-scale entropy values ​​may be relatively higher, leading to a fusion result that favors the smooth, large-scale reflectance characteristics. This pixel-level weighted summation ensures that the reflectance component image... It exhibits optimal detail preservation and noise suppression in different regions.

[0028] S7. The color restoration function is used to perform color correction on the result of the inverse exponential transformation of the reflection component images of each channel, restoring the local color constancy of the image and obtaining an image without reflection enhancement.

[0029] For the logarithmic reflectance component image of each color channel, the `exp` function from the NumPy library is used to perform exponential operations, transforming the image from the logarithmic domain back to the intensity domain to obtain the reflectance component image. A color restoration function is constructed; for each channel, the color restoration coefficient is calculated based on the color ratio information of the linear domain reflectance component image at the current pixel. Each channel of the reflectance component image is multiplied pixel-by-pixel with the calculated color restoration coefficient of the corresponding channel. The multiplication result is linearly stretched or truncated to scale the pixel values ​​to the range of 0 to 255, and then converted to a `uint8` unsigned 8-bit integer using the `astype` method to generate a non-reflective enhancement image.

[0030] In one possible implementation, the step of performing color correction on the result of the inverse exponential transform of the reflection component images of each channel using a color restoration function to restore the local color constancy of the image and obtain a reflection-free enhanced image includes: For each color channel of the image, a color restoration function is constructed. The value is calculated by a preset function. The input of the preset function is the pixel value of the channel divided by the sum of the pixel values ​​of the red, green and blue channels. The pixel values ​​are taken from the inverse exponential transform of the reflection component images of each channel obtained at the moment. The color restoration function is multiplied pixel-by-pixel with the result of the inverse exponential transform of the channel reflection component image; The result of multiplication is used as the corresponding color channel of the enhanced image. All channels are combined to obtain a non-reflective enhanced image.

[0031] The logarithmic domain reflection component image obtained in the previous step Transform back to the linear domain using the inverse exponential transform, i.e. For each color channel c∈{R,G,B}, construct a color recovery function. The function is calculated based on the relative intensities of each channel in the inverse exponential transform of the currently obtained reflection component images. For each pixel (x, y), the sum of the intensities of all channels at that point in the linear domain is calculated. In one embodiment, the color restoration function is specifically: ,in It is a constant, with an optimal value of 125, and log is the natural logarithm. Essentially, it adjusts the output using the proportions of the metallic base color after removing the effects of light and reflection, thus preventing the light source color in strongly reflective areas (such as the RGB proportions of highlight white spots) from re-contaminating and graying the restored metallic texture's original color. This involves using the color restoration function. Image of the reflection component in the linear domain Perform pixel-by-pixel multiplication to obtain a color-corrected channel image. Then, combine the three corrected channels... , , The results are merged and subjected to global linear stretching or truncation, scaling pixel values ​​to a standard integer range of [0, 255] to generate a reflective-free enhanced image.

[0032] S8. On the non-reflective enhanced image, a target detection network model is used to identify and locate metal micro-defects.

[0033] A YOLOv5s object detection model based on the PyTorch deep learning framework was implemented. A pre-collected and labeled dataset of anti-reflective enhanced images containing various types of metallic micro-defects was divided into training, validation, and test sets. YOLOv5s model weights pre-trained on the COCO large general dataset were loaded, and transfer learning and fine-tuning were performed on a custom defect dataset. During the detection phase, the real-time acquired anti-reflective enhanced images were scaled to the model's required input size, e.g., 640×640 pixels, and fed into the trained YOLOv5s network for forward propagation. The network outputs the bounding box coordinates, class confidence, and specific defect category for each detected target. Non-maximum suppression (NMS) was applied to filter out overlapping redundant detection boxes, obtaining the location and category information of the metallic micro-defects.

[0034] In one possible implementation, the step of identifying and locating metal micro-defects on the non-reflective enhanced image using a target detection network model includes: The YOLOv5s object detection network model was adopted, and the model has been trained on a dataset of metal surface images labeled with three types of defects: scratches, pits, and rust. The size of the non-reflective enhanced image is uniformly adjusted to 640×640 pixels and normalized. The processed image is then fed into the trained YOLOv5s model for forward propagation inference. Set confidence thresholds and non-maximum suppression thresholds for the model's output, and filter and output the defect category, location bounding box, and confidence score.

[0035] The YOLOv5s model was selected, which is a lightweight object detection network that achieves a good balance between speed and accuracy.

[0036] The YOLOv5s network model comprises three core components: a backbone network, a neck network, and a detection head. The backbone network employs the cross-stage local network CSPDarknet53 to extract multi-level feature maps from the input image. The neck network uses the path aggregation network PANet, which enhances information exchange between feature maps of different scales through a bidirectional feature fusion path from top to bottom and bottom to top, thereby improving the detection capability for defects of different sizes. The detection head is a YOLOv3 detection head that makes predictions on the feature maps at three different scales output by the neck network, with each scale corresponding to the detection of targets of different sizes. The input to the network model is a three-channel image tensor with dimensions of 3×640×640, with pixel values ​​normalized to the [0,1] interval. The output of the network model consists of three prediction tensors at different scales. Each tensor contains the predicted bounding box position, target confidence, and class probability for each grid cell of the feature map at the corresponding scale. The bounding box position information includes the horizontal and vertical coordinates of the center point. and ,width and height Target confidence This represents the probability that an object exists within the bounding box, and the class probability. This represents the probability that, given the existence of a target, the target belongs to the i-th type of defect.

[0037] The dataset already has manually labeled bounding boxes and categories for three typical micro-defects: scratches, dents, and rust. After training, the model weights are fixed. During the detection phase, the anti-reflective enhanced image generated in the algorithm's preceding steps is used as input. Before input, preprocessing is required: the image size is adjusted to 640×640 pixels, and to maintain the original aspect ratio, scaling is performed using gray-edge padding; the image data is normalized from an integer range of [0,255] to a floating-point range of [0.0,1.0], and the data dimensions are converted from HWC (height, width, channels) to the CHW (channels, height, width) format required by the model. The processed image tensor is then fed into the YOLOv5s network with the trained weights for one forward propagation. Post-processing is performed on the original network output: A confidence threshold of 0.5 and a non-maximum suppression threshold of 0.45 are set to filter out all candidate boxes with a confidence score below 0.5, eliminating most background noise and low-probability predictions. Non-maximum suppression (NMS) is applied, and an intersection-over-union (IoU) threshold of 0.45 is set to eliminate duplicate detection boxes for the same defect, retaining only the one with the highest confidence score. The output is a filtered list of defects, with each item clearly indicating the defect category, its bounding box coordinates on the original image, and the confidence score of the detection result.

[0038] The dataset was divided into training, validation, and test sets. The programming language was Python 3.12, the deep learning framework was PyTorch 1.10, and the parallel computing architecture was CUDA 11.3. The detection network was YOLOv5s, the input image size was adjusted to 640×640 pixels, the confidence threshold was set to 0.5, and the cross-union ratio (CUP) threshold for non-maximum suppression was set to 0.45. The mean accuracy (mAP@0.5), precision (P), recall (R), and F1 score at a CUP threshold of 0.5 were used as the evaluation criteria for model performance. Table 1 shows the experimental data.

[0039] Table 1 The baseline scheme, using the original image for detection, achieved a mAP@0.5 of 83.5 percentage points. The complete preprocessing scheme proposed in this invention improves this metric to 92.8 percentage points, achieving a performance gain of 9.3 percentage points. Ablation experiments show that removing or replacing any module in the preprocessing flow leads to a decrease in detection performance. Among them, removing the range adjustment module has the greatest impact on performance, reducing mAP@0.5 to 88.2 percentage points. Replacing anisotropic Gaussian filtering with conventional isotropic filtering, or replacing entropy weight fusion with simple average fusion, also causes performance loss. The absence of the color restoration module also leads to a performance decline, with mAP@0.5 dropping to 91.5 percentage points. The piecewise function enhances the details in the dark areas of the image while compressing the highlight areas, making small defect features that are obscured by insufficient lighting or overexposure clearly visible. This reduces the false negative rate and improves the model's recall and overall mAP. Unlike isotropic filtering, this invention can smooth along the defect edge direction while maintaining sharpness in the vertical edge direction. The edge-preserving property ensures that the defect outline is not blurred when estimating the illumination component, allowing the separated reflection component image to realistically retain the structural information of the defect, thereby improving detection accuracy. By calculating information entropy to assign weights to reflection components at different scales, this invention can fuse the advantageous information of each scale. For textured, small defect regions, smaller-scale components are given higher weights to preserve details; for smooth regions, larger-scale components are given higher weights to suppress noise. Compared to simple averaging fusion, this generates reflection images with better information content, helping the model to better identify defects of different sizes.

[0040] In this invention, an adaptive vision inspection system for inspection robots in highly reflective environments is also proposed, comprising the following modules: The compression module is used to acquire a highly reflective image of the metal surface to be inspected captured by the inspection robot; the color channels of the highly reflective image are adjusted using a piecewise function, pixels below a preset low threshold are linearly stretched, pixels between the low threshold and the high threshold are logarithmically transformed, and pixels above the preset high threshold are power-law compressed to obtain a preprocessed image. The determination module is used to convolve each color channel of the preprocessed image at multiple scales using an anisotropic Gaussian kernel guided by the structure tensor, to obtain the illumination component estimate of each color channel at each scale. The shape and orientation of the anisotropic Gaussian kernel are determined by the eigenvector of the local gradient covariance matrix of the image, so that the convolution is smooth along the image edge direction and suppressed in the vertical edge direction. The fusion module is used to calculate multiple single-scale reflectance component images of each color channel in the logarithmic domain based on the preprocessed image and the illumination component estimation at each scale; for each color channel at each scale, it calculates pixel-level neighborhood information entropy based on the single-scale reflectance component images to generate a multi-scale multi-channel entropy weight map; and uses the multi-scale multi-channel entropy weight map to perform weighted fusion of the multiple single-scale reflectance component images of each color channel to obtain the reflectance component images of each channel. The detection module is used to perform color correction on the result of the inverse exponential transformation of the reflection component images of each channel using the color restoration function, to restore the local color constancy of the image and obtain a non-reflective enhanced image; on the non-reflective enhanced image, a target detection network model is used to identify and locate metal micro-defects.

[0041] In one possible implementation, the step of adjusting the range of each color channel of the highly reflective image using a piecewise function includes: For each color channel: For pixel values ​​below the low threshold The pixels are processed using a linear stretching function, and the output pixel value = input pixel value × 1.5; For pixel values ​​between the low threshold and the high threshold The pixels between these points are processed using a logarithmic transformation function to output pixel values. ; For pixels with values ​​higher than a high threshold, a power-law compression transform function is used for processing, and the output pixel value is... .

[0042] In one possible implementation, the step of convolving each color channel of the preprocessed image using a structure tensor-guided anisotropic Gaussian kernel at multiple scales includes: Three scales are set, corresponding to the basic standard deviation of the Gaussian kernel as 15 for the small scale, 80 for the medium scale, and 200 for the large scale. At each scale, for each pixel in the image, the gradient covariance matrix, i.e. the structure tensor, of the pixel in a dynamic neighborhood that is proportional to the base standard deviation of the current scale is calculated. The structural tensor is decomposed into eigenvalues ​​to obtain the maximum eigenvalue, the minimum eigenvalue, and the corresponding first eigenvector and second eigenvector. The maximum eigenvalue and the minimum eigenvalue are then normalized to the interval between 0 and 1. An anisotropic Gaussian kernel is constructed based on the eigenvalues ​​and eigenvectors, and the anisotropic Gaussian kernel has a first principal axis and a second principal axis; Wherein, the direction of the first principal axis is determined by the second eigenvector corresponding to the minimum eigenvalue, and the standard deviation of the first principal axis is equal to the base standard deviation of the current scale; The direction of the second principal axis is determined by the first eigenvector corresponding to the largest eigenvalue, and the standard deviation of the second principal axis is equal to the base standard deviation divided by (1 + the absolute value of the difference between the normalized largest and smallest eigenvalues).

[0043] In one possible implementation, the step involves calculating pixel-level neighborhood information entropy based on the single-scale reflectance component image for each color channel at each scale, generating a multi-scale, multi-channel entropy weight map, including: For each single-scale reflectance component image of each color channel, the image pixel values ​​are linearly mapped to the integer range of 0 to 255, and a 9×9 neighborhood is taken with each pixel as the center. The probability distribution of pixels at 256 gray levels within the 9×9 neighborhood is statistically analyzed. The information entropy value of the center pixel is calculated according to the Shannon entropy formula, and the information entropy value is multiplied by a reflection suppression coefficient. The reflection suppression coefficient is inversely proportional to the brightness value of the corresponding pixel in the preprocessed image. All pixels are traversed to generate an entropy map of the corresponding scale and channel. Normalize the entropy maps at each scale for each pixel location to obtain the weight maps at each scale. The pixel value of each weight map is equal to the pixel value of the scale entropy map divided by the sum of the pixel values ​​of the small-scale, medium-scale, and large-scale entropy maps at that pixel location.

[0044] In one possible implementation, the step of weighted fusing multiple single-scale reflectance component images of each color channel using the multi-scale multi-channel entropy weighting map to obtain reflectance component images of each channel includes: For each color channel (red, green, and blue), the three scale weight maps of the channel are multiplied at the pixel level with the corresponding single-scale reflectance component image. The results of the multiplication are added pixel by pixel to obtain the reflection component image of the channel. The value is equal to the sum of the products of the weight maps at the small scale, medium scale and large scale and the corresponding single-scale reflection component images.

[0045] In one possible implementation, the step of performing color correction on the result of the inverse exponential transform of the reflection component images of each channel using a color restoration function to restore the local color constancy of the image and obtain a reflection-free enhanced image includes: For each color channel of the image, a color restoration function is constructed. The value is calculated by a preset function. The input of the preset function is the pixel value of the channel divided by the sum of the pixel values ​​of the red, green and blue channels. The pixel values ​​are taken from the inverse exponential transform of the reflection component images of each channel obtained at the moment. The color restoration function is multiplied pixel-by-pixel with the result of the inverse exponential transform of the channel reflection component image; The result of multiplication is used as the corresponding color channel of the enhanced image. All channels are combined to obtain a non-reflective enhanced image.

[0046] In one possible implementation, the step of identifying and locating metal micro-defects on the non-reflective enhanced image using a target detection network model includes: The YOLOv5s object detection network model was adopted, and the model has been trained on a dataset of metal surface images labeled with three types of defects: scratches, pits, and rust. The size of the non-reflective enhanced image is uniformly adjusted to 640×640 pixels and normalized. The processed image is then fed into the trained YOLOv5s model for forward propagation inference. Set confidence thresholds and non-maximum suppression thresholds for the model's output, and filter and output the defect category, location bounding box, and confidence score.

[0047] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referred to each other.

Claims

1. An adaptive vision inspection method for inspection robots in highly reflective environments, characterized in that, Includes the following steps: A strong reflective image of the metal surface to be inspected is acquired by the inspection robot; the range of each color channel of the strong reflective image is adjusted by a piecewise function, pixels below a preset low threshold are linearly stretched, pixels between the low threshold and the high threshold are logarithmically transformed, and pixels above the preset high threshold are power-law compressed to obtain a preprocessed image. At multiple scales, anisotropic Gaussian kernels guided by structure tensor are used to convolve each color channel of the preprocessed image to obtain the illumination component estimates of each color channel at each scale. The shape and orientation of the anisotropic Gaussian kernels are determined by the eigenvectors of the local gradient covariance matrix of the image, so that the convolution is smooth along the image edge direction and suppressed in the vertical edge direction. Based on the preprocessed image and the illumination component estimation at each scale, multiple single-scale reflectance component images of each color channel are calculated in the logarithmic domain. For each color channel at each scale, the pixel-level neighborhood information entropy is calculated based on the single-scale reflectance component image to generate a multi-scale multi-channel entropy weight map; the multi-scale multi-channel entropy weight map is used to perform weighted fusion of multiple single-scale reflectance component images of each color channel to obtain the reflectance component image of each channel. The color restoration function is used to perform color correction on the results of the inverse exponential transformation of the reflection component images of each channel to restore the local color constancy of the image and obtain a non-reflective enhanced image; on the non-reflective enhanced image, a target detection network model is used to identify and locate metal micro-defects.

2. The method according to claim 1, characterized in that, The step of adjusting the range of each color channel of the highly reflective image using a piecewise function includes: For pixel values Below the low threshold The pixels are processed using a linear stretching function, and the output pixel value = input pixel value × 1.5; For pixel values ​​between the low threshold and the high threshold The pixels between these points are processed using a logarithmic transformation function to output pixel values. ; For pixels with values ​​higher than a high threshold, a power-law compression transform function is used for processing, and the output pixel value is... .

3. The method according to claim 1, characterized in that, The step of convolving each color channel of the preprocessed image using a structure tensor-guided anisotropic Gaussian kernel at multiple scales includes: Three scales are set, corresponding to the basic standard deviation of the Gaussian kernel as 15 for the small scale, 80 for the medium scale, and 200 for the large scale. At each scale, for each pixel in the image, the gradient covariance matrix, i.e. the structure tensor, of the pixel in a dynamic neighborhood that is proportional to the base standard deviation of the current scale is calculated. The structural tensor is decomposed into eigenvalues ​​to obtain the maximum eigenvalue, the minimum eigenvalue, and the corresponding first eigenvector and second eigenvector. The maximum eigenvalue and the minimum eigenvalue are then normalized to the interval between 0 and 1. An anisotropic Gaussian kernel is constructed based on the eigenvalues ​​and eigenvectors, and the anisotropic Gaussian kernel has a first principal axis and a second principal axis; Wherein, the direction of the first principal axis is determined by the second eigenvector corresponding to the minimum eigenvalue, and the standard deviation of the first principal axis is equal to the base standard deviation of the current scale; The direction of the second principal axis is determined by the first eigenvector corresponding to the largest eigenvalue, and the standard deviation of the second principal axis is equal to the base standard deviation divided by (1 + the absolute value of the difference between the normalized largest and smallest eigenvalues).

4. The method according to claim 1, characterized in that, The first part refers to each color channel at each scale. Based on the single-scale reflectance component image, pixel-level neighborhood information entropy is calculated to generate a multi-scale, multi-channel entropy weight map, including: For each single-scale reflectance component image of each color channel, the image pixel values ​​are linearly mapped to the integer range of 0 to 255, and a 9×9 neighborhood is taken with each pixel as the center. The probability distribution of pixels at 256 gray levels within the 9×9 neighborhood is statistically analyzed. The information entropy value of the center pixel is calculated according to the Shannon entropy formula, and the information entropy value is multiplied by a reflection suppression coefficient. The reflection suppression coefficient is inversely proportional to the brightness value of the corresponding pixel in the preprocessed image. All pixels are traversed to generate an entropy map of the corresponding scale and channel. Normalize the entropy maps at each scale for each pixel location to obtain the weight maps at each scale. The pixel value of each weight map is equal to the pixel value of the scale entropy map divided by the sum of the pixel values ​​of the small-scale, medium-scale, and large-scale entropy maps at that pixel location.

5. The method according to claim 1, characterized in that, The step of using the multi-scale, multi-channel entropy weighting map to weight and fuse multiple single-scale reflectance component images of each color channel to obtain reflectance component images of each channel includes: For each color channel (red, green, and blue), the three scale weight maps of the channel are multiplied at the pixel level with the corresponding single-scale reflectance component image. The results of the multiplication are added pixel by pixel to obtain the reflection component image of the channel. The value is equal to the sum of the products of the weight maps at the small scale, medium scale and large scale and the corresponding single-scale reflection component images.

6. The method according to claim 1, characterized in that, The step of performing color correction on the result of the inverse exponential transform of the reflection component images of each channel using a color restoration function to restore the local color constancy of the image and obtain a reflection-free enhanced image includes: For each color channel of the image, a color restoration function is constructed. The value is calculated by a preset function. The input of the preset function is the pixel value of the channel divided by the sum of the pixel values ​​of the red, green and blue channels. The pixel values ​​are taken from the inverse exponential transform of the reflection component images of each channel obtained at the moment. The color restoration function is multiplied pixel-by-pixel with the result of the inverse exponential transform of the channel reflection component image; The result of multiplication is used as the corresponding color channel of the enhanced image. All channels are combined to obtain a non-reflective enhanced image.

7. The method according to claim 1, characterized in that, The step of using a target detection network model on the non-reflective enhanced image to identify and locate micro-defects in the metal includes: The YOLOv5s object detection network model was adopted, and the model has been trained on a dataset of metal surface images labeled with three types of defects: scratches, pits, and rust. The size of the non-reflective enhanced image is uniformly adjusted to 640×640 pixels and normalized. The processed image is then fed into the trained YOLOv5s model for forward propagation inference. Set confidence thresholds and non-maximum suppression thresholds for the model's output, and filter and output the defect category, location bounding box, and confidence score.

8. An adaptive vision inspection system for inspection robots in highly reflective environments, characterized in that, Includes the following modules: The compression module is used to acquire a highly reflective image of the metal surface to be inspected captured by the inspection robot; the color channels of the highly reflective image are adjusted using a piecewise function, pixels below a preset low threshold are linearly stretched, pixels between the low threshold and the high threshold are logarithmically transformed, and pixels above the preset high threshold are power-law compressed to obtain a preprocessed image. The determination module is used to convolve each color channel of the preprocessed image at multiple scales using an anisotropic Gaussian kernel guided by the structure tensor, to obtain the illumination component estimate of each color channel at each scale. The shape and orientation of the anisotropic Gaussian kernel are determined by the eigenvector of the local gradient covariance matrix of the image, so that the convolution is smooth along the image edge direction and suppressed in the vertical edge direction. The fusion module is used to calculate multiple single-scale reflectance component images of each color channel in the logarithmic domain based on the preprocessed image and the illumination component estimates at each scale. For each color channel at each scale, the pixel-level neighborhood information entropy is calculated based on the single-scale reflectance component image to generate a multi-scale multi-channel entropy weight map; the multi-scale multi-channel entropy weight map is used to perform weighted fusion of multiple single-scale reflectance component images of each color channel to obtain the reflectance component image of each channel. The detection module is used to perform color correction on the result of the inverse exponential transformation of the reflection component images of each channel using the color restoration function, to restore the local color constancy of the image and obtain a non-reflective enhanced image; on the non-reflective enhanced image, a target detection network model is used to identify and locate metal micro-defects.

9. The system according to claim 8, characterized in that, The step of adjusting the range of each color channel of the highly reflective image using a piecewise function includes: For pixel values ​​below the low threshold The pixels are processed using a linear stretching function, and the output pixel value = input pixel value × 1.5; For pixel values ​​between the low threshold and the high threshold The pixels between these points are processed using a logarithmic transformation function to output pixel values. ; For pixels with values ​​higher than a high threshold, a power-law compression transform function is used for processing, and the output pixel value is... .

10. The system according to claim 8, characterized in that, The step of convolving each color channel of the preprocessed image using a structure tensor-guided anisotropic Gaussian kernel at multiple scales includes: Three scales are set, corresponding to the basic standard deviation of the Gaussian kernel as 15 for the small scale, 80 for the medium scale, and 200 for the large scale. At each scale, for each pixel in the image, the gradient covariance matrix, i.e. the structure tensor, of the pixel in a dynamic neighborhood that is proportional to the base standard deviation of the current scale is calculated. The structural tensor is decomposed into eigenvalues ​​to obtain the maximum eigenvalue, the minimum eigenvalue, and the corresponding first eigenvector and second eigenvector. The maximum eigenvalue and the minimum eigenvalue are then normalized to the interval between 0 and 1. An anisotropic Gaussian kernel is constructed based on the eigenvalues ​​and eigenvectors, and the anisotropic Gaussian kernel has a first principal axis and a second principal axis; Wherein, the direction of the first principal axis is determined by the second eigenvector corresponding to the minimum eigenvalue, and the standard deviation of the first principal axis is equal to the base standard deviation of the current scale; The direction of the second principal axis is determined by the first eigenvector corresponding to the largest eigenvalue, and the standard deviation of the second principal axis is equal to the base standard deviation divided by (1 + the absolute value of the difference between the normalized largest and smallest eigenvalues).

Images