A glass inspection method, system, device, and medium

Abstract

This invention provides a glass inspection method, system, device, and medium, belonging to the field of computer vision technology. The method includes: real-time acquisition of raw images; image correction using a camera calibration file to eliminate lens distortion and crop the region of interest; brightness analysis of the corrected image to obtain the mean brightness, standard deviation, and highlight ratio; adaptive calculation of CLAHE contrast limit, highlight enhancement factor, and sharpening weight based on illumination characteristics; sequential CLAHE enhancement, highlight-oriented enhancement, denoising, and sharpening to generate a preprocessed image; inputting the preprocessed image into a segmentation model to obtain the original segmentation result; performing multi-step post-processing optimization on the segmentation result to obtain an optimized segmentation map; calculating the glass bounding box based on the segmentation map, drawing the segmentation mask and bounding box, and saving the result. This invention achieves high-precision, real-time detection of glass under different illumination conditions through intelligent preprocessing, dedicated reflectivity enhancement, and segmentation optimization.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, and more specifically relates to a glass detection method, system, device and medium. Background Technology

[0002] Glass inspection has significant application value in industrial production, intelligent cleaning of high-rise glass curtain walls, security monitoring, and smart buildings. Accurate and efficient glass inspection is a key technological link in achieving automated operations and intelligent management. Existing glass inspection technologies are mainly divided into two categories: traditional inspection methods and deep learning-based inspection methods.

[0003] Traditional detection methods primarily rely on manual visual inspection or edge detection algorithms based on simple image processing. Manual inspection suffers from inherent drawbacks such as low efficiency, high labor costs, and inconsistent detection standards. Furthermore, prolonged operation can lead to visual fatigue, resulting in missed and false detections. While methods based on simple image processing, such as threshold segmentation and Canny edge detection, can achieve a degree of automation, they are extremely sensitive to changes in lighting and exhibit poor robustness in complex lighting environments, making accurate glass edge identification difficult. Especially in scenes with strong light, low light, or reflections, traditional algorithms often fail to achieve stable detection results.

[0004] In recent years, with the development of deep learning technology, object detection and segmentation models such as YOLO and Mask R-CNN have been gradually applied to the field of glass detection, significantly improving detection accuracy and efficiency. However, existing deep learning-based glass detection methods still face multiple technical challenges in practical applications. First, poor adaptability to lighting conditions is a common problem. Existing methods typically use fixed image preprocessing parameters, such as contrast enhancement intensity and sharpening weights, which cannot be automatically adjusted according to dynamically changing lighting conditions, resulting in large fluctuations in detection accuracy under different environments. Second, the strong reflective properties of glass surfaces can lead to blurred edges. Traditional enhancement methods tend to over-enhance noise when processing reflective areas, which can negatively impact segmentation accuracy. In addition, the segmentation images output by existing segmentation models often have problems such as edge noise and internal holes, lacking specialized post-processing optimizations for glass characteristics, resulting in relatively coarse segmentation results. Finally, lens distortion of industrial or surveillance cameras can cause image distortion, and existing methods often lack real-time camera calibration and correction functions, affecting the geometric accuracy of the detection results.

[0005] In addition to the aforementioned technical issues, existing methods also suffer from significant shortcomings in balancing real-time performance and accuracy. High-precision detection methods typically rely on complex model structures, resulting in poor real-time performance on edge computing devices; while lightweight models can meet real-time requirements, their detection accuracy often falls short of practical application standards. Furthermore, the unique optical properties of glass, such as transmission, reflection, and refraction, make it difficult to distinguish glass areas from the background in images, and existing algorithms lack specific designs to address these characteristics. Summary of the Invention

[0006] To address the above problems, the present invention aims to provide a glass inspection method, system, device, and medium that achieves high-precision, real-time inspection of glass under different lighting conditions through intelligent preprocessing, dedicated reflectivity enhancement, and segmentation optimization.

[0007] To achieve the above objectives, the present invention employs the following technical solution: In a first aspect, embodiments of this application provide a glass testing method, including: The raw images of the area to be tested are acquired in real time using industrial cameras or surveillance cameras. The original image is corrected by loading the camera calibration file, calculating the latest camera matrix and distortion correction mapping table to eliminate the effects of lens distortion, and cropping the image according to the preset region of interest to obtain the corrected image. By performing statistical analysis of brightness information on the corrected image, quantified illumination feature data is generated; the illumination feature data includes the mean brightness, standard deviation, and highlight ratio. Based on the illumination feature data, preprocessing parameters are automatically calculated according to preset adjustment rules; the preprocessing parameters include CLAHE contrast limit, highlight enhancement factor, and sharpening weight. Based on the preprocessing parameters, the corrected image is sequentially subjected to CLAHE enhancement, specular enhancement, denoising and sharpening to generate the preprocessed image. The preprocessed image is input into a preloaded segmentation model for inference, and the NPU of the edge computing platform is used for acceleration to obtain the original segmentation result. The original segmentation result is post-processed and optimized to obtain an optimized segmentation map; Calculate the glass bounding box based on the optimized segmentation map, generate the result image by drawing the segmentation mask and bounding box, and save it.

[0008] In an optional implementation, the step of generating quantized illumination feature data by performing statistical analysis of brightness information on the corrected image includes: The corrected image is converted from the BGR color space to the LAB color space by color space transformation, and its L channel image is extracted as the luminance channel. Calculate the average grayscale value of all pixels in the brightness channel as the average brightness value; Calculate the grayscale standard deviation of all pixels in the brightness channel, and use it as the standard deviation; The number of pixels in the brightness channel whose grayscale value is greater than a preset highlight threshold is counted, and the ratio of this number to the total number of pixels is calculated as the highlight ratio.

[0009] In an optional implementation, preprocessing parameters are automatically calculated based on the illumination feature data according to preset adjustment rules, including: The preprocessing parameters are automatically calculated based on the illumination characteristic data according to the following rules, where the mean brightness is μ, the standard deviation is σ, and the highlight ratio is... : When μ < 80, the formula is used. Calculate the CLAHE contrast limiting parameter CL; where The default CLAHE contrast limit base value; When μ>150, according to the formula Calculate the CLAHE contrast limit parameter CL; when At this time, set the highlight enhancement factor HF to 1.0 to turn off highlight enhancement; when At that time, through the formula Calculate the specular enhancement factor HF; where This is the preset base value for the highlight enhancement factor; When σ < 30, it is calculated using the formula. Sharpening weights (SW); where This is the preset base value for sharpening weights; When σ > 60, according to the formula Calculate the sharpening weight SW.

[0010] In an optional implementation, the step of sequentially performing CLAHE enhancement, specular enhancement, denoising, and sharpening on the corrected image according to the preprocessing parameters to generate a preprocessed image includes: The corrected image is converted from the BGR color space to the LAB color space by color space transformation, and the L channel image, a channel image and b channel image are separated. The specular threshold is calculated using the following formula based on the mean luminance μ and the standard deviation σ. :

[0011] Where k is the preset highlight threshold multiplier; Based on the L-channel image, a specular mask is generated using the following formula. :

[0012] The L-channel image is enhanced using CLAHE, and the enhanced L-channel image is obtained by the following formula. :

[0013] Where CL is the CLAHE contrast limiting parameter; Through the specular mask Identify highlight regions in the enhanced L-channel image. The highlight regions in the image are targeted by the aforementioned highlight enhancement factor HF, and the enhanced L-channel image is obtained by the following formula. :

[0014] The L channel image after highlight enhancement The image is merged with the a-channel image and the b-channel image, and then converted back to the BGR color space through color space transformation to obtain the first intermediate image. ; For the first intermediate image Denoising is achieved by applying bilateral filtering to obtain the second intermediate image. ; For the second intermediate image Edge enhancement is performed by generating a blurred image using Gaussian blur. Then, using unsharpening masking techniques, the enhancement intensity is controlled by the sharpening weight SW, and the third intermediate image is obtained through the following formula. :

[0015] For the third intermediate image Gaussian blur is applied to remove sharpened noise, resulting in the fourth intermediate image. ; For the fourth intermediate image Perform letterboxing to generate a pre-processed image by adjusting the image size to a preset value while maintaining the aspect ratio.

[0016] In an optional implementation, the step of inputting the preprocessed image into a preloaded segmentation model for inference, and accelerating the process using the NPU of an edge computing platform to obtain the original segmentation result, includes: Load and initialize the segmentation model via the RKNN Lite API, and enable NPU multi-core acceleration; The preprocessed image is converted to RGB format and input into the loaded segmentation model for inference; Obtain the raw segmentation results output by the model, which include predicted bounding boxes, class confidence scores, and segmentation prototypes.

[0017] In an optional implementation, the post-processing optimization of the original segmentation result to obtain an optimized segmentation map includes: An initial segmentation map is generated by applying matrix multiplication to the segmentation prototype; The initial segmentation map is converted into a probability map by applying the sigmoid activation function. The size of the probability map is adjusted to a preset value by interpolation to generate a first optimized intermediate map; The first optimized intermediate image is cropped based on the predicted bounding box, retaining the target region, to generate the second optimized intermediate image. Gaussian blur is applied to smooth the second optimized intermediate image to generate a third optimized intermediate image; The third optimized intermediate image is binarized to generate the fourth optimized intermediate image; Morphological operations are performed on the fourth optimized intermediate image, using a 5×5 core to perform closing operations to fill internal holes and opening operations to remove noise, generating the fifth optimized intermediate image. The fifth optimized intermediate graph is smoothed by applying mean filtering and bilateral filtering in sequence to generate the sixth optimized intermediate graph; The Otsu algorithm is used to adaptively threshold the sixth optimized intermediate image, automatically determine the optimal segmentation threshold, and generate the optimized segmentation image.

[0018] In an optional implementation, the glass bounding box is calculated based on the optimized segmentation map, and a result image is generated and saved by drawing a segmentation mask and the bounding box, including: Connectivity analysis is performed on the optimized segmentation graph. All connected components are extracted using a contour extraction algorithm, and the minimum bounding rectangle of each connected component is calculated as the glass bounding box. The optimized segmentation image is converted into a color mask image, and the color mask image is superimposed on the preprocessed image in a semi-transparent manner through alpha fusion to generate a result image with segmentation mask; On the resulting image, draw a rectangle based on the glass bounding box, and label the rectangle with the corresponding category information and confidence score; The resulting image is saved to a local storage device in a specified format, and simultaneously output to a display device in real time via a display interface. The detection results are encapsulated into structured data through a standardized data interface and sent to an external system for further processing or display. The detection results include glass bounding box coordinates, category labels, confidence scores, and optimized segmentation maps.

[0019] Secondly, embodiments of this application also provide a glass inspection system, including: The image acquisition module is used to acquire raw images of the area to be tested in real time using an industrial camera or a surveillance camera. The camera correction module is used to correct the original image. By loading the camera calibration file, it calculates the latest camera matrix and distortion correction mapping table to eliminate the influence of lens distortion, and performs image cropping according to the preset region of interest to obtain the corrected image. The brightness analysis module is used to generate quantified illumination feature data by performing statistical analysis of brightness information on the corrected image; the illumination feature data includes the mean brightness, standard deviation, and highlight ratio. The parameter calculation module is used to automatically calculate preprocessing parameters based on the illumination feature data and through preset adjustment rules; the preprocessing parameters include CLAHE contrast limit, highlight enhancement factor, and sharpening weight. The image preprocessing module is used to sequentially perform CLAHE enhancement, specular enhancement, denoising and sharpening on the corrected image according to the preprocessing parameters to generate the preprocessed image; The model inference module is used to input the preprocessed image into a preloaded segmentation model for inference, and to accelerate the process using the NPU of the edge computing platform to obtain the original segmentation result. The segmentation optimization module is used to perform post-processing optimization on the original segmentation result to obtain an optimized segmentation map. The result output module is used to calculate the glass bounding box based on the optimized segmentation map, generate the result image by drawing the segmentation mask and bounding box, and save it.

[0020] Thirdly, embodiments of this application also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the glass detection method as described in any of the above.

[0021] Fourthly, embodiments of this application also provide a storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the glass detection method as described in any of the above claims.

[0022] As can be seen from the above technical solutions, the present invention has the following advantages: The glass detection method provided in this application significantly improves the accuracy, robustness, and real-time performance of glass detection through fully automated processing and intelligent adaptive adjustment mechanisms. First, the method integrates camera calibration and correction to eliminate the impact of lens distortion on detection accuracy at the source. Then, through the linkage of brightness analysis and parameter calculation, it dynamically adjusts the CLAHE contrast limit, highlight enhancement factor, and sharpening weight based on the image's mean brightness, standard deviation, and highlight ratio, achieving adaptation of preprocessing parameters to the lighting environment and solving the problem of poor lighting adaptability caused by fixed parameters in traditional methods. Simultaneously, by generating a highlight mask and performing targeted enhancement only on highlight areas, it effectively suppresses background noise amplification while highlighting glass edge features, overcoming the problem of severe glass reflection interference. Furthermore, through a series of post-processing optimization steps such as Gaussian smoothing, morphological operations, multi-level filtering, and Otsu adaptive thresholding, the method significantly improves the edge clarity and regional integrity of the segmentation image, solving the problem of coarse original segmentation results. Combined with the multi-core acceleration of the NPU on the edge computing platform, it meets real-time processing requirements while ensuring high-precision inference. Ultimately, this method can stably output result images with accurate bounding boxes and segmentation masks, and supports data encapsulation and interface with external systems. It can be widely used in complex scenarios such as industrial production, intelligent cleaning of high-altitude glass curtain walls, and security monitoring.

[0023] This application automatically calculates and adjusts preprocessing parameters (such as CLAHE contrast limit, highlight enhancement factor, and sharpening weight) by analyzing the brightness characteristics of images in real time (e.g., mean brightness, standard deviation, highlight ratio, and dark area ratio), achieving adaptability to different lighting conditions. For example, it enhances contrast in dark images, reduces enhancement intensity in bright environments, and enhances sharpness in low-contrast scenes. Compared to existing technologies that use fixed preprocessing parameters (which are prone to accuracy fluctuations in strong light, low light, or reflective environments), this application can maintain stable detection performance under various lighting conditions, significantly improving the system's environmental adaptability.

[0024] This application enhances the highlight areas (enhancing only areas with brightness exceeding a threshold) in a targeted manner, combined with bilateral filtering for noise reduction, thus highlighting the reflective characteristics of glass while avoiding excessive noise enhancement. Compared to existing technologies that uniformly enhance the entire image (which easily leads to noise amplification and blurred glass edges), this application effectively improves the clarity of glass edges, reduces missed detections caused by reflective interference, and enhances the ability to identify glass targets.

[0025] This application performs multi-step optimization processing on the output of the segmentation model, including Gaussian blur smoothing, morphological closing operation to fill internal holes, opening operation to remove noise, median filtering and bilateral filtering for further smoothing, and finally Otsu thresholding to generate a binary segmentation map. Compared with the output of existing segmentation models (which often have problems such as edge noise and internal holes), this application can significantly improve the edge smoothness and region integrity of the segmentation map, providing a more accurate basis for bounding box calculation.

[0026] This application integrates a real-time camera calibration and correction process. It loads the camera calibration file, calculates the optimal new camera matrix and distortion correction mapping table, performs distortion correction on the acquired image, and crops the effective area. Compared with existing technologies that often lack camera calibration (which are susceptible to image distortion and detection result offset due to lens distortion), this application can effectively eliminate the impact of camera distortion on detection, improve the accuracy of detection results, and is suitable for wide-angle lens scenarios such as industrial cameras.

[0027] This application optimizes the algorithm process (such as lightweighting parameter calculation and parallel processing of segmentation optimization) and adapts to the hardware acceleration capabilities of edge computing platforms (such as using NPU for model inference), reducing computational complexity while ensuring detection accuracy. Compared to existing high-precision segmentation methods (which have poor real-time performance on edge devices), this application can achieve efficient and real-time glass detection on edge devices, meeting the real-time requirements of scenarios such as industrial production lines and security monitoring. Attached Figure Description

[0028] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the description will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 This is a flowchart illustrating the glass testing method provided in this application.

[0030] Figure 2 This is a schematic diagram of the glass inspection system provided in this application.

[0031] Figure 3 A schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation

[0032] The various embodiments of this disclosure will be described more fully in the detailed steps of the glass testing method described below. This disclosure may have various embodiments, and adjustments and changes may be made therein. However, it should be understood that there is no intention to limit the various embodiments of this disclosure to the specific embodiments disclosed herein, but rather this disclosure should be understood to cover all adjustments, equivalents, and / or alternatives falling within the spirit and scope of the various embodiments of this disclosure.

[0033] In the following, the terms “comprising” or “may include”, which may be used in various embodiments of this disclosure, indicate the presence of the disclosed functions, operations, or elements, and do not limit the addition of one or more functions, operations, or elements. Furthermore, as used in various embodiments of this disclosure, the terms “comprising,” “having,” and their cognates are intended only to indicate a particular feature, number, step, operation, element, component, or combination of the foregoing, and should not be construed as primarily excluding the presence of one or more other features, numbers, steps, operations, elements, components, or combinations of the foregoing, or the possibility of adding one or more combinations of the foregoing.

[0034] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0035] Please see Figure 1 The diagram shown is a flowchart of a glass testing method in a specific embodiment. The method includes: S1: Acquire raw images of the area to be tested in real time using an industrial camera or surveillance camera.

[0036] In a specific implementation, raw images are acquired in real time using industrial cameras or surveillance cameras deployed in the area to be tested. Industrial cameras, such as high-resolution global shutter cameras, are suitable for high-speed glass inspection scenarios, such as quality inspection on glass production lines; surveillance cameras are suitable for glass inspection in fixed scenarios, such as periodic inspections of high-rise glass curtain walls. The cameras can be set with appropriate frame rates and resolutions according to actual needs, for example, acquiring RGB images with a resolution of 1920×1080 to ensure the accuracy of subsequent processing.

[0037] S2: Correct the original image by loading the camera calibration file, calculating the latest camera matrix and distortion correction mapping table to eliminate the influence of lens distortion, and cropping the image according to the preset region of interest to obtain the corrected image. In a specific implementation, lens distortion can negatively impact the accuracy of subsequent detection if the original image is used directly. Therefore, the original image needs to be corrected. First, a calibration file obtained beforehand through camera calibration is loaded, containing the camera's intrinsic parameter matrix and distortion coefficients. Based on these parameters, an optimal new camera matrix and distortion correction mapping table are calculated. Specifically, for each pixel in the original image, its corresponding position in the corrected image is found using the distortion correction mapping table, and the corrected image is generated using bilinear interpolation. After correction, the image is cropped according to a preset region of interest (ROI), removing useless edge areas and retaining the effective area containing the glass target, resulting in the corrected image. This step effectively eliminates the impact of lens distortion on detection accuracy and reduces the computational load of subsequent processing.

[0038] S3: By performing statistical analysis of brightness information on the corrected image, quantified illumination feature data is generated; the illumination feature data includes the mean brightness, standard deviation, and highlight ratio. In a specific implementation, to adaptively adjust subsequent preprocessing parameters, statistical analysis of brightness information is required on the corrected image. First, the corrected image is converted from the BGR color space to the LAB color space through color space transformation, and its L channel image is extracted as the brightness channel. The LAB color space has the characteristic of separating brightness and color, and can more accurately reflect the brightness distribution of the image. Then, the average grayscale value of all pixels in the L channel image is calculated as the average brightness value. Calculate the grayscale standard deviation of all pixels, and use it as the standard deviation. Simultaneously, count the number of pixels in the L-channel image whose grayscale value is greater than a preset highlight threshold (e.g., 200, which can be adjusted according to the actual scene), and calculate the ratio of this number to the total number of pixels, as the highlight ratio. These three quantitative indicators comprehensively characterize the illumination features of the image, providing a data foundation for subsequent parameter calculations.

[0039] S4: Based on the illumination feature data, preprocessing parameters are automatically calculated using preset adjustment rules; the preprocessing parameters include CLAHE contrast limit, highlight enhancement factor, and sharpening weight.

[0040] In a specific implementation, based on the illumination characteristic data obtained in S3, preprocessing parameters are automatically calculated according to preset adjustment rules. These rules aim to dynamically adapt to different illumination conditions, as follows: For CLAHE contrast limiting parameters : Preset base value (e.g., 3.0). When When the image is too dark, it indicates that the contrast needs to be increased. The calculation formula is:

[0041] when When the image is too bright, it indicates that the contrast needs to be reduced. The calculation formula is:

[0042] in and Function ensures Within a reasonable range, avoid excessive enhancement or weakening.

[0043] For highlight enhancement factor : Preset base value (e.g., 1.5). When This indicates that there are too many highlight areas in the image. To prevent over-enhancement from amplifying noise, [the image should be edited]. Set to 1.0 to turn off highlight enhancement; when If the highlight area is too small, it indicates that the highlights need to be enhanced to highlight the glass edges. The calculation formula is:

[0044] For sharpening weights : Preset base value (e.g., 1.0). When When the image contrast is low, it indicates that sharpening is needed. The calculation formula is:

[0045] when When the image contrast is too high, sharpening needs to be reduced. The calculation formula is:

[0046] By applying the above rules, the preprocessing parameters can be adaptively adjusted according to the illumination characteristics, providing the optimal parameter combination for subsequent image preprocessing.

[0047] S5: Based on the preprocessing parameters, perform CLAHE enhancement, specular enhancement, denoising, and sharpening on the corrected image in sequence to generate the preprocessed image.

[0048] In a specific implementation, after obtaining the adaptive parameters, a series of preprocessing operations are performed on the corrected image to improve image quality and highlight glass features. The specific implementation steps are as follows: S501, Color Space Conversion and Channel Separation: The corrected image is converted from the BGR color space to the LAB color space through color space transformation, and the L channel image, a channel image, and b channel image are separated. Subsequent processing mainly targets the L channel, because brightness information is crucial for enhancing the glass edges.

[0049] S502, Spectrum Mask Generation: Based on the average brightness value obtained in S3 and standard deviation Calculate the specular threshold:

[0050] in The preset specular threshold multiplier (e.g., 2.0, which can be adjusted according to the actual scene) is used to generate a specular mask based on the L-channel image. Its definition is: for each pixel ,like ,but If the pixel is in the highlight region, then it belongs to the highlight region; otherwise... This mask is used for subsequent targeted enhancement to ensure that only the highlight areas are processed.

[0051] S503, CLAHE Enhancement: Apply CLAHE (Contrast Limiting Adaptive Histogram Equalization) enhancement to the L channel image, and input the CLAHE contrast limiting parameters calculated in S4. The enhanced L-channel image is obtained. :

[0052] CLAHE can enhance local contrast while limiting noise amplification. (Parameters) Control the intensity of contrast enhancement.

[0053] S504, Specular Directional Enhancement: Through specular masking Identify highlight regions in the enhanced L-channel image. The highlight enhancement factor was calculated using S4 in the highlight region. Perform targeted enhancement. Specifically, for each pixel... ,

[0054] This step highlights the features of the glass edges while avoiding excessive amplification of noise in non-highlight areas.

[0055] S505, Channel Merging and Color Space Conversion: Improving the L channel image after highlight enhancement. The image is merged with the original a-channel and b-channel images, and then converted back to the BGR color space through color space transformation to obtain the first intermediate image. .

[0056] S506, Bilateral filtering for noise reduction: For the first intermediate image... Denoising is achieved by applying a bilateral filter, with the diameter of the bilateral filter set to 7, the color sigma set to 20, and the spatial sigma set to 20, resulting in a second intermediate image. Bilateral filtering can smooth noise while preserving edge information, effectively removing subtle noise from images.

[0057] S507, Unsharpened Masking Enhancement: For the second intermediate image Edge enhancement is performed. First, a blurred image is generated using Gaussian blur. The Gaussian kernel's sigma is set to 3.0. Then, using an unsharpened masking technique, the sharpening weights calculated by S4 are applied. By controlling the enhancement intensity, a third intermediate image is obtained. The calculation formula is:

[0058] This step enhances the edge details of the image, making the glass outline clearer.

[0059] S508, Remove sharpening noise: For the third intermediate image A slight Gaussian blur is applied to remove sharpening noise, with the sigma of the Gaussian blur set to 0.8, resulting in the fourth intermediate image. This step can eliminate minor noise caused by sharpening.

[0060] S509, Size Adjustment: Adjustment of the fourth intermediate image Perform letterbox processing by adjusting the image size to the model input size. It maintains the aspect ratio to generate the final preprocessed image. Letterbox processing ensures that the image content is not distorted by filling the image edges with gray borders, while also meeting the model input requirements.

[0061] Through the above preprocessing process, the generated preprocessed image has better contrast, clearer edges and less noise, providing high-quality input for subsequent model inference.

[0062] S6: Input the preprocessed image into the preloaded segmentation model for inference, and use the NPU of the edge computing platform for acceleration to obtain the original segmentation result.

[0063] In this specific implementation, the preprocessed image is input into a pre-loaded segmentation model for inference. This embodiment uses the YOLO11-seg segmentation model and optimizes it for the RK3588 edge computing platform. The specific implementation steps are as follows: S601, Model Loading and NPU Acceleration: Load and initialize the segmentation model through the RKNN Lite API, and enable NPU multi-core acceleration. Set the acceleration mode to CORE_0_1_2, which uses three NPU cores for parallel computation to maximize inference speed.

[0064] S602. Input Preprocessing: Convert the preprocessed image to RGB format (if the original is BGR), and input it into the loaded segmentation model for inference. The model input size is... The batch size is 1 to accommodate real-time detection requirements.

[0065] S603. Obtain the raw segmentation results: The model output includes predicted bounding boxes, class confidence scores, and segmentation prototypes. The predicted bounding boxes are used to locate the glass target, the class confidence scores are used to determine the target category (e.g., glass), and the segmentation prototypes are used to generate a fine-grained segmentation map.

[0066] With NPU acceleration, the model inference speed is greatly improved, which can meet the real-time requirements while ensuring high accuracy, making it suitable for industrial production and other scenarios.

[0067] S7: Perform post-processing optimization on the original segmentation result to obtain the optimized segmentation map.

[0068] In specific implementations, the raw segmentation results output by the model often suffer from issues such as edge noise and internal holes, requiring post-processing optimization to improve segmentation quality. The specific optimization steps are as follows: S701. Generate initial segmentation map: Generate an initial segmentation map corresponding to the size of the input image by applying matrix multiplication to the segmentation prototype.

[0069] S702, Probabilistic Map Transformation: Apply the sigmoid activation function to the initial segmentation map to transform it into a probabilistic map, where each pixel value represents the probability that the point belongs to the glass target.

[0070] S703, Size Adjustment: Adjust the size of the probability map using interpolation. (Model input size), generate the first optimized intermediate image, and align it with the original input image size.

[0071] S704. Boundary box cropping: Based on the predicted boundary box obtained in step S6, the first optimized intermediate image is cropped to retain the target area, remove background interference, and generate the second optimized intermediate image.

[0072] S705, Gaussian Blur Smoothing: Apply Gaussian blur to the second optimized intermediate image for smoothing, where the Gaussian blur kernel is set to... With sigma set to 1.0, a third optimized intermediate image is generated. This step eliminates jagged edges and noise at the segmentation edges.

[0073] S706. Binarization Processing: Perform binarization processing on the third optimized intermediate image, set the threshold to 0.5, that is, pixels with a probability greater than 0.5 are regarded as targets, otherwise they are background, and generate the fourth optimized intermediate image.

[0074] S707, Morphological Operations: Perform morphological operations on the fourth optimized intermediate image, first using... The kernel performs a closing operation (first expansion, then erosion) to fill the internal voids; then it employs... The kernel is opened (corrosion followed by expansion) to remove isolated noise points and generate the fifth optimized intermediate image.

[0075] S708, Multi-stage Filtering and Smoothing: The fifth optimized intermediate image is smoothed sequentially using median filtering and bilateral filtering. The median filter kernel is set to... This effectively removes salt and pepper noise; setting the bilateral filter diameter to 9, the color sigma to 75, and the spatial sigma to 75 can preserve edge details while smoothing the image, generating the sixth optimized intermediate image.

[0076] S709, Adaptive Thresholding: The Otsu algorithm is used to adaptively threshold the sixth optimized intermediate image, automatically determining the optimal segmentation threshold and generating the final optimized segmentation image. The Otsu algorithm can dynamically select the threshold based on the image's grayscale distribution, further improving segmentation accuracy.

[0077] Through the above multi-step optimization, the resulting segmentation map has clear edges, complete regions, no holes or noise, and can be directly used for subsequent result output.

[0078] S8: Calculate the glass bounding box based on the optimized segmentation map, generate the result image by drawing the segmentation mask and bounding box, and save it.

[0079] In a specific implementation, the glass bounding box is calculated based on the optimized segmentation map, and a visualization result is generated. The specific implementation steps are as follows: S801, Boundary box calculation: Perform connected component analysis on the optimized segmentation graph, extract all connected components using a contour extraction algorithm (such as OpenCV's findContours), and calculate the minimum bounding rectangle of each connected component as the glass bounding box.

[0080] S802, Mask Overlay: Convert the optimized segmentation image into a color mask image (e.g., using semi-transparent green), and overlay the color mask image onto the preprocessed image generated in S5 in a semi-transparent manner through alpha fusion to generate a result image with a segmentation mask.

[0081] S803. Bounding box drawing and annotation: On the result image, draw a rectangle based on the glass bounding box calculated in step 1, and annotate the corresponding category information (such as "glass") and confidence score (from model output) next to the rectangle to visually display the detection results.

[0082] S804. Result Saving and Display: The completed result image is saved to the local storage device in a specified format (such as JPEG or PNG), and simultaneously output to the display device in real time through the display interface for the operator to view.

[0083] S805, Data Integration: The detection results are encapsulated into structured data, including glass bounding box coordinates, category labels, confidence scores, and optimized segmentation maps, through standardized data interfaces (such as REST API or MQTT), and sent to external systems for further processing or display, such as for automatic sorting on production lines or path planning for glass cleaning robots.

[0084] Through the above steps, this invention realizes fully automated glass inspection from image acquisition to result output. It has the advantages of strong light adaptability, good reflection suppression effect, high segmentation accuracy, and strong real-time performance. It can be widely used in industrial production, intelligent cleaning of high-altitude glass curtain walls, security monitoring and other fields.

[0085] In this embodiment, the model first performs camera distortion correction on the acquired raw image to ensure the geometric accuracy of the input image. Then, through brightness analysis and adaptive parameter calculation, the CLAHE contrast limit, highlight enhancement factor, and sharpening weight can be dynamically adjusted according to the lighting characteristics, solving the problem of poor adaptability of fixed parameters in different environments. On this basis, through highlight mask generation and directional enhancement technology, only the highlight area is processed, effectively suppressing reflection interference while highlighting the glass edge. Subsequently, the original segmentation result output by the model is refined and optimized in multiple steps, significantly improving the edge clarity and regional integrity of the segmentation map. Finally, combined with the multi-core acceleration of the NPU of the edge computing platform, the real-time processing requirements are met while ensuring high detection accuracy. This provides a high-precision, robust, and real-time glass detection solution for scenarios such as industrial production, high-altitude cleaning, and security monitoring.

[0086] like Figure 2 As shown, the following are embodiments of the glass testing system provided in this disclosure. This system and the glass testing methods in the above embodiments belong to the same inventive concept. For details not described in detail in the embodiments of the glass testing system, please refer to the embodiments of the glass testing methods described above.

[0087] A glass inspection system, comprising: The image acquisition module is used to acquire raw images of the area under test in real time using an industrial camera or a surveillance camera, providing data input to the system. The output is connected to the camera calibration module.

[0088] The camera calibration module is used to correct the original image. By loading the camera calibration file, it calculates the latest camera matrix and distortion correction mapping table to eliminate the influence of lens distortion, and performs image cropping according to the preset region of interest to obtain the corrected image. Its output is connected to the brightness analysis module. The brightness analysis module is used to generate quantified illumination feature data by performing statistical analysis of brightness information on the corrected image. The illumination feature data includes the mean brightness, standard deviation, and highlight ratio. Specifically, the corrected image is converted to the LAB color space, the L channel is extracted, and the mean brightness, standard deviation, highlight ratio, and shadow ratio are calculated to quantify the illumination features. Its output is connected to the parameter calculation module.

[0089] The parameter calculation module is used to automatically calculate preprocessing parameters based on the illumination feature data and preset adjustment rules. These preprocessing parameters include CLAHE contrast limit, highlight enhancement factor, and sharpening weight. This module automatically adjusts the preprocessing parameters (CLAHE contrast limit, highlight enhancement factor, sharpening weight, etc.) according to brightness characteristics to achieve adaptive lighting conditions. Its output is connected to the image preprocessing module.

[0090] The image preprocessing module is used to sequentially perform CLAHE enhancement, specular directional enhancement, denoising, and sharpening on the corrected image according to the preprocessing parameters to generate a preprocessed image.

[0091] In a specific implementation, the image preprocessing module can perform CLAHE enhancement, specular enhancement, denoising, and sharpening on the image based on automatically calculated parameters to generate a high-quality input image and improve the model inference accuracy. Its output is connected to the model inference module.

[0092] The image preprocessing module is specifically used for: Convert the image to the LAB color space and separate the L, a, and b channels.

[0093] CLAHE enhancement is applied to the L channel using automatically calculated contrast limiting parameters.

[0094] Calculate the highlight threshold (mean brightness + standard deviation * highlight threshold multiplier) and generate a highlight mask.

[0095] Targeted enhancement is applied to highlight areas using an automatically calculated highlight enhancement factor.

[0096] Merge the channels and convert them back to the BGR color space.

[0097] Denoising was achieved using a bilateral filter (diameter=7, color sigma=20, space sigma=20).

[0098] Enhance edges using Gaussian blur (sigma=3.0) and unsharpened masking.

[0099] Apply a slight Gaussian blur (sigma=0.8) to remove sharpening noise.

[0100] Perform letterbox processing to adjust the image size to the model input size (640x640) while maintaining the aspect ratio.

[0101] The model inference module is used to input the preprocessed image into a preloaded segmentation model for inference, and to accelerate the process using the NPU of the edge computing platform to obtain the original segmentation result.

[0102] In a specific implementation, a segmentation model (such as YOLO11-seg) is loaded, and the preprocessed image is input into the model for inference. The NPU of an edge computing platform (such as RK3588) is used for acceleration to obtain the original segmentation result. The output of the model inference module is connected to the segmentation optimization module. Specifically, the RKNN Lite API is used to load and initialize the model, and multi-core NPU acceleration (CORE_0_1_2) is enabled. The model input is in RGB format, 640x640 pixels, with a batch size of 1. The model output includes predicted bounding boxes, class confidence scores, and segmentation prototypes.

[0103] The segmentation optimization module is used to perform post-processing optimization on the original segmentation results to obtain an optimized segmentation map.

[0104] In a specific implementation, the segmentation optimization module is used to perform Gaussian blurring, morphological operations, filtering, and thresholding on the segmentation results to optimize the edge sharpness and region integrity of the segmentation image. Its output is connected to the result output module.

[0105] The segmentation optimization module is specifically used for: Matrix multiplication is applied to the segmentation prototype to generate a segmentation graph.

[0106] The segmentation map is converted into a probability map by applying the sigmoid activation function.

[0107] Adjust the size of the segmentation image to the input dimensions (640x640).

[0108] The segmentation diagram is cut based on the bounding box.

[0109] Apply Gaussian blur (5x5 kernel, sigma=1.0) to smooth the segmentation map.

[0110] Binarized segmentation image (threshold=0.5).

[0111] Morphological operations: Closing operation (5x5 cores) fills internal holes, opening operation (5x5 cores) removes noise.

[0112] Median filtering (5x5 kernels) and bilateral filtering (diameter=9, color sigma=75, space sigma=75) further smooth the surface.

[0113] The results output module calculates the glass bounding box based on the optimized segmentation map, generates a result image by drawing a segmentation mask and bounding box, and saves it. This module supports real-time display and integration with external systems, and is the final output stage of the system.

[0114] Furthermore, as a refinement and extension of the specific implementation of the above embodiments, in order to fully illustrate the specific implementation process of the glass inspection system, an example is provided to illustrate the specific configuration, deployment and operation process of the glass inspection system based on the RK3588 development board and using a custom-trained glass inspection model to achieve real-time, high-precision glass inspection.

[0115] 1. The specific hardware configuration of this system is as follows: Processor: Xunwei RK3588 development board (8-core CPU, integrated NPU and GPU, NPU computing power 6 TOPS) Memory: 8GB Camera: OV13850 (13 megapixels, 1920x1920 resolution, 30 FPS, MIPI interface) Interfaces: Gigabit Ethernet, USB 3.0, HDMI (display), MIPICSI (camera connection) 2. The specific software environment of this system is as follows: Operating System: Ubuntu 20.04LTS (ARM64 architecture) Deep learning frameworks: RKNNToolkit 2.0 (model transformation), RKNNRuntime (model inference) Image processing library: OpenCV Programming language: Python 3. The specific deployment steps for this system are as follows: Step 1: Camera calibration.

[0116] Objective: To obtain camera intrinsic parameters and distortion coefficients for subsequent distortion correction.

[0117] The operation process includes: Print an A4 checkerboard calibration board (square size 20mm) and fix it to a flat surface.

[0118] Use an OV13850 camera to capture images of the calibration board from different angles (at least 15 viewpoints) to ensure the checkerboard pattern fills the frame and is clear.

[0119] The acquired calibration board images are processed using OpenCV calibration tools to calculate camera intrinsic parameters, distortion coefficients, and image size, and to generate calibration files.

[0120] Save the generated calibration file as "calib_result_1920x1920.npz", which contains the camera matrix (camera_matrix), distortion coefficients (dist_coeffs), and image size (image_size).

[0121] Copy the calibration file to the specified directory on the system.

[0122] Step 2: Model preparation.

[0123] First, we perform custom model training, including: Dataset preparation: Collect glass images (at least 2000 images) under different lighting conditions, and annotate the bounding boxes and segmentation masks of the glass targets.

[0124] Model training: Based on the YOLO11-seg framework, the category configuration was modified to recognize only the "glass" category, and appropriate batch size, number of training rounds and input size were set for training.

[0125] Model export: After training is complete, export the model file.

[0126] Then the model is converted to RKNN format, including: Use the RKNN Toolkit to convert your custom-trained model to RKNN format and enable quantization to improve inference speed.

[0127] Copy the generated RKNN model file to the specified directory on the system.

[0128] Step 3: System setup.

[0129] Install dependencies: Install system dependencies such as Python and OpenCV using the apt package manager.

[0130] Install Python dependency libraries such as NumPy and SciPy using pip.

[0131] Install the RKNN Runtime library for model inference.

[0132] Deployment code: Copy the code used to implement the glass detection method to the system's designated directory, including the main program (cap_test.py), camera calibration files, and model files.

[0133] The parameters in the configuration code include the camera device path, model path, calibration file path, etc.

[0134] Step 4: Parameter configuration.

[0135] Modify configuration parameters: Adjust the following parameters according to the actual hardware environment and scenario requirements: Camera calibration configuration: USE_CALIBRATION (whether to enable calibration), CALIB_FILE (calibration file path), CALIB_ALPHA (cropping parameters).

[0136] Model-related configurations: MODEL_PATH (model path), TARGET_PLAT (target platform).

[0137] Camera-related configurations: CAMERA (camera device path), CAMERA_RES (resolution), CAMERA_FPS (frame rate).

[0138] Inference-related configurations: IMG_SIZE (input size), OBJ_THRESH (confidence threshold), NMS_THRESH (non-maximum suppression threshold).

[0139] Image preprocessing configurations: PREPROC_CLAHE_CLIP (CLAHE contrast limit), PREPROC_HIGHLIGHT_FACTOR (highlight enhancement factor), etc.

[0140] 4. System operation process: Step 1: Initialize the system.

[0141] The startup message indicates that the RK3588 development board has been successfully launched, and the Ubuntu 20.04 LTS operating system has been logged in.

[0142] Connect the OV13850 camera to the MIPI CSI interface to ensure the camera is correctly recognized by the system.

[0143] Step 2: Start the detection program.

[0144] Once the main program is run, the system will automatically execute the core processing flow.

[0145] Step 3: Core Processing Flow: A. Camera distortion correction: Load the calibration file, calculate the optimal new camera matrix and distortion correction mapping table, correct the acquired image and crop the effective area.

[0146] B. Brightness Analysis: Convert the corrected image to the LAB color space, extract the L channel, and calculate the mean brightness, standard deviation, highlight ratio, and shadow ratio.

[0147] C. Parameter Calculation: Automatically adjust preprocessing parameters (such as CLAHE contrast limit, highlight enhancement factor, and sharpening weight) based on brightness characteristics.

[0148] D. Image preprocessing: CLAHE enhancement is performed on the L channel, highlight areas are enhanced in a targeted manner, and noise reduction and sharpening are performed after merging the channels.

[0149] E. Model Inference: Adjust the preprocessed image to the model input size, input it into the custom-trained RKNN model for inference, and use the NPU to accelerate the acquisition of segmentation results.

[0150] F. Segmentation Optimization: Gaussian blurring, morphological operations, filtering, and thresholding are applied to the segmentation results to generate an optimized binary segmentation image.

[0151] G. Output Results: Calculate the glass bounding box based on the segmentation map, draw the segmentation mask and bounding box, and save the result image. 5. The key code components of this system are as follows: 5.1 Camera calibration initialization: Python def init_calibration(): global mapx, mapy, roi if USE_CALIBRATION and os.path.exists(CALIB_FILE): try: calib_data = np.load(CALIB_FILE) camera_matrix = calib_data['camera_matrix'] dist_coeffs = calib_data['dist_coeffs'] image_size = calib_data['image_size'] new_camera_matrix, roi = cv2.getOptimalNewCameraMatrix( camera_matrix, dist_coeffs, image_size, CALIB_ALPHA, image_size ) mapx, mapy = cv2.initUndistortRectifyMap( camera_matrix, dist_coeffs, None, new_camera_matrix, image_size, 5 ) return True except Exception as e: print(f"Failed to load calibration result: {str(e)}") return False return False ``` 5.2 Automatic parameter calculation: Python def calculate_auto_params(l_channel): # Calculate image brightness features l_mean = np.mean(l_channel) l_std = np.std(l_channel) # Standard deviation, reflecting contrast # Calculate the proportion of the highlight area (brightness > 200) highlight_ratio = np.sum(l_channel>200) / l_channel.size # Calculate the proportion of dark areas (brightness < 50) dark_ratio = np.sum(l_channel<50) / l_channel.size # Automatically calculate CLAHE parameters: Adjust based on average brightness and contrast if l_mean < 80: # Dark image, requires stronger contrast enhancement clahe_clip = min(5.0, PREPROC_CLAHE_CLIP + (80 - l_mean) / 20) elif l_mean>150:# Bright image, requires weak contrast enhancement clahe_clip = max(0.5, PREPROC_CLAHE_CLIP - (l_mean - 150) / 25) else:# Medium brightness clahe_clip = PREPROC_CLAHE_CLIP # Automatically calculate highlight enhancement parameters: adjust according to highlight ratio If highlight_ratio > 0.3: # Too many highlight areas, turn off highlight enhancement. highlight_factor = 1.0 highlight_thresh = min(1.0, PREPROC_HIGHLIGHT_THRESH + 0.2) elif highlight_ratio<0.05: # Insufficient highlight area, enhance highlights. highlight_factor = min(2.0, PREPROC_HIGHLIGHT_FACTOR + (0.05 -highlight_ratio) * 10) highlight_thresh = max(0.1, PREPROC_HIGHLIGHT_THRESH - 0.1) else:# Normal highlight ratio highlight_factor = PREPROC_HIGHLIGHT_FACTOR highlight_thresh = PREPROC_HIGHLIGHT_THRESH # Automatically calculate sharpening parameters: adjust according to contrast if l_std<30: # Low contrast, requires stronger sharpening sharpen_weight = min(3.0, PREPROC_SHARPEN_WEIGHT + (30 - l_std) / 15) blur_weight = max(-1.0, PREPROC_BLUR_WEIGHT - (30 - l_std) / 30) elif l_std>60:# High contrast, requires weak sharpening sharpen_weight = max(0.5, PREPROC_SHARPEN_WEIGHT - (l_std - 60) / 30) blur_weight = min(-0.1, PREPROC_BLUR_WEIGHT + (l_std - 60) / 60) else:# Normal contrast sharpen_weight = PREPROC_SHARPEN_WEIGHT blur_weight = PREPROC_BLUR_WEIGHT return { 'clahe_clip': clahe_clip, 'highlight_factor': highlight_factor, 'highlight_thresh': highlight_thresh, 'sharpen_weight': sharpen_weight, 'blur_weight': blur_weight } ``` 5.3 Image Preprocessing: Python def preprocess_image(image, co_helper): # Automatic brightness adjustment enhances glass visibility img = image.copy() # Convert to LAB color space lab = cv2.cvtColor(img, cv2.COLOR_BGR2LAB) l, a, b = cv2.split(lab) # Save the original L channel for automatic parameter calculation l_original = l.copy() # Automatically calculate preprocessing parameters auto_params = calculate_auto_params(l_original) # Apply CLAHE to the L channel to enhance contrast (using automatic parameters) clahe = cv2.createCLAHE(clipLimit=auto_params['clahe_clip'],tileGridSize=PREPROC_CLAHE_GRID) l = clahe.apply(l) # Reflection Enhancement: Enhances highlight areas, emphasizing the reflective properties of the glass. # Calculate the mean and standard deviation of the L channel to determine the highlight threshold. l_mean = np.mean(l) l_std = np.std(l) highlight_threshold = l_mean + l_std * auto_params['highlight_thresh'] # Enhance highlight areas (using automatic parameters) l_highlight = l.copy() # Further enhance brightness in highlight areas. highlight_mask = l>highlight_threshold l_highlight[highlight_mask] = np.clip(l_highlight[highlight_mask]*auto_params['highlight_factor'], 0, 255) # Merge channels and convert back to BGR lab = cv2.merge((l_highlight, a, b)) img = cv2.cvtColor(lab, cv2.COLOR_LAB2BGR) # Noise Reduction Before Sharpening: Use bilateral filtering to remove noise while maintaining sharp edges. img = cv2.bilateralFilter(img, 7, 20, 20) # Sharpen glass edges (using automatic parameters) blur = cv2.GaussianBlur(img, (0, 0), PREPROC_BLUR_SIGMA) sharpened = cv2.addWeighted(img, auto_params['sharpen_weight'], blur,auto_params['blur_weight'], 0) # Denoising after sharpening: Use a slight Gaussian blur to remove noise generated during sharpening. sharpened = cv2.GaussianBlur(sharpened, (3, 3), 0.8) # Using letterbox processing img_letterbox = co_helper.letter_box(im=sharpened, new_shape=IMG_SIZE, pad_color=(114, 114, 114)) # Convert to RGB format and add batch dimension img_rgb = cv2.cvtColor(img_letterbox, cv2.COLOR_BGR2RGB) input_data = np.expand_dims(img_rgb, axis=0) return input_data.astype(np.float32), sharpened, img_letterbox ``` 5.4 Segmentation Result Optimization: Python def post_process(input_data): # Process the model output and generate a segmentation map # ... (Model output processing code)... # Smooth Segmentation Map seg_img_smoothed = [] for img in seg_img: blurred = cv2.GaussianBlur(img, (5, 5), 1.0) # Gaussian blur seg_img_smoothed.append(blurred) seg_img = np.array(seg_img_smoothed) # Binarized segmentation image seg_img = seg_img>0.5 # Further process the segmentation image to improve smoothness seg_img_final = [] for img in seg_img: img = img.astype(np.uint8) * 255 # Converts to the range 0-255 # Morphological operations: First close the operation (fill in internal holes), then open the operation (remove noise). kernel = np.ones((5, 5), np.uint8) img = cv2.morphologyEx(img, cv2.MORPH_CLOSE, kernel, iterations=1) img = cv2.morphologyEx(img, cv2.MORPH_OPEN, kernel, iterations=1) # Median filtering and bilateral filtering for further smoothing img = cv2.medianBlur(img, 5) img = cv2.bilateralFilter(img, 9, 75, 75) # Otsu thresholding, automatically determining the optimal threshold _, img = cv2.threshold(img, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU) seg_img_final.append(img.astype(bool)) seg_img = np.array(seg_img_final) return boxes, classes, scores, seg_img ``` The glass inspection system provided in this embodiment first eliminates lens distortion through a camera correction module, ensuring the geometric accuracy of the input data from the source. Then, a brightness analysis module extracts illumination features and drives a parameter calculation module to adaptively generate preprocessing parameters, achieving data-driven intelligent decision-making. Next, an image preprocessing module performs CLAHE enhancement, specular enhancement, denoising, and sharpening on the corrected image, generating high-quality input data to provide a feature-rich foundation image for model inference. Based on this, the model inference module uses an NPU to accelerate the acquisition of the original segmentation results, which are then smoothed, morphologically manipulated, and thresholded by a segmentation optimization module, significantly improving the edge clarity and regional integrity of the segmented data. Finally, the result output module transforms the optimized segmentation data into visualized bounding boxes and mask results, and encapsulates them as structured data for external systems to use. This system adopts a progressive processing logic, with clear data flow and explicit dependencies between modules, forming a complete data processing chain from physical world image input to semantic information output. It can stably output glass inspection result images with accurate bounding boxes and segmentation masks, and can be widely used in complex scenarios such as industrial production, high-altitude cleaning, and security monitoring.

[0152] Figure 3 A schematic diagram of the hardware structure of an electronic device for implementing various embodiments of the present invention.

[0153] The glass detection method provided in this application can be applied to electronic devices. Those skilled in the art will understand that the electronic device structure involved in the embodiments of this invention does not constitute a limitation on the electronic device. An electronic device may include more or fewer components than illustrated, or combine certain components, or have different component arrangements. In the embodiments of this invention, the electronic device includes, but is not limited to, laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the embodiments of this application described and / or claimed herein.

[0154] Electronic devices may include processors, external memory interfaces, internal memory, universal serial bus (USB) interfaces, charging management modules, power management modules, batteries, wireless communication modules, audio modules, speakers, microphones, sensor modules, buttons, cameras, displays, and SIM card interfaces, etc.

[0155] A processor may include one or more processing units, such as: a central processing unit (CPU), an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors.

[0156] The processor can serve as the nerve center and command center of an electronic device. The controller can generate operation control signals based on the instruction opcode and timing signals to control the fetching and execution of instructions.

[0157] The processor may also include memory for storing instructions and data. In some embodiments, the memory in the processor is a cache memory. This memory can store instructions or data that the processor has just used or that are used repeatedly. If the processor needs to use the instruction or data again, it can retrieve it directly from this memory. This avoids repeated accesses, reduces processor latency, and thus improves system efficiency.

[0158] An external storage interface (ESI) can be used to connect external memory cards, such as microSD cards, to expand the storage capacity of electronic devices. The external memory card communicates with the processor through the ESI to perform data storage functions, such as saving music and video files on the external memory card.

[0159] Internal memory can be used to store computer executable program code, which includes instructions. The processor executes various functional applications and data processing of electronic devices by running the instructions stored in internal memory. Internal memory can include a program storage area and a data storage area. Internal memory can include high-speed random access memory, and can also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0160] Wireless communication functionality in electronic devices can be achieved through antennas, wireless communication modules, modem processors, and baseband processors.

[0161] Wireless communication modules can provide solutions for wireless communication applications in electronic devices, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies.

[0162] Electronic devices can implement audio functions through audio modules, speakers, receivers, microphones, headphone jacks, and application processors.

[0163] Electronic devices can achieve shooting functions through ISPs, cameras, video codecs, GPUs, displays, and application processors.

[0164] Electronic devices can achieve display functions through GPUs, displays, and application processors.

[0165] A GPU is a microprocessor for image processing, connected to the display screen and application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering. A processor may include one or more GPUs, which execute program instructions to generate or modify display information.

[0166] A display screen is used to display images, videos, etc. A display screen includes a display panel.

[0167] The aforementioned electronic device realizes the glass inspection method of this application by integrating camera distortion correction, adaptive illumination parameter adjustment, highlight area directional enhancement, multi-step segmentation post-processing optimization, and edge computing NPU acceleration. It achieves the beneficial effects of eliminating lens distortion, dynamically adapting to complex illumination conditions, effectively suppressing reflection interference, significantly improving the clarity and integrity of segmentation edges, and meeting real-time processing requirements while ensuring high detection accuracy. It can be widely used in industrial production, high-altitude cleaning, and security monitoring scenarios.

[0168] The storage medium provided in this application stores a program product capable of implementing a glass testing method.

[0169] Glass testing methods include: The raw images of the area to be tested are acquired in real time using industrial cameras or surveillance cameras. The original image is corrected by loading the camera calibration file, calculating the latest camera matrix and distortion correction mapping table to eliminate the effects of lens distortion, and cropping the image according to the preset region of interest to obtain the corrected image. By performing statistical analysis of brightness information on the corrected image, quantified illumination feature data is generated; the illumination feature data includes the mean brightness, standard deviation, and highlight ratio. Based on the illumination feature data, preprocessing parameters are automatically calculated according to preset adjustment rules; the preprocessing parameters include CLAHE contrast limit, highlight enhancement factor, and sharpening weight. Based on the preprocessing parameters, the corrected image is sequentially subjected to CLAHE enhancement, specular enhancement, denoising and sharpening to generate the preprocessed image. The preprocessed image is input into a preloaded segmentation model for inference, and the NPU of the edge computing platform is used for acceleration to obtain the original segmentation result. The original segmentation result is post-processed and optimized to obtain an optimized segmentation map; Calculate the glass bounding box based on the optimized segmentation map, generate the result image by drawing the segmentation mask and bounding box, and save it.

[0170] In some possible implementations, the glass detection method of this disclosure can be implemented as a program product including program code that, when the program product is run on a terminal device, causes the terminal device to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure.

[0171] The storage medium disclosed herein may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0172] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for testing glass, characterized in that, include: The raw images of the area to be tested are acquired in real time using industrial cameras or surveillance cameras. The original image is corrected by loading the camera calibration file, calculating the latest camera matrix and distortion correction mapping table to eliminate the effects of lens distortion, and cropping the image according to the preset region of interest to obtain the corrected image. By performing statistical analysis of brightness information on the corrected image, quantified illumination feature data is generated; the illumination feature data includes the mean brightness, standard deviation, and highlight ratio. Based on the illumination feature data, preprocessing parameters are automatically calculated according to preset adjustment rules; the preprocessing parameters include CLAHE contrast limit, highlight enhancement factor, and sharpening weight. Based on the preprocessing parameters, the corrected image is sequentially subjected to CLAHE enhancement, specular enhancement, denoising and sharpening to generate the preprocessed image. The preprocessed image is input into a preloaded segmentation model for inference, and the NPU of the edge computing platform is used for acceleration to obtain the original segmentation result. The original segmentation result is post-processed and optimized to obtain an optimized segmentation map; Calculate the glass bounding box based on the optimized segmentation map, generate the result image by drawing the segmentation mask and bounding box, and save it.

2. The glass testing method according to claim 1, characterized in that, The step of generating quantized illumination feature data by performing statistical analysis of brightness information on the corrected image includes: The corrected image is converted from the BGR color space to the LAB color space by color space transformation, and its L channel image is extracted as the luminance channel. Calculate the average grayscale value of all pixels in the brightness channel as the average brightness value; Calculate the grayscale standard deviation of all pixels in the brightness channel, and use it as the standard deviation; The number of pixels in the brightness channel whose grayscale value is greater than a preset highlight threshold is counted, and the ratio of this number to the total number of pixels is calculated as the highlight ratio.

3. The glass testing method according to claim 1, characterized in that, Based on the illumination feature data, preprocessing parameters are automatically calculated according to preset adjustment rules, including: The preprocessing parameters are automatically calculated based on the illumination characteristic data according to the following rules, where the mean brightness is μ, the standard deviation is σ, and the highlight ratio is... : When μ < 80, the formula is used. Calculate the CLAHE contrast limiting parameter CL; where The default CLAHE contrast limit base value; When μ>150, according to the formula Calculate the CLAHE contrast limit parameter CL; when At this time, set the highlight enhancement factor HF to 1.0 to turn off highlight enhancement; when At that time, through the formula Calculate the specular enhancement factor HF; where This is the preset base value for the highlight enhancement factor; When σ < 30, it is calculated using the formula. Sharpening weights (SW); where This is the preset base value for sharpening weights; When σ > 60, according to the formula Calculate the sharpening weight SW.

4. The glass testing method according to claim 3, characterized in that, The process involves sequentially performing CLAHE enhancement, specular enhancement, denoising, and sharpening on the corrected image based on the preprocessing parameters to generate a preprocessed image, including: The corrected image is converted from the BGR color space to the LAB color space by color space transformation, and the L channel image, a channel image and b channel image are separated. The specular threshold is calculated using the following formula based on the mean luminance μ and the standard deviation σ. : Where k is the preset highlight threshold multiplier; Based on the L-channel image, a specular mask is generated using the following formula. : The L-channel image is enhanced using CLAHE, and the enhanced L-channel image is obtained by the following formula. : Where CL is the CLAHE contrast limiting parameter; Through the specular mask Identify highlight regions in the enhanced L-channel image. The highlight regions in the image are targeted by the aforementioned highlight enhancement factor HF, and the enhanced L-channel image is obtained by the following formula. : The L channel image after highlight enhancement The image is merged with the a-channel image and the b-channel image, and then converted back to the BGR color space through color space transformation to obtain the first intermediate image. ; For the first intermediate image Denoising is achieved by applying bilateral filtering to obtain the second intermediate image. ; For the second intermediate image Edge enhancement is performed by generating a blurred image using Gaussian blur. Then, using unsharpening masking techniques, the enhancement intensity is controlled by the sharpening weight SW, and the third intermediate image is obtained through the following formula. : For the third intermediate image Gaussian blur is applied to remove sharpened noise, resulting in the fourth intermediate image. ; For the fourth intermediate image Perform letterboxing to generate a pre-processed image by adjusting the image size to a preset value while maintaining the aspect ratio.

5. The glass testing method according to claim 1, characterized in that, The step of inputting the preprocessed image into a preloaded segmentation model for inference, and accelerating the process using the NPU of an edge computing platform to obtain the original segmentation result, includes: Load and initialize the segmentation model via the RKNN Lite API, and enable NPU multi-core acceleration; The preprocessed image is converted to RGB format and input into the loaded segmentation model for inference; Obtain the raw segmentation results output by the model, which include predicted bounding boxes, class confidence scores, and segmentation prototypes.

6. The glass testing method according to claim 5, characterized in that, The post-processing optimization of the original segmentation result to obtain the optimized segmentation map includes: An initial segmentation map is generated by applying matrix multiplication to the segmentation prototype; The initial segmentation map is converted into a probability map by applying the sigmoid activation function. The size of the probability map is adjusted to a preset value by interpolation to generate a first optimized intermediate map; The first optimized intermediate image is cropped based on the predicted bounding box, retaining the target region, to generate the second optimized intermediate image. Gaussian blur is applied to smooth the second optimized intermediate image to generate a third optimized intermediate image; The third optimized intermediate image is binarized to generate the fourth optimized intermediate image; Morphological operations are performed on the fourth optimized intermediate image, using a 5×5 core to perform closing operations to fill internal holes and opening operations to remove noise, generating the fifth optimized intermediate image. The fifth optimized intermediate graph is smoothed by applying mean filtering and bilateral filtering in sequence to generate the sixth optimized intermediate graph; The Otsu algorithm is used to adaptively threshold the sixth optimized intermediate image, automatically determine the optimal segmentation threshold, and generate the optimized segmentation image.

7. The glass testing method according to claim 6, characterized in that, Calculate the glass bounding box based on the optimized segmentation map, generate and save the resulting image by drawing the segmentation mask and bounding box, including: Connectivity analysis is performed on the optimized segmentation graph. All connected components are extracted using a contour extraction algorithm, and the minimum bounding rectangle of each connected component is calculated as the glass bounding box. The optimized segmentation image is converted into a color mask image, and the color mask image is superimposed on the preprocessed image in a semi-transparent manner through alpha fusion to generate a result image with segmentation mask; On the resulting image, draw a rectangle based on the glass bounding box, and label the rectangle with the corresponding category information and confidence score; The resulting image is saved to a local storage device in a specified format, and simultaneously output to a display device in real time via a display interface. The detection results are encapsulated into structured data through a standardized data interface and sent to an external system for further processing or display. The detection results include glass bounding box coordinates, category labels, confidence scores, and optimized segmentation maps.

8. A glass inspection system, characterized in that, The system employs the glass detection method as described in any one of claims 1 to 7; The system includes: The image acquisition module is used to acquire raw images of the area to be tested in real time using an industrial camera or a surveillance camera. The camera correction module is used to correct the original image. By loading the camera calibration file, it calculates the latest camera matrix and distortion correction mapping table to eliminate the influence of lens distortion, and performs image cropping according to the preset region of interest to obtain the corrected image. The brightness analysis module is used to generate quantified illumination feature data by performing statistical analysis of brightness information on the corrected image; the illumination feature data includes the mean brightness, standard deviation, and highlight ratio. The parameter calculation module is used to automatically calculate preprocessing parameters based on the illumination feature data and through preset adjustment rules; the preprocessing parameters include CLAHE contrast limit, highlight enhancement factor, and sharpening weight. The image preprocessing module is used to sequentially perform CLAHE enhancement, specular enhancement, denoising and sharpening on the corrected image according to the preprocessing parameters to generate the preprocessed image; The model inference module is used to input the preprocessed image into a preloaded segmentation model for inference, and to accelerate the process using the NPU of the edge computing platform to obtain the original segmentation result. The segmentation optimization module is used to perform post-processing optimization on the original segmentation result to obtain an optimized segmentation map. The result output module is used to calculate the glass bounding box based on the optimized segmentation map, generate the result image by drawing the segmentation mask and bounding box, and save it.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the glass detection method as described in any one of claims 1 to 7.

10. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the glass inspection method as described in any one of claims 1 to 7.

Images