AI-based identification system and method for oversize

Abstract

The present invention belongs to the field of image processing and discloses an AI-based oversize recognition system and method. The system and method comprise the following steps: collecting a video stream in a slag pit, extracting key frames from the video stream using an adaptive frame rate, performing preliminary noise reduction processing on the image using a hybrid filtering algorithm, performing noise smoothing processing using an improved NLM filter, performing image clarity processing using an improved Laplace operator, performing image deformation correction using a distortion correction model, outputting slag block information through an improved YOLOv8 model and a SAM model using an integrated model, introducing a CBAM attention mechanism into the improved YOLOv8 model to enhance the model's attention to slag block features, fine-tuning the model using optimized K-means clustering, arranging the outputted slag block contour, area, and shape information on a visualization interface, judging the slag accumulation state, providing early warning information, and realizing intelligent recognition of oversize.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and more specifically, to an AI-based recognition system and method for oversize objects. Background Art

[0002] During the slag pit production process, the furnace bottom slag discharge device consists of a screen, extrusion head, housing, drive hydraulic cylinder, support frame, and camera monitoring system. This adapts to the highly variable quality of coal used in boilers and its tendency to coke. Therefore, the oversize material after the screen passes through the screen screen is also called slag. The composition of slag is complex and may include other solids such as stones and iron. Existing technology primarily assesses the slag pit's slag status through manual observation of the slag pit camera video feed. For example, the brightness of the cold ash hopper can be used to determine if coke has formed, or the temperature of the slag pit and the steel strip head can be used to determine if the slag is falling normally. Manual observation is highly subjective and inefficient, especially during long operations or night shifts, and fatigue can easily lead to missed detections or misjudgments. Furthermore, manual observation cannot quantify the slag accumulation status (such as area, height, and accumulation trend), making it difficult to provide accurate early warnings. Consequently, undetected slag can lead to slag pit blockage, reduced slag discharge efficiency, and even equipment failure or unplanned downtime.

[0003] The complex environment inside a slag pit (e.g., high temperature, smoke, dust, and vibration) can interfere with the quality of images captured by industrial cameras (e.g., insufficient brightness, excessive noise), hindering the effectiveness of manual observation. The image quality is insufficient to meet subsequent analysis requirements. The irregular shape and size distribution of slag blocks make it difficult for manual observation to accurately identify and quantify them.

[0004] In view of this, the present invention proposes an AI-based identification system for oversize material to solve the above problems. Summary of the Invention

[0005] In order to overcome the above-mentioned defects of the prior art and to achieve the above-mentioned objectives, the present invention provides the following technical solutions: an AI-based oversize recognition system, comprising: an image data acquisition module: for collecting a video stream in a slag pit, extracting key frames from the video stream using an adaptive frame rate, and obtaining an image to be processed;

[0006] Image data enhancement module: Use hybrid filtering algorithm to perform preliminary noise reduction on the image, use improved NLM filtering to smooth the noise, use improved Laplace operator to clear the image, use distortion correction model to correct the image deformation, and output the image after image data enhancement;

[0007] Object detection module: Design an integrated model that outputs information about slag blocks using an improved YOLOv8 model and SAM model. The improved YOLOv8 model introduces the CBAM attention mechanism to enhance the model's focus on slag block features. The model is fine-tuned using optimized K-means clustering to output the ground truth bounding box and the number of slag blocks in the image.

[0008] Visualization module: The output is the outline, area and shape information of the slag block arranged on the visualization interface, and the slag accumulation status is judged to provide early warning information.

[0009] Preferably, the method of collecting the video stream in the slag pit and extracting key frames from the video stream using an adaptive frame rate includes:

[0010] Install high-precision infrared camera equipment on the top or side wall of the slag pit to collect real-time video stream inside the slag pit;

[0011] The initial frame rate is set to Z_1fps, so the initial frame interval is

[0012] Collect the real-time speed of slag accumulation changes in the slag pit, dynamically adjust the frame rate by setting the speed interval, customize R speed intervals, set a frame rate for each speed interval, and the frame rate increases as the speed interval increases. Match the collected real-time speed with the speed interval, and use the frame rate corresponding to the matched speed interval as the real-time frame rate. According to the real-time frame rate, obtain the real-time frame interval, and extract image frames from the video stream according to the frame interval;

[0013] Set the standard frame and change threshold, calculate the frame difference between the image frame and the standard frame, if the frame difference is greater than or equal to the change threshold, mark the image frame as a key frame, extract the image, and set it as the image to be processed, otherwise, do not extract the image.

[0014] Preferably, the method of performing noise reduction processing on an image using a hybrid filtering algorithm includes:

[0015] In view of the thermal noise in the image, the Gaussian filter function is used to perform preliminary smoothing on the image. The standard deviation σ of the Gaussian filter function is set to be dynamically adjusted according to the smoke concentration, then σ=σ0+e k1+Δc ×C smooke , Δc represents the frame difference between the image frame and the standard frame, σ0 is the basic standard deviation, k1 is the adjustment coefficient, C smoke is the smoke concentration, and the standard frame is a predefined ideal image;

[0016] For salt and pepper noise, the median filter method is used, and the median filter window is set to W = w0 + k2 × log (1 + C smoke ), w0 is the basic window size, and k2 is the adjustment coefficient, which is used to control the influence of smoke concentration on the window size.

[0017] Preferably, the noise smoothing process is performed using an improved NLM filter, and the method comprises:

[0018] Apply NLM filtering to the input image to obtain the denoised image INLM;

[0019] Perform wavelet decomposition on the image INLM after NLM filtering to obtain low-frequency components and high-frequency components;

[0020] A denoising threshold is preset and threshold denoising is applied to the high frequency component, that is, if the absolute value of the wavelet coefficient is greater than the threshold, the wavelet coefficient is retained, otherwise, the wavelet coefficient is set to zero;

[0021] The image is reconstructed using the denoised high-frequency components and the unchanged low-frequency components, and the reconstructed image is output as the final denoising result.

[0022] Preferably, the method for performing image sharpening processing using the improved Laplace operator includes:

[0023] The CLAHE method was used for preprocessing;

[0024] Applying the improved Laplacian operator

[0025] Among them, I sh (x, y) is the input image, I en (x, y) is the enhanced image, is the Laplace operator, k3 is the sharpening coefficient, which is used to control the intensity of sharpening. k3 is designed as

[0026] N is the total number of pixels in the input image,

[0027] is the gradient of the original image I(x, y) at point (x, y), ∈ is a small positive number used to avoid the denominator being zero, and I_σ is the gradient of the image I sh The variance of (x, y);

[0028] Enhance edges using morphological gradients.

[0029] Preferably, the method of performing deformation correction on an image using a distortion correction model includes:

[0030] Step A1: Build a distortion correction model

[0031] Among them, x d Represents the horizontal coordinate in the distorted image, y d Represents the vertical coordinate in the distorted image, x uis the distortion-free horizontal coordinate output by the distortion correction model, y u is the distortion-free ordinate output by the distortion correction model, θ1 and θ2 are the radial distortion coefficients;

[0032] Step A2: Use the checkerboard calibration plate to collect T_O groups of images, calculate the distortion parameters θ1 and θ2, and set the objective function to minimize the error between the input image and the ideal image.

[0033]

[0034] (x d，i ,y d，i ) are the horizontal and vertical coordinates of the i-th distorted coordinate point,

[0035] (x d，i (θ1,θ2),y d，i (θ1, θ2)) is the horizontal and vertical coordinates of the i-th distortion-free coordinate point output by the distortion correction model, and M is the number of checkerboard corner points;

[0036] Introducing temperature compensation model Among them, T is the current temperature in the slag pit, T0 is the calibration temperature, α and β are temperature compensation coefficients, θ 1，0 and θ 2，0 is the initial value, θ1(T) and θ2(T) are the temperature compensated θ1 and θ2;

[0037] Step A3: Use θ1 and θ2 in the distortion correction model, perform reverse mapping correction on the image, calculate the corrected image by interpolation, and output the image.

[0038] Preferably, the integrated model outputs information of the slag block through an improved YOLOv8 model and a SAM model, and the method includes:

[0039] In the feature extraction network of YOLOv8, the CBAM module is inserted after the output of each residual block or CSP block of CSPDarknet53, and before the feature fusion operation of PANet;

[0040] Use optimized K-means clustering to perform multi-scale stratification of the target and output the size and number of the true box;

[0041] In the feature pyramid of YOLOv8, a high-resolution feature map is set, and the true frames of small objects are assigned to the high-resolution feature map, and the true frames of medium and large objects are assigned to the low-resolution feature map;

[0042] The output of YOLOv8 is used as the input of the SAM model, and the output is the contour, area and shape information of the slag block.

[0043] Preferably, the method of performing multi-scale stratification of targets using optimized K-means clustering includes:

[0044] Step B1: Extract the true box sizes of all slag blocks, pre-define the true boxes into three categories: small, medium, and large, define the size range of each category, and assign the true boxes to small, medium, and large target categories;

[0045] Step B2: Use IoU distance instead of Euclidean distance in the optimized K-means, then the objective function is Among them, A is the total number of true boxes, j is the index of the cluster center, a is the index of the true box, J is the number of cluster centers, IoU(s j , g j ) is the IoU distance between the jth ground truth box and the jth cluster center;

[0046] Step B3: Calculate the variance of the real box size distribution F_C of the image and dynamically adjust Among them, γ is the adjustment coefficient, J0 is the basic quantity;

[0047] Step B4: For each level of targets, output the clustered true box size.

[0048] Preferably, the method of arranging the outputted contour, area and shape information of the slag block on a visual interface, judging the slag accumulation state and providing early warning information comprises:

[0049] Predefine the area threshold and regular shape, and compare them with the output contour, area and shape information. If it is greater than the area threshold or does not conform to the regular shape, it is determined to be a slag accumulation state and an alarm signal is activated; otherwise, continue working.

[0050] An AI-based identification method for oversize material, comprising:

[0051] Step S1: Install a high-precision infrared camera on the top or side wall of the slag pit to collect real-time video streams inside the slag pit;

[0052] Step S2: using a hybrid filtering algorithm to perform preliminary noise reduction on the image, using an improved NLM filter to perform noise smoothing, using an improved Laplace operator to perform image clarity processing, using a distortion correction model to perform image deformation correction, and outputting an image after image data enhancement;

[0053] Step S3: Design an integrated model. The integrated model outputs information about slag blocks through the improved YOLOv8 model and the SAM model. The image after image data enhancement is input into the integrated model, and the real box and number of slag blocks in the image are output.

[0054] Step S4: The output is the outline, area and shape information of the slag block arranged on a visual interface, interacting with the user, judging the slag accumulation state and providing early warning information.

[0055] The technical effects and advantages of the present invention based on the AI ​​recognition system for oversize materials are as follows:

[0056] By extracting keyframes from the video stream at an adaptive frame rate, the system dynamically adjusts the frame rate based on the real-time speed of slag accumulation within the slag pit, ensuring efficient image data acquisition at varying speeds. This improves the efficiency and specificity of data acquisition, reduces unnecessary data processing, and ensures high frame rate capture of rapidly changing areas.

[0057] The hybrid filtering algorithm, combined with an improved NLM filter and the Laplacian operator, effectively removes thermal and salt-and-pepper noise from images. It also uses a distortion correction model to correct image distortion. This significantly improves image quality, enhancing clarity and detail, and provides higher-quality input for subsequent object detection.

[0058] The improved YOLOv8 model introduces the CBAM attention mechanism, which enhances the focus on slag features. The optimized K-means clustering performs multi-scale stratification of targets, improving the detection accuracy of small targets. This improves detection accuracy, especially for small slag blocks. Multi-scale feature fusion also enhances the model's adaptability to targets of different sizes.

[0059] The contour, area, and shape of the slag blocks are displayed on a visual interface. The system then determines the slag accumulation status based on predefined thresholds and provides early warning information. This provides operators with an intuitive monitoring interface, facilitating the timely detection and resolution of slag accumulation issues, improving the system's practicality and user experience.

[0060] By replacing the Euclidean distance with the Intersection over Union (IoU) distance, the clustering results better meet the actual requirements of object detection tasks. The number of cluster centers, K, is dynamically adjusted to accommodate data of varying size distributions. This improves the match between anchor boxes and ground truth boxes, optimizing detection performance, especially for small objects. The SCConv module and CoordinateAttention mechanism are introduced to enhance feature extraction and spatial attention capabilities.

[0061] The temperature compensation model dynamically adjusts distortion parameters to ensure accurate correction of image distortion under different temperature conditions. This improves the accuracy and adaptability of image correction, especially in high-temperature environments.

[0062] This AI recognition system significantly improves the accuracy and efficiency of detecting slag blocks within slag pits by optimizing modules such as image acquisition, enhancement, object detection, and visualization. The detection capability for small slag blocks has been significantly enhanced, while dynamic adjustment and multi-scale processing have also improved the system's adaptability and robustness. These improvements not only enhance detection accuracy but also provide a more efficient and intuitive solution for practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 This is a schematic structural diagram of an AI-based identification system for oversize material according to the present invention;

[0064] Figure 2 The figure is a schematic diagram of the steps of the AI ​​identification method based on the screen material of the present invention. DETAILED DESCRIPTION

[0065] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0066] Example 1

[0067] See also Figure 1 and Figure 2 As shown, the AI ​​recognition system based on oversize material described in this embodiment includes:

[0068] Main research contents:

[0069] We propose an intelligent oversize identification and control method. This method leverages AI technology to achieve real-time detection, identification, and early warning of slag accumulation within the slag pit, thereby ensuring the normal operation of the equipment and reducing the workload and risk of unplanned downtime accidents for operators and maintenance personnel. The following is a detailed description of the method's design, the technologies used, and the improvements it addresses.

[0070] Based on the background of the existing technology and the requirements for operation monitoring and adjustment, the dry slag discharge system has the following defects and problems in actual operation. By analyzing the deficiencies of the existing technology and combining the characteristics of this technical solution, the technical problems to be solved by this technical solution are described in detail below.

[0071] With existing technical means, the slag accumulation status of the slag pit is mainly judged by manual observation of the slag pit camera video, such as judging whether the cold ash hopper is coked by the brightness, or judging whether the coke slag is falling normally by observing the temperature of the slag pit and the steel strip head. Manual observation has the problems of strong subjectivity and low efficiency. Especially during long-term operation or night shifts, it is easy to miss detection or misjudgment due to fatigue. Moreover, manual observation cannot quantify the slag accumulation status (such as area, height, accumulation trend), making it difficult to achieve accurate early warning. Therefore, if the slag accumulation is not discovered in time, it may lead to slag pit blockage, reduced slag discharge efficiency, and even equipment failure or non-stop accidents.

[0072] When judging the state of slag accumulation, in the prior art, the judgment of the state of slag accumulation (such as whether it needs to be cleaned or whether it is abnormal) mainly relies on the experience of the operation and maintenance personnel. For example, the background technology mentioned that the state of slag accumulation can be judged by observing parameters such as the brightness and temperature of the slag pit, but the judgment criteria for these parameters are unclear and lack quantification and automation. Inaccurate judgment of the state of slag accumulation may lead to untimely or erroneous start-up of the cleaning equipment, affecting the operating efficiency of the equipment. In addition, the environment inside the slag pit is complex (such as high temperature, smoke, dust, vibration, etc.), and the image quality collected by the industrial camera may be disturbed (such as insufficient brightness, high noise), affecting the effect of manual observation. The image quality is difficult to meet the needs of subsequent analysis. The shape and size distribution of the slag blocks are irregular, and manual observation is difficult to accurately identify and quantify.

[0073] Image data acquisition module: used to collect video streams in the slag pit, extract key frames from the video streams using an adaptive frame rate, and obtain images to be processed;

[0074] Image data enhancement module: Use hybrid filtering algorithm to perform preliminary noise reduction on the image, use improved NLM filtering to smooth the noise, use improved Laplace operator to clear the image, use distortion correction model to correct the image deformation, and output the image after image data enhancement, which is recorded as high-quality image;

[0075] Methods for collecting video streams in slag pits and extracting key frames from the video streams using adaptive frame rates include:

[0076] Install high-precision infrared cameras on the top or side walls of the slag pit to ensure full coverage. For example, use high-precision industrial-grade cameras with an operating temperature range of -20°C to 80°C, IP67 dust and water resistance, and vibration resistance that meets IEC 60068-2-6 standards. The cameras must have a resolution of 1080p or higher and a frame rate of 30fps or higher. These cameras are suitable for harsh environments such as high temperatures, smoke, and vibration. They capture real-time video streams from the slag pit.

[0077] Frame rate refers to the number of frames displayed per second. For example, a frame rate of 5 fps means that 5 frames are captured per second. A higher frame rate means more frames are captured per unit time, allowing for greater detail to be captured. A lower frame rate means fewer frames are captured, reducing the computational effort.

[0078] The initial frame rate is set to Z_1fps, so the initial frame interval is

[0079] At the beginning, an initial frame rate is set. For example, the initial frame rate is set to Z_l = 5fps, which means that 5 frames of images are extracted from the video stream per second, and the time interval between each frame (frame interval) is That is to say, one frame of image is extracted every 0.2 seconds.

[0080] Collect the real-time speed of the accumulated slag in the slag pit, dynamically adjust the frame rate by setting the speed interval, customize R speed intervals, set a frame rate for each speed interval, and the frame rate increases with the increase of the speed interval. Match the collected real-time speed with the speed interval, and use the frame rate corresponding to the matched speed interval as the real-time frame rate. According to the real-time frame rate, obtain the real-time frame interval, and extract image frames from the video stream according to the frame interval; for example, use OpenCV or FFmpeg to extract image frames according to the frame interval.

[0081] The real-time speed of slag accumulation change in the slag pit can be collected by the following methods:

[0082] Industrial-grade high-definition cameras (e.g., 50 frames per second CCD cameras) are installed at observation windows on both sides of the slag pit to capture real-time video images of accumulated slag in the pit. Differences between adjacent frames (e.g., using the frame difference method) are calculated to determine the rate of change of accumulated slag.

[0083] For example, if the speed intervals are set to 0 to 5 and 5 to 10, the corresponding frame rates are 10 and 20. For example, when the slag accumulation change speed is 2, it belongs to 0 to 5, and the frame rate is increased from 5fps to 10fps. At this time, the frame interval becomes 0.1 seconds. The core purpose of this step is to ensure that sufficient details can be captured by increasing the frame rate when the slag accumulation changes rapidly. When the slag accumulation changes slowly, the frame rate is reduced to reduce redundant data and the amount of calculation. By dynamically adjusting the frame rate, the system's computing efficiency and resource consumption can be optimized while ensuring monitoring accuracy.

[0084] A motion detection algorithm (such as the frame difference method) is introduced to extract only frames with significant changes in the slag accumulation area. The frame difference method detects motion by comparing the pixel value differences between consecutive frames. The usage steps in the present invention are to set a standard frame and a change threshold, calculate the frame difference between the image frame and the standard frame, and if the frame difference is greater than or equal to the change threshold, mark the image frame as a key frame, extract the image, and set it as the image to be processed. Otherwise, do not extract the image.

[0085] Due to the impact of harsh environments such as high temperature, smoke and dust in slag pits on images captured by infrared cameras, the images may have the following problems:

[0086] 1. Low clarity: Smoke and dust cause image blur, and the infrared image itself has low contrast and serious loss of details.

[0087] 2. Image deformation: High temperature environments may cause slight deformation of the camera lens or sensor, resulting in geometric distortion of the image (such as barrel distortion or pincushion distortion).

[0088] 3. Noise interference: Infrared images are easily affected by thermal noise and smoke scattering noise, which manifests as salt and pepper noise or Gaussian noise.

[0089] 4. Uneven brightness: Infrared images may have uneven brightness distribution, with some areas being too bright or too dark, affecting the detection accuracy of subsequent AI models.

[0090] To address the above issues, and considering the characteristics of infrared images (low contrast, high noise, and thermal imaging characteristics), the following provides a complete and detailed image preprocessing technology solution:

[0091] Infrared images contain thermal noise, smoke scattering noise, and salt and pepper noise. To effectively remove these different types of noise from infrared images, multi-scale noise reduction can be performed by combining Gaussian filtering, median filtering, and non-local means filtering (NLM). Furthermore, wavelet transform can be introduced for further noise suppression. The following are specific implementation methods and steps:

[0092] Methods for image noise reduction using hybrid filtering algorithms include:

[0093] In view of the thermal noise in the (infrared) image, the Gaussian filter function is used to perform preliminary smoothing on the image. The Gaussian filter function is Where (x, y) is the horizontal and vertical coordinates of the pixel point in the image. The standard deviation σ of the Gaussian filter function is set to be dynamically adjusted according to the smoke concentration (in the slag pit), then σ=σ0+e k1+Δc ×C smoke , Δc represents the frame difference between the image frame and the standard frame, σ0 is the basic standard deviation, k1 is the adjustment coefficient, C smoke is the smoke concentration, and the standard frame is a predefined ideal image; for example, when the smoke concentration is high, increase σ (such as σ=2.0) to enhance the smoothing effect. The method for obtaining the smoke concentration can be through an optical measurement method or a gas-sensitive smoke sensor.

[0094] To determine the optimal value for the adjustment coefficient k1, follow these steps: Select an initial value, such as 0.1. Gradually increase the adjustment coefficient k1 and observe the effect of image smoothing. If the smoothing effect is insufficient, increase k1 appropriately. If the smoothing effect is too strong, resulting in loss of image detail, decrease k1. Evaluate the smoothing effect at different k1 values ​​using visual evaluation or quantitative metrics such as mean square error (MSE) and peak signal-to-noise ratio (PSNR). The base standard deviation is pre-set by management based on experience or historical data analysis.

[0095] In order to deal with the salt and pepper noise caused by smoke scattering, the median filter method is used. The most important thing in the median filter is how big the window is to process the image. Different window processing effects are also different. According to the technical problem proposed by the present invention, the window of the median filter is set to W = w0 + k2 × log (1 + C smoke ), w0 is the basic window size (for example, W0=3 corresponds to a 3×3 window), and k2 is an adjustment coefficient used to control the influence of smoke concentration on the window size.

[0096] The minimum window size to use when the smoke concentration is 0. A 3×3 window is typically selected, as this is the minimum window size commonly used in median filtering. The adjustment factor k2 is adjusted based on actual needs. For example, if you want the window size to increase by 1 for every 1 unit increase in smoke concentration, then k2 = 1. Using a logarithmic function can slow the growth of the window size as smoke concentration increases, preventing excessive window size growth that could lead to loss of image detail.

[0097] Use improved NLM filtering for noise smoothing, including the following methods:

[0098] Improved NLM filtering combined with wavelet transform for multi-scale denoising. To combine non-local means filtering (NLM) and wavelet transform for multi-scale denoising, you can follow the following steps. These steps combine the advantages of NLM filtering in preserving edge details with the ability of wavelet transform in multi-scale analysis to achieve better denoising results.

[0099] Apply NLM filtering to the input image to obtain the denoised image INLM;

[0100] Perform wavelet decomposition on the image INLM after NLM filtering to obtain low-frequency components and high-frequency components;

[0101] A preset noise reduction threshold is used to apply the threshold noise reduction to the high-frequency components. That is, if the absolute value of the wavelet coefficient is greater than the threshold, the wavelet coefficient is retained; otherwise, the wavelet coefficient is set to zero. This is pre-set by the management personnel.

[0102] The image is reconstructed using the denoised high-frequency components and the unchanged low-frequency components, and the reconstructed image is output as the final denoising result.

[0103] By combining NLM filtering and wavelet transform, image noise can be effectively removed while preserving edges and details. NLM filtering excels at removing Gaussian noise and preserving detail, while wavelet transform excels at multiscale analysis and high-frequency noise removal. This combined approach is particularly suitable for processing complex noisy environments, such as the heat blooming effect in infrared images.

[0104] Methods for image sharpening using the improved Laplacian operator include:

[0105] The goal is to enhance the contrast and edge details of infrared images and improve the visibility of slag blocks.

[0106] To address the low contrast problem of infrared images, the CLAHE method is used for preprocessing to enhance local contrast. CLAHE can effectively enhance the details of infrared images and is suitable for scenes with uneven temperature distribution in slag pits.

[0107] To solve the problem of edge blur in infrared images, an improved Laplace operator is applied Among them, I sh (x, y) is the input image, I en (x, y) is the enhanced image, is the Laplace operator, which is used to detect edges and details in the image. K3 is the sharpening coefficient, which is used to control the intensity of sharpening. Therefore, K3 is the key to the quality of image sharpening. Too large a K3 value may introduce noise, so it needs to be adjusted according to the actual situation. K3 is designed to be N is the total number of pixels in the input image, is the gradient of the original image I(x, y) at point (x, y). The original image here can be regarded as the image to be processed in the final output of the image data acquisition module. ∈ is a small positive number used to avoid the denominator being zero. I_σ is the gradient of the image I sh The variance of (x, y);

[0108] In the design of k3, first, we need a metric to evaluate the edge blur of the image. We can use the gradient information of the image to achieve this. For example, we can calculate the average value of the gradient modulus of the image. Second, we need a metric to assess the noise level of an image. We can use the variance of the image to do this.

[0109] in, I(x, y) represents the value of the pixel located at the xth row and yth column in the image, I(x+1, y) represents the value of the pixel located at the x+1th row and yth column in the image, I(x-1, y) represents the value of the pixel located at the x-1th row and yth column in the image, I(x, y+1) represents the value of the pixel located at the xth row and y+1th column in the image, and I(x, y-1) represents the value of the pixel located at the xth row and y-1th column in the image.

[0110] Aiming at the thermal blooming effect (edge ​​diffusion) of infrared images, morphological gradient is used to enhance the edge.

[0111] Process the heat blooming effect (edge ​​diffusion) in infrared images and enhance the edges, using morphological gradient to enhance the edges. The following are the complete usage steps:

[0112] Use the structure element to transform the image I sh (x, y) is expanded to enhance the edge; the structural element is used to sh (x, y) is corroded to weaken the edge; the morphological gradient is calculated by the difference between the dilation and corrosion operations, the edge is highlighted, and a gradient image is formed. The structural element is usually a 3×3 rectangle. The denoised image is logarithmically transformed to better separate the illumination and reflection components.

[0113] The logarithmic transformed image is convolved with a Gaussian filter function to estimate the illumination component. The reflection component U(x, y) is obtained by subtracting the logarithmic transformed image from the logarithm of the illumination component. The reflection component is enhanced and adjusted by exp(U(x, y)-1) to improve the brightness and contrast of the image and obtain a clear final image.

[0114] Methods for correcting image deformation using distortion correction models include:

[0115] Goal: Correct geometric distortion (such as barrel distortion or pincushion distortion) caused by deformation of the camera lens or sensor due to high temperature.

[0116] Step A1: Build a distortion correction model Among them, X d Represents the horizontal coordinate in the distorted image, y d Represents the vertical coordinate in the distorted image, x u is the distortion-free horizontal coordinate output by the distortion correction model, y u is the distortion-free ordinate output by the distortion correction model, θ1 and θ2 are the radial distortion coefficients, which are the parameters that need to be optimized;

[0117] Step A2: Use the checkerboard calibration plate to collect T_O groups of images, calculate the distortion parameters θ1 and θ2, and set the objective function to minimize the error between the input image and the ideal image.

[0118]

[0119] (x d，i ,y d，i ) is the horizontal and vertical coordinates of the ith distortion coordinate point, (x d，i (θ1,θ2),y d，i (θ1, θ2)) is the horizontal and vertical coordinates of the i-th undistorted coordinate point output by the distortion correction model, which depends on the distortion coefficients θ1 and θ2. M is the number of checkerboard corner points, that is, the total number of data points;

[0120] In high temperature environment, the distortion parameters change with temperature, so a temperature compensation model is introduced. Where T is the current temperature in the slag pit, T0 is the calibration temperature, and α and β are temperature compensation coefficients;

[0121] Step A3: Use θ1 and θ2 in the distortion correction model, perform reverse mapping correction on the image, calculate the corrected image through interpolation (such as bilinear interpolation), and output the image.

[0122] Using a checkerboard calibration plate, collecting T_O sets of images, and calculating the distortion parameters θ1 and θ2 are as follows:

[0123] Use a checkerboard calibration plate with known grid spacing. Take multiple images of the checkerboard at different angles and positions.

[0124] Use OpenCV's cv2.cvtColor() function to convert the image from RGB to grayscale.

[0125] Use OpenCV's cv2.findChessboardCorners() function to detect the corners of the chessboard.

[0126] Enter the size of the checkerboard (for example, an 8x8 checkerboard) and the image, and the function will return the locations of the detected corners.

[0127] Use OpenCV's cv2.calibrateCamera() function to calculate the distortion parameters.

[0128] Input the corner positions of the chessboard, the actual size of the chessboard, and the image size, and the function will return the camera's intrinsic parameter matrix and distortion coefficients.

[0129] This technical solution aims to address the impact of harsh environments such as high temperature and smoke in slag pits on infrared images, and has designed a complete pre-processing module covering frame extraction, noise reduction, clarity enhancement, deformation correction, and brightness adjustment. The solution combines the low contrast and high noise characteristics of infrared images, and adopts multi-stage noise reduction (Gaussian filtering, median filtering, NLM filtering), multi-method clarity enhancement (CLAHE, Laplace sharpening, morphological gradient) and dynamic distortion correction (temperature compensation) to significantly improve image quality. The system ensures real-time and robustness through parameter adaptation and parallel processing to meet the detection requirements of subsequent AI models. This solution is efficient, adaptable, and reliable, and can effectively solve the pre-processing problem of infrared images in slag pit scenarios, providing high-quality input data for slag accumulation identification.

[0130] Object detection module: Design an integrated model that outputs information about slag blocks using an improved YOLOv8 model and SAM model. The improved YOLOv8 model introduces the CBAM attention mechanism to enhance the model's focus on slag block features. The model is fine-tuned using optimized K-means clustering to output the ground truth bounding box and the number of slag blocks in the image.

[0131] The ground truth box refers to the position, size and area of ​​the slag block in the image.

[0132] The integrated model outputs slag information through the improved YOLOv8 model and SAM model. The method includes:

[0133] In YOLOv8's feature extraction network, the CBAM module is inserted after the output of each feature extraction layer (to enhance the focus of multi-scale features) and before the feature fusion layer (such as PANet) (to optimize the fusion of multi-scale features). Specifically, the CBAM module is inserted after the output of each residual block or CSP block in CSPDarknet53. This enhances the focus of each scale feature map and improves the expressiveness of features. Inserting the CBAM module before the feature fusion operation in PANet optimizes the fusion of multi-scale features and ensures that the fused feature maps are more representative.

[0134] In order to optimize the size and number of anchor boxes in YOLOv8 to adapt to the size distribution of slag targets and improve detection accuracy, an improved K-means clustering algorithm can be used.

[0135] Use optimized K-means clustering to perform multi-scale stratification of the target and output the size and number of the true box;

[0136] In the feature pyramid (FPN) of YOLOv8, high-resolution feature maps (such as the P2 layer) are set to adapt to small-sized slag blocks. The true frames of small objects are assigned to high-resolution feature maps (such as P3 and P2), and the true frames of medium and large objects are assigned to low-resolution feature maps (such as P4 and P5).

[0137] Set up a high-resolution feature map (such as P2 layer) and implement the following steps:

[0138] New P2 layer: Upsamples the C2 feature map (output of the second stage of CSPDarknet53, stride = 4) and fuses it with the P3 layer features;

[0139] Input the P2 layer features into the detection head: generate the prediction results of the small target ground truth box and enhance the feature expression of small-sized slag blocks.

[0140] The output of YOLOv8 is used as the input of the SAM model, and the output is the contour, area and shape information (aspect ratio, boundary complexity, etc.) of the slag block.

[0141] A training dataset is constructed to mark the locations and contours of slag blocks for pre-training of the YOLOv8 and SAM models. The models are not put into practical use until they meet the preset standards.

[0142] Methods for multi-scale stratification of targets using optimized K-means clustering include:

[0143] Step B1: Extract the true box sizes of all slag blocks, pre-define the true boxes into three categories: small, medium, and large, define the size range of each category, and assign the true boxes to small, medium, and large target categories;

[0144] Step B2: Use IoU distance instead of Euclidean distance in the optimized K-means, then the objective function is Among them, A is the total number of true boxes, j is the index of the cluster center, a is the index of the true box, J is the number of cluster centers, IoU(s j , g j ) is the IoU distance between the jth true box and the jth cluster center; the goal of the formula is to find the optimal cluster center so that the sum of the IoU distances between all true boxes and the nearest cluster center is minimized.

[0145] Step B3: Calculate the variance of the real box size distribution F_C of the image and dynamically adjust Among them, γ is the adjustment coefficient, which is predefined by the management personnel, and J0 is the basic quantity;

[0146] Step B4: For each level of targets, output the clustered true box size.

[0147] Assigning ground truth boxes to detection heads: Assign the ground truth boxes of small objects to high-resolution feature maps (such as P3 and P2). Assign the ground truth boxes of medium and large objects to low-resolution feature maps (such as P4 and P5).

[0148] Visualization module: Arranges the outline, area and shape information of the output slag block on the visualization interface, judges the slag accumulation status, and provides early warning information.

[0149] Arranging the outline, area, and shape information of the output slag block on a visual interface, judging the slag accumulation state, and providing early warning information include:

[0150] Predefine the area threshold and regular shape, and compare them with the output contour, area and shape information. If it is greater than the area threshold or does not conform to the regular shape, it is determined to be a slag accumulation state and an alarm signal is activated; otherwise, continue working.

[0151] The criteria for determining whether a slag block meets the regular shape criteria is that the regular shape can contain the outline and shape of the slag block. When an alarm signal is issued, cleaning equipment (such as vibrating screens and mechanical slag removal equipment) is automatically activated. The alarm signal can be sent in the form of SMS, email, or app push. The alarm content includes the location, area, and height of the slag accumulation.

[0152] Example 2

[0153] See also Figure 1 As shown, the parts not described in detail in this embodiment are described in Example 1, which provides an AI-based identification method for oversize, including:

[0154] Step S1: Install a high-precision infrared camera on the top or side wall of the slag pit to collect real-time video streams inside the slag pit;

[0155] Step S2: using a hybrid filtering algorithm to perform preliminary noise reduction on the image, using an improved NLM filter to perform noise smoothing, using an improved Laplace operator to perform image clarity processing, using a distortion correction model to perform image deformation correction, and outputting an image after image data enhancement;

[0156] Step S3: Design an integrated model. The integrated model outputs information about slag blocks through the improved YOLOv8 model and the SAM model. The image after image data enhancement is input into the integrated model, and the real box and number of slag blocks in the image are output.

[0157] Step S4: The output is the outline, area and shape information of the slag block arranged on a visual interface, interacting with the user, judging the slag accumulation state and providing early warning information.

[0158] Example 3

[0159] This embodiment discloses an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the operating mode of the AI ​​recognition system based on screen material provided above is realized.

[0160] Since the electronic device introduced in this embodiment is an electronic device used to implement an AI recognition system based on screen material in the embodiment of this application, based on the AI ​​recognition system based on screen material introduced in the embodiment of this application, those skilled in the art can understand the specific implementation of the electronic device of this embodiment and its various variations, so how the electronic device implements the method in the embodiment of this application will not be described in detail here. As long as those skilled in the art implement an electronic device used in an AI recognition system based on screen material in the embodiment of this application, it falls within the scope of protection of this application.

[0161] The above formulas are all dimensionless and numerical calculations. The formula is a formula that is closest to the actual situation obtained by collecting a large amount of data and performing software simulation. The preset parameters and thresholds in the formula are set by technicians in this field according to actual conditions.

[0162] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiment. All technical solutions based on the principles of the present invention are within the scope of protection of the present invention. It should be noted that for users of ordinary skill in the art, various improvements and modifications that do not depart from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. An AI-based identification system for oversize material, characterized in that: include: Image data acquisition module: used to collect video streams in the slag pit, extract key frames from the video streams using an adaptive frame rate, and obtain images to be processed; Image data enhancement module: Use hybrid filtering algorithm to perform preliminary noise reduction on the image, use improved NLM filtering to smooth the noise, use improved Laplace operator to clear the image, use distortion correction model to correct the image deformation, and output the image after image data enhancement; The method for correcting image deformation using a distortion correction model includes: Step A1: Build a distortion correction model Among them, x d Represents the horizontal coordinate in the distorted image, y d Represents the vertical coordinate in the distorted image, x u is the distortion-free horizontal coordinate output by the distortion correction model, y u is the distortion-free ordinate output by the distortion correction model, θ1 and θ2 are the radial distortion coefficients; Step A2: Use the checkerboard calibration plate to collect T_O groups of images, calculate the distortion parameters θ1 and θ2, and set the objective function to minimize the error between the input image and the ideal image. (x d,i ,y d,i ) is the horizontal and vertical coordinates of the ith distortion coordinate point, (x d,i (θ1,θ2),y d,i (θ1,θ2)) is the horizontal and vertical coordinates of the i-th distortion-free coordinate point output by the distortion correction model, and M is the number of checkerboard corner points; Introducing temperature compensation model Among them, T is the current temperature in the slag pit, T0 is the calibration temperature, α and β are temperature compensation coefficients, θ 1,0 and θ 2,0 is the initial value, θ1(T) and θ2(T) are the temperature compensated θ1 and θ2; Step A3: Use θ1 and θ2 in the distortion correction model to perform reverse mapping correction on the image, calculate the corrected image through interpolation, and output the image; Object detection module: Design an integrated model that outputs information about slag blocks using an improved YOLOv8 model and SAM model. The improved YOLOv8 model introduces the CBAM attention machine to enhance the model's focus on slag block features. The model is fine-tuned using optimized K-means clustering to output the ground truth bounding box and the number of slag blocks in the image. Visualization module: The output is the outline, area and shape information of the slag block arranged on the visualization interface, and the slag accumulation status is judged to provide early warning information.

2. The AI ​​recognition system based on oversize according to claim 1, characterized in that: The method of collecting the video stream in the slag pit and extracting key frames from the video stream using an adaptive frame rate includes: Install high-precision infrared camera equipment on the top or side wall of the slag pit to collect real-time video stream inside the slag pit; The initial frame rate is set to Z_1fps, so the initial frame interval is Collect the real-time speed of slag accumulation changes in the slag pit, dynamically adjust the frame rate by setting the speed interval, customize R speed intervals, set a frame rate for each speed interval, and the frame rate increases as the speed interval increases. Match the collected real-time speed with the speed interval, and use the frame rate corresponding to the matched speed interval as the real-time frame rate. According to the real-time frame rate, obtain the real-time frame interval, and extract image frames from the video stream according to the frame interval; Set the standard frame and change threshold, calculate the frame difference between the image frame and the standard frame, if the frame difference is greater than or equal to the change threshold, mark the image frame as a key frame, extract the image, and set it as the image to be processed, otherwise, do not extract the image.

3. The AI-based identification system for oversize material according to claim 2, characterized in that: The method for performing noise reduction processing on an image using a hybrid filtering algorithm comprises: In view of the thermal noise in the image, the Gaussian filter function is used to perform preliminary smoothing on the image. The standard deviation σ of the Gaussian filter function is set to be dynamically adjusted according to the smoke concentration, then σ=σ0+e k1+Δc ×C smoke , Δc represents the frame difference between the image frame and the standard frame, σ0 is the basic standard deviation, k1 is the adjustment coefficient, C smoke is the smoke concentration, and the standard frame is a predefined ideal image; For salt and pepper noise, the median filter method is used, and the median filter window is set to W = w0 + k2 × log (1 + C smoke ), w0 is the basic window size, and k2 is the adjustment coefficient, which is used to control the influence of smoke concentration on the window size.

4. The AI-based identification system for oversize material according to claim 3, characterized in that: The improved NLM filtering is used to perform noise smoothing, and the method includes: Apply NLM filtering to the input image to obtain the denoised image INLM; Perform wavelet decomposition on the image INLM after NLM filtering to obtain low-frequency components and high-frequency components; A denoising threshold is preset and threshold denoising is applied to the high frequency component, that is, if the absolute value of the wavelet coefficient is greater than the threshold, the wavelet coefficient is retained, otherwise, the wavelet coefficient is set to zero; The image is reconstructed using the denoised high-frequency components and the unchanged low-frequency components, and the reconstructed image is output as the final denoising result.

5. The AI-based identification system for oversize material according to claim 4, characterized in that: The method for performing image sharpening processing using the improved Laplace operator comprises: The CLAHE method was used for preprocessing; Applying the improved Laplacian operator Among them, I sh (x,y) is the input image, I en (x,y) is the enhanced image, is the Laplace operator, k3 is the sharpening coefficient, which is used to control the intensity of sharpening. k3 is designed as N is the total number of pixels in the input image, is the gradient of the original image I(x,y) at point (x,y), ∈ is a small positive number used to avoid the denominator being zero, and I_σ is the gradient of the image I sh The variance of (x,y); Enhance edges using morphological gradients.

6. The AI-based identification system for oversize material according to claim 5, characterized in that: The integrated model outputs information about the slag block through an improved YOLOv8 model and a SAM model, and the method includes: In the feature extraction network of YOLOv8, the CBAM module is inserted after the output of each residual block or CSP block of CSPDarknet53, and before the feature fusion operation of PANet; Use optimized K-means clustering to perform multi-scale stratification of the target and output the size and number of the true box; In the feature pyramid of YOLOv8, a high-resolution feature map is set, and the true frames of small objects are assigned to the high-resolution feature map, and the true frames of medium and large objects are assigned to the low-resolution feature map; The output of YOLOv8 is used as the input of the SAM model, and the output is the contour, area and shape information of the slag block.

7. The AI-based identification system for oversize material according to claim 6, characterized in that: The method for performing multi-scale stratification of targets using optimized K-means clustering includes: Step B1: Extract the true box sizes of all slag blocks, pre-define the true boxes into three categories: small, medium, and large, define the size range of each category, and assign the true boxes to small, medium, and large target categories; Step B2: Use IoU distance instead of Euclidean distance in the optimized K-means, then the objective function is Among them, A is the total number of true boxes, j is the index of the cluster center, a is the index of the true box, J is the number of cluster centers, IoU(s j , g j ) is the IoU distance between the jth ground truth box and the jth cluster center; Step B3: Calculate the variance of the real box size distribution F_C of the image and dynamically adjust J= Among them, γ is the adjustment coefficient, J0 is the basic quantity; Step B4: For each level of targets, output the clustered true box size.

8. The AI-based identification system for oversize material according to claim 7, characterized in that: The method of arranging the outputted contour, area and shape information of the slag block on a visual interface, judging the slag accumulation state and providing early warning information includes: Predefine the area threshold and regular shape, and compare them with the output contour, area and shape information. If it is greater than the area threshold or does not conform to the regular shape, it is determined to be a slag accumulation state and an alarm signal is activated; otherwise, continue working.

9. A method for AI-based identification of oversize, applied to the AI-based identification system for oversize according to any one of claims 1 to 8, characterized in that: The AI ​​identification method based on oversize material includes: Step S1: Install a high-precision infrared camera on the top or side wall of the slag pit to collect real-time video streams inside the slag pit; Step S2: using a hybrid filtering algorithm to perform preliminary noise reduction on the image, using an improved NLM filter to perform noise smoothing, using an improved Laplace operator to perform image clarity processing, using a distortion correction model to perform image deformation correction, and outputting an image after image data enhancement; Step S3: Design an integrated model. The integrated model outputs information about slag blocks through the improved YOLOv8 model and the SAM model. The image after image data enhancement is input into the integrated model, and the real box and number of slag blocks in the image are output. Step S4: The output is the outline, area and shape information of the slag block arranged on a visual interface, interacting with the user, judging the slag accumulation state and providing early warning information.

Images