A method and system for obtaining key frames of images at the tail of a pellet belt induration machine

Abstract

This invention discloses a method and system for keyframe acquisition and image restoration of the tail section of a pellet belt roaster. The method, based on joint constraints of target detection and spatial-temporal continuity, accurately selects keyframe image groups with consistent temporal sequence and optimal clarity from the video stream of the tail section under harsh operating conditions. It extracts multi-dimensional fog features and calculates the pixel-level dust transmittance of each frame using a Gaussian mixture model. To overcome the technical bottleneck of traditional single-frame defogging algorithms failing in heterogeneous high-dust environments, this invention utilizes the high dynamic flow velocity characteristics of tail dust, transforming multi-frame transmittance into temporal visual trust weights. By constructing a multi-frame joint defogging model for global accumulation and single-time restoration, it transforms single-frame spatial defogging into multi-frame temporal optimization and complementarity, improving the usability and restoration fidelity of material layer images under conditions of strong dust and sudden changes in illumination.

Description

Technical Field

[0001] This invention relates to the field of pellets and image recognition, and in particular to a method and system for acquiring keyframes and restoring images of the tail section of a pellet belt roaster. Background Technology

[0002] Large-scale belt calciner processes have become a major development trend in pellet production technology in recent years. They offer high heat utilization, large single-unit capacity, and strong raw material adaptability, making them suitable for the large-scale, intensive needs of the steel industry. The cross-sectional image of the belt calciner's tail section contains rich color and texture information, serving as a crucial basis for real-time perception of calcination quality and optimization of thermal regimes. However, obtaining high-quality tail section images presents significant challenges.

[0003] The operating conditions at the tail end of the belt roaster are extremely harsh, often accompanied by high concentrations of randomly distributed clustered dust, and drastic fluctuations in lighting, resulting in degradation phenomena such as non-uniform fogging, low contrast, and localized exposure anomalies in the acquired video sequences. Simultaneously, the periodic movement of the trolley causes the spatiotemporal position of the observed target in the material layer to constantly change, making the effective observation window short and unstable. Furthermore, existing imaging technologies are insufficiently applicable to this specific scenario:

[0004] First, in terms of frame selection, existing methods mostly rely on human experience or a single brightness / sharpness threshold rule. When encountering dust obscuring or detection jitter, it is very easy to misselect or miss the selection, making it difficult to obtain key frames that are consistent in time and representative.

[0005] Secondly, in terms of image restoration, traditional dehazing algorithms are usually based on the assumption that "transmittance varies with depth of field" in natural atmospheric scenes. However, the tail section scene has the unique characteristics of "approximately constant physical depth of field, but extremely uneven spatial distribution of dust scattering coefficients," coupled with the high dynamic flow velocity of dust brought by the trolley unloading. This means that in most single-frame images, some material layer areas will be completely obscured by high-concentration dust, resulting in the complete loss of physical optical information. Directly applying existing single-frame dehazing algorithms, when faced with dense dust pixels with transmittance approaching zero, will lead to severe truncation errors and noise amplification when performing forced division operations, resulting in large areas of color spots and artifacts.

[0006] In summary, there is an urgent need for a method to acquire and restore keyframes of pellet belt roaster tail images that can automatically select representative and temporally consistent keyframes from video sequences and improve image clarity and texture detail discernibility, thereby providing reliable image data support for online evaluation of pellet roasting quality and thermal regime control. Summary of the Invention

[0007] To address the problems of high manual dependence, low selection accuracy, and poor temporal consistency in existing keyframe acquisition technologies for the tail section images of belt calciners, as well as the technical problems of insufficient restoration, over-enhancement, or artifacts caused by uneven dust scattering in existing dehazing methods, this invention provides a method and system for keyframe acquisition and image restoration of the tail section images of a pellet belt calciner. The method achieves stable acquisition and clear restoration of tail section images under complex working conditions through a temporal consistency keyframe screening mechanism based on target detection and an adaptive dehazing restoration strategy combining multiple frames.

[0008] To achieve the above technical objectives, the technical solution adopted by the present invention is as follows: In a first aspect, the present invention provides a method for acquiring keyframes and restoring images of the tail section of a pellet belt roaster, comprising: S1. Obtain the historical video stream of the tail of the pellet belt roaster and perform frame extraction processing, and construct a pellet material layer detection dataset by combining it with manually labeled information; use the dataset to iteratively train a deep learning network with preset initial parameters and update the network weights to obtain a detection model that can output the target box of the pellet material layer. S2. Obtain the video sequence of the trolley's motion trajectory, use the detection model to predict the video sequence, and obtain the target box of the target ball material layer; extract the frame sequence that continuously meets the target box screening conditions as a key frame group, and select one frame image from it as a reference frame; obtain the reference frame and its temporally adjacent images to construct a target frame set; S3. Extract the material layer feature points of each frame image in the target frame set, calculate the corresponding perspective transformation matrix based on the material layer feature points; use the perspective transformation matrix to perform inverse perspective transformation on each frame image in the target frame set, and uniformly map the images with perspective distortion to the preset reference coordinate system to obtain a spatially aligned transformed image sequence. S4. Extract the dust concentration distribution features of each frame in the transformed image sequence, and then calculate the transmittance map of each frame; use the transmittance map of each frame as the fusion weight to construct a global denominator aggregation matrix, calculate the residual value after deducting the atmospheric light component from the observed image, and construct a global numerator aggregation matrix; finally, use the numerator aggregation matrix and the denominator aggregation matrix to perform a single division operation to output the pellet layer reconstruction image.

[0009] Furthermore, in S1, the pellet material layer detection dataset specifically includes a positive sample subset and a negative sample subset; wherein, the positive sample subset consists of images containing complete material layer texture features and geometric shapes obtained during the horizontal operation phase of the trolley; the negative sample subset consists of images of material layer collapse, gradual disappearance and eventual complete loss obtained during the tail feeding phase and the no-load rotation phase of the trolley.

[0010] Furthermore, in S2, the specific process of acquiring the keyframe group is as follows: S21. Based on the target bounding box of the acquired frame t image, obtain the lower boundary ordinate and vertical height of the target bounding box; S22. Position constraint condition: The absolute value of the difference between the ordinate of the lower boundary and the ordinate of the preset observation baseline is less than or equal to the preset position deviation threshold. S23. Longitudinal dimensional constraint: The longitudinal height of the material layer shall not exceed the preset effective height range; S24. Temporal Consistency Constraint: The current state is determined to be a stable observation state only when the target boxes of N consecutive frames of images simultaneously satisfy the positional constraint and the longitudinal scale constraint, and the corresponding set of frame sequences is extracted as a keyframe group.

[0011] Furthermore, in step S2, a cascading filtering strategy is used to select the frame with the highest clarity from the keyframe image group as the reference frame. The specific process is as follows: S25. Exposure Filtering: Count the number of pixels in each image of the key frame image group whose gray value is lower than the preset black level threshold, calculate the proportion of them to the total number of pixels in the image, and when the proportion exceeds the preset black tolerance threshold, determine the current frame as an invalid exposure frame and remove it to obtain valid exposure frames and construct a set of valid exposure frames. S26. Sharpness optimization: Based on the lower boundary ordinate and vertical height of the target box in each frame of the effective frame set, the target analysis area is obtained; the Laplacian operator is used to perform local convolution operation on the target analysis area of ​​the effective exposure frame to extract the material layer edge gradient information, the variance of the gradient magnitude is calculated as the image sharpness score, and the frame with the highest score is selected as the benchmark frame. Furthermore, image sharpness scoring The calculation formula is: ; in, The Laplace operator, where Var represents variance operation. For the input image, In the horizontal direction, It is in the vertical direction.

[0012] Furthermore, in step S3, the process of obtaining the spatially aligned transformed image sequence is as follows: S31. Blind calibration distortion correction: Using a blind calibration algorithm based on straight line features or texture distribution, nonlinear perspective distortion correction is performed on each original image in the target frame set to obtain a corrected image sequence that eliminates lens distortion. S32.ROI Spatial Constraint: The target bounding box of the pellet layer determined in S2 is taken as the region of interest; S33. Stable protrusion feature extraction: Within the preset ROI, morphological operators are used to specifically extract the physical protrusion geometric vertices that characterize the boundary of the pellet entity as candidate feature points; a zero displacement spatial constraint is established based on the relative coordinate system of the trolley, and transient pseudo feature points with relative displacement greater than the preset small rigidity tolerance threshold are filtered out, and the entity protrusion points that always remain relatively stationary are established as material layer feature points. S34. Initialization of the periodic prior matrix: Extract the historical annotation data of the single-cycle operation of the belt roasting machine trolley, and pre-calculate and construct the initial reference perspective transformation matrix by performing polygon fitting analysis on the reference plane within a single cycle. S35. Inter-frame dynamic matrix calculation: The material layer feature points extracted in the current frame are spatially matched with the corresponding feature points in the reference frame, and the dynamic affine matrix representing the local relative motion between frames is calculated. S36. Cascaded Update and Spatial Mapping: The reference perspective transformation matrix and the dynamic affine matrix are cascaded and multiplied to obtain the target perspective transformation matrix updated in the current frame; the target perspective transformation matrix is ​​used to perform inverse perspective transformation on the corrected image sequence, and the image is uniformly projected into a preset orthogonal top-view coordinate system to obtain a spatially aligned transformed image sequence.

[0013] Furthermore, the specific process in S4 is as follows: S41. Physical Prior Decoupling: Extract the prior features of color attenuation, color ellipsoid, dark channel, wavelet texture energy, and fog line from the image, and use the Gaussian mixture model to decouple and obtain the pixel-level dust concentration distribution map. S42. Perform a linear transformation on the obtained pixel-level dust concentration distribution map to obtain a transmittance map; S43. Constructing a spatiotemporal trust evaluation index: Based on the high-frequency transient non-stationary occlusion characteristics of tail dust driven by high-temperature thermal convection, the transmittance map of the acquired current frame image is directly mapped to the fusion weight of that frame in the multi-frame fusion matrix. S44. Constructing the Global Physical Residual Tensor: Obtain the observed image and atmospheric light value of the current frame, and calculate the intrinsic physical residual after deducting the dust scattering interference component; using the dust transmittance of each frame image as a prior constraint, aggregate the intrinsic physical residuals in the time sequence at the pixel level to construct a global physical residual tensor containing spatiotemporal complementary information, i.e., the global molecular aggregation matrix N. sum : ; Where N is the number of frames. For pixels, Given the input image i, Atmospheric light value, For image frames The fusion weights; S45. Constructing the global confidence matrix: Aggregate the spatiotemporal trust evaluation indicators corresponding to each frame of the image in the time series pixel by pixel to construct the global confidence matrix, i.e., the global denominator aggregation matrix D. sum : ; S46. Physical Reconstruction: After all the transformed image sequences have been traversed, the global physical residual tensor is divided element by element by the global confidence matrix to obtain the reconstructed image of the pellet layer.

[0014] Furthermore, in step S42, the formula for calculating the transmittance map is as follows: ; in, This is a transmittance diagram. To preserve the coefficient of variation, This is a dust concentration distribution map.

[0015] Furthermore, dynamic atmospheric light calculation is employed for industrial site lighting flicker: each frame in the transformed image sequence is calculated independently: the region with the highest dust concentration in the current frame image is extracted according to a preset ratio; subsequently, the average RGB three-channel pixel value corresponding to this region in the original observation image is calculated as the local atmospheric light value A specific to the current frame image. i When using it, simply substitute the local atmospheric light value into the molecular aggregation matrix.

[0016] Secondly, the present invention provides a system for acquiring keyframe images and restoring images of the tail section of a pellet belt roaster, wherein the system performs the method described above, including: The target bounding box detection model construction module is used to acquire the historical video stream of the tail of the pellet belt roaster and perform frame extraction processing, and combine it with manually labeled information to construct a pellet material layer detection dataset; the dataset is used to iteratively train a deep learning network with preset initial parameters and update the network weights to obtain a detection model that can output the target bounding box of the pellet material layer. Target frame set acquisition module: acquires the motion trajectory video sequence of the trolley, uses a detection model to predict the video sequence to obtain the target box of the target ball material layer; extracts the frame sequence that continuously meets the target box screening conditions as a key frame group, and selects one frame image from it as a reference frame; acquires the reference frame and its temporally adjacent images to construct the target frame set; Transformed image sequence acquisition module: extracts the material layer feature points of each frame image in the target frame set, calculates the corresponding perspective transformation matrix based on the material layer feature points, and uses the perspective transformation matrix to perform inverse perspective transformation on each frame image in the target frame set, uniformly mapping the images with perspective distortion to a preset reference coordinate system to obtain a spatially aligned transformed image sequence. The pellet layer image reconstruction module extracts the dust concentration distribution features of each frame in the transformed image sequence, and then calculates the transmittance map of each frame; using the transmittance map of each frame as the fusion weight, a global denominator aggregation matrix is ​​constructed, and the residual value after subtracting the atmospheric light component from the observed image is calculated to construct a global numerator aggregation matrix; finally, a single division operation is performed between the numerator aggregation matrix and the denominator aggregation matrix to output the reconstructed pellet layer image.

[0017] This invention proposes a method and system for keyframe acquisition and image restoration of the tail section of an iron ore pellet belt roaster. Compared with existing technologies, the method has the following advantages:

[0018] 1. This invention achieves high-quality automated selection of keyframes under complex working conditions. By employing joint constraints of geometric position, longitudinal scale, and temporal consistency, it effectively suppresses detection jitter and occasional false detections, ensuring the stability of the observation state. Simultaneously, it introduces an exposure validity verification and sharpness cascade filtering mechanism to automatically eliminate low-quality frames caused by lighting failure or dust obstruction. This multi-dimensional filtering strategy significantly reduces the cost of manual intervention and ensures a high degree of consistency in the spatiotemporal distribution and texture quality of keyframes.

[0019] 2. This invention overcomes the bottleneck of image restoration in non-uniform dust environments, improving the discernibility of material layer details. Addressing the unique physical characteristics of near-constant depth of field but non-uniform dust scattering at the tail of the aircraft, this invention abandons the traditional assumption of depth-of-field dehazing and constructs an adaptive transmittance estimation algorithm based on multi-dimensional feature perception and GMM probabilistic soft clustering. Combined with the edge-preserving properties of guided filtering, this method can accurately perceive and remove clustered dust occlusions, effectively avoiding halo artifacts while enhancing image contrast and visibility.

[0020] 3. Utilizing the high dynamic characteristics of dust, this invention solves the pathological amplification problem of single-frame dehazing algorithms in dense dust areas. Addressing the transient occlusion phenomenon caused by the rapid movement of dust at the tail of a belt roaster, this invention transforms the single-frame dehazing formula into a multi-frame numerator-denominator joint accumulation model. This mechanism converts transmittance into a trust weight for temporal fusion, adaptively extracting the optimal physical observation moment for each pixel on the time axis. This not only avoids the division-to-zero overflow and exponential noise amplification problems caused by transmittance approaching zero in traditional single-frame dehazing algorithms, but also leverages the complementary advantages of multi-frame temporal information to achieve high-quality material layer texture reconstruction exceeding the physical limits of a single frame under extremely dense and rapidly moving dust interference. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart of a method for acquiring keyframes and restoring images of the tail section of an iron ore pellet belt roaster, provided by an embodiment of the present invention. Figure 2 This is a schematic diagram of the target detection verification set results at the tail section of the roasting machine provided in an embodiment of the present invention; wherein, Figure 2 (a) is a set of truth graphs for the validation set labels. Figure 2 (b) is a set of plots showing the prediction results for the validation set; Figure 3 These are the keyframe groups for runtime cycle A and the corresponding reference frame diagrams for the keyframe groups provided in this embodiment of the invention; wherein, Figure 3 (a) is a schematic diagram of a keyframe group. Figure 3 (b) is a schematic diagram of the reference frame for the keyframe group; Figure 4 The embodiments of the present invention provide an image of the original material layer in period A and a multidimensional fog feature vector map; wherein, Figure 4 (a) is the original material layer image. Figure 4 (b) is the dust concentration perception map obtained by GMM soft clustering; Figure 5 This is a composite diagram of the dust concentration sensing map provided in the embodiments of the present invention; wherein, Figure 5 (a) represents the prior features of color attenuation. Figure 5 (b) represents the prior features of the color ellipsoid. Figure 5 (c) represents the prior features of the dark channel. Figure 5 (d) represents the wavelet texture energy feature. Figure 5 (e) represents the prior features of the fog line; Figure 6 These are the reference frame and the distortion-free effect diagram provided in this embodiment of the invention for calculating the distortion model and the reference perspective transformation matrix; wherein, Figure 6 (a) is a schematic diagram of the reference frame for calculating the distortion model and the reference perspective transformation matrix. Figure 6 (b) is the result after distortion correction; Figure 7 These are the original image and the restored clear image provided in the embodiments of the present invention; wherein, Figure 7 (a) is the original image. Figure 7 (b) is the restored clear image. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be described in detail below. Obviously, the described embodiments are merely some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other implementation methods obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0024] Example 1

[0025] This embodiment provides a method for acquiring keyframes and restoring images of the tail section of an iron ore pellet belt roaster, including: S1: Obtain the historical video stream of the tail of the pellet belt roaster and perform frame extraction processing, and construct a pellet material layer detection dataset by combining it with manually labeled information; use the dataset to iteratively train a preset deep learning network and update the network weights to obtain a detection model that can output accurate target boxes of pellet material layers.

[0026] Specifically, the pellet material layer detection dataset includes a positive sample subset and a negative sample subset. The positive sample subset consists of images containing complete material layer texture features and geometric shapes acquired during the horizontal operation phase of the trolley. The negative sample subset consists of images of material layer collapse, gradual disappearance, and eventual complete loss acquired during the tail feeding phase and the no-load rotation phase of the trolley.

[0027] In this specific implementation, the pellet material layer detection dataset covers the detection datasets of multiple machine cycles. This dataset uses the material layer during the stable conveying stage as positive samples and the background during the tail discharge collapse and no-load rotation stages as negative samples. The YOLOv5 model is used for iterative training so that the network can fully learn the visual boundary between the material layer and the background of the machine under harsh tail conditions and obtain the target detection model weights.

[0028] S2: Obtain the video sequence of the trolley's motion trajectory, use the detection model to predict the video sequence, and obtain the target box of the target ball material layer; extract the frame sequence that continuously meets the target box screening conditions as a key frame group, and select one frame image from it as a reference frame; obtain the reference frame and its temporally adjacent images to construct a target frame set; In practice, the acquisition of the video sequence of the trolley's motion trajectory involves the continuous movement of the belt roaster trolley from far to near within the field of view of the tail camera. Continuous video frames at the end of the horizontal translation phase and before the rotational tilting action are extracted from the motion trajectory. The trained detection model is used to predict the video sequence, obtaining the target bounding box of the target ball material layer. Based on the lower boundary of the target bounding box and the physical material layer thickness (i.e., the height of the target bounding box), the target analysis region is determined. The image sharpness within the target analysis region is calculated, and the frame with the highest sharpness is selected as the reference frame. The reference frame and its temporally adjacent images are acquired to construct a target frame set.

[0029] Specifically, the process of acquiring keyframe groups is as follows: S21: Obtain the target bounding box of the t-th frame image. Its coordinates are defined as ,in The coordinates of the top left corner Using the coordinates of the bottom right corner, we obtain the ordinate of the bottom boundary and the vertical height of the target box; S22: Positional Constraints: To ensure uniformity of the acquisition perspective, the algorithm constrains the bottom edge of the material layer to be located near the preset observation baseline of the image. Therefore, the lower boundary ordinate is calculated. ordinate with the preset observation baseline The absolute value of the difference is used to determine whether it is less than or equal to a preset position deviation threshold. ,Right now: ; in, This represents the absolute value operation. The tolerance is set according to the field of view size (in this embodiment, it is set to 15 pixels); S23: Longitudinal Dimension Constraint: To eliminate incompletely inspected or abnormally shaped material layers, the visible thickness of the material layer needs to be constrained. The material layer height is defined as... The preset effective height range is It is determined whether the vertical height is within the preset effective height range, that is, the effective frame must satisfy: ; S24: Temporal Consistency Constraint: To eliminate detection jitter, a temporal stability criterion is introduced. The current state is determined to be a stable observation state only when the target bounding boxes of N consecutive frames simultaneously satisfy both the positional constraint and the longitudinal scale constraint, and the corresponding frame sequence set {F} is extracted. t , F t+1 , …, F t+N-1} as a keyframe group; Define the indicator function I(t): ; The conditions for obtaining keyframe groups are: ; Wherein, N is the preset number of consecutive frames (N=5 in this embodiment); This indicates a multiplication operation, meaning that all N consecutive frames starting from frame t must be valid frames in order to trigger the saving mechanism.

[0030] The image quality of the belt roaster tail section is affected by the automatic exposure of the camera. During the rapid movement of dust, the exposure parameters cannot be accurately tracked, resulting in overall image quality degradation.

[0031] Specifically, a cascading filtering strategy can be used to select the frame with the highest resolution from the keyframe image group as the reference frame. The specific process is as follows: S25: Exposure Filtering: Count the number of pixels in the keyframe image group whose grayscale value is lower than the preset black level threshold, calculate the proportion of the total number of pixels in the image, and when the proportion exceeds the preset black level tolerance threshold, determine the current frame as an invalid exposure frame and remove it to obtain valid exposure frames, and construct a set of valid exposure frames.

[0032] In specific implementation, for image I i First, it is converted to a grayscale image. To remove "dead black" images caused by dust obscuring the light source or dark corners, a pixel grayscale threshold T is defined. black (e.g. T) black = 5), in the statistical image, pixel(x, y) ≤ T black Number of pixels N dead Calculate the percentage of black pixels (R). black : ; Set the maximum tolerance θ tol (e.g., 1%). If R black > θ tol Then determine the image I i If a frame is an invalid exposure frame, the subsequent calculation is terminated and the frame is discarded; otherwise, it is marked as a valid exposure frame and saved to the valid frame set {I}. valid}

[0033] S26: Sharpness Optimization: Based on the lower boundary ordinate and vertical height of the target box in each frame of the effective frame set, the target analysis area is obtained; the Laplacian operator is used to perform local convolution operation on the target analysis area of ​​the effective exposure frame to extract the gradient information of the material layer edge, the variance of the gradient magnitude is calculated as the image sharpness score, and the frame with the highest score is selected as the benchmark frame.

[0034] In practice, for all valid frames {I} that pass the exposure screening... valid The texture richness of the material surface is evaluated using a second-order differential operator. The variance of the Laplacian convolution result is used as the image sharpness score S. clarity The calculation formula is: ; in, The Laplace operator, where Var represents variance operation. For the input image, In the horizontal direction, In the vertical direction, a larger variance value indicates sharper edges and crack details in the image. Finally, the image frame with the highest score is selected as the baseline frame. ; S27: Construct the target frame set: Using the reference frame as the last frame, select the preset number of adjacent image frames in time sequence as the target frame set to ensure that it is strictly within the translational movement range of the trolley.

[0035] In this specific implementation, a first-in-first-out (FIFO) buffer queue of length 5 is maintained synchronously in the background during image filtering. When the highest resolution reference frame is confirmed, the system immediately freezes and exports the current queue, directly obtaining the target frame set consisting of the reference frame and its four immediately preceding frames.

[0036] S3: Extract the material layer feature points of each frame image in the target frame set, calculate the corresponding perspective transformation matrix based on the material layer feature points; use the perspective transformation matrix to perform inverse perspective transformation on each frame image in the target frame set, and uniformly map the images with perspective distortion to the preset reference coordinate system to obtain a spatially aligned transformed image sequence.

[0037] To address the physical bottleneck of highly disordered surface texture of pellet material layers and the failure of traditional feature matching due to strong dust interference, this paper extracts material layer feature points from each frame image in the target frame set, calculates the perspective transformation matrix based on the prior knowledge of mechanical rigid body motion, and performs inverse perspective transformation, specifically including: S31: Blind calibration distortion correction: Using a blind calibration algorithm based on straight line features or texture distribution, nonlinear perspective distortion correction is performed on each original image in the target frame set to obtain a corrected image sequence that eliminates lens distortion.

[0038] Specifically, based on the physical prior that a spatial straight line remains a straight line after projection, a set of straight line edge feature points of the frame or trolley in the image is extracted. An inverse distortion iterative mapping function based on error feedback is constructed using the Brown-Conrady distortion model, and global optimization is performed with the objective of minimizing the sum of squared residuals of the two-dimensional straight line fitting of the correction point set to calculate the optimal radial distortion coefficient. Finally, this coefficient is used to perform global pixel remapping of the target frame set to eliminate lens optical distortion and obtain a geometrically restored corrected image sequence. The global optimization objective function used is: ; Where E(K) is the total error energy that varies with the set of distortion parameters K (such as the k1 parameter of the dominant radial distortion), S is the total number of point sets, (x i , y i Let be the i-th corrected coordinate point after iterative mapping, and g be the slope of the straight line fitted to this point set. Let be the intercept of the line fitted to this set of points.

[0039] S32: ROI spatial constraint: The target frame of the pellet material layer determined in S2 is used as the ROI, and the stray background and trolley edge interference outside the target frame are shielded. S33: Stable Protrusion Feature Extraction: Due to factors such as particle stacking, light flickering, and dust obstruction, the surface of the pellet material layer exhibits a highly random and chaotic texture distribution. If traditional global texture feature extraction algorithms (such as SIFT, SURF, or ORB operators) are directly used, it is very easy to extract false feature points generated by changes in light and shadow, which will lead to a large number of mismatches in subsequent inter-frame matching, resulting in the failure of perspective transformation matrix calculation. Within the preset ROI, morphological operators are used to specifically extract the physical protrusion geometric vertices representing the boundary of the pellet entity as candidate feature points; a zero-displacement spatial constraint is established based on the relative coordinate system of the trolley, and the candidate feature points are tracked across frames in continuous temporal video frames to forcibly filter out transient false feature points (such as free dust or light spots) with relative displacement greater than a preset small rigidity tolerance threshold, and the solid protrusion points that always remain relatively stationary are established as the feature points of the material layer. The specific process is as follows: S3-3-1: Inside the target box, skip the detection of messy surface textures and focus on the physical spatial morphology of the boundary of the spherical particle entity. Use morphological operators to extract the set of geometric vertices of its convex contour as the initial candidate feature points. S3-3-2: Zero-Displacement Spatial Constraints and Transient False Point Removal: A local relative reference system is established based on the target frame position (representing the trolley's reference position). The relative coordinates of the candidate feature points in continuous temporal video frames are tracked and calculated, retaining candidate feature points with zero relative displacement (or less than a preset small rigidity tolerance threshold). The rigid kinematic characteristics of the belt roasting machine trolley are transformed into spatial constraints for image processing (the spatial constraint condition is that the relative displacement is zero or less than a preset small rigidity tolerance threshold). Transient false feature points with relative displacement caused by sudden changes in light and shadow, free dust obstruction, or local small slippage of the material layer are directly filtered out using the spatial constraint conditions.

[0040] S3-3-3: After screening, the physical convex points that remain absolutely stationary relative to the trolley in multiple consecutive images are established as the material layer feature points for the final inter-frame alignment calculation.

[0041] S34: Periodic Prior Matrix Initialization: Extract the single-cycle historical annotation data of the belt roasting machine trolley operation, and pre-calculate and construct the initial reference perspective transformation matrix by performing polygon fitting analysis on the reference plane within a single cycle.

[0042] Specifically, a two-dimensional reference plane is fitted using historical annotation data from a single cycle of the belt roasting machine trolley, and the initial reference perspective transformation matrix H is pre-calculated. base This matrix is ​​a 3×3 homography matrix, and its mathematical expression is: ; Where (x, y) are the original image pixel coordinates, (x', y', w') are the homogeneous coordinates after projection, and h 11 to h 32 These are the prior perspective parameters that characterize the camera pose and the plane.

[0043] S35: Inter-frame dynamic matrix calculation: Spatial position matching of the material layer feature points extracted in the current frame with the corresponding feature points in the reference frame, and calculation of the dynamic affine matrix representing the local relative motion between frames.

[0044] Specifically, the dynamic affine matrix A, which represents the local relative motion between frames (i.e., translation and small rotation), is solved using the least squares method. dyn Its formula is defined as: ; Among them, (u ref , v ref ) and (u t , v t ) represent the pixel coordinates of the same physical convex point in the reference frame and the current frame, respectively, t x , t yLet a be the translation amount. 11 To a 22 Includes rotation and scaling parameters.

[0045] S36: Cascaded Update and Spatial Mapping: The reference perspective transformation matrix H is... base With the dynamic affine matrix A dyn Perform concatenated multiplication to obtain the target perspective transformation matrix H after the current frame is updated. target Using the target perspective transformation matrix H target An inverse perspective transformation is performed on the corrected image sequence, and the images are uniformly projected into a preset orthogonal top-view coordinate system to obtain a spatially aligned transformed image sequence. The nonlinear motion of the material layer caused by the camera's perspective projection is strictly mapped into a spatiotemporally steady-state sequence with absolutely consistent spatial pose in the reconstructed plane, providing a spatial alignment prerequisite for subsequent multi-frame temporal trust weight allocation.

[0046] S4: Extract the dust concentration distribution features of each frame in the transformed image sequence, and then calculate the transmittance map of each frame; use the transmittance map of each frame as the fusion weight to construct a global denominator aggregation matrix, calculate the residual value after deducting the atmospheric light component from the observed image, and construct a global numerator aggregation matrix; finally, use the numerator aggregation matrix and the denominator aggregation matrix to perform a single division operation to output a clear image of the pellet layer reconstruction.

[0047] To address the issues of non-uniform dust occlusion, low contrast, and color distortion in the images of the tail material layer of a pellet belt roaster, the specific process for extracting the dust concentration distribution features of each frame in the transformed image sequence is as follows: S41: Physical Prior Decoupling: Extract the color attenuation prior (CAP), color ellipsoid prior (CEP), dark channel prior (DCP), wavelet texture energy features, and fog line prior features from the image. Use a Gaussian mixture model (GMM) to decouple these features and obtain a pixel-level dust concentration distribution map. This concentration distribution map characterizes the transient occlusion thickness of free dust in the three-dimensional space of the tail section on the two-dimensional projection surface.

[0048] Specifically, in order to comprehensively capture the visual characteristics of different forms of dust (such as dense smoke, thin floating dust, and colored dust), the five-dimensional feature vector of the material layer image I(m) is first extracted, and a high-dimensional feature tensor F(m) = {f 1, f 2, f 3, f 4, f5}: Color attenuation prior feature f1: The difference between saturation and brightness in the HSV color space is used to characterize the fog concentration, that is: ; in, Let m be the lightness component of pixel m in the HSV color space. represents the saturation component of pixel m in the HSV color space.

[0049] Color ellipsoidal prior feature f2: The ellipsoidality of the RGB pixel vector distribution within the local window in the color space, reflecting the degree to which a pixel deviates from the center of the color cluster. Define pixel m within the local window... The mean of the color channels c∈{R, G, B} is μ. c (m), with standard deviation σ c (m), then the CEP feature f2(m) is defined as: .

[0050] Dark channel prior feature f3: Calculate the minimum value of the RGB three channels within the local window, i.e.: ; in, Let c be the grayscale / intensity value of the c-th color channel at pixel position m in image I.

[0051] Wavelet texture energy feature f4: Perform discrete wavelet transform on the image to extract the energy modulus values ​​of the high-frequency subbands (LH, HL, HH). Since dust blurs the texture, the lower the high-frequency energy, the higher the corresponding fog concentration feature value. Mathematically defined as: ; in, For normalization function, High-frequency energy: ; Where LH(m), HL(m), and HH(m) are the high-frequency subband coefficients.

[0052] Fog line prior feature f5: Based on the characteristic that pixels tend to cluster along fog lines in RGB space, the projected distance from the fog line to the atmospheric light point is estimated. Mathematically defined as: ; Where I(m) is the foggy image, A is the atmospheric light value, and H(m) is the set of pixels that are located on the same fog line as pixel m.

[0053] A Gaussian mixture model (GMM) is used for unsupervised learning of the aforementioned five-dimensional feature space to achieve soft sensing of dust concentration. A preset number of concentration levels K is set (K=5 in this embodiment). The five-dimensional feature vector of each pixel is input into the GMM for fitting, and the posterior probability of each pixel m belonging to the Gaussian component of the k-th concentration level is calculated. ,satisfy .

[0054] Index value w for each concentration level k Using ∈{0,1,…,K-1} as weights, a weighted sum is performed using the posterior probability to generate a continuous initial dust concentration distribution map D. raw (m): .

[0055] S42: Use with edge protection factor The pixel-level dust concentration distribution map is linearly transformed to obtain the transmittance map: ; in, Transmittance, To preserve the edge coefficient, This is a dust concentration distribution map.

[0056] To address the extremely dense dust obstruction at the tail of the belt roaster and the complex dynamic lighting environment, this step abandons the traditional single-frame independent dehazing logic and constructs a joint dehazing model that includes multi-frame weighted fusion terms and dehazing terms. Using the transmittance map calculated in previous steps, processing is performed synchronously within the same optimization plane.

[0057] Preferably, in the tail section of the belt roaster, the periodic flickering of the furnace flame and the automatic exposure adjustment of the camera in response to strong light cause high-frequency brightness abrupt changes in the video frame sequence. The static assumption of using a single fixed atmospheric light value globally in traditional dehazing algorithms completely fails in this scenario. Therefore, this step employs dynamic atmospheric light calculation: each frame in the transformed image sequence is calculated independently: the region with the highest dust concentration (i.e., extremely low transmittance) in the current frame image is extracted (e.g., the top 0.1% of the total image pixels); subsequently, the average RGB three-channel pixel value corresponding to this region in the original observed image is calculated as the local atmospheric light value A specific to the current frame image. i In use, the local atmospheric light value is substituted into the construction of subsequent molecular accumulation terms to dynamically compensate for the interference of brightness abrupt changes caused by the high-frequency light flicker at the tail of the belt roaster and the automatic exposure of the camera. This mechanism realizes the dynamic adaptive adjustment of local atmospheric light according to the ambient light, effectively offsetting the interference of brightness abrupt changes.

[0058] S43: Constructing a spatiotemporal trust evaluation index: Based on the high-frequency transient non-stationary obstruction characteristics of tail dust driven by high-temperature thermal convection, the transmittance map calculated in S42 is upgraded into a spatiotemporal trust evaluation index characterizing absolute physical visibility.

[0059] Specifically, the dust transmittance t of the acquired current frame image i (m) is directly mapped to the frame fusion weight W in the multi-frame fusion matrix. i (m), that is The lower the dust concentration and the higher the physical visibility (the closer the transmittance is to 1) in the region where a pixel is located, the greater its information contribution should be in the final fused image. Through this mapping, transmittance is forcibly elevated from a simple dehazing parameter to a trust weight for temporal fusion.

[0060] S44: Constructing the Global Physical Residual Tensor: Obtain the observed image and local atmospheric light values ​​of the current frame, and calculate the intrinsic physical residual after deducting the dust scattering interference component; using the dust transmittance of each frame image as a priori constraint, aggregate the intrinsic physical residuals in the time sequence at the pixel level to construct a global physical residual tensor containing spatiotemporal complementary information, i.e., the global molecular aggregation matrix N. sum .

[0061] S45: Constructing the global confidence matrix: Aggregate the spatiotemporal trust evaluation indicators corresponding to each frame of the image in the time series pixel by pixel to construct the global confidence matrix, i.e., the global denominator aggregation matrix D. sum This matrix utilizes the non-stationary flow characteristics of high-speed dust to ensure that as long as there is an extremely brief moment of dust thinning (high transmittance) in multiple consecutive frames, the cumulative sum of the temporal confidence of a single pixel has sufficient mathematical support, thus eliminating the possibility of division by zero overflow from the physical level.

[0062] In S4-4~S4-5, the traditional atmospheric scattering model single-frame dehazing formula is: ; in, For fog-free images, For images with fog, Atmospheric light value, Transmittance. In the high-dust area at the tail of the aircraft, the transmittance t is extremely small (approaching 0). If a division operation is performed directly within a single frame, it will cause serious truncation errors and exponentially amplified noise, resulting in large areas of black or color spots in the image.

[0063] To avoid the risk of division by zero in a single frame, this embodiment constructs a global aggregation matrix in the time dimension. Specifically, a global molecular aggregation matrix N is initialized. sum and a global denominator aggregation matrix D sumFor each frame in the sequence, calculate its observed image I. i The residual value after subtracting the atmospheric light component affected by scattering is then accumulated pixel-by-pixel into the numerator matrix; simultaneously, the fusion weights are accumulated pixel-by-pixel into the denominator matrix. The accumulation process is represented as follows: .

[0064] S46: Physical Reconstruction: After all the transformed image sequences (N frames in total) have been traversed, the denominator aggregation matrix D... sum Having accumulated a sufficiently large weight base, the global physical residual tensor is divided element-wise by the global confidence matrix, i.e., the global numerator aggregation matrix is ​​divided element-wise by the global denominator aggregation matrix, performing a unique division operation: .

[0065] in, This is the fused, fog-free image.

[0066] This single-step joint solution process is based on a time-series information penetration mechanism driven by prior dust concentration. By extracting high-confidence window segments in the time series, it can achieve high-fidelity reconstruction of the texture of the pellet material layer obscured by random high-frequency dust fields. After calculation, out-of-bounds values ​​are truncated to upper and lower limits and restored to the standard image format. Through a joint solution mechanism of first accumulating and then dividing in a single step, it perfectly integrates the two processes of multi-frame temporal information complementarity and physical dehazing and noise reduction. It not only completely eliminates the noise amplification problem of single-frame dehazing, but also restores the real texture and clear boundaries of the pellet material layer surface obscured by dense fog with high fidelity.

[0067] Example 2

[0068] This embodiment provides a system for keyframe acquisition and image restoration of the tail section of a pellet belt roaster. The system performs the method described above, including: The target bounding box detection model construction module is used to acquire the historical video stream of the tail of the pellet belt roaster and perform frame extraction processing, and combine it with manually labeled information to construct a pellet material layer detection dataset; the dataset is used to iteratively train a deep learning network with preset initial parameters and update the network weights to obtain a detection model that can output the target bounding box of the pellet material layer. Target frame set acquisition module: acquires the motion trajectory video sequence of the trolley, uses a detection model to predict the video sequence to obtain the target box of the target ball material layer; extracts the frame sequence that continuously meets the target box screening conditions as a key frame group, and selects one frame image from it as a reference frame; acquires the reference frame and its temporally adjacent images to construct the target frame set; Transformed image sequence acquisition module: extracts the material layer feature points of each frame image in the target frame set, calculates the corresponding perspective transformation matrix based on the material layer feature points, and uses the perspective transformation matrix to perform inverse perspective transformation on each frame image in the target frame set, uniformly mapping the images with perspective distortion to a preset reference coordinate system to obtain a spatially aligned transformed image sequence. The pellet layer image reconstruction module extracts the dust concentration distribution features of each frame in the transformed image sequence, and then calculates the transmittance map of each frame; using the transmittance map of each frame as the fusion weight, a global denominator aggregation matrix is ​​constructed, and the residual value after subtracting the atmospheric light component from the observed image is calculated to construct a global numerator aggregation matrix; finally, a single division operation is performed between the numerator aggregation matrix and the denominator aggregation matrix to output the reconstructed pellet layer image.

[0069] To further illustrate this implementation, this application selects a complete trolley operation cycle (cycle A) during the operation of the pellet belt roaster as the data object, and fully demonstrates the entire process from video stream detection and key frame selection to image restoration.

[0070] Step 1: Object Detection and Model Validation. First, the tail video stream was acquired and discretized into frames to construct an object detection dataset containing multiple operating conditions. The trained YOLOv5 model was then used to predict on the validation set, and the results are as follows: Figure 2 As shown. Comparison Figure 2 (a) The truth value of the label and Figure 2 (b) shows that the model can accurately identify the material layer area and output a high-confidence prediction box, providing a precise positioning basis for subsequent processing. Figure 2 (a) in (a1)-(a16) and Figure 2 In (b), the positions of images (b1)-(b16) correspond one-to-one, and Material_layer is the material layer label. Figure 2 In (b), the number 0.9 represents the label confidence level.

[0071] Step 2: Keyframe Group Acquisition and Optimal Frame Selection. The aforementioned weight parameters are applied to perform real-time detection of the video stream with period A. The algorithm performs validity filtering based on the lower boundary coordinates and vertical height of the predicted bounding box, extracting frame sequences that satisfy the continuous spatiotemporal consistency constraint to form a keyframe image group, such as... Figure 3 (a) shows (a1)-(a16). Subsequently, the candidate images are subjected to exposure validity verification (removing completely black / overexposed frames) and texture sharpness index calculation (Laplacian variance optimization). Finally, the frame with the sharpest texture is automatically selected as the best processing frame, as shown in (a). Figure 3 As shown in (b).

[0072] Step 3: Multi-frame image spatial consistency processing. First, extract linear features or texture distribution from the target frame to eliminate image distortion. The processing result is as follows: Figure 6 As shown in (b), the reference frame for calculating the distortion model and the reference perspective transformation matrix is ​​as follows. Figure 6 As shown in (a). Next, within the material layer detection frame, morphological operators are used to specifically extract and track the solid protrusions that are stationary relative to the trolley as alignment references. In spatial mapping, the system first loads single-cycle data to fit a preset reference perspective matrix, then combines the small relative motion of the protrusions between frames to calculate the dynamic affine matrix. Finally, the two are concatenated and multiplied to obtain the latest target matrix. By performing an inverse perspective transformation, each frame is uniformly mapped to an orthogonal top-view plane, completely eliminating camera viewpoint errors and outputting a high-quality frame-aligned image sequence.

[0073] Step 4: Image restoration based on fog concentration perception. The region of interest (ROI) of the dust layer in the best-processed frame is extracted, and its color attenuation prior (CAP), color ellipsoid prior (CEP), dark channel prior (DCP), wavelet texture energy, and fog line prior features are extracted. After normalization and stacking, a concentration-perceived map characterizing dust distribution is fitted using a Gaussian mixture model (GMM), as shown below. Figure 4 The figure shown illustrates the process from the original image to feature extraction and then to concentration perception. Figure 4 (a) is the original material layer image. Figure 4 (b) shows the dust concentration perception map obtained from GMM soft clustering, where color cards L1~L5 describe 5 concentration levels. Figure 4 (b) The pixel values ​​are obtained by weighting the density level index values ​​and using the posterior probability for weighted summation. Figure 5 (a) represents the prior features of color attenuation. Figure 5 (b) represents the prior features of the color ellipsoid. Figure 5 (c) represents the prior features of the dark channel. Figure 5 (d) represents the wavelet texture energy feature. Figure 5 (e) represents the prior features of the fog line, where color cards 0~1 are continuous values ​​describing dust concentration; it can be seen that the dust concentration perception map fitted by GMM is more accurate than the other five-dimensional features and can better characterize the dust concentration distribution features of the tail image.

[0074] Step 5: Multi-frame joint dehazing and restoration. First, local atmospheric light is independently calculated frame by frame to compensate for exposure abrupt changes; then, utilizing the temporal complementarity of multiple frames, the transmittance of a single frame is directly mapped to the fusion weight, and the effective features of each frame are simultaneously accumulated in the global numerator and denominator matrices; after the keyframe sequence has been traversed, only one global division operation is performed to output a high-fidelity, clear image of the material layer in one step, completely avoiding the noise surge defect of single-frame dehazing. Figure 7 As shown, Figure 7 (a) is the original image. Figure 7 (b) is the restored clear image.

[0075] Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned readable storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0076] It should be understood that, in the embodiments of the present invention, the processor may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor. The memory may include read-only memory and random access memory, and provides instructions and data to the processor. A portion of the memory may also include non-volatile random access memory. For example, the memory may also store device type information.

[0077] The readable storage medium is a computer-readable storage medium, which can be an internal storage unit of the controller described in any of the foregoing embodiments, such as the controller's hard drive or memory. The readable storage medium can also be an external storage device of the controller, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the controller. Further, the readable storage medium can include both the controller's internal storage unit and external storage devices. The readable storage medium is used to store the computer program and other programs and data required by the controller. The readable storage medium can also be used to temporarily store data that has been output or will be output.

[0078] It is understood that the same or similar parts in the above embodiments can be referred to each other, and the contents not described in detail in some embodiments can be referred to the same or similar contents in other embodiments.

[0079] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for acquiring keyframes and restoring images of the tail section of a pellet belt roaster, characterized in that, include: S1. Obtain the historical video stream of the tail of the pellet belt roaster and perform frame extraction processing, and construct a pellet material layer detection dataset by combining manually labeled information; The dataset is used to iteratively train a deep learning network with preset initial parameters and update the network weights to obtain a detection model that can output target boxes of pellet material layers. S2. Obtain the video sequence of the trolley's motion trajectory, and use the detection model to predict the video sequence to obtain the target bounding box of the target ball material layer; Extract a sequence of frames that continuously meet the target box filtering conditions as a key frame group, and select one frame from it as a reference frame; obtain the reference frame and its temporally adjacent images to construct a target frame set. S3. Extract the material layer feature points of each frame image in the target frame set, and calculate the corresponding perspective transformation matrix based on the material layer feature points; The perspective transformation matrix is ​​used to perform inverse perspective transformation on each frame image in the target frame set, and the images with perspective distortion are uniformly mapped to a preset reference coordinate system to obtain a spatially aligned transformed image sequence. S4. Extract the dust concentration distribution features of each frame in the transformed image sequence, and then calculate the transmittance map of each frame; use the transmittance map of each frame as the fusion weight to construct a global denominator aggregation matrix, calculate the residual value after deducting the atmospheric light component from the observed image, and construct a global numerator aggregation matrix; finally, use the numerator aggregation matrix and the denominator aggregation matrix to perform a single division operation to output the pellet layer reconstruction image.

2. The method according to claim 1, characterized in that, In S1, the pellet material layer detection dataset specifically includes a positive sample subset and a negative sample subset; wherein, the positive sample subset consists of images containing complete material layer texture features and geometric shapes obtained during the horizontal operation phase of the trolley; the negative sample subset consists of images of material layer collapse, gradual disappearance and eventual complete loss obtained during the tail feeding phase and the no-load rotation phase of the trolley.

3. The method according to claim 1, characterized in that, In S2, the specific process of acquiring the keyframe group is as follows: S21. Based on the target bounding box of the acquired frame t image, obtain the lower boundary ordinate and vertical height of the target bounding box; S22. Position constraint condition: The absolute value of the difference between the ordinate of the lower boundary and the ordinate of the preset observation baseline is less than or equal to the preset position deviation threshold. S23. Longitudinal dimensional constraint: The longitudinal height of the material layer shall not exceed the preset effective height range; S24. Temporal Consistency Constraint: The current state is determined to be a stable observation state only when the target boxes of N consecutive frames of images simultaneously satisfy the positional constraint and the longitudinal scale constraint, and the corresponding set of frame sequences is extracted as a keyframe group.

4. The method according to claim 1, characterized in that, In step S2, a cascaded filtering strategy is used to select the frame with the highest clarity from the keyframe image group as the reference frame. The specific process is as follows: S25. Exposure Filtering: Count the number of pixels in each image of the key frame image group whose gray value is lower than the preset black level threshold, calculate the proportion of them to the total number of pixels in the image, and when the proportion exceeds the preset black tolerance threshold, determine the current frame as an invalid exposure frame and remove it to obtain valid exposure frames and construct a set of valid exposure frames. S26. Sharpness Optimization: Based on the lower boundary ordinate and vertical height of the target box in each frame of the effective frame set, the target analysis region is obtained; the Laplacian operator is used to perform local convolution operation on the target analysis region of the effective exposure frame to extract the gradient information of the material layer edge, and the variance of the gradient magnitude is calculated as the image sharpness score. The frame with the highest score is selected as the benchmark frame.

5. The method according to claim 4, characterized in that, Image sharpness rating The calculation formula is: ； in, The Laplace operator, where Var represents variance operation. For the input image, In the horizontal direction, It is in the vertical direction.

6. The method according to claim 1, characterized in that, In step S3, the process of obtaining the spatially aligned transformed image sequence is as follows: S31. Blind calibration distortion correction: Using a blind calibration algorithm based on straight line features or texture distribution, nonlinear perspective distortion correction is performed on each original image in the target frame set to obtain a corrected image sequence that eliminates lens distortion. S32.ROI Spatial Constraint: The target bounding box of the pellet layer determined in S2 is taken as the region of interest; S33. Stable protrusion feature extraction: Within the preset ROI, morphological operators are used to specifically extract the physical protrusion geometric vertices that characterize the boundary of the pellet entity as candidate feature points; a zero displacement spatial constraint is established based on the relative coordinate system of the trolley, and transient pseudo feature points with relative displacement greater than the preset small rigidity tolerance threshold are filtered out, and the entity protrusion points that always remain relatively stationary are established as material layer feature points. S34. Initialization of the periodic prior matrix: Extract the historical annotation data of the single-cycle operation of the belt roasting machine trolley, and pre-calculate and construct the initial reference perspective transformation matrix by performing polygon fitting analysis on the reference plane within a single cycle. S35. Inter-frame dynamic matrix calculation: The material layer feature points extracted in the current frame are spatially matched with the corresponding feature points in the reference frame, and the dynamic affine matrix representing the local relative motion between frames is calculated. S36. Cascaded Update and Spatial Mapping: The reference perspective transformation matrix and the dynamic affine matrix are cascaded and multiplied to obtain the target perspective transformation matrix updated in the current frame; the target perspective transformation matrix is ​​used to perform inverse perspective transformation on the corrected image sequence, and the image is uniformly projected into a preset orthogonal top-view coordinate system to obtain a spatially aligned transformed image sequence.

7. The method according to claim 1, characterized in that, The specific process in S4 is as follows: S41. Physical Prior Decoupling: Extract the prior features of color attenuation, color ellipsoid, dark channel, wavelet texture energy, and fog line from the image, and use the Gaussian mixture model to decouple and obtain the pixel-level dust concentration distribution map. S42. Perform a linear transformation on the obtained pixel-level dust concentration distribution map to obtain a transmittance map; S43. Constructing a spatiotemporal trust evaluation index: Based on the high-frequency transient non-stationary occlusion characteristics of tail dust driven by high-temperature thermal convection, the transmittance map of the acquired current frame image is directly mapped to the fusion weight of that frame in the multi-frame fusion matrix. S44. Constructing the Global Physical Residual Tensor: Obtain the observed image and atmospheric light value of the current frame, and calculate the intrinsic physical residual after deducting the dust scattering interference component; using the dust transmittance of each frame image as a prior constraint, aggregate the intrinsic physical residuals in the time sequence at the pixel level to construct a global physical residual tensor containing spatiotemporal complementary information, i.e., the global molecular aggregation matrix N. sum : ； Where N is the number of frames. For pixels, For input image frame i, Atmospheric light value, For image frames The fusion weight; S45. Constructing the global confidence matrix: Aggregate the spatiotemporal trust evaluation indicators corresponding to each frame of the image in the time series pixel by pixel to construct the global confidence matrix, i.e., the global denominator aggregation matrix D. sum : ； S46. Physical Reconstruction: After all the transformed image sequences have been traversed, the global physical residual tensor is divided element by element by the global confidence matrix to obtain the reconstructed image of the pellet layer.

8. The method according to claim 7, characterized in that, In step S42, the formula for calculating the transmittance map is as follows: ； in, This is a transmittance diagram. As a regulating factor, This is a dust concentration distribution map.

9. The method according to claim 1, characterized in that, For industrial site lighting flicker, dynamic atmospheric light calculation is adopted: each frame in the transformed image sequence is calculated independently: the area with the highest dust concentration in the current frame image is extracted according to a preset ratio. Subsequently, the mean value of the RGB three-channel pixels in this region in the original observation image is calculated as the local atmospheric light value A specific to the current frame image. i When using it, simply substitute the local atmospheric light value into the molecular aggregation matrix.

10. A system for keyframe acquisition and image restoration of the tail section of a pellet belt roaster, wherein the system executes the method according to any one of claims 1-9, characterized in that, include: The target bounding box detection model construction module is used to acquire the historical video stream of the tail of the pellet belt roaster and perform frame extraction processing, and combine it with manually labeled information to construct a pellet material layer detection dataset; the dataset is used to iteratively train a deep learning network with preset initial parameters and update the network weights to obtain a detection model that can output the target bounding box of the pellet material layer. Target frame set acquisition module: acquires the video sequence of the trolley's motion trajectory, uses a detection model to predict the video sequence, and obtains the target bounding box of the target ball material layer; Extract a sequence of frames that continuously meet the target box filtering conditions as a key frame group, and select one frame from it as a reference frame; obtain the reference frame and its temporally adjacent images to construct a target frame set. Transformed image sequence acquisition module: extracts the material layer feature points of each frame image in the target frame set, calculates the corresponding perspective transformation matrix based on the material layer feature points, and uses the perspective transformation matrix to perform inverse perspective transformation on each frame image in the target frame set, uniformly mapping the images with perspective distortion to a preset reference coordinate system to obtain a spatially aligned transformed image sequence. The pellet layer image reconstruction module extracts the dust concentration distribution features of each frame in the transformed image sequence, and then calculates the transmittance map of each frame; using the transmittance map of each frame as the fusion weight, a global denominator aggregation matrix is ​​constructed, and the residual value after subtracting the atmospheric light component from the observed image is calculated to construct a global numerator aggregation matrix; finally, a single division operation is performed between the numerator aggregation matrix and the denominator aggregation matrix to output the reconstructed pellet layer image.

Images