Metal melting end point determination method based on video feature extraction

Abstract

The present application relates to the technical field of metal smelting, and particularly relates to a metal smelting end point determination method based on video feature extraction. The method comprises the following steps: acquiring a smelting real-time image sequence, performing log-hyperbolic tangent color conversion on a current frame, and adjusting a slope by using a maximum Lyapunov exponent of a previous time window; calculating a space-time joint gradient and determining a high-temperature region, combining local entropy and a main direction to generate an anisotropic spatial weighted kernel; generating a spatial modulation factor and a color weight matrix from a high-temperature region Fourier phase spectrum and a log amplitude spectrum; assigning weights by using a maximum Lyapunov exponent of a previous time window, fusing the spatial weighted kernel and the color weight matrix, and generating a global feature vector; constructing a phase space trajectory and calculating a maximum Lyapunov exponent, and determining an end point according to an index across zero maintenance and a second derivative norm. The scheme can reduce interference and false triggering, and improve end point determination accuracy and reliability.

Description

Technical Field

[0001] This invention relates to the field of metal smelting technology, and more specifically to a method for determining the endpoint of metal smelting based on video feature extraction. Background Technology

[0002] Metal smelting is a crucial process in metallurgical production, significantly impacting product quality, production efficiency, and energy consumption. Accurate determination of the smelting endpoint directly affects the stability of subsequent casting, composition control, and energy management. Traditional methods for determining the smelting endpoint rely primarily on operator observation based on experience, such as flame color, molten pool brightness, and surface flow patterns, or on obtaining process information through contact temperature measurement and periodic sampling and testing. These methods are challenging to implement in high-temperature, high-radiation, and dusty environments, and human judgment is susceptible to variability based on experience. Sampling and testing also suffer from inherent lag, making it difficult to meet the real-time and consistency requirements of continuous and automated smelting processes. In recent years, non-contact visual monitoring methods based on industrial cameras have been increasingly applied to smelting status identification. These methods typically acquire images of the flame or molten pool surface, extracting visual features such as color, brightness, texture, and edges, and then combining these with threshold rules or empirical models to estimate the smelting progress.

[0003] However, the metal smelting process is accompanied by intense thermal radiation, multiphase flow disturbances, molten metal agitation, smoke and dust obscuring, and metal splashing. The acquired images are prone to problems such as local overexposure, color saturation, texture masking, and enhanced background noise. Existing visual determination methods mostly employ fixed feature extraction patterns, which have limited adaptability to changes in illumination, texture, and dynamic disturbances at different smelting stages, making it difficult to stably focus on effective changes in the core high-temperature region of the molten pool. Simultaneously, the smelting process exhibits significant nonlinear and time-varying characteristics; its reaction state typically gradually stabilizes from violent fluctuations. Simply relying on static image features or shallow temporal changes is insufficient to fully represent this dynamic evolution. Without a comprehensive weighting mechanism for spatial noise, color channel differences, and temporal chaos, the extracted visual features are easily interfered with by strong light, smoke, and invalid backgrounds, leading to premature or delayed endpoint determination. Therefore, it is necessary to propose a metal smelting endpoint determination method that can integrate video image features with nonlinear temporal evolution information to improve the stability and reliability of endpoint identification under complex conditions.

[0004] Chinese patent document CN106513879B discloses a spark discharge state identification and detection method based on chaos theory. This method determines the embedding dimension and delay time for the spark discharge process, calculates the maximum Lyapunov exponent through phase space reconstruction, and identifies and detects the discharge state accordingly. However, this method has specific processing objects and signal conditions. In visual monitoring scenarios of metal smelting involving strong light, smoke, and splashes, it is difficult to stably characterize the evolution of the molten pool state amidst complex interference. Furthermore, the method of judging the state based on the exponent has limited ability to distinguish between short-term fluctuations and continuously converging states, and may still result in premature or delayed judgments when applied to determining the smelting endpoint. Summary of the Invention

[0005] To address the problems of existing technologies being susceptible to interference from strong light, smoke, and splashes in determining the endpoint of metal smelting, lacking stability in feature extraction, and failing to characterize the nonlinear evolution of the smelting state, leading to premature or delayed endpoint determination, this invention provides a method for determining the endpoint of metal smelting based on video feature extraction. The method includes the following steps: acquiring a real-time image sequence of the metal smelting process; converting the current frame of the image sequence from RGB to a log-hyperbolic tangent color space to obtain a multi-channel feature map; using preset parameters as slope parameters in the initial stage, and generating a slope parameter greater than zero using the maximum Lyapunov exponent of the previous time window in the non-initial stage; calculating the spatiotemporal joint gradient field of the multi-channel feature map; determining the high-temperature region based on the multi-channel feature map; and generating an anisotropic spatial weighted kernel by combining local entropy with the principal direction of the high-temperature region. The local entropy is used as... Entropy decay adjustment is performed on the anisotropic spatial weighted kernel; the two-dimensional Fourier phase spectrum and logarithmic amplitude spectrum of the circumscribed rectangle of the high-temperature region are extracted to construct a spatial modulation factor, and the spatial modulation factor is used to modulate each monochromatic channel to generate a color weight matrix; after aligning the anisotropic spatial weighted kernel and the color weight matrix in terms of spatial dimensions, they are fused through a nonlinear coupling function to obtain a composite weighted matrix, and the weights in the fusion process are allocated by the maximum Lyapunov exponent of the previous time window; the composite weighted matrix is ​​used to weight and sum the feature maps of multiple channels to generate a global feature vector; the continuous global feature vectors are constructed into phase space trajectories and the current maximum Lyapunov exponent is calculated. When the maximum Lyapunov exponent decreases from a positive value to a non-positive value within multiple consecutive predetermined time windows and the second derivative norm is lower than the convergence threshold, the endpoint is determined to be reached.

[0006] By using the maximum Lyapunov exponent of the previous time window to adjust the color mapping slope and the fusion weight of the composite weighted matrix, and combining the spatiotemporal joint gradient, local entropy, main direction of the high-temperature region and spectral information to extract global features, the image features can be adjusted with the chaotic state of the molten pool and centrally represent the high-temperature region. Furthermore, by using the maximum Lyapunov exponent continuously decreasing from a positive value to a non-positive value range and the second derivative norm being lower than the convergence threshold as joint conditions, the endpoint judgment is avoided due to single-frame changes or instantaneous fluctuations in the exponent, thus improving the accuracy and reliability of endpoint determination.

[0007] Furthermore, the focal length and three-dimensional attitude angle of the fixed image acquisition device are fixed, the mechanical position adjustment command is blocked to lock the center of the liquid molten pool, and image frames are acquired at a fixed sampling frequency of 30fps to 60fps, stored in a circular buffer according to the timestamp, and real-time image sequences are extracted with a sliding time window of fixed size.

[0008] By fixing the focal length and three-dimensional attitude angle of the image acquisition device and shielding mechanical position adjustment commands, the monitoring field of view is continuously locked onto the center of the molten pool, reducing background displacement interference caused by mechanical micro-movements. By adopting a fixed sampling frequency, timestamp sorting, a circular buffer, and a fixed-size sliding time window, the temporal continuity of the image sequence and the consistency of the window data scale can be maintained, providing a stable input for spatiotemporal joint gradient calculation and phase space trajectory construction.

[0009] Furthermore, the red, green, and blue monochrome channel matrices of the current frame are separated, and the pixel values ​​are normalized to the range of 0 to 1 to obtain... proceed in sequence Logarithmic transformation and tanh Hyperbolic tangent transform, where This is the result of a logarithmic transformation. The hyperbolic tangent transform results are then stacked along the channel dimension to form the multi-channel feature map, where Greater than zero, and in the initial stage Set to 1.5 to 2.5, update the maximum Lyapunov exponent based on the previous time window during non-initial phases. And apply a positive value limit.

[0010] By sequentially performing logarithmic and hyperbolic tangent transformations on the normalized monochrome channels, the glare response in the high-temperature region can be compressed while preserving the texture details of the dark slag edges. In the non-initial stage, the slope parameter is updated based on the maximum Lyapunov exponent of the previous time window and positive value limiting is applied, so that the color response sensitivity can adapt to the changes in the chaos of the molten pool, while maintaining the stability of the mapping direction, thereby improving the ability of the multi-channel feature map to represent changes in the melting state.

[0011] Furthermore, the Sobel operator is used to calculate the horizontal and vertical spatial gradients of the multi-channel feature map. The temporal gradient is obtained by pixel-by-pixel difference between two adjacent frames. The square roots of the sum of the squares of the three gradients are then taken and averaged over each channel to obtain a two-dimensional spatiotemporal joint gradient intensity matrix. The two-dimensional single-channel representative map is obtained by weighted summation of each channel. A high-temperature region mask is generated by Otsu threshold segmentation, and the high-temperature connected region with the largest area is retained. The main direction of the high-temperature region is determined by the eigenvector corresponding to the largest eigenvalue of the coordinate covariance matrix of the region.

[0012] By synthesizing the horizontal spatial gradient, vertical spatial gradient, and temporal gradient of adjacent frames into a spatiotemporal joint gradient intensity matrix, the edge structure and dynamic changes of the melt pool can be reflected simultaneously. By using Otsu thresholding to segment and retain the high-temperature connected region with the largest area, the interference of scattered bright regions on the localization of high-temperature regions can be reduced. Then, the main direction is determined by the eigenvector of the coordinate covariance matrix, providing a stable regional and directional basis for the spatial weighting of subsequent direction alignment.

[0013] Further, a local Shannon entropy matrix is ​​calculated within a preset neighborhood window of the two-dimensional single-channel representative image, and the major axis of the two-dimensional Gaussian kernel is rotated to be consistent with the main direction of the high-temperature region. The local Shannon entropy matrix is ​​normalized, and an entropy decay coefficient matrix is ​​generated according to the negative exponential relationship of the normalized local entropy values. This coefficient matrix is ​​then multiplied element-wise with the high-temperature region mask, the two-dimensional spatiotemporal joint gradient intensity matrix, and the rotated two-dimensional Gaussian kernel, and normalized to generate the anisotropic spatial weighted kernel.

[0014] By rotating the major axis of the two-dimensional Gaussian kernel to the main direction of the high-temperature region, the spatial weight distribution is matched with the extension direction of the high-temperature region. The weights are attenuated according to the negative exponential relationship of local entropy and weighted together with the high-temperature region mask and the spatiotemporal joint gradient intensity matrix. This reduces the weights of local disordered high-entropy disturbances caused by smoke, splashes or overexposure, highlights the characteristics of continuous direction and effective dynamic change in the high-temperature region, and improves the targeting of spatial feature screening.

[0015] Further, the minimum bounding rectangle of the high-temperature region is obtained, and image blocks are extracted from the multi-channel feature map and weighted to generate a two-dimensional single-channel representative matrix. A two-dimensional Fourier transform is performed on the representative matrix to obtain the phase spectrum and logarithmic amplitude spectrum. After excluding the central low frequency, continuous or adjacent high-energy ring frequency bands with a cumulative energy ratio of not less than 60% are selected to construct a frequency domain filter mask. The complex frequency domain matrix is ​​reconstructed and inversely transformed to generate the spatial domain modulation factor.

[0016] Further, after aligning the spatial modulation factor size, it is multiplied element-wise with the three monochrome channel matrices to generate the color weight matrix; the anisotropic spatial weighting kernel is broadcast as a three-dimensional spatial weighting matrix, and the dimensionless maximum Lyapunov exponent of the previous time window is mapped to the fusion weight using the Sigmoid function. The three-dimensional spatial weighting matrix and the color weight matrix are then weighted with the fusion weight and the value obtained by subtracting the fusion weight from 1, respectively, and the sum and normalization are used to generate the composite weighting matrix.

[0017] Furthermore, the composite weighted matrix is ​​multiplied element-wise and channel-wise by the multi-channel feature map, and the resulting three-dimensional tensor is subjected to global average pooling along its width and height. The pooled scalars are then concatenated in channel order and normalized using the L2 norm to generate the global feature vector.

[0018] Furthermore, the mutual information algorithm is invoked to estimate the optimal delay time for the global feature vectors of multiple consecutive frames, and the pseudo-nearest neighbor algorithm is invoked to determine the optimal embedding dimension. Multivariate delay embedding is performed according to both to form the phase space trajectory. In the phase space trajectory, adjacent trajectory points that satisfy the time interval constraint are searched, and the linear growth interval of the average logarithmic divergence curve is fitted. The fitting slope is used as the maximum Lyapunov exponent of the current time window.

[0019] Furthermore, the maximum Lyapunov exponent is recorded for 3 to 5 consecutive predetermined time windows to construct an exponential time series. The second derivative sequence is calculated using the central difference method or the Savitzky-Golay filtering algorithm. When the exponential time series decreases from a positive value to a non-positive value range and remains there, and the L2 norm of the second derivative sequence is lower than the convergence threshold in the range of 0.001 to 0.005, the signal that has reached the endpoint is output.

[0020] The beneficial effects of this invention are as follows: By locking the acquisition field of view and forming a continuous image sequence at a fixed frequency, the interference of mechanical displacement and sampling fluctuations on the characterization of the molten pool state is reduced; by dynamically adjusting the slope of the log-hyperbolic tangent mapping and the fusion weights of spatial and color features using the maximum Lyapunov exponent, feature extraction adapts to the state change of the smelting process from chaos to stability. An anisotropic spatial weighted kernel is constructed using the spatiotemporal joint gradient, the principal direction of the high-temperature region, and local entropy, and the spatial modulation factor is reconstructed using the phase spectrum and logarithmic amplitude spectrum, which can highlight the effective texture and dynamic changes of the core reaction area and attenuate the influence of high-entropy disturbances, background thermal radiation, and low-frequency components. A stable global feature vector is obtained through composite weighting, global average pooling, and norm normalization. Then, the endpoint is jointly determined by phase space reconstruction, maximum Lyapunov exponent zero-crossing maintenance, and second-order derivative norm constraints, reducing false triggering caused by local fluctuations and improving the accuracy, reliability, and execution stability of endpoint determination. Attached Figure Description

[0021] Figure 1 This is a flowchart of the metal smelting endpoint determination method based on video feature extraction according to an embodiment of the present invention;

[0022] Figure 2 This is a schematic diagram comparing the endpoint detection performance in an embodiment of the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this specification will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of them. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this specification.

[0024] In the first embodiment, the present invention proposes a method for determining the endpoint of metal smelting based on video feature extraction, such as... Figure 1 As shown, it includes the following steps: S1, acquire a real-time image sequence of the metal smelting process.

[0025] The system connects to an industrial area array CCD camera via a gigabit Ethernet interface to acquire video streams from the surface of a metal smelting pool in real time. The video streams are read frame by frame, and a double-ended queue data structure is created in the system memory to cache the continuously acquired image frames, thus constructing a sliding time window data pool with a fixed capacity.

[0026] In an optional embodiment, acquiring a real-time image sequence of the metal smelting process includes: The focal length and three-dimensional attitude angle of the fixed image acquisition device are shielded from mechanical position adjustment commands to keep the monitoring field of view locked on the center of the liquid molten pool; The image acquisition device is triggered according to a preset fixed sampling frequency to extract continuous image frames as a real-time image sequence.

[0027] In the environment of a metal smelting site, such as when the temperature can reach... to In a high-temperature industrial furnace, images are acquired using a radiation-resistant and high-temperature-resistant high-speed industrial camera. The image acquisition equipment undergoes initial calibration, locking the focal length to, for example... or The fixed-focus mode is used, and the three-dimensional attitude angles are fixed, such as the pitch angle being set to... Yaw angle Roll angle This allows the camera's monitoring field of view to be locked on the central area of ​​the molten pool, such as an area with a diameter of approximately... to The core reaction zone. Mechanical position adjustment commands, including Pan, Tilt, and Zoom commands, issued by the gimbal controller or motor drive module are shielded at the system control layer to maintain spatial coordinate stillness throughout the entire melting cycle, typically lasting 30 to 60 minutes, reducing background displacement noise caused by mechanical micro-motions in subsequent spatiotemporal joint gradient field calculations.

[0028] Hardware triggers or timers send trigger level signals to the image acquisition device at a preset fixed sampling frequency. Considering the high-frequency dynamic characteristics of the surface physicochemical reactions during the melting process and the requirement of the maximum Lyapunov exponent for the length of the phase-space time series, the fixed sampling frequency is usually preferably set to [value missing]. to For example, typical The acquisition system continuously acquires data at this frequency with a resolution of [resolution value missing]. Image frames with pixel counts and bit depths of 8 or 16 bits are generated and stored in a circular buffer memory in chronological order according to their timestamps. These consecutive image frames are then extracted using a fixed-size sliding window, for example, each window containing 500 frames corresponding to a 10-second time period, to form a real-time image sequence input matrix for subsequent color space conversion and phase space trajectory analysis.

[0029] S2 converts the current frame of the image sequence from RGB to the log-hyperbolic tangent color space to obtain a multi-channel feature map.

[0030] Separate the red, green, and blue monochrome channels of the current frame image, divide all pixel values ​​by 255, and normalize them to a floating-point number between 0 and 1. To avoid zero singularities in extremely dark areas during logarithmic operations and to prevent the mapping direction from reversing, perform pixel-by-pixel processing on each normalized channel matrix. Logarithmic transformation, where The adjustment coefficient is greater than zero. The hyperbolic tangent is calculated to complete the nonlinear color mapping, and three independent feature matrices corresponding to the red, green and blue channels are output. The matrices are stacked according to the channel dimension to form a three-dimensional multi-channel feature map.

[0031] In an optional embodiment, the current frame of the image sequence is converted from RGB to a log-hyperbolic tangent color space to obtain a multi-channel feature map, including: Separate the red, green, and blue monochrome channel matrices of the current frame in the image sequence; Logarithmic and hyperbolic tangent transformations are performed on the pixel values ​​of each monochrome channel matrix; In the non-initial stage, the maximum Lyapunov exponent of the previous time window is multiplied by the preset gain coefficient and the offset is added to obtain the updated slope parameter. The updated slope parameter is then positively limited to make it greater than zero. The updated slope parameter is then used to replace the default slope in the hyperbolic tangent function. The transformed monochrome channel matrices are merged to generate a multi-channel feature map.

[0032] The current image frame with a resolution of H×W is split in memory along the channel dimension to obtain three independent monochrome channel matrices: R, G, and B, each with dimensions of H×W. To suppress localized glare during high-temperature melting and enhance texture details at the edges of molten slag in dark areas, normalization is performed on each extracted monochrome channel matrix, mapping pixel values ​​to the range of 0.0 to 1.0, and applying the formula [formula missing] to each pixel. The logarithmic transformation, where the constant The preferred value is 10 to 50, for example The pixel matrix after logarithmic compression is further subjected to a hyperbolic tangent transform, calculated using the following formula: tanh ,in This is the slope parameter. In the initial stages of the system, such as before a sliding time window length has accumulated, the slope parameter... Use empirical default values, which are generally preferred to be between 1.5 and 2.5, for example. This ensures that the initial transformation is within the linear buffer mapping region.

[0033] As the smelting process enters a non-initial stage, the slope parameter update mechanism is activated due to the drastic changes in the chaotic state of the molten pool. The maximum Lyapunov exponent calculated in the previous time window is extracted. The maximum Lyapunov exponent measures the rate at which the trajectories of adjacent states of a system diverge or converge over time. A larger value indicates a more chaotic and unstable system, while a value approaching zero or being negative indicates that the system is gradually becoming stable. The system is then processed according to a preset time base. Dimensionless processing is performed, with the preset time base being the sampling period or the length of the sliding time window, to obtain the dimensionless exponent value. Substitute into the linear update equation The temporary slope parameter is calculated and then used as a positive limiting function. The updated slope parameters are obtained, where This is the lower limit of the slope, which is greater than zero. The preset gain coefficient is also included. Preferably between 0.5 and 1.5, for example Offset Then, based on the device's basic response characteristics, set it to between 0.5 and 1.0, for example... If the dimensionless exponent value calculated in the previous time window... =1.25, then the temporary slope parameter is calculated. =2.0, and it is greater than 2.0. At that time, =2.0 replaces the original default slope k and applies it to the nonlinear mapping calculation of the current frame. The three channel matrices processed by the above nonlinear transformation are re-stacked and concatenated in the channel dimension to output a multi-channel feature map with a dimension of H×W×3. At this time, the dynamic mapping maps the values ​​of the multi-channel feature map to the interval [0,1).

[0034] S3 uses preset parameters as slope parameters in the initial stage, and generates slope parameters greater than zero by using the maximum Lyapunov exponent of the previous time window in the non-initial stage.

[0035] During the algorithm startup phase, before accumulating enough data to satisfy a sliding time window length, a preset default slope parameter is used as the multiplication coefficient of the independent variable in the hyperbolic tangent mapping function. After accumulating enough data to satisfy a sliding time window length, the Rosenstein algorithm based on chaotic time series analysis is called to calculate the maximum Lyapunov exponent of features extracted within the previous sliding time window. The maximum Lyapunov exponent is then dimensionless to obtain a dimensionless exponent value. This dimensionless exponent value is substituted into the linear update equation to calculate a temporary slope parameter. The temporary slope parameter is then positively limited to obtain a slope parameter greater than zero after the current frame update, thus achieving real-time control of color response sensitivity.

[0036] S4 calculates the spatiotemporal joint gradient field of the multi-channel feature map, determines the high-temperature region based on the multi-channel feature map, and generates an anisotropic spatial weighting kernel by combining the local entropy and the main direction of the high-temperature region. The local entropy is used to adjust the entropy decay of the anisotropic spatial weighting kernel.

[0037] The Sobel operator is called to calculate the spatial gradients of the multi-channel feature maps in the horizontal and vertical directions respectively, and pixel-by-pixel difference is performed on the multi-channel feature maps of two adjacent frames to obtain the temporal gradient. The horizontal spatial gradient, vertical spatial gradient and temporal gradient are squared at the corresponding pixel positions and then squared to obtain the spatiotemporal joint gradient intensity map corresponding to each channel. The results of each channel are averaged to form a two-dimensional spatiotemporal joint gradient intensity matrix with the same spatial size as the image.

[0038] The eigenvalues ​​of corresponding pixel positions in each channel of the multi-channel feature map are weighted and summed to obtain a two-dimensional single-channel representative image with the same spatial size as the image. Each channel weight is a preset non-negative weight, and the sum of all channel weights is 1. Based on the two-dimensional single-channel representative image, the gray-level distribution is statistically analyzed within a preset neighborhood window, and the local Shannon entropy is calculated to obtain the local Shannon entropy matrix. The single-channel representative image is binarized using the Otsu thresholding algorithm to generate a high-temperature region mask. High-temperature connected regions in the mask are then filtered, retaining the high-temperature connected regions with the largest area. Based on the pixel coordinates within the retained regions, image moments and coordinate covariance matrices are calculated. The eigenvector corresponding to the largest eigenvalue is obtained, and its direction is determined as the principal direction angle of the high-temperature region.

[0039] A two-dimensional Gaussian kernel function is constructed, and its major axis is rotated to align with the principal direction using a rotation matrix to obtain an anisotropic Gaussian kernel with aligned orientations. The local Shannon entropy matrix is ​​normalized and then... The entropy decay coefficient matrix is ​​generated in the form of , where Let η be the normalized local entropy value, and η be the decay coefficient greater than zero. The entropy decay coefficient matrix, the high-temperature region mask, the two-dimensional spatiotemporal joint gradient intensity matrix, and the orientation-aligned anisotropic Gaussian kernel are adjusted to the same spatial size and normalized to the interval between 0 and 1. Then, the four are multiplied element-wise to obtain the initial spatial weight matrix, and the initial spatial weight matrix is ​​normalized to generate the anisotropic spatial weighted kernel.

[0040] S5. Extract the two-dimensional Fourier phase spectrum and logarithmic amplitude spectrum of the circumscribed rectangle of the high-temperature region to construct a spatial modulation factor. Use the spatial modulation factor to modulate each monochrome channel to generate a color weight matrix.

[0041] Obtain the minimum bounding rectangle of the high-temperature connected region segmented by the Otsu method, and extract the corresponding image block from the multi-channel feature map. Perform a weighted summation of the feature values ​​of corresponding pixels in each channel of the image block to generate a two-dimensional single-channel representative matrix, which is then used as the input for a two-dimensional Fourier transform. Call the `fft2` function to perform a two-dimensional fast Fourier transform on the two-dimensional single-channel representative matrix, and call the `fftshift` function to shift the low-frequency components to the center of the spectrum. Extract the phase and amplitude matrices from the frequency domain matrix, and add 1 to all values ​​in the amplitude matrix and take the logarithm to obtain the logarithmic amplitude spectrum. Construct a frequency domain filter mask based on the target frequency band with concentrated energy in the logarithmic amplitude spectrum. Use the frequency domain filter mask to filter the logarithmic amplitude spectrum element by element, retaining the amplitude information within the target frequency band and suppressing frequency components outside the target frequency band. Then, combine the filtered amplitude information with the original phase matrix to reconstruct a new complex frequency domain matrix. Perform a two-dimensional inverse Fourier transform on the new complex frequency domain matrix, extract the real part of the inverse transform result, and construct the spatial modulation factor. The spatial modulation factor is backfilled into the spatial modulation matrix with the same width and height as the multi-channel feature map according to the spatial position of the bounding rectangle of the high-temperature region in the original image, and the non-high-temperature region is assigned a preset low weight value; or the spatial modulation factor is interpolated and scaled to the same spatial size as the multi-channel feature map.

[0042] The size-aligned spatial modulation matrix is ​​then multiplied element-wise with the three monochrome channel matrices of the log-hyperbolic tangent color space to generate a three-channel color weight matrix with frequency domain enhancement characteristics.

[0043] In an optional embodiment, the spatial modulation factor is constructed by extracting the two-dimensional Fourier phase spectrum and logarithmic amplitude spectrum of the circumscribed rectangle of the high-temperature region, including: The pixel matrix of the circumscribed rectangle of the high-temperature region is extracted and subjected to a two-dimensional discrete Fourier transform to separate the real and imaginary parts of the frequency domain matrix. The amplitude and phase of each frequency component are calculated using the real and imaginary parts, and the logarithmic amplitude spectrum is obtained by performing a logarithmic operation on the amplitude values. A frequency domain filter mask is constructed based on the target frequency band in the logarithmic amplitude spectrum used to characterize the high-temperature texture changes of the molten pool. The frequency domain filter mask is combined with the two-dimensional Fourier phase spectrum and the logarithmic amplitude spectrum to construct a complex matrix and perform an inverse Fourier transform to generate a spatial modulation factor.

[0044] By using binarization thresholding, such as for an 8-bit grayscale image or an 8-bit equivalent grayscale image obtained by normalization of a 16-bit image, combined with Otsu's method and morphological connected component analysis of highlight segments with pixel values ​​above 200, the high-temperature region generated by the chemical reaction in the image is located. The smallest orthogonal bounding rectangle pixel matrix that can enclose this region is then extracted, with its size automatically zero-padded or cropped to an integer power of 2, such as 128×128 or 256×256 pixels to meet the cardinality requirements of the Fast Fourier Transform (FFT). A two-dimensional discrete Fourier transform (2D-DFT) is then performed on this bounding rectangle matrix to convert the spatial domain image to the frequency domain, separating the real part R(u,v) and imaginary part I(u,v) corresponding to each spatial frequency coordinate (u,v) in the output complex frequency domain matrix. Based on the extracted real and imaginary parts, the amplitude spectrum A(u,v) of each frequency component is calculated using the Euclidean norm, and the phase spectrum P(u,v) of each frequency component is calculated using the arctangent function. To mitigate the energy abrupt change caused by the central zero-frequency DC component and highlight the high-frequency texture structure, a compression operation based on the natural logarithm was performed on the amplitude values ​​to obtain the logarithmic amplitude spectrum. .

[0045] Using the center of the spectrum as the origin, the logarithmic amplitude spectrum is divided into multiple concentric ring bands according to the frequency radius, and the sum of the squares of the amplitude values ​​within each ring band is calculated as the band energy. After excluding the low-frequency ring band containing the central DC component, continuous or adjacent high-energy ring bands are selected according to the band energy, such that their cumulative energy accounts for a preset proportion of the total energy after excluding the central low frequency, preferably more than 60%, thereby determining a specific high-energy ring band interval; for example, the spatial cutoff frequency is concentrated in the radius. arrive The frequency domain annular region is defined. The frequency coordinates within this region are set to 1, and the other frequency coordinates are set to 0, thus constructing a size-matched binarized bandpass frequency domain filter mask M(u,v).

[0046] The frequency domain filter mask is element-wise multiplied with the logarithmic amplitude spectrum. Then, the filtered amplitude information is combined with the original phase spectrum, so that each frequency coordinate is determined by the filtered amplitude value and the original phase angle, thus reconstructing a new complex frequency domain matrix. A two-dimensional inverse Fourier transform (2D-IFFT) is performed on this reconstructed complex matrix, extracting the real part of the output and normalizing it in the spatial domain to between 0 and 1, resulting in a two-dimensional spatial modulation factor matrix with high-temperature texture detail enhancement characteristics. Subsequently, this two-dimensional spatial modulation factor matrix is ​​backfilled to the corresponding position of the high-temperature region's bounding rectangle in the original image, forming a spatial modulation matrix with the same spatial dimensions as the multi-channel feature map, thus ensuring dimensional consistency during subsequent element-wise operations with each monochrome channel matrix.

[0047] S6 aligns the anisotropic spatial weighting kernel and the color weight matrix in terms of spatial dimensions, and then fuses them through a nonlinear coupling function to obtain a composite weighting matrix. The weights in the fusion process are allocated by the maximum Lyapunov exponent of the previous time window.

[0048] The two-dimensional anisotropic spatial weighting kernel is adjusted to have the same width and height as the color weighting matrix, and then copied or broadcast along the channel dimension to form a three-dimensional spatial weighting matrix with the same number of channels as the color weighting matrix. The maximum Lyapunov exponent, calculated and dimensionless from the previous time window, is read and input into the Sigmoid function to obtain fusion weights ranging from 0 to 1. These fusion weights are used as the first weight of the three-dimensional spatial weighting matrix, and the value obtained by subtracting the fusion weights from 1 is used as the second weight of the color weighting matrix. The three-dimensional spatial weighting matrix and the color weighting matrix are weighted according to the first and second weights, respectively. The corresponding elements of the two weighted matrices are then added to obtain the initial composite weighting matrix. The initial composite weighting matrix is ​​normalized to generate the three-dimensional composite weighting matrix.

[0049] S7 uses a composite weighted matrix to weight and sum the multi-channel feature maps to generate a global feature vector.

[0050] A pixel-level filtering operation based on joint spatial and frequency domain attention is achieved by performing channel-by-channel, element-by-element multiplication on the 3D composite weighted matrix and the transformed multi-channel feature map. Global average pooling is then performed along the width and height spatial dimensions of the filtered 3D tensor to obtain a one-dimensional array with a length equal to the number of color channels. This one-dimensional array is then normalized using the L2 norm to generate a global feature vector that reduces the influence of absolute illumination values ​​and represents the current frame's melting evolution state.

[0051] In an optional embodiment, a global feature vector is generated by weighting and summing the multi-channel feature maps using a composite weighting matrix, including: Extract the weight coefficients from the composite weighted matrix; Perform element-wise multiplication of the composite weighted matrix with the pixel feature values ​​of the corresponding spatial locations and channels in the multi-channel feature map; The results of element-wise multiplication are subjected to global spatial pooling, and the pooling results of each channel are combined into a global feature vector.

[0052] Because the preceding steps involved spatial dimension alignment and nonlinear coupling, the generated composite weighted matrix already matches the multi-channel feature map to be processed in terms of dimension, i.e., both remain constant. A three-dimensional tensor, for example, with a resolution of Number of channels Traverse and analyze the storage space of the composite weighted matrix to extract the total number of elements contained in the tensor. Each floating-point weight coefficient. Through parallel acceleration instructions, such as CUDA core matrix operations or SIMD instruction sets, the Hadamard product is calculated between the composite weighted matrix and the multi-channel feature map, i.e., for each spatial coordinate ( , ) and channel index The corresponding modulation feature tensor is calculated. This element-wise multiplication operation can assign higher weights to key regions of the molten pool that are located in high-temperature areas, continuously distributed along the main direction, and have effective spatiotemporal gradient changes. At the same time, it attenuates the local disordered high-entropy disturbance regions caused by smoke, splash, or overexposure, thereby reducing the feature values ​​of the background furnace wall and the ineffective interference regions.

[0053] After weighted modulation of the feature maps, a global spatial pooling operation is performed on the 3D tensor generated by the multiplication operation in the spatial dimension. Global average pooling is typically preferred, which involves calculating the arithmetic mean of the element values ​​at all spatial locations in the weighted feature tensor within each single-channel plane. After calculating the single scalar representation for each channel, the pooled scalars output from all channels (e.g., three monochrome channels) are concatenated and fused in the original channel order. For example, if channel 0 outputs 0.45, channel 1 outputs 0.72, and channel 2 outputs 0.51, then the pooled scalars are concatenated to form a dimensional tensor. The one-dimensional real number array [0.45, 0.72, 0.51] is used as a global feature vector representing the overall state of the melting dynamics in the current frame. This vector is then stored in a buffer and concatenated into a high-dimensional phase space time series.

[0054] S8 constructs the continuous global eigenvectors into phase space trajectories and calculates the current maximum Lyapunov exponent. When the maximum Lyapunov exponent decreases from a positive value to a non-positive value within multiple consecutive predetermined time windows and the second derivative norm is lower than the convergence threshold, it is determined that the endpoint has been reached.

[0055] For the time series of global feature vectors extracted from multiple consecutive frames, the mutual information algorithm is used to estimate the optimal delay time, and the pseudo nearest neighbor algorithm is used to determine the optimal embedding dimension. Based on the multidimensional time series composed of continuous global feature vectors, multivariate delayed embedding is performed according to the optimal delay time and the optimal embedding dimension. The global feature vectors at different times are concatenated into high-dimensional state points to form a phase space trajectory matrix.

[0056] The Rosenstein maximum Lyapunov exponent estimation algorithm is employed. In the reconstructed high-dimensional phase space trajectory, neighboring trajectory points satisfying the time interval constraint are searched. The average logarithmic divergence curve of the neighboring trajectories over time is calculated, and a least-squares straight line is fitted to the linear growth interval of this average logarithmic divergence curve. The slope of the fitted line is taken as the maximum Lyapunov exponent corresponding to the current time window, and it is converted into a dimensionless exponent value that is easy to compare with a threshold according to the sampling period or the length of the sliding time window. Monitors for five consecutive sliding time windows are set in the buffer queue. When the value of the maximum Lyapunov exponent crosses zero, decreasing from a value greater than zero representing a chaotic and violently reactive state to a steady-state interval less than or equal to zero and maintained thereafter, the maximum Lyapunov exponent time series recorded within multiple consecutive sliding time windows are continuously numerically differencing twice to obtain the second derivative sequence. The L2 norm of this second derivative sequence is then calculated. If the norm is less than the preset convergence threshold set by the system, it is determined that the kinetic state of the smelting reaction tends to converge, triggering the GPIO interface at the bottom layer of the industrial control computer to send a relay closing signal, and determining that the smelting process has reached the end point.

[0057] In an optional embodiment, the current maximum Lyapunov exponent is calculated. When the maximum Lyapunov exponent decreases from a positive value to a non-positive value range within a series of predetermined time windows and the second derivative norm is below a convergence threshold, it is determined that the endpoint has been reached, including: Record the maximum Lyapunov exponent calculated within multiple consecutive predetermined time windows to construct an exponent time series; Calculate the first and second derivatives of an exponential time series; When the exponential time series decreases from a positive value to a non-positive value within three or more consecutive time windows, and the second derivative norm is below the convergence threshold, a signal is output to determine that the endpoint has been reached.

[0058] The main control program maintains a first-in-first-out (FIFO) cache queue to collect data in real time for each time window, for example, each window is 10 seconds long, and to calculate the parsed maximum Lyapunov exponent. As the smelting process advances, data such as the output of the most recent 20 to 50 consecutive time windows are recorded and accumulated. The value is thus used to construct a discrete exponential time series L(t) in the time domain, representing the decay and change law of the degree of chaos in the system dynamics. Calculus analysis is performed on this discrete series using the central difference method or the Savitzky-Golay filtering differentiation algorithm. The time base for differentiation is the sliding time window index or the normalized window time, so that the first derivative, second derivative, and their threshold comparisons have a consistent dimensional basis. The first derivative sequence of the exponential time series in the current window and its neighborhood is calculated. and second derivative sequence This represents the rate and acceleration changes in chaotic evolution. Built-in real-time monitoring logic jointly assesses these three levels of time series indicators.

[0059] To verify the positive and negative transition properties of the base sequence L(t), since a positive maximum Lyapunov exponent represents chaotic behavior with local exponential divergence, typically during a period of intense reaction, while a negative value indicates convergence of the system's phase space trajectory, the system requires L(t) to stably transition from a positive interval such as +0.15 and remain at 0 or negative values, such as -0.05 or -0.08, within at least three consecutive sliding time windows, typically three to five. Based on this, to reduce local false triggering artifacts caused by measurement noise, the second derivative sequence is calculated. The L2 norm is calculated and compared with a pre-set convergence threshold, such as 0.001 to 0.005, for example, 0.002. Only when the second derivative norm is lower than this convergence threshold, that is, when it is determined that the oscillation of melt flow and thermal convection tends to weaken and the system tends to be in a macroscopic steady state, will the main controller send a high-level digital interrupt signal of "reaching the end point" to the programmable logic controller through the I / O port to trigger the finishing process actions such as shutting off the flame and slag removal.

[0060] When the temperature reaches to At the high-temperature industrial furnace site, radiation-resistant, high-temperature-resistant, high-speed industrial cameras were used to collect real-time video data of 60 batches of metal smelting. The camera focal length was fixed at [missing information]. The three-dimensional attitude angles remained stationary, and the sampling frequency was fixed at 50 frames per second. Three comparative schemes were used for ablation experiments. Scheme 1 was the basic model, using a fixed slope parameter for color space conversion without spatial modulation factor enhancement. Scheme 2 was a single-module model, adding only a slope update mechanism based on the maximum Lyapunov exponent to the basic model. Scheme 3 was the complete model of this invention, combining Scheme 2 with a spatial modulation factor generation and feature weighting mechanism based on the logarithmic amplitude spectrum.

[0061] After continuous monitoring and analysis of 60 smelting cycles, the average signal-to-noise ratio of the high-temperature region feature map extracted by Scheme 1 was [value missing]. The average time error for endpoint determination is The false alarm rate is The average signal-to-noise ratio of the feature maps in Scheme 2 is improved to [value missing]. The average time error for endpoint determination has been reduced to The false alarm rate dropped to The average signal-to-noise ratio of the feature maps in Scheme 3 reaches [a certain value]. The average time error for endpoint determination was reduced to Furthermore, the false alarm rate for endpoint determination within a continuous time window is only [missing information]. The endpoint determination for most batches falls near the macroscopic steady-state region where the second derivative norm is below the convergence threshold, such as... Figure 2 As shown.

[0062] The difference between Scheme 2 and Scheme 1 shows that binding the maximum Lyapunov exponent to the slope of the hyperbolic tangent function allows for adjustment of the nonlinear mapping interval according to the chaotic dynamic state of the system. A comparison between Scheme 3 and Scheme 2 further demonstrates that separating specific high-energy ring frequency bands and reconstructing the spatial modulation factor through a frequency domain filtering mask can reduce the influence of energy abrupt changes in the central DC component and background thermal radiation displacement noise, thus significantly improving the accuracy of the global feature vector representation of the spatiotemporal changes in the core reaction zone of the molten pool.

Claims

1. A method for determining the endpoint of metal smelting based on video feature extraction, characterized in that, Includes the following steps: Acquire real-time image sequences of the metal smelting process; convert the current frame of the image sequence from RGB to the log-hyperbolic tangent color space to obtain a multi-channel feature map; In the initial stage, preset parameters are used as slope parameters, and in the non-initial stage, the maximum Lyapunov exponent of the previous time window is used to generate slope parameters greater than zero. The spatiotemporal joint gradient field of the multi-channel feature map is calculated. Based on the multi-channel feature map, the high-temperature region is determined. An anisotropic spatial weighting kernel is generated by combining the local entropy and the principal direction of the high-temperature region. The local entropy is used to adjust the entropy decay of the anisotropic spatial weighting kernel. The two-dimensional Fourier phase spectrum and logarithmic amplitude spectrum of the circumscribed rectangle of the high-temperature region are extracted to construct a spatial modulation factor. The spatial modulation factor is used to modulate each monochrome channel to generate a color weight matrix. After aligning the anisotropic spatial weighting kernel and the color weight matrix in terms of spatial dimensions, they are fused through a nonlinear coupling function to obtain a composite weighting matrix. The weights in the fusion process are allocated by the maximum Lyapunov exponent of the previous time window. A global feature vector is generated by weighted summation of multi-channel feature maps using a composite weighted matrix. The continuous global eigenvectors are constructed into phase space trajectories and the current maximum Lyapunov exponent is calculated. When the maximum Lyapunov exponent decreases from a positive value to a non-positive value within multiple consecutive predetermined time windows and the second derivative norm is lower than the convergence threshold, the endpoint is determined to have been reached.

2. The method for determining the endpoint of metal smelting based on video feature extraction according to claim 1, characterized in that, The focal length and three-dimensional attitude angle of the fixed image acquisition device are used to lock the center of the liquid molten pool by shielding the mechanical position adjustment command. Image frames are acquired at a fixed sampling frequency of 30fps to 60fps and stored in a circular buffer according to the timestamp. Real-time image sequences are extracted by a sliding time window of fixed size.

3. The method for determining the endpoint of metal smelting based on video feature extraction according to claim 2, characterized in that, Separate the red, green, and blue monochrome channel matrix of the current frame, and normalize the pixel values ​​to the range of 0 to 1 to obtain... proceed in sequence Logarithmic transformation and tanh Hyperbolic tangent transform, where This is the result of a logarithmic transformation. The hyperbolic tangent transform results are then stacked along the channel dimension to form the multi-channel feature map, where Greater than zero, and in the initial stage Set to 1.5 to 2.5, update the maximum Lyapunov exponent based on the previous time window during non-initial phases. And apply a positive value limit.

4. The method for determining the endpoint of metal smelting based on video feature extraction according to claim 3, characterized in that, The horizontal and vertical spatial gradients of the multi-channel feature map are calculated using the Sobel operator. The temporal gradient is obtained by pixel-by-pixel difference between two adjacent frames. The square root of the sum of the squares of the three gradients is then taken and averaged over each channel to obtain a two-dimensional spatiotemporal joint gradient intensity matrix. The two-dimensional single-channel representative image is obtained by weighted summation of each channel. Otsu threshold segmentation is used to generate a high-temperature region mask, and the high-temperature connected region with the largest area is retained. The main direction of the high-temperature region is determined by the eigenvector corresponding to the largest eigenvalue of the coordinate covariance matrix of the region.

5. The method for determining the endpoint of metal smelting based on video feature extraction according to claim 4, characterized in that, Calculate the local Shannon entropy matrix within a preset neighborhood window of the two-dimensional single-channel representative image, and rotate the major axis of the two-dimensional Gaussian kernel to be consistent with the main direction of the high-temperature region; The local Shannon entropy matrix is ​​normalized, and an entropy decay coefficient matrix is ​​generated according to the negative exponential relationship of the normalized local entropy values. This coefficient matrix is ​​then multiplied element-wise with the high-temperature region mask, the two-dimensional spatiotemporal joint gradient intensity matrix, and the rotated two-dimensional Gaussian kernel, and normalized to generate the anisotropic spatial weighted kernel.

6. The method for determining the endpoint of metal smelting based on video feature extraction according to claim 5, characterized in that, Obtain the minimum bounding rectangle of the high-temperature region, extract image blocks from the multi-channel feature map and generate a two-dimensional single-channel representative matrix by weighting; A two-dimensional Fourier transform is performed on the representative matrix to obtain the phase spectrum and the logarithmic amplitude spectrum. After excluding the central low frequency, continuous or adjacent high-energy ring frequency bands with a cumulative energy ratio of not less than 60% are selected to construct a frequency domain filter mask. The complex frequency domain matrix is ​​reconstructed and inversely transformed to generate the spatial domain modulation factor.

7. The method for determining the endpoint of metal smelting based on video feature extraction according to claim 6, characterized in that, The spatial modulation factor is aligned in size and then multiplied element-wise with the three monochrome channel matrices to generate the color weight matrix. The anisotropic spatial weighting kernel is broadcast as a three-dimensional spatial weighting matrix. The dimensionless maximum Lyapunov exponent of the previous time window is mapped to a fusion weight using the Sigmoid function. The three-dimensional spatial weighting matrix and the color weighting matrix are weighted by the fusion weight and the value obtained by subtracting the fusion weight from 1, respectively. The results are then added together and normalized to generate the composite weighting matrix.

8. The method for determining the endpoint of metal smelting based on video feature extraction according to claim 7, characterized in that, The composite weighted matrix is ​​multiplied element-wise and channel-wise by the multi-channel feature map. The resulting three-dimensional tensor is then subjected to global average pooling along its width and height. The pooled scalars are concatenated in channel order and normalized using the L2 norm to generate the global feature vector.

9. The method for determining the endpoint of metal smelting based on video feature extraction according to claim 8, characterized in that, The mutual information algorithm is called to estimate the optimal delay time for the global feature vectors of multiple consecutive frames, and the pseudo nearest neighbor algorithm is called to determine the optimal embedding dimension. Multivariate delay embedding is performed according to the two to form the phase space trajectory. Search for adjacent trajectory points that satisfy the time interval constraint in the phase space trajectory, fit the linear growth interval of the average logarithmic divergence curve, and use the fitting slope as the maximum Lyapunov exponent of the current time window.

10. The method for determining the endpoint of metal smelting based on video feature extraction according to claim 9, characterized in that, The maximum Lyapunov exponent is recorded for 3 to 5 consecutive predetermined time windows to construct an exponential time series, and the second derivative series is calculated using the central difference method or Savitzky-Golay filtering algorithm. When the exponential time series decreases from a positive value to a non-positive value range and remains there, and the L2 norm of the second derivative series is lower than the convergence threshold in the range of 0.001 to 0.005, the signal that has reached the endpoint is output.

Images