Real-time intelligent monitoring method for tuyere state based on computational vision

Abstract

The invention provides a calculation vision-based tuyere state real-time intelligent monitoring method, which comprises the following steps of: acquiring a sample tuyere image at a self-adaptive frame rate, and preprocessing to obtain a preprocessed tuyere image; feature discrimination and label labeling are carried out on the preprocessed tuyere image, and a tuyere image data set is built; based on a computer vision target detection algorithm, establishing an initial tuyere state monitoring model, and training and optimizing the initial tuyere state monitoring model by using the tuyere image data set to obtain a trained tuyere state monitoring model; the current tuyere image is collected at the self-adaptive frame rate based on the industrial camera installed in front of the tuyere and is input into the trained tuyere state monitoring model, the tuyere state monitoring result is obtained, the monitoring precision and real-time performance are improved, then the operation stability of the blast furnace is improved, meanwhile, the active damping-down time can be shortened, and the production efficiency of the blast furnace is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of metallurgy, and particularly relates to a tuyere state real-time intelligent monitoring method based on computer vision. BACKGROUND

[0002] As a nonlinear, large time delay, multiphase coupling complex reactor, the high furnace is always undergoing high temperature and high pressure physical and chemical reactions. The abnormal state occurs frequently due to the severe physical and chemical reactions and multiphase flow movement in the furnace, which affects the product quality and the stable operation of the high furnace. In severe cases, it can lead to active wind shutdown, increase production cost and labor intensity, and greatly reduce the production efficiency of the high furnace.

[0003] The tuyere of the high furnace is the only window through which the internal smelting progress of the high furnace can be observed. The operators of the high furnace observe the movement state, light and dark degree and coal cloud shape of the tuyere, and combine their own production experience to judge and evaluate the furnace condition, and make timely feedback on abnormal furnace condition to reduce the probability of production accidents. However, this judgment and evaluation method has great limitations. The human observation has strong subjectivity, and the judgment of the same furnace condition may vary from person to person, which leads to different control methods. In addition, the working condition of the tuyere of the high furnace is complex, and the optical properties of the flame are affected by many factors, including the increase of the artificial judgment difficulty of the pressure in the furnace, the coal powder injection amount, and the like. The high temperature, high pressure, dust and other harsh conditions in the production of the high furnace bring great safety hazards to artificial observation.

[0004] Therefore, how to timely, accurately and safely monitor the tuyere of the high furnace and improve the running stability and production efficiency of the high furnace is a technical problem to be solved by the person skilled in the art. SUMMARY

[0005] The present application provides a tuyere state real-time intelligent monitoring method based on computer vision, which solves the defects of low monitoring precision, real-time performance and safety of the tuyere of the high furnace in the prior art, and leads to poor running stability and low production efficiency of the high furnace.

[0006] In one aspect, the present application provides a tuyere state real-time intelligent monitoring method based on computer vision, which comprises: An industrial camera installed in front of the tuyere is used to collect sample tuyere images at an adaptive frame rate; A digital image processing technology is used to pre-process the sample tuyere images to obtain pre-processed tuyere images; An unsupervised clustering algorithm and production experience are combined to analyze the features and label the pre-processed tuyere images, and a tuyere image dataset is built; An initial tuyere state monitoring model is established based on a computer vision target detection algorithm, and the initial tuyere state monitoring model is trained and optimized using a tuyere image dataset to obtain a trained tuyere state monitoring model; An industrial camera installed in front of the tuyere is used to collect a current tuyere image at an adaptive frame rate, and the collected tuyere image is input into the trained tuyere state monitoring model to obtain a monitoring result of the tuyere state.

[0007] According to the tuyere state real-time intelligent monitoring method based on computer vision provided by the application, the sample tuyere image is preprocessed using a digital image processing technology to obtain a preprocessed tuyere image, including: The sample tuyere image is converted into a gray-scale image; The gray-scale image is denoised and enhanced to obtain a denoised and enhanced tuyere image; The denoised and enhanced tuyere image is morphologically processed to obtain the preprocessed tuyere image.

[0008] According to the tuyere state real-time intelligent monitoring method based on computer vision provided by the application, the sample tuyere image is converted into a gray-scale image, including: The weight coefficients of different colors are set according to the sensitivity of the human eye to different colors; The pixel values of each color in the sample tuyere image are weighted and averaged with the respective weight coefficients to obtain the gray-scale image.

[0009] According to the tuyere state real-time intelligent monitoring method based on computer vision provided by the application, the gray-scale image is denoised and enhanced to obtain a denoised and enhanced tuyere image, including: An adaptive median filter algorithm is used to denoise the gray-scale image to obtain a denoised tuyere image; A power transformation algorithm is used to enhance the denoised tuyere image to obtain the denoised and enhanced tuyere image.

[0010] According to the tuyere state real-time intelligent monitoring method based on computer vision provided by the application, the denoised and enhanced tuyere image is morphologically processed to obtain the preprocessed tuyere image, including: The edges of the denoised and enhanced tuyere image are deburred by opening operation to smooth the image boundary to obtain a first morphologically processed image; The target holes in the first morphologically processed image are filled by closing operation to connect the target dark areas together to obtain the preprocessed tuyere image; wherein the hole diameter of the target hole is less than a preset size, and the area of the target dark area is less than a preset area.

[0011] According to the wind port state real-time intelligent monitoring method based on computer vision provided by the application, the sample wind port image includes sub-sample wind port images of different wave bands, and the preprocessed wind port image includes sub-preprocessed wind port images of different wave bands; Before building the wind port image dataset, the preprocessed wind port image is subjected to feature analysis and label annotation in combination with the unsupervised clustering algorithm and production practice experience. The sub-preprocessed wind port images of different wave bands are subjected to multi-scale decomposition to obtain low-frequency components and high-frequency components of the sub-preprocessed wind port images of different wave bands. Each low-frequency component is subjected to weighted average with the respective wave band weight to obtain a low-frequency fusion component. The largest high-frequency component in absolute value is selected from all high-frequency components as a high-frequency fusion component. The low-frequency fusion component and the high-frequency fusion component are subjected to inverse multi-scale transformation, recombined and restored to a complete image to obtain a fusion wind port image. Correspondingly, the preprocessed wind port image is subjected to feature analysis and label annotation in combination with the unsupervised clustering algorithm and production practice experience to build the wind port image dataset. The fusion wind port image is subjected to feature analysis and label annotation in combination with the unsupervised clustering algorithm and production practice experience to build the wind port image dataset.

[0012] According to the wind port state real-time intelligent monitoring method based on computer vision provided by the application, the sample wind port image includes sub-sample wind port images of different wave bands, and the preprocessed wind port image includes sub-preprocessed wind port images of different wave bands; The method further includes: The sharpness deviation and the signal-to-noise ratio deviation of the visible light wave band sub-sample wind port image are calculated. The sharpness deviation and the signal-to-noise ratio deviation are subjected to weighted average to obtain a comprehensive deviation. If the comprehensive deviation is greater than a preset deviation, the original weight of the visible light, the comprehensive deviation and a preset compensation intensity are multiplied to obtain a weight decrement of the visible light. The wave band weight of the visible light is obtained by subtracting the weight decrement of the visible light from the original weight of the visible light. According to the synergy coefficient of infrared and ultraviolet, the weight decrement of the visible light is distributed to obtain a weight increment of infrared and a first weight increment of ultraviolet. The wave band weight of the infrared is obtained by summing the original weight of the infrared and the weight increment of the infrared. The wave band weight of the ultraviolet is obtained by summing the original weight of the ultraviolet and the first weight increment of the ultraviolet.

[0013] The method for real-time intelligent monitoring of the state of a tuyere based on computer vision provided by the application further comprises: acquiring a temperature anomaly detection rate in historical data of an infrared wave band and a pollutant detection rate in historical data of an ultraviolet wave band; if the temperature anomaly detection rate is greater than a first preset detection rate and the pollutant detection rate in the ultraviolet wave band is less than or equal to a second preset detection rate, the synergy coefficient is increased by a preset value; if the temperature anomaly detection rate is less than or equal to the first preset detection rate and the pollutant detection rate in the ultraviolet wave band is greater than the second preset detection rate, the synergy coefficient is decreased by a preset value; if the temperature anomaly detection rate is greater than the first preset detection rate and the pollutant detection rate in the ultraviolet wave band is greater than the second preset detection rate, the synergy coefficient is maintained unchanged; if the temperature anomaly detection rate is less than or equal to the first preset detection rate and the pollutant detection rate in the ultraviolet wave band is less than or equal to the second preset detection rate, the synergy coefficient is maintained unchanged.

[0014] The method for real-time intelligent monitoring of the state of a tuyere based on computer vision provided by the application further comprises: acquiring a temperature deviation degree and a humidity deviation degree of a current environment and a fluctuation rate of a historical state of a tuyere; performing weighted averaging on the temperature deviation degree, the humidity deviation degree and the fluctuation rate of the historical state of the tuyere to determine an environmental entropy of the current environment; multiplying the environmental entropy, a preset wave band sensitivity coefficient and an environmental coefficient of the current environment to obtain a second weight increment of the ultraviolet wave.

[0015] The method for real-time intelligent monitoring of the state of a tuyere based on computer vision provided by the application further comprises: if the monitoring result of the state of the tuyere indicates an abnormal state of the tuyere, a rectangular boundary box is used to mark a position corresponding to the abnormal state of the tuyere and output corresponding label information, and the category and position of the abnormal state of the tuyere are recorded in the background.

[0016] In another aspect, the application further provides a system for real-time intelligent monitoring of the state of a tuyere based on computer vision, which comprises: a collection module configured to collect sample tuyere images based on an industrial camera installed in front of a tuyere at an adaptive frame rate; a preprocessing module configured to preprocess the sample tuyere images by using a digital image processing technology to obtain preprocessed tuyere images; The building module is used for combining an unsupervised clustering algorithm with production practice experience to perform feature analysis and label annotation on the preprocessed tuyere image, and build a tuyere image dataset; The training module is used for establishing an initial tuyere state monitoring model based on a computer vision target detection algorithm, training and optimizing the initial tuyere state monitoring model using the tuyere image dataset, and obtaining a trained tuyere state monitoring model. The monitoring module is used for collecting a current tuyere image based on an industrial camera installed in front of the tuyere at an adaptive frame rate, inputting the current tuyere image into the trained tuyere state monitoring model, and obtaining a monitoring result of the tuyere state.

[0017] In another aspect, the application further provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the real-time intelligent tuyere state monitoring method based on computer vision according to any one of the above aspects when executing the program.

[0018] In another aspect, the application further provides a non-transitory computer readable storage medium having a computer program stored thereon, and the computer program implements the real-time intelligent tuyere state monitoring method based on computer vision according to any one of the above aspects when executed by a processor.

[0019] In another aspect, the application further provides a computer program product including a computer program, and the computer program implements the real-time intelligent tuyere state monitoring method based on computer vision according to any one of the above aspects when executed by a processor.

[0020] The real-time intelligent tuyere state monitoring method based on computer vision provided by the application collects sample tuyere images at an adaptive frame rate, pre-processes the sample tuyere images using a digital image processing technology to obtain preprocessed tuyere images, combines an unsupervised clustering algorithm with production practice experience to perform feature analysis and label annotation on the preprocessed tuyere images, builds a tuyere image dataset, establishes an initial tuyere state monitoring model based on a computer vision target detection algorithm, trains and optimizes the initial tuyere state monitoring model using the tuyere image dataset to obtain a trained tuyere state monitoring model, collects a current tuyere image based on an industrial camera installed in front of the tuyere at an adaptive frame rate, inputs the current tuyere image into the trained tuyere state monitoring model, and obtains a monitoring result of the tuyere state, thereby solving the problems of strong subjectivity, poor safety, and large time lag of traditional manual observation, improving monitoring accuracy and real-time performance, and further improving the stability of blast furnace operation, reducing the length of active idle time, and improving the production efficiency of the blast furnace. BRIEF DESCRIPTION OF DRAWINGS

[0021] In order to make the technical solutions in the present application or prior art clearer, the accompanying drawings needed in the embodiments or prior art description will be briefly introduced. Obviously, the accompanying drawings in the following description are only some embodiments of the present application, and all other embodiments obtained by a person of ordinary skill in the art without creative work based on the accompanying drawings also belong to the protection scope of the present application.

[0022] Figure 1 is a flowchart of a real-time intelligent monitoring method for a tuyere state based on computer vision provided by an embodiment of the present application; Figure 2 is a structural schematic diagram of a real-time intelligent monitoring system for a tuyere state based on computer vision provided by an embodiment of the present application; Figure 3 is a structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0023] In order to make the technical solutions in the present application or prior art clearer, the accompanying drawings needed in the embodiments or prior art description will be briefly introduced. Obviously, the accompanying drawings in the following description are only some embodiments of the present application, and all other embodiments obtained by a person of ordinary skill in the art without creative work based on the accompanying drawings also belong to the protection scope of the present application.

[0024] In the prior art, the monitoring of the tuyere state of a blast furnace mainly relies on manual observation. An operator evaluates the furnace condition by judging the movement of the furnace charge, the change in brightness and the shape of the coal mass in the tuyere image through the naked eye. This method has strong subjectivity and large response delay, and is difficult to adapt to the complex and variable working conditions in the blast furnace. Manual observation is easily disturbed by high temperature, high pressure and dust, and has safety hazards. In addition, experience differences lead to inconsistent judgment results, affecting the timeliness and accuracy of the control measures.

[0025] In order to solve the above problems, it is found through research that the tuyere of the blast furnace, as the only visible window of the internal state, has strong correlation between its image features and the furnace condition. Traditional manual monitoring cannot meet the real-time and objectivity requirements, and automation technology needs to be introduced. Considering that an industrial camera can collect images in real time, combined with digital image processing technology to extract effective features, but pure algorithm classification is difficult to adapt to the complex scene in actual production. Therefore, unsupervised clustering is combined with production experience to build a dataset with actual guiding significance as the key. Further, the target detection algorithm can realize state recognition and position positioning at the same time, providing double information support for the operator, forming a closed-loop solution from data acquisition to intelligent decision-making.

[0026] Therefore, the present application proposes to collect the tuyere image based on an industrial camera at an adaptive frame rate, to obtain a preprocessed image through digital image processing for gray scale conversion, noise reduction enhancement and morphological processing. The image features are analyzed and labeled in combination with an unsupervised clustering algorithm and production experience to construct a special data set. A monitoring model is established based on a computer vision target detection algorithm, and after training and optimization, the current tuyere image is processed in real time to output the state category and position information.

[0027] Specifically, Figure 1 is a flowchart of the tuyere state real-time intelligent monitoring method based on computer vision provided by the embodiment of the present application.

[0028] As Figure 1 shown, the execution subject of the tuyere state real-time intelligent monitoring method based on computer vision provided by the embodiment of the present application can be an electronic device, and the method mainly includes the following steps: 101. Collecting a sample tuyere image based on an industrial camera installed in front of the tuyere at an adaptive frame rate; 102. Preprocessing the sample tuyere image by using a digital image processing technology to obtain a preprocessed tuyere image; 103. Analyzing and labeling the features of the preprocessed tuyere image in combination with an unsupervised clustering algorithm and production experience to build a tuyere image data set; 104. Establishing an initial tuyere state monitoring model based on a computer vision target detection algorithm, training and optimizing the initial tuyere state monitoring model using the tuyere image data set to obtain a trained tuyere state monitoring model; 105. Collecting a current tuyere image based on an industrial camera installed in front of the tuyere at an adaptive frame rate, inputting the current tuyere image into the trained tuyere state monitoring model to obtain the monitoring result of the tuyere state.

[0029] In a specific implementation process, adaptive frame rate collection refers to dynamically adjusting the image capture frequency according to the blast furnace working condition, for example, it can be selected according to the actual production situation in units of frames / second, frames / minute, frames / hour, frames / day, according to the fluctuation of blast furnace production, wherein, when the furnace condition fluctuates violently, 60 frames per minute can be used, and in the stable stage, it can be adjusted to 10 frames per minute, which not only ensures the data integrity but also avoids resource waste.

[0030] The gray scale conversion adopts a weighted average method, for example, the red, green and blue channels are multiplied by 0.299, 0.587 and 0.114 coefficients respectively to synthesize a gray scale image, which retains the sensitive features of the human eye. The morphological processing includes a combination of open operation and close operation, for example, a 3x3 circular structure element is used to remove burrs, and then a 5x5 rectangular structure element is used to fill holes to enhance the feature continuity.

[0031] The unsupervised clustering algorithm can select the K-means method. For example, according to the area of the coal mass in the tuyere image, the area of the shadow at the edge of the raceway, and the high-dimensional features of the color space of the tuyere image, combined with the common abnormal working state operation experience of the blast furnace in the production practice, the image dataset is divided into three clusters according to the shadow area of the tuyere raceway. The LabelImg image labeling tool can be used as the tuyere image labeling tool of the embodiment to label the abnormal state category and position of each frame of image obtained by clustering, and a tuyere state real-time intelligent monitoring dataset is built. The tuyere image dataset is divided into a training set, a validation set and a test set in a ratio of 6:2:2.

[0032] The target detection algorithm can adopt an improved YOLO architecture, such as a YOLOv8 model. In this way, an attention mechanism module is added to the feature extraction layer to improve the detection accuracy of small targets. The basic network structure of the YOLOv8 model is composed of a backbone, a neck, and a head. The backbone layer is an improved CSPDarknet network structure, which uses residual blocks and CSP to extract multi-scale features of images, improve the feature extraction capability of the model, reduce the computational complexity of the model, and solve the problem of gradient disappearance. The neck layer is a network structure similar to the FPN network, which connects image feature maps of different scales horizontally from top to bottom to generate multi-scale and multi-semantic feature maps containing rich semantic information and detailed features. The head layer predicts the target and the category of the target at each position of the feature map based on the feature vector corresponding to the position, predicts the parameters of the target bounding box, and locates and determines the target. That is, if the monitoring result of the tuyere state indicates an abnormal tuyere state, the position corresponding to the abnormal tuyere state can be marked and labeled information corresponding to the abnormal tuyere state can be output using a rectangular bounding box, and the category and position of the abnormal tuyere state can be recorded in the background.

[0033] Specifically, the industrial camera can be installed in front of the tuyere peephole, and the acquisition frequency can be dynamically adjusted according to the data of the in-furnace pressure sensor. After the original image is weighted and grayed, adaptive median filtering is performed, for example, the window size is dynamically expanded between 3x3 and 7x7 to effectively remove impulse noise. Then, the gamma value is set to a power transformation value to enhance the dark details, making the coal mass outline clearer. In the morphological processing stage, the open operation eliminates isolated noise points with a particle size less than 5 pixels, and the closed operation can connect dark areas with an area exceeding 200 pixels to form a complete feature region.

[0034] The preprocessed image is input into a clustering model, and a histogram distribution of a saturation channel in an HSV color space is taken as a feature to automatically divide the image into three categories of normal, slight abnormality and serious abnormality, and the three categories are reviewed and labeled by an experienced engineer. The trained detection model can output coordinates of an abnormal region in real time, for example, positioning accuracy reaches a range of ±15 pixels in a 1280*720 resolution image.

[0035] Compared with the prior art, the traditional method relies on fixed frequency image acquisition and cannot adapt to fluctuations in the blast furnace working condition, resulting in loss of key frames or data redundancy. The scheme automatically increases the sampling rate when the furnace condition changes, for example, when the temperature sensor detects a sudden change of 50℃ / min, the frame rate is immediately increased from 1fps to 10fps, ensuring the capture of transient abnormalities. The traditional morphological processing uses a fixed structure element size, which can easily cause feature distortion. The scheme dynamically adjusts the operation parameters according to the image resolution, for example, a 7*7 structure element is used for a 2 million pixel image and an 11*11 structure element is used for a 5 million pixel image, keeping the consistency of feature processing. The existing clustering method simply relies on algorithm classification. The scheme introduces an expert experience correction mechanism, for example, when the algorithm misclassifies a normal coal powder injection image as an abnormal class, the operator can manually adjust the classification label through the labeling platform to improve the reliability of the data set.

[0036] Through the above technical scheme, the present application realizes the automatic real-time monitoring of the tuyere state, and eliminates the subjective bias of manual observation. The adaptive image processing flow effectively extracts key features, and the clustering labeling with experience correction ensures the quality of the data set, so that the detection model can accurately identify abnormal states such as slag hanging and nodulation and locate the fault area. The abnormal information is recorded in the system log in real time to provide visual data support for the operator and shorten the fault response time. The method shortens the time-consuming of abnormal identification from minutes to milliseconds, greatly improves the positioning accuracy, significantly reduces the number of non-scheduled blowdowns caused by tuyere damage, and ensures the continuous and stable operation of the blast furnace.

[0037] In some embodiments, the present application further proposes converting a sample tuyere image into a grayscale image; denoising and enhancing the grayscale image to obtain a denoised and enhanced tuyere image; and performing morphological processing on the denoised and enhanced tuyere image to obtain a preprocessed tuyere image.

[0038] The grayscale image refers to converting a color image into a single-channel image containing only brightness information. The weighted average method can be used to achieve this, and the weight coefficients of different colors are set according to the sensitivity of the human eye to different colors, for example, the weight coefficients of red, green and blue are set to 0.299, 0.587 and 0.114 respectively, and more visual information is retained through weighted average.

[0039] The denoising and enhancement refer to eliminating image noise and improving feature contrast, and specifically, an adaptive median filtering algorithm can be used to remove impulse noise, and a power transformation algorithm is used to adjust the gray scale distribution to enhance dark details.

[0040] Specifically, the following definitions are first made: Sxy represents a filter rectangular window size; Zmax and Zmin represent the maximum and minimum values of the gray scale values in Sxy, respectively; Zxy is a gray scale value at coordinates (x, y); Smax is the maximum size of the rectangular window Sxy; and Zmed represents the median of the gray scale values in Sxy.

[0041] First, it is determined whether the median is valid. If the median satisfies the condition relationship Zmin<Zmed<Zmax, the following execution is continued, if not, it is determined that the current median contains noise points, and then neighborhood expansion is performed. If the size of the expanded window exceeds Smax, the value Zmed obtained by the algorithm is forced to be output. Finally, pixel value judgment is performed. If the original pixel value satisfies the limited condition Z_min<Z_xy<Z_max, the original gray scale value of the pixel value is taken as the output value. Otherwise, the verified output Zmed is taken as the final output value. Such a judgment rule can not only remove impulse noise, but also save some detail information contained in the image itself to the maximum extent.

[0042] The power transformation algorithm can be used to perform image enhancement processing on the gray scale image, which can enhance the contrast of the image and highlight the combustion state feature information contained in the image. The mathematical expression of the power transformation is:

[0043] wherein, is the original gray scale value; is the output gray scale value of the power transformation; is a gray scale scaling coefficient, which is usually taken as 1; is a scaling factor. Through such gray scale compensation, the filtered image is subjected to nonlinear range compression (Zout=Zin^k) to compress the highlight area, to enlarge the shadow details), so as to improve the clarity of the visible image.

[0044] In a specific implementation process, the morphological processing refers to improving the image structure through mathematical morphological operation. Specifically, the open operation can be used to remove edge burrs, and the closed operation can be used to fill holes. The size of the structure element can be set to 3*3 circular kernel, which is used to smooth the boundary and connect the discrete dark area.

[0045] Specifically, the denoising and enhancing tuyere image is morphologically processed to obtain a preprocessed tuyere image, including removing burrs of edges of the denoising and enhancing tuyere image by an opening operation, smoothing the image boundary to obtain a first morphologically processed image; and filling target holes in the first morphologically processed image by a closing operation to connect target dark areas together to obtain the preprocessed tuyere image; wherein a hole diameter of the target hole is less than a preset size, and an area of the target dark area is less than a preset area.

[0046] The opening operation refers to a morphological processing process of performing an erosion operation first and then performing a dilation operation, and specifically, a rectangular or circular structural element can be used to scan the image pixel by pixel to eliminate small protrusions and smooth the boundaries of large objects. The closing operation refers to a morphological processing process of performing a dilation operation first and then performing an erosion operation, and specifically, a structural element similar to the opening operation can be used to fill holes and connect adjacent regions. The target hole refers to a blank part inside a closed region in the image, and the hole diameter size can be quantified by the number of pixels, for example, set to not more than 5 pixel diameters. The target dark area refers to a connected region with a gray value lower than a set threshold, and the area can be limited by the number of pixels, for example, set to not more than 0.5% of the total area of the image.

[0047] Specifically, after the image denoising and enhancement are completed, the opening operation is first processed: a rectangular structural element with a size of 3x3 can be selected to scan the image row by row, the edge burrs are removed by the erosion operation, and the main profile is restored by the dilation operation to eliminate isolated noise points and smooth the tuyere profile line. Then the closing operation is processed: the same size structural element is selected, the holes are filled by the dilation operation on the opening operation result, and the target shape is restored by the erosion operation to merge the dark areas with an area less than a preset value into a continuous region. In this process, the hole diameter size can be set to, for example, a range of 3-8 pixels, and the dark area area threshold can be set to 0.1%-1% of the total area of the image, and the specific value is dynamically adjusted according to the actual tuyere image characteristics.

[0048] In this embodiment, by setting the dynamic threshold of the hole diameter and the area, the processing intensity can be automatically adjusted according to the image characteristics under different working conditions, avoiding feature distortion caused by excessive filling or detail loss caused by insufficient processing. In this way, the feature blur problem caused by noise residue or insufficient contrast in the morphological processing process can be effectively solved, and the key working state features in the tuyere image, such as the coal powder injection trajectory and the boundary profile of the whirl region, can be accurately separated, providing a clear image data basis for subsequent feature extraction and model recognition.

[0049] Compared with the prior art, the existing method generally adopts fixed-size median filtering or histogram equalization, which is difficult to adapt to the dynamically changing noise level and complex lighting conditions in the blast furnace tuyere image. The adaptive median filtering adjusts through a dynamic window, which can still effectively denoise in a strong noise environment; the power transformation flexibly controls the gray scale distribution through the parameterized gamma value, and specifically enhances the features of different brightness regions; the morphological processing combines opening and closing operations to remove interference while retaining key structural information, overcoming the problem of edge blurring or detail loss caused by traditional filtering.

[0050] Through the above technical solutions, the present application can effectively eliminate image noise, enhance feature contrast and optimize structural integrity, providing high-quality preprocessed images for subsequent feature discrimination and model training, thereby improving the accuracy and robustness of tuyere state monitoring.

[0051] In one specific implementation process, for the application scenario of blast furnace tuyere state monitoring, although the real-time intelligent monitoring method based on computer vision solves many problems of traditional manual observation, it still faces a specific technical challenge. Due to the complex and variable internal environment of the blast furnace, the lighting conditions and smoke density at the tuyere often change dramatically, which leads to unstable image quality captured by the camera. At certain times, the image may become overexposed or blurred due to strong light or thick smoke, seriously affecting the recognition accuracy of computer vision algorithms. This image quality fluctuation problem is caused by the special working environment of the blast furnace and is not common in other application scenarios. If this problem cannot be effectively solved, even if advanced computer vision technology is used, misjudgment or missed judgment may occur at critical moments, affecting the safe operation and production efficiency of the blast furnace.

[0052] Therefore, in order to solve the problem of unstable image quality in blast furnace tuyere monitoring, adaptive multi-spectral fusion imaging technology can be used. This technology adds infrared and ultraviolet band imaging devices based on the original visible light camera to realize multi-spectral data acquisition and fusion.

[0053] Specifically, three camera devices can be installed at each tuyere to capture images in the visible light, infrared and ultraviolet bands. The images in these three bands have different sensitivities to environmental changes and can be used complementarily. Among them, installing three camera devices is only exemplary, and the present embodiment does not limit other ways of capturing images in the visible light, infrared and ultraviolet bands.

[0054] That is, the collected sample tuyere image includes sub-sample tuyere images of different wavebands. Before the pre-processed tuyere image is subjected to feature analysis and label annotation in combination with an unsupervised clustering algorithm and production experience, the sub-sample tuyere images of different wavebands can be pre-processed to obtain sub-pre-processed tuyere images of different wavebands, and the sub-pre-processed tuyere images of different wavebands can be fused.

[0055] Specifically, the sub-pre-processed tuyere images of different wavebands can be subjected to multi-scale decomposition to obtain low-frequency components and high-frequency components of the sub-pre-processed tuyere images of different wavebands; each low-frequency component is weighted and averaged with a respective waveband weight to obtain a low-frequency fusion component; the high-frequency component with the largest absolute value is selected from all high-frequency components as a high-frequency fusion component; and the low-frequency fusion component and the high-frequency fusion component are subjected to inverse multi-scale transformation to be recombined and restored to a complete image to obtain a fused tuyere image. Correspondingly, the feature analysis and label annotation of the pre-processed tuyere image in combination with the unsupervised clustering algorithm and the production experience to build the tuyere image dataset include the feature analysis and label annotation of the fused tuyere image in combination with the unsupervised clustering algorithm and the production experience to build the tuyere image dataset.

[0056] The multi-scale decomposition refers to decomposition of an image into components of different spatial frequencies, which can be realized by wavelet transform or pyramid decomposition method. The low-frequency component reflects the overall structure of the image, and the high-frequency component contains detailed information. The low-frequency fusion component is realized by weighted averaging. The waveband weight can be dynamically adjusted according to the feature importance of different wavebands. For example, when the visible light waveband weight is higher, more texture information can be retained. The high-frequency fusion component selects the component with the largest absolute value, which can retain the most prominent edge and detail features in each waveband. The inverse multi-scale transformation restores the fused low-frequency and high-frequency components to a complete image by a reconstruction algorithm, such as wavelet inverse transform or pyramid reconstruction algorithm.

[0057] Specifically, after the sub-pre-processed tuyere images of different wavebands are subjected to multi-scale decomposition, the low-frequency components represent the overall brightness and structure of the image, and the high-frequency components reflect the detail changes. When the low-frequency components are subjected to weighted averaging, the waveband weight can be allocated according to the contribution of each waveband in feature expression. For example, the visible light waveband accounts for a higher proportion in texture recognition, and the infrared waveband is more critical in temperature anomaly detection. The maximum value selection strategy of the high-frequency components can effectively retain the most prominent edge information in each waveband and avoid detail loss. The fused image restores the spatial structure through inverse transformation to form a unified expression containing multi-waveband feature information. This fusion process enables the subsequent unsupervised clustering algorithm to classify the state based on more comprehensive features, improving the accuracy and robustness of the dataset construction.

[0058] Compared with the prior art, the traditional method usually uses a single band image for analysis, and cannot fully utilize the complementarity of multi-band information. In the prior art, multi-band fusion is usually simple superposition or fixed weight fusion, without considering the dynamic importance difference of different bands in different scenes. However, the scheme realizes the optimized integration of multi-source information through low-frequency component dynamic weight fusion and high-frequency component selective reservation, and effectively improves the completeness and adaptability of feature expression.

[0059] Through the above technical scheme, the present application solves the problem of insufficient classification accuracy caused by low utilization rate of multi-band image features, enhances the recognition of image features through an adaptive fusion method, and enables subsequent clustering analysis and label marking to more accurately reflect the actual working state of the tuyere. The multi-scale decomposition and reconstruction process effectively retains the key information of different bands, avoids the problems of information loss or interference superposition in traditional methods, and provides a reliable data basis for constructing a high-quality data set. The dynamic weight adjustment mechanism can automatically optimize the feature fusion strategy according to different production environments, and improves the adaptability of the monitoring system under different working conditions.

[0060] In some embodiments, the present application further proposes a method for adjusting the weights of sub-sample tuyere images of different bands, wherein the sub-sample tuyere images of different bands include visible light band sub-sample tuyere images, infrared band sub-sample tuyere images and ultraviolet band sub-sample tuyere images. The clarity deviation and signal-to-noise ratio deviation of the visible light band sub-sample tuyere images can be calculated, the clarity deviation and the signal-to-noise ratio deviation are weighted and averaged to obtain a comprehensive deviation, if the comprehensive deviation is greater than a preset deviation, the original weight of visible light, the comprehensive deviation and a preset compensation intensity are multiplied to obtain a weight decrement of visible light, the original weight of visible light and the weight decrement are used to obtain the band weight of visible light, the weight decrement of visible light is distributed according to the synergy coefficient of infrared and ultraviolet to obtain a weight increment of infrared and a first weight increment of ultraviolet, the original weight of infrared and the weight increment are summed to obtain the band weight of infrared, and the original weight of ultraviolet and the first weight increment are summed to obtain the band weight of ultraviolet.

[0061] The definition of the clarity deviation is the difference between the blurring degree of the edge details in the visible light band image and the ideal clarity, which can be realized by using the Laplace gradient modulus calculation method. The edge sharpness is measured by calculating the second derivative of the image, and the larger the deviation is, the lower the image clarity is. The definition of the signal-to-noise ratio deviation is the deviation of the power ratio of the useful signal and the noise in the visible light band image from the target signal-to-noise ratio, which can be realized by using the local variance analysis method. The deviation is calculated by comparing the signal intensity and the noise fluctuation degree of the local area of the image, and the higher the deviation is, the more serious the image noise interference is. The comprehensive deviation is a composite index obtained by integrating the clarity and the signal-to-noise ratio deviation according to a preset proportion, which can be realized by using the linear weighting method, and is used for comprehensive evaluation of the visible light image quality degradation degree. The coordination coefficient is a quantitative parameter of the complementary relationship between the infrared and ultraviolet bands in the anomaly detection task, which can be adjusted according to the dynamic relationship between the anomaly detection rates of the two bands in the historical data, and is used to guide the weight distribution strategy.

[0062] Specifically, when the clarity and the signal-to-noise ratio of the visible light band image decrease due to environmental dust or optical interference, the image quality is judged by calculating the weighted value of the deviation of the two indexes whether it is lower than a preset threshold. If the comprehensive deviation exceeds the threshold, the visible light weight reduction is generated according to the deviation degree and the preset compensation strength, and the contribution proportion of the visible light in the multi-band fusion is reduced. At the same time, according to the coordination effect of the infrared and ultraviolet bands in the historical data, the weight of the visible light is dynamically distributed to the infrared and ultraviolet bands according to the coordination coefficient. For example, when the infrared band performs well in temperature anomaly detection and the pollutant detection rate of the ultraviolet band is low, the coordination coefficient is adjusted to make more weight transferred to the infrared band; on the contrary, the coordination coefficient is adjusted to increase the weight distribution of the ultraviolet band. This mechanism dynamically adjusts the weight of each band to ensure that the key feature information is effectively retained in the multi-source image fusion process.

[0063] Compared with the prior art, the traditional multi-band image fusion method usually uses fixed weight or simple adjustment strategy based on a single index, which cannot adapt to the dynamic changes of the quality of each band image in complex industrial scenes. The weight adjustment of the visible light band in the prior art does not consider the comprehensive influence of the clarity and the signal-to-noise ratio, and the weight distribution of the infrared and ultraviolet bands lacks a coordination mechanism driven by historical data, resulting in loss of feature information or aggravation of noise interference in the fused image. The present scheme realizes the precise distribution of the multi-band weight by establishing a comprehensive deviation evaluation model and a coordination coefficient dynamic adjustment mechanism, and solves the problem of unstable feature extraction caused by the quality fluctuation of the visible light image.

[0064] By the technical scheme, the application effectively solves the feature recognition error problem caused by the decline of the definition and signal-to-noise ratio of the visible light image under the condition of dust interference or light change, reduces the influence of the low-quality visible light image on the overall monitoring result by dynamically reducing the fusion weight of the low-quality visible light image. Meanwhile, the weight redistribution mechanism based on the synergy coefficient makes full use of the complementary advantages of the infrared and ultraviolet bands in anomaly detection, ensures the integrity of the key feature information in the multi-source data fusion, and improves the recognition accuracy of the tuyere state monitoring model for complex working conditions such as temperature anomaly and pollutant accumulation.

[0065] In some embodiments, the application further proposes a tuyere state real-time intelligent monitoring method based on computer vision, which comprises obtaining the temperature anomaly detection rate in the historical data of the infrared band and the pollutant detection rate in the historical data of the ultraviolet band; if the temperature anomaly detection rate is greater than a first preset detection rate and the ultraviolet band pollutant detection rate is less than or equal to a second preset detection rate, the synergy coefficient is increased by a preset value; if the temperature anomaly detection rate is less than or equal to the first preset detection rate and the ultraviolet band pollutant detection rate is greater than the second preset detection rate, the synergy coefficient is decreased by a preset value; if the temperature anomaly detection rate is greater than the first preset detection rate and the ultraviolet band pollutant detection rate is greater than the second preset detection rate, the synergy coefficient is maintained unchanged; if the temperature anomaly detection rate is less than or equal to the first preset detection rate and the ultraviolet band pollutant detection rate is less than or equal to the second preset detection rate, the synergy coefficient is maintained unchanged.

[0066] The synergy coefficient is a dynamic parameter for adjusting the weight distribution relationship of the infrared and ultraviolet bands, which can be obtained by comprehensive calculation of the historical data statistics and environmental factors, and its role is to dynamically balance the contribution of different bands to the state judgment according to the actual monitoring demand. The temperature anomaly detection rate is the proportion of temperature anomaly events detected in the infrared band data, which can be calculated by the ratio of the number of times exceeding the preset temperature threshold to the total number of detections in the historical data, and is used to reflect the sensitivity of the infrared data to the abnormal state. The pollutant detection rate is the proportion of the existence of pollutants identified in the ultraviolet band data, which can be calculated by the ratio of the frequency of detecting pollutant characteristics to the total number of detections in the historical data, and is used to evaluate the capture ability of the ultraviolet data to the pollution state. The first preset detection rate and the second preset detection rate are the preset judgment thresholds, which can be the benchmark value determined based on long-term production experience, and are used to judge whether the synergy coefficient needs to be adjusted.

[0067] Specifically, in the implementation process, first, the frequency of temperature anomaly events detected by the infrared band and the frequency of pollutant events identified by the ultraviolet band are extracted from the historical database. When the infrared data shows a higher temperature anomaly detection capability and the ultraviolet data does not reach the expected pollution detection, the synergy coefficient is appropriately adjusted upwards, for example, increased by a value range of 0.1 to 0.3, to strengthen the proportion of the infrared band in the weight distribution. Conversely, when the ultraviolet data has a significant identification capability for pollutants and the infrared data does not adequately reflect temperature anomalies, the synergy coefficient is adjusted downwards, for example, reduced by a value range of 0.1 to 0.2, thereby increasing the influence weight of the ultraviolet band. When both bands show high or low detection rates, the synergy coefficient remains stable to avoid weight fluctuations caused by frequent adjustments. This dynamic adjustment mechanism optimizes the fusion strategy of multi-source data by evaluating the actual monitoring efficiency of each band in real time.

[0068] Compared with the prior art, the traditional method usually adopts a fixed weight distribution strategy and cannot adaptively adjust the contribution of different band data according to environmental changes or device performance fluctuations. For example, in a high-temperature environment, the sensitivity of the infrared sensor may decrease, and the fixed weight distribution will cause the monitoring result to deviate. The present scheme can automatically optimize the band weight for different working conditions by introducing a synergy coefficient dynamic adjustment mechanism based on historical detection rates, thereby improving the adaptability of multi-modal data fusion.

[0069] Through the above technical scheme, the present application realizes dynamic balance of the infrared and ultraviolet band weights, and solves the problem of insufficient adaptability of the traditional fixed weight distribution in complex working conditions. Specifically, when the monitoring efficiency of a certain band decreases due to environmental interference or device performance changes, the system can automatically reduce its weight proportion to avoid the negative impact of low-quality data on the overall judgment. At the same time, the role of the high-performance band is strengthened in specific scenarios, for example, relying on ultraviolet data preferentially in a high-pollution period, or focusing on infrared data analysis in a temperature fluctuation period, thereby significantly improving the accuracy and reliability of abnormal state identification.

[0070] In some embodiments, the present application further proposes obtaining a historical state fluctuation rate of the tuyere, a temperature deviation degree and a humidity deviation degree of the current environment, performing weighted average on the temperature deviation degree, the humidity deviation degree and the tuyere historical state fluctuation rate, determining an environmental entropy of the current environment, and multiplying the environmental grade of the environmental entropy, the preset band sensitivity coefficient and the environmental coefficient of the current environment to obtain a second weight increment of the ultraviolet.

[0071] The tuyere historical state fluctuation rate refers to the degree of change in the working state of the tuyere in the historical operation process, and can be specifically realized by calculating the standard deviation or variance of the historical working state parameters of the tuyere, and is used to reflect the stability of the tuyere operation. The temperature deviation degree refers to the difference between the current environmental temperature and the preset standard temperature, and can be specifically calculated by using the difference value or ratio value, and is used to quantify the influence of temperature abnormalities on monitoring accuracy. The humidity deviation degree refers to the difference between the current environmental humidity and the preset standard humidity, and can be specifically calculated by collecting data in real time through the humidity sensor and calculating the relative deviation, and is used to represent the influence of humidity abnormalities on image quality. The weighted average refers to an operation method of linearly combining multiple parameters according to preset weights, and can be specifically realized by using a normalized weight coefficient distribution method, and is used to comprehensively evaluate the combined influence of environmental factors on the monitoring system. The environmental entropy refers to an evaluation index obtained by quantifying the comprehensive influence degree of environmental factors, and can be specifically calculated by using an information entropy calculation model combined with the probability distribution of environmental parameters, and is used to represent the influence level of environmental complexity on the monitoring system. The waveband sensitivity coefficient refers to the response sensitivity of different spectral wavebands to specific environmental conditions, and can be specifically obtained by laboratory calibration or historical data analysis, and is used to adjust the contribution of different wavebands in image fusion. The environmental coefficient refers to a correction parameter reflecting the comprehensive characteristics of the current environment, and can be specifically calculated dynamically according to the environmental sensor data, and is used to compensate for the influence of environmental changes on the monitoring model. The second weight increment of ultraviolet refers to the additional weight value that needs to be added to the ultraviolet waveband in the image fusion process, and can be specifically calculated by using a linear or nonlinear function, and is used to dynamically optimize the fusion effect of multi-waveband images.

[0072] Specifically, by collecting the monitoring data of the temperature sensor and the humidity sensor in real time, the absolute deviation of each from the preset reference value is calculated as the temperature deviation and the humidity deviation. At the same time, the tuyere working state parameters in the recent set time period are extracted from the historical database, for example, the tuyere brightness value or the coal powder concentration data in the past 24 hours, and the standard deviation thereof is calculated as the historical state fluctuation rate. The three parameters are weighted and summed according to the preset weight ratio, for example, the temperature deviation weight is set to 0.4, the humidity deviation weight is 0.3, and the historical fluctuation rate weight is 0.3, to obtain the environmental entropy value which comprehensively reflects the environmental complexity. According to the numerical interval of the environmental entropy, the environmental grade is divided, for example, when the entropy value exceeds the threshold value, it is determined as a severe environmental grade. Combined with the characteristic coefficient of the ultraviolet waveband which is sensitive to pollutants and the dust concentration correction factor contained in the current environmental coefficient, the additional weight value of the ultraviolet waveband is calculated by multiplication operation. The incremental weight will be added to the first weight increment of the ultraviolet obtained by the synergy coefficient distribution to form the total weight of the ultraviolet waveband finally applied to the multi-waveband image fusion.

[0073] In some embodiments, the temperature deviation calculation can adopt the absolute value of the difference between the current temperature and the standard temperature of 25 degrees Celsius, and the weight compensation mechanism is triggered when the temperature exceeds 30 degrees Celsius. The humidity deviation can be calculated by the percentage deviation of the data measured by the relative humidity sensor from the standard value of 60%. The historical state fluctuation rate can be selected as the standard deviation of the sliding window of the tuyere brightness data in the past 12 hours, and the window length can be set to one sampling point per hour, for example. The grade division of environmental entropy can adopt a three-level classification, and the ultraviolet band weight enhancement mode is activated when the weighted calculation result exceeds 0.7. The band sensitivity coefficient can be obtained through laboratory calibration, for example, the sensitivity coefficient of ultraviolet light to tar pollutants is set to 1.2, and the sensitivity coefficient to water mist interference is set to 0.8. The environmental coefficient can integrate real-time dust concentration monitoring data, and automatically increase the correction factor by 0.15 when PM2.5 exceeds 100 μg / m 3 .

[0074] Compared with the prior art, the traditional method usually adopts fixed weight for multi-band image fusion, and does not consider the influence of dynamic change of environmental parameters on monitoring accuracy. The prior art is prone to insufficient feature extraction of the ultraviolet band due to environmental interference when dealing with complex working conditions, for example, in high temperature and high humidity environment, the pollutant features are easy to be covered by background noise. The present scheme combines real-time monitoring data such as temperature and humidity with historical data of device running state, dynamically adjusts the weight proportion of ultraviolet band in image fusion, and effectively solves the problem of insufficient adaptability of fixed weight strategy in complex working conditions.

[0075] Through the above technical scheme, the present application can automatically optimize the feature extraction intensity of the ultraviolet band according to the real-time environmental conditions and the device running state, enhance the recognition ability of the pollutant features in high temperature and high humidity or in severe working condition fluctuations, and reduce the interference of invalid signals in stable environment, thereby improving the accuracy of multi-band image fusion and providing more reliable image data basis for tuyere abnormal state monitoring.

[0076] Based on the same overall inventive concept, the present application also protects a tuyere state real-time intelligent monitoring system based on computer vision. The tuyere state real-time intelligent monitoring system based on computer vision provided by the present application is described below, and the tuyere state real-time intelligent monitoring system based on computer vision described below can be mutually corresponding and referred to the tuyere state real-time intelligent monitoring method based on computer vision described above.

[0077] Figure 2 is a structural schematic diagram of the tuyere state real-time intelligent monitoring system based on computer vision provided by the embodiments of the present application, as Figure 2 shown, the tuyere state real-time intelligent monitoring system based on computer vision of the present embodiment includes an acquisition module 21, a preprocessing module 22, a building module 23, a training module 24 and a monitoring module 25.

[0078] The acquisition module 21 is configured to acquire sample tuyere images at an adaptive frame rate based on an industrial camera installed in front of the tuyere. The preprocessing module 22 is configured to preprocess the sample tuyere images by using a digital image processing technology to obtain preprocessed tuyere images. The building module 23 is configured to perform feature analysis and label annotation on the preprocessed tuyere images by combining an unsupervised clustering algorithm with production practice experience to build a tuyere image dataset. The training module 24 is configured to establish an initial tuyere state monitoring model based on a computer vision target detection algorithm, train and optimize the initial tuyere state monitoring model by using the tuyere image dataset, and obtain a trained tuyere state monitoring model. The monitoring module 25 is configured to acquire current tuyere images at an adaptive frame rate based on an industrial camera installed in front of the tuyere, input the current tuyere images to the trained tuyere state monitoring model, and obtain a monitoring result of the tuyere state.

[0079] Figure 3 is a structural schematic diagram of an electronic device provided by an embodiment of the present application. The electronic device can include a processor 310, a communications interface 320, a memory 330, and a communications bus 340. The processor 310, the communications interface 320, and the memory 330 can communicate with each other through the communications bus 340. The processor 310 can invoke logical instructions in the memory 330 to execute a real-time intelligent monitoring method for tuyere states based on computer vision.

[0080] In addition, the logical instructions in the memory 330 described above can be implemented in the form of a software functional unit and sold or used as an independent product. When used, the logical instructions can be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the present application or parts of the technical solutions that essentially contribute to the prior art can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device) to execute all or part of the steps of the method described in the various embodiments of the present application. The aforementioned storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0081] In another aspect, the present application also provides a computer program product, which comprises a computer program, the computer program being stored on a non-transitory computer readable storage medium, and the computer program being executable by a processor to enable a computer to perform the real-time intelligent monitoring method of a tuyere state based on computer vision.

[0082] In yet another aspect, the present application also provides a non-transitory computer readable storage medium, which stores a computer program, and the computer program is executable by a processor to implement the real-time intelligent monitoring method of a tuyere state based on computer vision.

[0083] It should be noted that the related information involved in each embodiment of the present application is strictly in accordance with the requirements of laws and regulations, and follows the principles of legality, legitimacy and necessity, and is based on the reasonable purpose of the business scene, and processes the information provided by the user in the process of using the product / service or generated due to the use of the product / service, and the information authorized by the user.

[0084] The related information processed by the present application will be different due to the specific product / service scene, and the specific scene of the user using the product / service should be used as the standard, which may involve the user's account information, device information or other related information. The present application will treat the related information and its processing with high diligence and obligation.

[0085] The present application attaches great importance to the security of the related information, and has taken reasonable and feasible security protection measures to protect the related information in accordance with the industry standards, to prevent the related information from being accessed, disclosed, used, modified, damaged or lost without authorization.

[0086] The device embodiments described above are only schematic, and the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, i.e. they may be located in one place, or distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the present embodiment scheme according to the actual needs. Those skilled in the art can understand and implement it without creative labor.

[0087] Those skilled in the art can clearly understand the technical solutions of the various embodiments from the above description of the embodiments, and the various embodiments can be implemented by means of software with the necessary general hardware platforms, and of course, can also be implemented by hardware. Based on such understanding, the above technical solutions, essentially or in other words, the part of the prior art that makes a contribution, can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, and the like, and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0088] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for some technical features therein; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for real-time intelligent monitoring of air vent status based on computer vision, characterized in that, include: Sample images of the air vents are acquired using an industrial camera installed in front of the vents at an adaptive frame rate. The sample air vent image is preprocessed using digital image processing technology to obtain a preprocessed air vent image; By combining unsupervised clustering algorithms with production practice experience, feature analysis and labeling are performed on the preprocessed wind vent images to build a wind vent image dataset; Based on computer vision target detection algorithms, an initial vent status monitoring model is established. The initial vent status monitoring model is then trained and optimized using a vent image dataset to obtain a trained vent status monitoring model. An industrial camera installed in front of the air vent captures images of the current air vent at an adaptive frame rate, which are then input into the trained air vent status monitoring model to obtain the monitoring results of the air vent status.

2. The real-time intelligent monitoring method for air vent status based on computer vision according to claim 1, characterized in that, The sample air vent image is preprocessed using digital image processing technology to obtain a preprocessed air vent image, including: Convert the sample air vent image into a grayscale image; The grayscale image is denoised and enhanced to obtain a denoised and enhanced air vent image; The denoised and enhanced air vent image is subjected to morphological processing to obtain the preprocessed air vent image.

3. The real-time intelligent monitoring method for air vent status based on computer vision according to claim 2, characterized in that, Converting the sample air vent image into a grayscale image includes: The weighting coefficients for each color are set according to the human eye's sensitivity to different colors; The grayscale image is obtained by weighting the pixel values ​​of each color in the sample vent image with their respective weight coefficients.

4. The real-time intelligent monitoring method for air vent status based on computer vision according to claim 2, characterized in that, The grayscale image is denoised and enhanced to obtain a denoised and enhanced air vent image, including: An adaptive median filtering algorithm is used to denoise the grayscale image to obtain a denoised air vent image; The noise-reduced air vent image is enhanced using a power transform algorithm to obtain the noise-reduced and enhanced air vent image.

5. The real-time intelligent monitoring method for air vent status based on computer vision according to claim 2, characterized in that, The morphological processing of the denoised and enhanced air vent image to obtain the preprocessed air vent image includes: By performing an opening operation to remove the jagged edges of the denoised and enhanced air vent image, the image boundaries are smoothed to obtain the first morphologically processed image. The target holes in the first morphologically processed image are filled using a closing operation, and the target dark areas are connected together to obtain the preprocessed air vent image; wherein the diameter of the target holes is smaller than a preset size, and the area of ​​the target dark areas is smaller than a preset area.

6. The real-time intelligent monitoring method for air vent status based on computer vision according to claim 1, characterized in that, The sample air vent image includes sub-sample air vent images of different bands, and the pre-processed air vent image includes sub-pre-processed air vent images of different bands. Combining unsupervised clustering algorithms with practical production experience, feature analysis and labeling are performed on the preprocessed wind vent images. Before building the wind vent image dataset, the following steps are also included: Multi-scale decomposition is performed on the sub-preprocessed air vent images of different bands to obtain the low-frequency and high-frequency components of the sub-preprocessed air vent images of different bands. Each low-frequency component is weighted and averaged with its respective band weight to obtain the low-frequency fused component. Select the high-frequency component with the largest absolute value from all high-frequency components as the high-frequency fusion component; The low-frequency fusion component and the high-frequency fusion component are subjected to inverse multi-scale transformation and recombined to restore the complete image, thus obtaining the fused wind vent image. Correspondingly, combining unsupervised clustering algorithms with production practice experience, feature analysis and labeling are performed on the preprocessed wind vent images to build a wind vent image dataset, including: By combining unsupervised clustering algorithms with production practice experience, feature analysis and labeling are performed on the fused wind vent images to build a wind vent image dataset.

7. The real-time intelligent monitoring method for air vent status based on computer vision according to claim 6, characterized in that, Sub-sample images of wind tunnels in different bands include sub-sample images of wind tunnels in the visible light band, sub-sample images of wind tunnels in the infrared band, and sub-sample images of wind tunnels in the ultraviolet band. The method further includes: Calculate the sharpness deviation and signal-to-noise ratio deviation of the wind vent images in the visible light band sub-samples; The overall deviation is obtained by weighting the sharpness deviation and the signal-to-noise ratio deviation. If the overall deviation is greater than the preset deviation, the original weight of the visible light, the overall deviation, and the preset compensation intensity are multiplied to obtain the weight reduction of the visible light. The band weight of the visible light is obtained by subtracting the original weight of the visible light from the weight of the visible light itself. Based on the synergy coefficient between infrared and ultraviolet, the weight reduction of visible light is allocated to obtain the weight increment of infrared and the first weight increment of ultraviolet. The original weight of the infrared radiation is summed with the weight increment of the infrared radiation to obtain the band weight of the infrared radiation. The original weight of the ultraviolet light is summed with the first weight increment of the ultraviolet light to obtain the band weight of the ultraviolet light.

8. The real-time intelligent monitoring method for air vent status based on computer vision according to claim 7, characterized in that, Also includes: Obtain the detection rate of temperature anomalies from historical data in the infrared band and the detection rate of pollutants from historical data in the ultraviolet band; If the temperature anomaly detection rate is greater than the first preset detection rate, and the ultraviolet band pollutant detection rate is less than or equal to the second preset detection rate, the synergy coefficient is increased by a preset value. If the temperature anomaly detection rate is less than or equal to the first preset detection rate, and the ultraviolet band pollutant detection rate is greater than the second preset detection rate, the synergy coefficient is lowered by a preset value. If the temperature anomaly detection rate is greater than the first preset detection rate, and the ultraviolet pollutant detection rate is greater than the second preset detection rate, the synergy coefficient remains unchanged; If the temperature anomaly detection rate is less than or equal to the first preset detection rate, and the ultraviolet pollutant detection rate is less than or equal to the second preset detection rate, the synergy coefficient remains unchanged.

9. The real-time intelligent monitoring method for air vent status based on computer vision according to claim 7, characterized in that, Also includes: Obtain the historical volatility of the wind vent, and the current temperature and humidity deviations of the environment; The environmental entropy of the current environment is determined by weighting the temperature deviation, the humidity deviation, and the historical fluctuation rate of the air outlet. The second weight increment of ultraviolet radiation is obtained by multiplying the environmental level, the preset band sensitivity coefficient, and the environmental coefficient of the current environment by the environmental entropy.

10. The real-time intelligent monitoring method for air vent status based on computer vision according to any one of claims 1-9, characterized in that, Also includes: If the monitoring result of the air vent status indicates an abnormal air vent status, the location corresponding to the abnormal air vent status is marked using a rectangular bounding box and the corresponding label information is output. The category and location of the abnormal air vent status are recorded in the background.

Images