Steel ladle thermal state detection method

Abstract

The invention relates to a steel ladle thermal state detection method. The invention relates to the technical field of steel ladle thermal state detection, which is characterized in that a steel ladle thermal state detection area is arranged below a back platform of a converter, and a steel ladle lining, a ladle bottom, an air brick and a water gap part are photographed and visually identified from different angles through 3-4 high-definition thermal infrared imagers which are arranged in a distributed manner and are combined into a complete steel ladle lining model; the sample library is gradually improved through comparison with an initial package type and self-learning ability in a later operation process, and thermal detection capable of accurately identifying various defects is formed; and in combination with a human-computer interaction interface, state tracking is carried out on the steel ladle thermal detection process, a defect alarm mechanism is established, and abnormal state feedback of the steel ladle is realized.

Description

Technical Field

[0001] The invention relates to the technical field of ladle hot state detection, in particular to a ladle hot state detection method. Background Art

[0002] Ladle, also known as steel ladle or big ladle, is an important equipment for receiving molten steel and pouring in steelmaking and continuous casting plants. Ladle is also an important container for transporting molten steel and carrying out secondary metallurgical reactions during the converter, refining and continuous casting processes.

[0003] During the use of the ladle, the ladle brick lining is in contact with high-temperature molten steel and slag for a long time. In addition, during the steel-making and refining process outside the furnace, the erosion will be more serious due to the dual effects of molten steel flow scouring and metallurgical reaction. The erosion rate of the ladle refractory is closely related to the temperature of the converter molten steel, oxidizability, slag amount, refining time outside the furnace, argon blowing time, LF furnace submerged arc effect and RH refining time. The complex physical and chemical environment of the slag line makes the lining of this part most susceptible to chemical erosion and physical iron seepage, resulting in damage. The damage process of the ladle refractory is generally that the graphite on the working surface of the refractory is oxidized to form a decarburization layer, and the magnesia in the decarburization layer is eroded and falls off under the conditions of thermal stress, chemical erosion and mechanical scouring. After the refractory falls off, graphite is exposed again, and continues to be oxidized to form a decarburization layer, and then to the dissolution of magnesia, which causes the damage of the ladle refractory over and over again. This process can be summarized as oxidation, decarburization, loosening, erosion, scouring, shedding and damage. When local damage to refractory materials is severe and undetected, serious steel leakage accidents will occur.

[0004] The erosion of the ladle brick lining will not only reduce the service life of the ladle, but also increase the content of inclusions in the molten steel. In addition, since the temperature of molten steel is higher than that of molten iron and it has good fluidity, accidents of ladle leakage will be more serious after brick falling, cracks, etc., and there are people, equipment, and machinery crossing the molten steel lifting route, which poses a huge safety hazard.

[0005] Through visual recognition technology, the hot state inspection of the ladle will be changed from relying on manual observation and experience judgment to an intelligent, digital and information-based management model using machine vision. It can not only timely and accurately detect safety hazards such as brick falling, peeling and brick damage, but also analyze the operating data of the ladle during its life cycle, thereby reducing production and operation costs and safety costs.

[0006] When the ladle is manually inspected hot during use, the workers are exposed to high-risk risks such as high temperature, dust, thermal radiation, and heat wave shock. The visual ladle thermal inspection management system can directly observe the condition of the ladle lining and perform visual identification to promptly and accurately detect safety hazards such as brick falling, peeling, and brick damage. This project is planned to be implemented in the hot rolling department of Hesteel Tangshan. In order to realize the visual ladle thermal inspection management system, the thermal inspection management system uses visual recognition to identify the falling, erosion, and cracking of the brick lining of the online ladle. At the same time, the thermal imager steel shell temperature measurement technology is used to correspond the local high temperature points of the ladle steel shell to the lining erosion, and the corresponding prompts and warnings are given on the host computer to effectively avoid the occurrence of leakage accidents. During the operation, self-learning can also be sampled to improve the quality of hot ladle inspection. The application of the ladle hot state detection management system not only solves the risk of personal injury and health damage to personnel during operation, but also effectively improves the recognition rate of ladle lining defects. At the same time, it can realize online detection without affecting the rhythm of steelmaking and continuous casting production.

[0007] At present, the hot inspection of steel ladles is still mainly done by manual inspection. The ladle is turned to a horizontal state by a ladle turning machine. Under the condition of high temperature heat radiation, the personnel use steel plates, anti-scalding clothes, protective masks and other personal safety protection measures to check the erosion of the brick lining in the ladle. Due to the poor working environment and short inspection window, the brick lining is not properly inspected, causing the ladle to operate with a disease, which poses a great safety hazard. On the basis of manual visual inspection, some companies use temperature measuring guns to measure the temperature of key parts of the steel shell of the ladle where leakage is prone to occur in experience, and require the ladle to be forced off the line when the steel shell temperature reaches 350℃. Although the temperature measuring gun can directly measure the steel shell temperature, it is still inevitable to combine it with manual inspection. Advanced units use more infrared thermal imaging technology to detect the overall temperature of the ladle steel shell, forming a more comprehensive ladle steel shell temperature image, and then use the finite element method to establish a ladle temperature distribution model, and calculate the residual thickness of the lining through the outer surface temperature and heat transfer model, so as to obtain the erosion of the ladle lining. This method can comprehensively observe the temperature of the ladle steel shell, but it is greatly affected by environmental factors and cannot truly reflect the erosion of the ladle brick lining. In addition, many companies have newly installed ladle management systems, but the research on ladle management systems only involves the realization of technologies in normal production processes such as ladle tracking, ladle distribution and molten steel temperature control, and there is no specific implementation form in the hot state detection and control of the ladle. Summary of the invention

[0008] In view of the shortcomings of the prior art, the present invention provides a hot state detection method for a ladle. Through visual and infrared detection technology inside the ladle, brick falling, peeling, cracking, local erosion, etc. are discovered, and a warning is issued. Manual judgment is made and the information is entered into the system to implement database updates and iterations, and an alarm is issued when the warning value is reached.

[0009] The present invention provides the following technical solutions:

[0010] A method for detecting hot state of a ladle, the method comprising:

[0011] The ladle hot state inspection area is arranged below the platform behind the converter. 3-4 distributedly installed high-definition and infrared thermal imagers are used to take photos of the ladle lining, ladle bottom, air bricks and nozzle from different angles, visually identify and combine them into a complete ladle lining model. By comparing with the initial ladle shape and the self-learning ability in the later operation process, the sample library is gradually improved to form a thermal inspection system that can accurately identify various defects. In combination with the human-computer interaction interface, the status of the ladle hot inspection process is tracked, a defect alarm mechanism is established, and feedback on the abnormal status of the ladle is realized.

[0012] Preferably, the visible light image is converted into a color space from an RGB image to an HSV image; the brightness information of the color image is extracted as the input of the image fusion, and its original hue and saturation are retained:

[0013] Create filter kernel template: construct the corresponding filter kernel template according to the image size of infrared and visible light;

[0014] Construct image pyramid using filter kernel template convolution;

[0015] Use linear interpolation method to extract details at different scales of the pyramid;

[0016] The saliency of the details is used to discriminate the infrared and visible light details of each layer of the pyramid. The designed sliding window is used to convolve the image to generate a weight matrix. The neighborhood information of each pixel is compared with the weight matrix to extract a more credible detail map.

[0017] The smallest scale infrared and visible light images are linearly added to obtain a smooth background image. The extracted detail image is then fused to the background image, and the image is super-resolution upscaled. The detail information of the upper scale is then added, and the process is repeated until the top of the pyramid is reached.

[0018] Color space conversion: convert the fused image back to RGB image and add the hue and saturation that were previously preserved;

[0019] Color enhancement: Perform color enhancement on the fused image to generate a fused image with better clarity and contrast. Perform pixel-level image enhancement based on the contrast of each pixel.

[0020] Preferably, multiple binocular stereo cameras and infrared cameras are used to achieve blind-angle observation of the inner and outer walls of the ladle; the field of view of the single vision module is not less than 45°; each single vision module ensures field of view angle coupling, and uses multi-target positioning and optical flow calculation methods to ensure the accurate acquisition of full-view 3D multimodal information.

[0021] Preferably, obtaining registered infrared and visible light images;

[0022] Perform multi-scale pyramid transformation on the image, and use the designed filter template to perform down-convolution and down-sampling on the infrared image and visible light image respectively;

[0023] Perform detail extraction of infrared and visible light images based on multi-scale pyramid transform, and use the high frequency of the image obtained by linear interpolation method as the fused detail layer;

[0024] Image color space conversion: To address the problem that visible light images are three-channel RGB, we convert RGB color space to HSV color space, extract the V information of the visible light image and fuse it with the infrared image, retaining its original H and S; extract the V channel as the input of visible light, retain the H and S channels to the corresponding matrix, and retain the color information for the subsequent color restoration after fusion;

[0025] Use filter convolution details to filter infrared and visible light detail images, and fuse the infrared and visible light detail images according to the weights of the generated matrix to generate a detail image with richer texture.

[0026] Image super-resolution reconstruction technology is used to perform multi-scale pyramid inverse transform.

[0027] Preferably, a visual information encoding and decoding network is used for training: dual-scale decomposition of low-frequency and high-frequency information: infrared and visible light real-time fusion algorithm based on multi-scale pyramid transformation, image pyramid is constructed by designing filter templates, image information at different scales is obtained, and image super-resolution and significant enhancement are performed, and finally GPU acceleration is used to achieve real-time;

[0028] Infrared images have strong penetration and are not affected by light, while visible light has fine texture information and high spatial resolution. The complementary characteristics are combined to produce high-quality images.

[0029] Fusion image: highlights radiation information; detailed texture information; clearly, completely and accurately describes the target.

[0030] A ladle hot state detection system adopts a three-layer architecture. The physical layer is supported by computing equipment, electrical systems, mechanical auxiliary systems, and communication systems; the intermediate layer is software middleware that provides functional support in the system, including images, communications, algorithms, third-party libraries, and drivers; the application layer directly provides user-oriented functions, and is mainly composed of various detection systems and interactive interfaces.

[0031] Preferably, infrared imaging detection equipment, infrared imaging detection equipment, optical imaging detection equipment, optical imaging detection lens, computing server and computing card

[0032] The 360° visual coverage of the outer wall of the ladle is to collect the outer wall status of the ladle to determine the degree of wear of the insulation bricks inside the ladle, whether they have fallen off and the location of the falling off. Four infrared imaging sensors need to be arranged around the ladle at 90 degrees to each other.

[0033] In order to observe and detect the insulation bricks in the ladle in real time, 4 optical detection devices and 4 optical lenses are set up;

[0034] In order to detect the situation at the bottom of the ladle, an infrared camera needs to be set up at the bottom.

[0035] Preferably, a detection position is set up at the ladle hanging ear, which can effectively detect the ladle status of the process with high fault incidence and reduce the risk of accidents;

[0036] The thermal protection platform provides overheat protection for all sensors on site, ensuring that the equipment operates within the range of 0-50℃ and has a dust removal function.

[0037] A computer-readable storage medium stores a computer program, which is executed by a processor to implement a ladle hot state detection method.

[0038] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements a ladle hot state detection method when executing the computer program.

[0039] The present invention has the following beneficial effects:

[0040] Compared with the prior art, the present invention has the following advantages:

[0041] The present invention detects the state of the ladle shell in real time: at the refining ladle dropping position, the shell of the ladle is detected in real time at 360 degrees, and real-time detection and analysis are completed for abnormal states, and an alarm is issued to the operator.

[0042] The present invention detects the bottom of the ladle in real time: performs real-time detection on the outer side of the bottom of the ladle at the position of the ladle turner, completes real-time detection and analysis of abnormal conditions of the bottom of the ladle, and alarms operators.

[0043] The present invention performs real-time detection on the ladle lining: after the cover is opened at the refining position, a surround-view camera is used to complete real-time photography, detection and analysis of the lining and the bottom of the ladle, and real-time detection and analysis of abnormal conditions of the lining are completed, and an alarm is issued to the operator.

[0044] The present invention can detect the ventilation status of the ladle in real time: after the cover is opened at the refining position, the ventilation brick part at the bottom of the ladle and the surrounding of the ventilation bricks are detected in real time, the ventilation depth can be determined, and the current use status of the ventilation bricks can be given. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0046] Figure 1 for image processing of the present invention;

[0047] Figure 2 It is the system architecture design diagram;

[0048] Figure 3 is a detection schematic diagram;

[0049] Figure 4 It is a flow chart of implementation steps. DETAILED DESCRIPTION

[0050] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] The present invention is described in detail below in conjunction with specific embodiments. Specific embodiment one:

[0053] according to Figure 1 As shown, the specific optimized technical solution adopted by the present invention to solve the above technical problems is: the present invention relates to a hot state detection method of a ladle.

[0054] A method for detecting hot state of a ladle, the method comprising:

[0055] The ladle hot state inspection area is arranged below the platform behind the converter. 3-4 distributedly installed high-definition and infrared thermal imagers are used to take photos of the ladle lining, ladle bottom, air bricks and nozzle from different angles, visually identify and combine them into a complete ladle lining model. By comparing with the initial ladle shape and the self-learning ability in the later operation process, the sample library is gradually improved to form a thermal inspection system that can accurately identify various defects. In combination with the human-computer interaction interface, the status of the ladle hot inspection process is tracked, a defect alarm mechanism is established, and feedback on the abnormal status of the ladle is realized. Specific embodiment 2:

[0057] The difference between the second embodiment of the present invention and the first embodiment is that:

[0058] The visible light image is converted into a color space from an RGB image to an HSV image; the brightness information of the color image is extracted as the input for image fusion, retaining its original hue and saturation:

[0059] Create filter kernel template: construct the corresponding filter kernel template according to the image size of infrared and visible light;

[0060] Construct image pyramid using filter kernel template convolution;

[0061] Use linear interpolation method to extract details at different scales of the pyramid;

[0062] The saliency of the details is used to discriminate the infrared and visible light details of each layer of the pyramid. The designed sliding window is used to convolve the image to generate a weight matrix. The neighborhood information of each pixel is compared with the weight matrix to extract a more credible detail map.

[0063] The smallest scale infrared and visible light images are linearly added to obtain a smooth background image. The extracted detail image is then fused to the background image, and the image is super-resolution upscaled. The detail information of the upper scale is then added, and the process is repeated until the top of the pyramid is reached.

[0064] Color space conversion: convert the fused image back to RGB image and add the hue and saturation that were previously preserved;

[0065] Color enhancement: Perform color enhancement on the fused image to generate a fused image with better clarity and contrast. Perform pixel-level image enhancement based on the contrast of each pixel. Specific embodiment three:

[0067] The difference between the third embodiment of the present invention and the second embodiment is that:

[0068] Multiple binocular stereo cameras and infrared cameras are used to achieve blind-angle observation of the inner and outer walls of the ladle; the field of view of the single vision module is not less than 45°; each single vision module ensures the coupling of the field of view angle, and uses multi-target positioning and optical flow calculation methods to ensure the accurate acquisition of full-view 3D multimodal information. Specific embodiment four:

[0070] The difference between the fourth embodiment of the present invention and the third embodiment is that:

[0071] Acquire registered infrared and visible light images;

[0072] Perform multi-scale pyramid transformation on the image, and use the designed filter template to perform down-convolution and down-sampling on the infrared image and visible light image respectively;

[0073] Perform detail extraction of infrared and visible light images based on multi-scale pyramid transform, and use the high frequency of the image obtained by linear interpolation method as the fused detail layer;

[0074] Image color space conversion: To address the problem that visible light images are three-channel RGB, we convert RGB color space to HSV color space, extract the V information of the visible light image and fuse it with the infrared image, retaining its original H and S; extract the V channel as the input of visible light, retain the H and S channels to the corresponding matrix, and retain the color information for the subsequent color restoration after fusion;

[0075] Use filter convolution details to filter infrared and visible light detail images, and fuse the infrared and visible light detail images according to the weights of the generated matrix to generate a detail image with richer texture.

[0076] Image super-resolution reconstruction technology is used to perform multi-scale pyramid inverse transform. Specific embodiment five:

[0078] The difference between the fifth embodiment of the present invention and the fourth embodiment is that:

[0079] Use visual information encoding and decoding networks for training: dual-scale decomposition of low-frequency and high-frequency information: infrared and visible light real-time fusion algorithm based on multi-scale pyramid transformation, construct image pyramid by designing filter templates, obtain image information at different scales, perform image super-resolution and significant enhancement, and finally achieve real-time through GPU acceleration;

[0080] Infrared images have strong penetration and are not affected by light, while visible light has fine texture information and high spatial resolution. The complementary characteristics are combined to produce high-quality images.

[0081] Fusion image: highlights radiation information; detailed texture information; clearly, completely and accurately describes the target. Specific embodiment six:

[0083] The difference between the sixth embodiment of the present invention and the fifth embodiment is that:

[0084] The present invention provides a ladle hot state detection system, which adopts a three-layer architecture. The physical layer is supported by computing equipment, electrical systems, mechanical auxiliary systems, and communication systems; the intermediate layer is software middleware that provides functional support in the system, including images, communications, algorithms, third-party libraries, and drivers; the application layer directly provides user-oriented functions, and is mainly composed of various detection systems and interactive interfaces. Specific embodiment seven:

[0086] The difference between the seventh embodiment of the present invention and the sixth embodiment is that:

[0087] The 360° visual coverage of the outer wall of the ladle is to collect the outer wall status of the ladle to determine the degree of wear of the insulation bricks inside the ladle, whether they have fallen off and the location of the falling off. Four infrared imaging sensors need to be arranged around the ladle at 90 degrees to each other.

[0088] In order to observe and detect the insulation bricks in the ladle in real time, 4 optical detection devices and 4 optical lenses are set up;

[0089] In order to detect the situation at the bottom of the ladle, an infrared camera needs to be set up at the bottom. Specific embodiment eight:

[0091] The difference between the eighth embodiment of the present invention and the seventh embodiment is that:

[0092] Setting up a detection position at the ladle hanging ear can effectively detect the status of the ladle in the process with high fault incidence and reduce the risk of accidents;

[0093] The thermal protection platform provides overheat protection for all sensors on site, ensuring that the equipment operates within the range of 0-50℃ and has a dust removal function. Specific embodiment nine:

[0095] The difference between the ninth embodiment of the present invention and the eighth embodiment is that:

[0096] The present invention provides a computer-readable storage medium on which a computer program is stored. The program is executed by a processor to implement a method for detecting hot state of a steel ladle. Specific embodiment ten:

[0098] The difference between the tenth embodiment of the present invention and the ninth embodiment is that:

[0099] The present invention provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements a ladle hot state detection method when executing the computer program. Specific implementation method eleven:

[0101] according to Figure 2-Figure 4 It is shown that the present invention uses in-package visual and infrared detection technology to detect brick falling, peeling, cracking, local erosion, etc., and issues warnings. Manual judgment is made and entered into the system to achieve database update and iteration, and an alarm is issued when the warning value is reached.

[0102] The ladle hot state inspection area is arranged below the platform behind the converter. 3-4 distributed high-definition and infrared thermal imagers are used to take photos of the ladle lining, ladle bottom, air bricks, nozzles and other parts from different angles, visually identify and combine them into a complete ladle lining model. By comparing with the initial ladle model and self-learning ability in the later operation process, the sample library is gradually improved to form a thermal inspection system that can accurately identify various defects. Combined with the human-computer interaction interface, the status of the ladle hot inspection process is tracked, a defect alarm mechanism is established, and abnormal status feedback of the ladle is realized.

[0103] Fusion of infrared and visible light

[0104] (1) Convert the visible light image into a color space from an RGB image to an HSV image; extract the brightness information of the color image as the input for image fusion, retaining its original hue and saturation.

[0105] (2) Create filter kernel template: Construct the corresponding filter kernel template according to the image size of infrared and visible light;

[0106] (3) Use filter kernel template convolution to construct an image pyramid.

[0107] (4) Use the linear interpolation method to extract details at different scales of the pyramid.

[0108] (5) The saliency of the details is used to discriminate the infrared and visible light details of each layer of the pyramid. The designed sliding window is used to convolve the image to generate a weight matrix. The neighborhood information of each pixel is compared with the weight matrix to extract a more credible detail map.

[0109] (6) The smallest-scale infrared and visible light images are linearly added to obtain a smooth background image. The extracted detail image is then fused with the background image, and the image is super-resolution upscaled. The detail information of the upper scale is then added, and the process is iterated until the top of the pyramid is reached.

[0110] (7) Color space conversion: Convert the fused image back to an RGB image and add the previously preserved hue and saturation.

[0111] (8) Color enhancement: Enhance the color of the fused image to generate a fused image with better clarity and contrast. Perform pixel-level image enhancement based on the contrast of each pixel.

[0112] It is planned to use multiple binocular stereo cameras and infrared cameras to achieve blind-angle observation of the inner and outer walls of the ladle; the field of view of the single vision module is not less than 45°; each single vision module ensures the coupling of the field of view, and uses multi-target positioning and optical flow calculation methods to ensure the accurate acquisition of full-view 3D multimodal information. To ensure real-time performance, NVIDIA computing card groups are used for parallel computing. On this basis, a method for fusion of infrared and visible light is provided to achieve accurate acquisition of observation features. The method comprises the following steps:

[0113] (1) Obtain registered infrared and visible light images.

[0114] (2) Perform a multi-scale pyramid transform on the image and use the designed filter template to down-convolve and down-sample the infrared image and visible light image respectively.

[0115] (3) Detail extraction of infrared and visible light images is performed based on multi-scale pyramid transform, and the high frequency of the image obtained by linear interpolation method is used as the fused detail layer.

[0116] (4) Image color space conversion: To address the problem that visible light images are composed of three RGB channels, we convert the RGB color space to the HSV color space, extract the V (brightness) information of the visible light image and fuse it with the infrared image, retaining its original H (hue) and S (saturation); extract the V channel as the input of the visible light, retain the H channel and S channel to the corresponding matrix, and retain the color information for the subsequent color restoration after fusion.

[0117] (5) Use filter convolution details to filter the infrared and visible light detail images, and fuse the detail images of the infrared and visible light images according to the weights of the generated matrix to generate a detail image with richer texture.

[0118] (6) Use image super-resolution reconstruction technology to perform multiscale pyramid inverse transform (Multiscale contrast enhancement for radiographies: Laplacian pyramid versus fast wavelet transform).

[0119] Ladle Inspection Training Baseline Method

[0120] It mainly uses visual information encoding and decoding networks for training: dual-scale decomposition of low-frequency (background) and high-frequency information (details): real-time fusion algorithm of infrared and visible light based on multi-scale pyramid transformation, constructing image pyramid by designing filter templates, obtaining image information at different scales, and performing image super-resolution and significant enhancement, and finally achieving real-time through GPU acceleration.

[0121] (1) Infrared images have strong penetration and are not affected by light, while visible light has fine texture information and high spatial resolution. The complementary characteristics are fused to produce high-quality images.

[0122] (2) Fusion image: highlighting radiation information (brightness and contrast); detailed texture information (gradient and edge); clearly, completely and accurately describing the target.

[0123] Feasibility analysis and verification of ladle inspection scheme

[0124] According to the existing ladle image data, it is observed and analyzed that there are two problems when testing the ladle lining:

[0125] (1) The refractory bricks of the ladle are falling off, missing, worn or cracked.

[0126] (2) Residue at the ladle mouth.

[0127] Existing visual inspection technologies can be used for defective images of ladle linings, such as using visual recognition, infrared temperature measurement and other technologies to conduct a comprehensive analysis of the ladle lining temperature, which can promptly detect brick drop, peeling, wear and tear, and crack points for early warning. However, this technical solution for ladle lining inspection also has many defects. For example, infrared temperature measurement of steel residues inside the ladle may cause visual judgment errors, and judge that refractory bricks are missing or broken. In order to solve the defects of the above-mentioned detection schemes, this scheme has proposed the following solutions and conducted quantitative testing and analysis of their experimental data.

[0128] Through observation of the above images and analysis of traditional detection methods, this program proposed the following method to inspect and test the ladle lining.

[0129] System architecture design

[0130] The system design adopts a three-layer architecture. The physical layer supports computing equipment, electrical systems, mechanical auxiliary systems, communication systems, etc.; the middle layer is the software middleware that provides functional support in the system, including images, communications, algorithms, third-party libraries, drivers, etc.; the application layer directly provides user-oriented functions, which is mainly composed of various detection systems and interactive interfaces.

[0131] Hardware system architecture (physical architecture)

[0132] (1) Optical equipment selects focal length, fixed focus lens, and global shutter according to on-site requirements;

[0133] (2) The acquisition equipment has an Ethernet interface;

[0134] (3) The acquisition device can provide the working status of the device;

[0135] (4) The image resolution generated by the thermal imager is 640 x 480, and the acquisition frame rate is greater than 5fps;

[0136] (5) The target temperature captured by the thermal imager is greater than 1600°C and the temperature resolution is less than 50°C;

[0137] (6) The optical camera resolution is greater than 300M and the acquisition frame rate is greater than 15fps.

[0138] Without affecting the lifting and dropping of the ladle, 360° visual coverage of the outer wall of the ladle is required. The purpose is to collect the outer wall status of the ladle to determine the degree of wear of the insulation bricks inside the ladle, whether they have fallen off and the location of the falling off. Four infrared imaging sensors of model HITHMAIC-SLHDS-TOD-1 need to be arranged around the ladle, and the arrangement angles are approximately 90 degrees to each other. In order to observe in real time and detect the insulation bricks inside the ladle, it is necessary to set up four HITHMAIC-SLHDS-OD-201 optical detection equipment and match them with four optical lenses of model HITHMAIC-SLHDS-OD-101.

[0139] In order to detect the condition of the bottom of the ladle, a HITHMAIC-SLHDS-TOD-2 infrared camera needs to be set up at the bottom. Setting up a detection position at the ladle hanging ear can effectively detect the status of the ladle in the process with high fault incidence and reduce the risk of accidents.

[0140] Thermal protection platform: The thermal protection platform provides overheat protection for all sensors on site, ensuring that the equipment works within the range of 0-50℃ and has a dust removal function.

[0141] (1) Dealing with dust conditions on site;

[0142] (2) Ensure the normal operation of electronic equipment in thermal radiation scenarios;

[0143] (3) Ensure the normal operation of electronic equipment under ambient temperature scenarios;

[0144] (4) Consider the heat dissipation of electronic equipment during operation;

[0145] (5) The temperature inside the protective cover should be below 50°C;

[0146] (6) The replacement cycle of dust prevention equipment is greater than 2 months.

[0147] Data transmission (Ethernet): Data transmission uses a combination of optical fiber and network cables to ensure data reliability and real-time transmission rate over long distances.

[0148] (1) Long-distance transmission is possible;

[0149] (2) Shielding electromagnetic signals;

[0150] (3) Heat-resistant wire;

[0151] (4) Consider backup lines;

[0152] (5) The total bandwidth is greater than 1.5 times the bandwidth required by the acquisition equipment.

[0153] Computing and storage platform: The computing and storage platform mainly performs real-time detection, processing and analysis of the collected data at each detection point, and can display and store the calculation results.

[0154] (1) Perform multi-threaded data collection and data processing;

[0155] (2) Graphics computing card;

[0156] (3) Data storage and export interface;

[0157] (4) a separate data storage disk;

[0158] (5) Image acquisition card;

[0159] (6) Data storage disk capacity;

[0160] Feedback device: On-site feedback equipment can provide operators with abnormal prompts and alarms in noisy environments, and automatically eliminate the alarm after the operator confirms it.

[0161] (1) Sound and light indicate serious problems;

[0162] (2) Able to work normally in noisy environments on site.

[0163] Fixing mechanism: Due to the complex on-site environment, all sensor equipment needs to be fixed in angle and position. The fixing mechanism can provide the required position of the sensor and ensure safe and reliable use.

[0164] (1) Fixed image acquisition equipment;

[0165] (2) Fixed transmission lines;

[0166] (3) Fixed cooling equipment;

[0167] (4) Acquisition equipment pan / tilt.

[0168] According to the project objectives and requirements, based on the acquisition of sample images, the project team conducted tests and verifications on multispectral image analysis and processing, and completed preliminary feature extraction and classification experiments based on different spectral responses and local features. The designed features can be used to locate and extract designated high-temperature object areas.

[0169] Defect judgment module: (see Appendix 7: camera intrinsic parameter acquisition, network framework, target detection).

[0170] (1) Defect determination is completed based on the ladle data passing through the inspection area;

[0171] (2) Identify and locate side wall refractory brick defects.

[0172] (3) Identify and locate bottom refractory brick defects;

[0173] (4) Estimate the depth of air bricks;

[0174] (5) Single image processing time is less than 200ms;

[0175] (6) The accuracy of refractory brick drop determination is greater than 99.5%.

[0176] Defect determination method: Through artificial intelligence learning, normal data and abnormal data are labeled to detect abnormal areas. For the small sample ladle defect detection scenario, the project team intends to use machine learning and neural network methods for supervised learning, and detect abnormal conditions through pixel-level feature classification. Because its principle is to divide the area through semantic analysis, and use manual labeling to complete the segmentation and abnormality identification of different areas.

[0177] In order to effectively complete the detection of ladle shell, the following network results are designed to extract and classify the features of ladle shell.

[0178] According to the characteristics of the image, an attention network is designed to optimize feature extraction. The attention mechanism can generally be divided into three steps: first, information input; second, calculating the attention distribution; and third, processing the input information according to the calculated attention distribution.

[0179] make:

[0180] x∈R^d is the input vector

[0181] X=[x_1,x_2,…,x_N] is N input samples

[0182] q∈R^k is the query vector and feature vector

[0183] z∈[1,N] is the attention variable, which indicates the position of the selected information. For example, z=i indicates that the i-th input vector is selected.

[0184] Given q and X, the probability of selecting the i-th input vector α_i is:

[0185]

[0186] in:

[0187] α_i is called the attention distribution

[0188] s(xi,q) is the attention scoring function

[0189] Based on the image region of interest, the watershed algorithm is proposed to be further segmented.

[0190] For the ladle shell defect detection parameters, the accuracy of defect detection and user credibility are further improved by calculating the global features and local features and completing the analysis of fusion features.

[0191] During the project, preliminary data collection of the ladle shell was completed and manual annotation was carried out.

[0192] The above-mentioned AI-based learning method was used to train and verify the data set, and the results are as follows.

[0193] Based on the above training model, the ladle shell is tested, which can fully capture the key areas of concern in the ladle shell image and extract the regional range for further feature analysis. At present, adaptive learning is carried out on the existing data to complete pixel clustering, which can effectively observe the pixel features of abnormal areas. For the extracted ladle shell area of ​​interest, the features are calculated independently and in parallel, and the feature distribution pattern of the specific area of ​​the ladle shell can be obtained. After on-site debugging, through the production line process guidance and the auxiliary calibration of the operator, different types of anomalies can be classified and labeled for more detailed classification.

[0194] Usually, when the gradient information of the normal ladle outer wall infrared image is analyzed, a stable and evenly distributed gradient histogram will be obtained, and the distribution trend of the histogram has a fixed regular trend. Therefore, this paper analyzes and statistics the gradient histogram of the ladle outer wall image area collected on site. When the ladle surface is damaged or the ladle lining is abnormal, the highlighted part can be seen in the local area of ​​the ladle outer wall. By analyzing the gradient histogram trend of the abnormal area and putting it into the network for gradient feature extraction and calculation, and then comparing the similarity with the gradient information of the normal ladle outer wall, the abnormal situation of the changed area is finally obtained.

[0195] Spatial position calibration of the inner wall of the ladle: Since the ladle is a rigid body with a fixed shape, a fixed camera is used to calibrate the running object to obtain the relationship between the running speed and position. During the inspection process, the relative position of the defect in the ladle can be calculated by tracking the trajectory of the same defect in the image. Calibration tool: a marker with a variable running speed.

[0196] (1) The marker is placed on a plane at a vertical distance of h1 so that it runs at the same speed as the ladle. Each camera detects it at the same time and marks the correspondence between the timestamp and the position.

[0197] (2) Place the marker on a plane with a vertical distance of h2, perform the same operation, and record the second set of data

[0198] Repeat the above steps several times.

[0199] (3) Obtain the relationship between running speed and image displacement at different vertical distances.

[0200] (4) Since the camera viewing angle is a perspective relationship, the defect trajectory of the ladle is brought into the running speed-running distance model, and the spatial position of the defect can be calculated.

[0201] Defect detection: First use feature extraction algorithms such as SIFT to extract the features of the ladle lining image, and use the kd-tree model to calculate the Euclidean distance between the feature points of the two images to match the feature points, so as to find an image pair with the required number of feature point matches. For each image matching pair, calculate the epipolar geometry, estimate the F matrix, and optimize and improve the matching pair through the ransac algorithm. If there are feature points that can be passed down in a chain in such a matching pair and are always detected, then a trajectory can be formed. Then enter the motion positioning part. The key first step is to select a good image pair to initialize the entire BA process. First, perform the first BA on the two initialization selected ladle lining pictures, and then add new pictures in a loop for new BA, and finally until there are no suitable pictures to continue to add, BA ends. Obtain the camera estimation parameters and scene geometry information, that is, the geometry information of the ladle lining. The specific steps are as follows:

[0202] First, select a suitable initialization ladle lining image. Then use the homography model between the two image transformations to find the initialization ladle lining image pair. If it does not fit the homography model well, it means that the camera center is still a certain distance away. The RANSAC method is also used to reduce noise and improve the reliability of matching. Try to select a low percentage of inliers, but at least ensure 100 matching inliers.

[0203] This system uses the 5-point method to estimate the external parameters of the initialized matching pair, and then the trajectory triangulation can provide the initialized 3D points. The initialized two frames of pictures can start the first bundle adjustment. The sparse bundle adjustment (SBA) method is used here. Bundle Adjustment is an iterative process. After one iteration, all 3D points are back-projected to the pixel coordinates of each photo and compared with the initial coordinates. If it is greater than a certain threshold, it should be removed from the track. If there are less than 2 points in the track, the entire track will be removed, and the optimization will continue until there are no points to remove.

[0204] Finally, new cameras and 3D points are added continuously for BA. After obtaining the scene geometry information and inputting it into the multi-eye system, the names of each defect and their location coordinates are obtained. The results are as follows:

[0205] For the depth detection of air bricks, the following two algorithms are proposed to provide support:

[0206] (1) Multi-view visual measurement and positioning;

[0207] (2) Fixed-position parallax measurement.

[0208] Multi-eye vision measurement mainly uses multi-camera joint calibration to obtain the spatial coordinate information within the detectable area, and then extracts features from the breathable surface to locate the feature spatial coordinates. This spatial coordinate can represent the depth information of the breathable brick.

[0209] Fixed-camera parallax measurement: The surface area of ​​the breathable bricks changes continuously at different depths. When the breathable bricks are at the highest height, the appearance time, position, and size of the breathable bricks at the same position in the image sequence under a fixed viewing angle are all different. Through real-time synchronous shooting of four cameras, each camera can independently measure the depth of the breathable bricks. After evaluating the measurement results of multiple cameras, the depth of the breathable bricks can be accurately estimated.

[0210] Lining erosion trend analysis: The analysis of lining erosion can be measured in several stages. First, the surface characteristics of the lining are analyzed. When the surface of the ladle lining is flat, the characteristics of the inner wall are relatively uniform. When the lining gradually begins to erode, the surface characteristics will be uneven. The system will record this unevenness to form a coefficient for storage. Secondly, the system can perform statistical analysis on the statistical information of the ladle shell temperature and the lining surface, and derive a relative coefficient to represent the ladle erosion degree. After completing the communication with the computer secondary system, the curve relationship between the ladle number information and the ladle erosion degree can be obtained, and the trend analysis of the ladle erosion degree can be further obtained.

[0211] On-site installation design:

[0212] (1) Ladle bottom inspection station: In order to detect the condition of the ladle bottom, an infrared camera needs to be installed at the bottom of the ladle turner for real-time detection. The No. 2 ladle turner was selected for visual installation. An infrared camera device needs to be installed at the bottom of the No. 2 ladle turner to observe the condition of the ladle bottom and monitor from top to bottom.

[0213] There is an inclined support near the top of the bottom of the bag turning machine. When the bag turning machine lies flat, the center line of the bottom of the bag turning machine intersects with the upper inclined support at one point. From this point, a one-meter I-beam is led downward. The I-beam is perpendicular to the ground, and the flat surface of the I-beam faces the bottom of the ladle.

[0214] The installation location requires air cooling, power supply, and optical fiber. The specific requirements are as follows:

[0215] The No. 2 bag turning machine is convenient for installing an electrical cabinet to house the power supply, terminal box, photoelectric converter and switch, etc. The electrical cabinet and the ground are connected with expansion screws.

[0216] (2) Ladle side wall inspection station: In order to detect the wear of the outer wall of the ladle without affecting the ladle lifting and dropping work, it is necessary to install 4 infrared cameras around the ladle at the refining ladle dropping position, also installed through camera protective covers and brackets.

[0217] The installation location requires air cooling, power supply, and optical fiber. Installation locations A and B are a group, and C and D are a group. The specific requirements are as follows:

[0218] Two electrical cabinets are installed at the refining and dropping positions, close to positions A, B and C, D, respectively, to house power supplies, terminal boxes, photoelectric converters, switches, etc. Expansion screws are used to connect the electrical cabinets to the ground.

[0219] (3) Ladle lining inspection station: In order to inspect the condition of the ladle lining insulation bricks, considering the actual construction difficulty and its high temperature, four cameras are installed on the second-floor platform of the converter to perform lining defect detection.

[0220] (4) Control room

[0221] Add a display, the display positions are as follows:

[0222] Logic control module: The logic control module is the system control middleware. It does not directly provide services to end users. As an important component of the system's automatic operation, its main contents are the interfaces and logic control units of each software module in the system.

[0223] (1) Read configuration parameters; (2) Transfer configuration parameters; (3) Transfer processing quantities.

[0224] Human-computer interaction interface: As a direct medium for user operations, the human-computer interaction interface provides display, viewing and other functions, and can provide real-time sound and light alarms based on the type and location of abnormalities.

[0225] (1) Display real-time processing quantity; (2) Display real-time captured data; (3) Prompt detection results; (4) Abnormal detection result alarm, sound and light alarm; (5) Manual result intervention; (6) Generate detection log; (7) Equipment abnormality alarm; (8) Equipment maintenance reminder.

[0226] Data storage module: The function of the storage system is to save and view the test results. The stored content is the test result data, result description, and test image for secondary verification.

[0227] (1) Store captured data; (2) Store test results; (3) Store test logs; (4) Store data for no less than one year.

[0228] The ladle is an important container for transporting molten steel and performing secondary metallurgical reactions during the converter, refining and continuous casting processes. During use, the ladle brick lining is in contact with high-temperature molten steel and slag for a long time. In addition, during the steel tapping and refining outside the furnace, the erosion caused by the flow of molten steel and metallurgical reactions will be more serious. The erosion of the ladle brick lining will not only reduce the service life of the ladle, but also increase the content of inclusions in the molten steel. In addition, since the temperature of molten steel is higher than that of molten iron and it has good fluidity, the accident of ladle leakage will be more serious after bricks fall off, cracks, etc., and there are personnel, equipment, and machinery crossing under the molten steel lifting route, which poses a great safety hazard.

[0229] Through visual recognition technology, the hot state inspection of the steel ladle is changed from relying on experience and manual work to an intelligent, digital and information-based management model using machine vision, which can timely and accurately detect safety hazards such as brick falling, peeling, brick damage, etc., and reduce production and operation costs and safety costs. System function list:

[0230] (1) Detection of falling and damage of refractory bricks on the side wall of the ladle; (2) Detection of falling and damage of refractory bricks on the bottom of the ladle; (3) Depth estimation of the air-permeable bricks on the bottom of the ladle; (4) The system operates stably in a high-temperature environment; (5) Human-computer interaction interface displays the detection status and can intervene in the results; (6) Alarm for serious defects; (7) Description of the ladle status and data storage; (8) Maintenance and overhaul after project implementation must take safety and operability into consideration.

[0231] Functional mode design: Automatic mode: The system runs automatically and completes accurate and rapid ladle health status detection based on data models and measurement data. Expert mode: Experts intervene through the system human-machine interaction interface, provide online guidance to the machine, and complete continuous optimization.

[0232] Process design: Process description: Development and application of ladle hot state detection management system The system is started by the ladle in place signal, and collects data through spectral imaging equipment, optical imaging equipment and laser equipment in turn, and calculates various ladle health status guarantee values ​​in the background, reconstructs the ladle space data and health data, and finally completes the interface display and recording.

[0233] Since the system detects at multiple detection positions simultaneously, the detection process is aimed at:

[0234] Bag turner position: The bag turner position process starts and the ladle is in place at the bag turner. After the ladle is in place, the computer secondary system sends the bag turner ladle position signal and the ladle number to the ladle hot state detection system. After receiving the signal, the system displays and records the ladle number and starts detection. The system detects abnormal conditions of the outer shell at the bottom of the ladle and marks the key areas at the bottom of the ladle (air permeability, steel outlet). At the same time, after detecting abnormalities, sound and light alarms will be issued and abnormal areas will be marked.

[0235] Position of the second-floor platform of the converter: After the ladle is placed at the converter position, the secondary computer system needs to send the positioning signal and the ladle number to the ladle thermal management system. Since the movement speed of the ladle is relatively fixed, the detection process is started by time triggering after the ladle is uncovered through a fixed time delay. During the detection process, the abnormal situation of the ladle lining will be detected, and the depth of the air permeable part at the bottom of the ladle will be judged, and the results will be stored and displayed. When an abnormal problem occurs, an audible and visual alarm will be issued.

[0236] Refining ladle drop position: During the refining ladle drop process, there is a brief pause of 3-5 seconds when the bottom of the ladle is 5-8 meters from the ground. The computer secondary system sends a refining ladle drop position signal to the ladle hot state detection system, together with the ladle number. The ladle hot state detection system displays the ladle number and performs simultaneous detection at four angles. After the pause time, the ladle hot state detection system will complete the detection and display and record the results in the interface. If any abnormality occurs, the detection system will sound an audible and visual alarm.

[0237] The user interface adopts a design method that combines real-time image display, real-time data display and manual interaction interface, which includes the following parts:

[0238] (1) 3D model: abnormal points of the ladle can be marked in the 3D model; (2) data information: real-time display of ladle temperature information; (3) image information: real-time display of ladle bottom and side image information, and real-time detection image information of the lining status; (4) manual correction interface: the ladle detection results can be corrected by manual marking; (5) ladle status information: visual display of the ladle lining status: defect status, loss status, etc. The ladle lining model can be viewed through operation and compared with the detection results; (6) historical detection data: the historical data of ladle detection can be traced back in the form of visual information. The traceback data includes detection values, detection time, detection results and other information; (7) the system will mark the ladle number corresponding to the detection position and detection time, which is conducive to the staff to grasp the status tracking of a single ladle.

[0239] Power supply system: The power supply system includes the power supply for the visual module and the upper computer console. The industrial visual camera is powered by a 12V DC power supply; the thermal imager is powered by a 24V DC power supply; the upper computer is powered by a 220V AC power supply and UPS power supply protection.

[0240] Network system: The network system uses TCP / IP communication to complete data transmission and control signal transmission. The system network communication is mainly connected to the system host computer by sensor signal acquisition. Since the acquisition site is far away from the host computer, a Category 6 shielded network cable will be used to lead out from the sensor, connected to the electric-to-optical module, and connected to the optical-to-electrical module after long-distance optical fiber transmission. After the network cable is connected, it is connected to the switch and then transferred to the video capture card to connect to the host computer.

[0241] Server and host computer system: The server is used to store a large amount of production process data and can be managed through the host computer. The host computer is used to complete the human-machine interface control and database operations. The host computer system provides the following functions:

[0242] (1) Display interface; (2) Human-computer operation interface; (3) Data operation interface; (4) The server provides the following functions: (5) Store data; (6) Modify data; (7) Export data.

[0243] Storage, backup and archiving system: Every time the program is run, a log will be generated and stored and backed up.

[0244] The system will store raw data, process data and result data according to demand. Since different types of data require different memory spaces, the system can adaptively change the storage strategy for different types of data. For example, if the memory space of raw data is large, it needs to be exported regularly.

[0245] The process data and test data of hot ladle testing are important production materials and system output information respectively, and the data collection and management system design will be completed in various ways. Test labels such as time, status, ladle number and other characteristic information will be used as design elements for data storage; important information content such as images, defects, logs, etc. will be designed as the main units of data storage. The system will reserve the data and command communication interface of the computer secondary system to enhance the interactive performance of the system, build the basic capabilities of data interaction, and avoid the formation of information islands. The system supports user backup interface, and batch data can be exported and backed up through operation.

[0246] Interface design: The hardware interface supports display interface, network cable interface, USB interface, etc. The business function structure includes real-time data display interface, which supports real-time digital display of detection process, ladle health status monitoring, ladle space status information, etc.; human-computer interaction interface, which can view real-time data, drill down data, and correct data through the software interface; data interface, which can operate, store, modify, export, and backup the stored data.

[0247] Design of communication interface between ladle hot state detection system and computer secondary system:

[0248] (1) Communication method. The ladle hot state detection system will communicate with the computer secondary system through TCP / IP. The IP is fixed and the communication port needs to be reserved by the computer secondary system. The ladle hot state detection system is the socket client and the computer secondary system is the server. The server has the function of reconnecting after disconnection.

[0249] The system will adopt a three-channel communication mode, channel one sends heartbeat data, channel two sends command data, and channel three sends business data.

[0250] (2) Communication content. Heartbeat data: System status: After successful connection, the ladle thermal state detection system will send a heartbeat packet to the computer secondary system, indicating the online status of the ladle thermal state detection system. Command data: In-position signal: After the ladle is in position at the three detection points, the computer secondary system sends a ladle in-position signal to the ladle thermal state detection system. Business data: Ladle number: After the computer secondary system is in position, it sends serial data of the ladle number to the ladle thermal state detection system. Other business data: According to business needs, business data can be sent through a mutually confirmed coding structure.

[0251] (3) Communication format. Heartbeat: Data that pulses 0 / 1 according to a mutually confirmed coding format, with 1 bit of system status attached. On-location signal: 1 bit of on-location information is sent according to a mutually confirmed coding format, with 2 bits of location information attached. Ladle number: 1 byte of ladle number length is sent according to a mutually confirmed coding format, with N bytes of data indicating the ladle number attached, and no check bit.

[0252] Security design: The strictest and most violent security solution is to prevent intrusion by disconnecting the networks. In our production practice, this is mainly reflected in several aspects: machines that do not provide public services do not allocate external IP addresses; office networks and production networks cannot communicate directly with each other; gateway design.

[0253] User permission design: The user permission module is mainly used to control user access to the interface, background function menu, interface resources, and the permission management function for roles, menus, and resources. The management background permission management function includes role management, menu management, resource management, and assigning roles to users, assigning menus to roles, and assigning resources to roles. The specific functions are as follows.

[0254] A role is a collection of resource or menu permissions. Different administrators are assigned different resource and menu permissions through roles. Users with the same permissions can access the same menus and resources. This can be understood as permission grouping. By assigning roles to users, user access rights control can be ultimately achieved.

[0255] The above is only a preferred implementation of a method for detecting the hot state of a steel ladle. The protection scope of a method for detecting the hot state of a steel ladle is not limited to the above embodiment. All technical solutions under this idea belong to the protection scope of the present invention. It should be pointed out that for those skilled in the art, several improvements and changes without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A method for detecting hot state of a ladle, characterized in that: The method comprises: The ladle hot state inspection area is arranged below the platform behind the converter. 3-4 distributedly installed high-definition and infrared thermal imagers are used to take photos of the ladle lining, ladle bottom, air bricks and nozzle from different angles, visually identify and combine them into a complete ladle lining model. By comparing with the initial ladle shape and the self-learning ability in the later operation process, the sample library is gradually improved to form a thermal inspection system that can accurately identify various defects. In combination with the human-computer interaction interface, the status of the ladle hot inspection process is tracked, a defect alarm mechanism is established, and feedback on the abnormal status of the ladle is realized.

2. The method according to claim 1, characterized in that: The visible light image is converted into a color space from an RGB image to an HSV image; the brightness information of the color image is extracted as the input for image fusion, retaining its original hue and saturation: Create filter kernel template: construct the corresponding filter kernel template according to the image size of infrared and visible light; Construct image pyramid using filter kernel template convolution; Use linear interpolation method to extract details at different scales of the pyramid; The saliency of the details is used to discriminate the infrared and visible light details of each layer of the pyramid. The designed sliding window is used to convolve the image to generate a weight matrix. The neighborhood information of each pixel is compared with the weight matrix to extract a more credible detail map. The smallest scale infrared and visible light images are linearly added to obtain a smooth background image. The extracted detail image is then fused to the background image, and the image is super-resolution upscaled. The detail information of the upper scale is then added, and the process is repeated until the top of the pyramid is reached. Color space conversion: convert the fused image back to RGB image and add the hue and saturation that were previously preserved; Color enhancement: Perform color enhancement on the fused image to generate a fused image with better clarity and contrast. Perform pixel-level image enhancement based on the contrast of each pixel.

3. The method according to claim 2, characterized in that: Multiple binocular stereo cameras and infrared cameras are used to achieve blind-angle observation of the inner and outer walls of the ladle; the field of view of the single vision module is not less than 45°; each single vision module ensures the coupling of the field of view angle, and uses multi-target positioning and optical flow calculation methods to ensure the accurate acquisition of full-view 3D multimodal information.

4. The method according to claim 3, characterized in that: Acquire registered infrared and visible light images; Perform multi-scale pyramid transformation on the image, and use the designed filter template to perform down-convolution and down-sampling on the infrared image and visible light image respectively; Perform detail extraction of infrared and visible light images based on multi-scale pyramid transform, and use the high frequency of the image obtained by linear interpolation method as the fused detail layer; Image color space conversion: To address the problem that visible light images are three-channel RGB, we convert RGB color space to HSV color space, extract the V information of the visible light image and fuse it with the infrared image, retaining its original H and S; extract the V channel as the input of visible light, retain the H and S channels to the corresponding matrix, and retain the color information for the subsequent color restoration after fusion; Use filter convolution details to filter infrared and visible light detail images, and fuse the infrared and visible light detail images according to the weights of the generated matrix to generate a detail image with richer texture. Image super-resolution reconstruction technology is used to perform multi-scale pyramid inverse transform.

5. The method according to claim 4, characterized in that: Use visual information encoding and decoding networks for training: dual-scale decomposition of low-frequency and high-frequency information: infrared and visible light real-time fusion algorithm based on multi-scale pyramid transformation, construct image pyramid by designing filter templates, obtain image information at different scales, perform image super-resolution and significant enhancement, and finally achieve real-time through GPU acceleration; Infrared images have strong penetration and are not affected by light, while visible light has fine texture information and high spatial resolution. The complementary characteristics are combined to produce high-quality images. Fusion image: highlight radiation information; detailed texture information; Describe your goals clearly, completely, and accurately.

6. A ladle hot state detection system, characterized by: The system adopts a three-layer architecture. The physical layer supports computing devices, electrical systems, mechanical auxiliary systems, and communication systems. The intermediate layer is software middleware that provides functional support in the system, including images, communications, algorithms, third-party libraries, and drivers. The application layer directly provides user-oriented functions, which is mainly composed of various detection systems and interactive interfaces.

7. The system according to claim 6, characterized in that: Infrared imaging detection equipment, infrared imaging detection equipment, optical imaging detection equipment, optical imaging detection lens, computing server and computing card have 360° visual coverage of the outer wall of the ladle. The purpose is to collect the outer wall status of the ladle to determine the degree of wear of the insulation bricks inside the ladle, whether they have fallen off and the location of the falling off. Four infrared imaging sensors need to be arranged around the ladle, and the arrangement angles are 90 degrees to each other. In order to observe and detect the insulation bricks in the ladle in real time, 4 optical detection devices and 4 optical lenses are set up; In order to detect the situation at the bottom of the ladle, an infrared camera needs to be set up at the bottom.

8. The system according to claim 7, characterized in that: Setting up a detection position at the ladle hanging ear can effectively detect the status of the ladle in the process with high fault incidence and reduce the risk of accidents; The thermal protection platform provides overheat protection for all sensors on site, ensuring that the equipment operates within the range of 0-50℃ and has a dust removal function.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the method according to claims 1 to 5.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the method of claims 1-5 is implemented.

Images