Method and device for judging insufficient sintering ignition intensity based on machine vision

Abstract

The present invention discloses a method and device for judging insufficient sintering ignition intensity based on machine vision. The method includes: training a semantic segmentation model to segment and obtain the material surface area of the sintering trolley; performing perspective transformation to obtain a top-down view of the material surface image; obtaining a template for the yellow mud area and calculating the histogram; traversing the image and comparing the histogram with the template, and combining the morphological judgment conditions of the yellow mud to determine whether there is yellow mud, so as to determine whether there is insufficient sintering ignition intensity. Yellow mud is a common defect in the sintering process of the iron and steel industry, which reflects the insufficient strength of the corresponding surface sintered cake caused by the low ignition temperature of the igniter. The effective detection of yellow mud can be used as a judgment criterion for whether there is insufficient sintering ignition intensity. At present, the research on the intelligent detection of yellow mud on the sintering material surface is still blank. The present invention has good effects and practical value for judging insufficient sintering ignition intensity by detecting yellow mud.

Description

Technical Field

[0001] The present invention belongs to the field of industrial process monitoring, perception and detection, and relates to a method and device for judging insufficient sintering ignition intensity based on machine vision, which is an application of machine vision in industrial production. Background Art

[0002] The iron and steel industry is a pillar industry of the national economy and has made outstanding contributions to the growth of China's gross national product. Blast furnace ironmaking, as the main method of modern ironmaking, is an important link in steel production. In the steel smelting process, sinter is one of the important raw materials for blast furnace ironmaking. Iron ore sintering is a pretreatment process that provides sinter for the blast furnace and is an indispensable key step in the ironmaking production process. Sintering ignition is to use high-temperature flue gas generated by the combustion of blast furnace gas or coke oven gas in the ignition furnace to ignite the fuel in the sintering mixture, forming an initial combustion zone. At the same time, under the action of suction, the combustion zone burns downward. Ignition is the beginning of the combustion process in sintering production. Insufficient ignition intensity will lead to insufficient strength of the sinter and excessive return fines; excessive ignition intensity will weaken the air permeability of the material surface, reduce productivity, and increase energy consumption. Therefore, the ignition situation directly affects the quality of sinter, the yield, and the ignition energy consumption. When the ignition intensity of the igniter is insufficient, it will lead to insufficient strength of the corresponding surface sinter cake, resulting in a defect of yellowing on the surface of the material surface, which is called "yellow mud". Effective detection of yellow mud can be used as a judgment standard for whether there is insufficient sintering ignition intensity.

[0003] Currently, for the yellow mud defect on the sintering pallet surface, the defect detection still relies on the visual observation of the on-duty workers. This method is inefficient and labor-consuming. In the existing technical research, the automated detection means for yellow mud is still blank. Summary of the Invention

[0004] The purpose of the present invention is to provide a method and device for judging insufficient sintering ignition intensity based on machine vision in view of the blank in the research of the existing related fields. This method can intelligently process the monitoring images of the sintering plant and effectively and real-time judge whether there is insufficient sintering ignition intensity.

[0005] To achieve the above purpose, the present invention adopts the following technical solutions: A method for judging insufficient sintering ignition intensity based on machine vision, the method includes the following steps:

[0006] S1, train a semantic segmentation model, input the monitoring image of the sintering plant, and output the area of the sintering pallet surface to be monitored in the image;

[0007] S2, extract the four corners of the sintering pallet surface area, perform perspective transformation, and obtain an overhead view image of the sintering pallet surface;

[0008] S3. Take an image of an area known as yellow mud as a template, calculate the HSV histogram of the template and normalize it;

[0009] S4. Traverse each corresponding area of the template and the sintering pallet surface image from the top-down perspective at a specific step size, perform histogram correlation comparison, and based on the correlation calculation result, combined with the morphological judgment conditions of yellow mud, determine whether there is yellow mud, so as to determine whether there is a situation of insufficient sintering ignition intensity.

[0010] Further, step S1 specifically includes the following sub-steps:

[0011] S1.1. Obtain the monitoring screen of the sintering plant, save several historical pictures from the monitoring video, and use the annotation tool to annotate the sintering pallet surface area in the pictures as the training set;

[0012] S1.2. Based on the DeepLabv3+ semantic segmentation model, take the monitoring screen picture as the input and the picture with the annotated sintering pallet surface area as the ground truth value of the model output, and train the weight parameters of the DeepLabv3+ semantic segmentation model;

[0013] S1.3. Take the weight scheme with the minimum loss value as the model weight parameters and import them into the DeepLabv3+ semantic segmentation model to obtain the trained semantic segmentation model. Input the monitoring screen of the sintering plant and output the estimation result of the sintering pallet surface area, so as to achieve the purpose of segmenting the sintering pallet surface area from the monitoring screen.

[0014] Further, step S2 specifically includes the following sub-steps:

[0015] S2.1. Represent the obtained sintering pallet surface area and the background area of the non-sintering pallet surface with white and black respectively to obtain a temporary picture;

[0016] S2.2. Extract the contours of the temporary picture, approximate the boundary of the largest contour, that is, the surface area, as a contour with four vertices, obtain the coordinates of the approximate four vertices of the surface area, and sort them in ascending order of their x-direction coordinates;

[0017] S2.3. Map the four sorted vertices on the original monitoring screen to the lower left corner, upper left corner, upper right corner, and lower right corner of the picture in turn, and perform perspective transformation on the picture to obtain the sintering pallet surface image from the top-down perspective.

[0018] Further, step S3 specifically includes the following steps:

[0019] Take an image containing the yellow mud area from a top-down perspective as a template, convert the original image of the template into an image in the HSV color space, and then calculate and standardize the histograms for each of the three channels respectively; the color histogram refers to the color distribution in an image, which has nothing to do with specific objects in the image and is only used to represent the color distribution in the image observed by human eyes; when standardizing, linear normalization is adopted to scale the original histogram to a specified range, and the formula is as follows:

[0020]

[0021] where dst(p) is the standardized histogram, src(p) is the original histogram, α is the lower limit of the standardized value, and β is the upper limit of the standardized value.

[0022] Furthermore, step S4 specifically includes the following sub-steps:

[0023] S4.1, establish a sliding window on the sintering pallet surface image from a top-down perspective with the same size as the template area, and traverse the entire surface area with a certain step size;

[0024] S4.2, during the sliding process, calculate and standardize the HSV histogram of the small surface area covered by the sliding window, and calculate the correlation coefficient between it and the standardized HSV histogram of the template. The formula for the correlation coefficient is as follows:

[0025]

[0026] where, is the label of the histogram, and H k (I) represents the pixel value at pixel point I of the standardized HSV histogram H k , and N is the total number of corresponding image pixel points, I = 1, 2,..., N;

[0027] If the correlation coefficient d(H 1 , H 2 ) is greater than the pre-set correlation coefficient threshold γ, then preliminarily record and mark this area in the image, otherwise no operation is performed;

[0028] S4.3, define the morphological judgment conditions for yellow mud. If the sliding windows with a correlation coefficient greater than the correlation coefficient threshold continuously exist in the vertical direction and the number is greater than the set value, that is, the number n of continuously marked areas in the vertical direction is greater than or equal to the pre-set sliding window number threshold μ, then it is judged that there is yellow mud here and there is a situation of insufficient sintering ignition intensity, otherwise it is judged that there is no yellow mud here and there is no situation of insufficient sintering ignition intensity.

[0029] The present invention also provides a device for judging insufficient sintering ignition intensity based on machine vision, which includes a memory and one or more processors. Executable code is stored in the memory. When the processor executes the executable code, it is used to implement the above-mentioned method for judging insufficient sintering ignition intensity based on machine vision.

[0030] The present invention also provides a computer-readable storage medium, on which a program is stored. When the program is executed by a processor, it implements the above-mentioned method for judging insufficient sintering ignition intensity based on machine vision.

[0031] The beneficial effects of the present invention are as follows: The present invention solves the problem of the lack of existing research on the automated judgment technology for insufficient sintering ignition intensity in the scenario of iron ore sintering. Combining with the monitoring equipment deployed in the sintering plant by cooperative iron and steel enterprises, on the basis of obtaining the sintering burden surface image information, the method of machine vision is used to segment the sintering burden surface and detect yellow mud, showing excellent detection accuracy. At the same time, it breaks through the limitations of manual visual recognition, liberates human resources, and ensures the feasibility, real-time and continuity of detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 is the flowchart of the method for judging insufficient sintering ignition intensity based on machine vision in the embodiment of the present invention.

[0033] Figure 2 is the result of obtaining the sintering pallet burden surface area through the semantic segmentation model in the embodiment of the present invention.

[0034] Figure 3 is the temporary picture representing the burden surface and the background area in white and black respectively in the embodiment of the present invention.

[0035] Figure 4 is the burden surface image from a top-down perspective in the embodiment of the present invention.

[0036] Figure 5 is the monitoring image after marking the possible yellow mud areas in the embodiment of the present invention.

[0037] Figure 6 is the structure diagram of the device for judging insufficient sintering ignition intensity based on machine vision in the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0038] The following further elaborates on the present invention in conjunction with the accompanying drawings and specific embodiments.

[0039] Figure 1The overall process of the method for judging insufficient sintering ignition intensity based on machine vision in the embodiments of the present invention is given. First, a semantic segmentation model is trained. By inputting the monitoring images of the sintering plant, the area of the sintering pallet surface to be monitored in the image is output. Then, the four corners of the area of the sintering pallet surface are extracted, and perspective transformation is performed to obtain an image of the sintering pallet surface from a top-down perspective. An image of a known yellow mud area is used as a template, and the HSV histogram of the template is calculated and normalized. According to a specific step size, each corresponding area between the template and the image of the sintering pallet surface from a top-down perspective is traversed, and histogram correlation comparison is performed. Finally, based on the correlation calculation result and combined with the morphological judgment conditions of yellow mud, it is judged whether there is yellow mud, so as to judge whether there is a situation of insufficient sintering ignition intensity.

[0040] For the method for judging insufficient sintering ignition intensity based on machine vision in the embodiments of the present invention, each step is specifically described as follows:

[0041] (1) Train the DeepLabv3+ semantic segmentation model for extracting the area of the sintering pallet surface. The specific steps include:

[0042] (1.1) Obtain the monitoring images of the sintering plant, and save hundreds of historical images with yellow mud from the monitoring video; use a labeling tool to label the area of the sintering pallet surface in the images, and the unlabeled part is the background area. The labeled images are regarded as the training set;

[0043] (1.2) Based on the DeepLabv3+ semantic segmentation model, use the monitoring image as the input, and the image with the labeled area of the sintering pallet surface as the true value output by the model; set the parameter information of the model, such as the number of iterative training times, the categories of objects to be classified, the number of frozen parameter layers, etc. After setting the above parameters, start training the weight parameters of the model;

[0044] (1.3) During the training process, save the weight parameters used in each iteration in a separate weight file. After the training is completed, select the weight file with the smallest loss value as the weight parameters of the model to obtain a trained semantic segmentation model; input the monitoring image into the model, and output the estimation result of the area of the sintering pallet surface as Figure 2 shown.

[0045] (2) Obtain an image of the sintering pallet surface from a top-down perspective. The specific steps include:

[0046] (2.1) Represent the obtained area of the sintering pallet surface and the background area of the non-sintering pallet surface with white and black respectively to obtain a temporary image as Figure 3 shown. This is to avoid the interference of other contours at the boundary of the non-pallet surface area in the original image when extracting contours in step (2.2);

[0047] (2.2) Extract the contour of the temporary image, select the largest contour, which is the boundary of the material surface area, approximate it as a contour with four vertices, obtain the coordinates of the approximate four vertices of the material surface area, and sort them in ascending order of their x-direction coordinates; (2.3) Map the sorted four vertices on the original monitoring screen to the lower left corner, upper left corner, upper right corner, and lower right corner of the image in turn, project the image to a new perspective for perspective transformation, and obtain the material surface image from the top-down perspective as Figure 4 shown; among them, the perspective transformation formula is as follows:

[0048]

[0049] Among them, (x′, y′, z′) T is the coordinate after perspective transformation, (x, y, z) T is the coordinate before perspective transformation, and a ij (i = 1, 2, 3; j = 1, 2, 3) are the elements in the transformation matrix.

[0050] (3) Take an image of a known yellow mud area as a template, calculate and standardize the HSV histogram of the template. The specific steps include:

[0051] Take an image containing the yellow mud area from the top-down perspective as a template, convert the original image of the template into an image in the HSV color space, and then calculate and standardize the histograms of the three channels respectively; the color histogram refers to the color distribution in an image, which has nothing to do with specific objects in the image, but is only used to represent the color distribution in the image observed by the human eye; when standardizing, linear normalization is adopted to scale the original histogram to a specified range. The formula is as follows:

[0052]

[0053] Among them, dst(p) is the standardized histogram, src(p) is the original histogram, α is the lower limit of the standardized value, and β is the upper limit of the standardized value; in this embodiment, the lower limit α of the histogram after standardization is 0, and the upper limit β is 1.

[0054] (4) According to the comparison of the HSV histogram correlation and combined with the morphological judgment conditions of the yellow mud, judge whether there is yellow mud. The specific steps include:

[0055] (4.1) Establish a sliding window with the same size as the template area on the material surface image from the top-down perspective, and traverse the entire material surface area with a certain step size;

[0056] (4.2) During the sliding process, calculate the HSV histogram of the small-scale material surface area covered by the sliding window and standardize it, and calculate the correlation coefficient with the standardized HSV histogram of the template. The correlation coefficient calculation formula is as follows:

[0057]

[0058] Among them, is the label of the histogram, and H k (I) represents the pixel value at pixel point I of the standardized HSV histogram H k , N is the total number of corresponding image pixel points, and I = 1, 2,..., N;

[0059] If the correlation coefficient d(H 1 , H 2 ) is greater than or equal to the pre-set correlation coefficient threshold γ. In this embodiment, γ = 0.5 is set, then preliminarily record and mark, as shown in Figure 5 , and the formula is expressed as follows:

[0060]

[0061] (4.3) Define the morphological judgment conditions of yellow mud. If the sliding windows with a correlation coefficient greater than or equal to the correlation coefficient threshold continuously exist in the vertical direction and the number is greater than or equal to the set value, that is, the number n of continuously marked areas in the vertical direction is greater than or equal to the pre-set sliding window number threshold μ, then it conforms to the reason for the appearance of yellow mud and the most likely form, and it is judged that there is yellow mud here; in this embodiment, μ = 6 is set. Figure 5 If there is a situation where the number of continuously marked sliding windows in the vertical direction in

[0062] is greater than or equal to, it indicates that yellow mud is detected through the detection, and there is a situation of insufficient sintering ignition strength.

[0063] In this embodiment, the monitoring video data of a sintering plant of a certain iron and steel enterprise in China in a certain month is used. 700 pictures are selected as the training set, and 100 pictures are selected as the test set. A semantic segmentation model based on DeepLabv3+ is trained to segment the sintering trolley material surface area.

[0064]

[0065]

[0066] Further analyze the detection results obtained by the present invention (unit: piece):

[0067]

[0068] Using precision and recall as evaluation metrics, the calculation results of the evaluation metrics are as follows:

[0069]

[0070]

[0071] From the evaluation metrics, it can be seen that for the detection of yellow mud in the present invention, the precision is 100.0% and the recall is 94.8%. It can better detect yellow mud in the monitoring images and judge that there is a situation of insufficient sintering ignition intensity, intuitively proving the effectiveness of the method of the present invention.

[0072] Corresponding to the foregoing embodiments of the method for judging insufficient sintering ignition intensity based on machine vision, the present invention also provides an embodiment of a device for judging insufficient sintering ignition intensity based on machine vision.

[0073] See Figure 6 , an embodiment of a device for judging insufficient sintering ignition intensity based on machine vision provided by the embodiments of the present invention includes a memory and one or more processors. An executable code is stored in the memory. When the processor executes the executable code, it is used to implement the method for judging insufficient sintering ignition intensity based on machine vision in the above embodiments.

[0074] The embodiments of the device for judging insufficient sintering ignition intensity based on machine vision of the present invention can be applied to any device with data processing capabilities. The any device with data processing capabilities can be a device or apparatus such as a computer. The device embodiments can be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a logically meaningful device, it is formed by the processor of any device with data processing capabilities where it is located reading the corresponding computer program instructions in the non-volatile memory into the memory for operation. From the hardware level, as Figure 6 shown, it is a hardware structure diagram of any device with data processing capabilities where the device for judging insufficient sintering ignition intensity based on machine vision of the present invention is located. In addition to Figure 6 the processor, memory, network interface, and non-volatile memory shown, usually according to the actual functions of the any device with data processing capabilities where the device in the embodiments is located, other hardware may also be included, which will not be elaborated here.

[0075] The implementation processes of the functions and roles of each unit in the above device are specifically detailed in the implementation processes of the corresponding steps in the above method, which will not be elaborated here.

[0076] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the descriptions in the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of the present invention. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0077] An embodiment of the present invention also provides a computer-readable storage medium, on which a program is stored. When the program is executed by a processor, the method for judging insufficient sintering ignition intensity based on machine vision in the above embodiments is implemented.

[0078] The computer-readable storage medium may be an internal storage unit of any device with data processing capabilities described in any of the foregoing embodiments, such as a hard disk or memory. The computer-readable storage medium may also be an external storage device of any device with data processing capabilities, such as a plug-in hard disk, a Smart Media Card (SMC), an SD card, a Flash Card, etc. equipped on the device. Further, the computer-readable storage medium may also include both an internal storage unit and an external storage device of any device with data processing capabilities. The computer-readable storage medium is used to store the computer program and other programs and data required by any device with data processing capabilities, and can also be used to temporarily store the data that has been output or will be output.

[0079] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention, and they should all be covered by the scope of the claims of the present invention.

[0080] The specific embodiments described above have detailed the technical solutions and beneficial effects of the present invention. It should be understood that the above is only the most preferred embodiment of the present invention and is not used to limit the present invention. Any modifications, supplements, equivalent replacements, etc. made within the scope of the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for judging insufficient sintering ignition intensity based on machine vision, characterized in that, it includes the following steps: S1. Train a semantic segmentation model, input the monitoring screen of the sintering plant, and output the area of the sintering pallet surface to be monitored in the screen; specifically including the following sub-steps: S1.

1. Obtain the monitoring screen of the sintering plant, save several historical pictures from the monitoring video, and use a labeling tool to label the area of the sintering pallet surface in the pictures as the training set; S1.

2. Based on the DeepLabv3+ semantic segmentation model, use the picture of the monitoring screen as the input, and the picture with the labeled sintering pallet surface area as the true value output by the model to train the weight parameters of the DeepLabv3+ semantic segmentation model; S1.

3. Import the weight scheme with the minimum loss value as the model weight parameters into the DeepLabv3+ semantic segmentation model to obtain a trained semantic segmentation model, input the monitoring screen of the sintering plant, and output the estimation result of the sintering pallet surface area; S2. Extract the four corners of the sintering pallet surface area, perform perspective transformation, and obtain the sintering pallet surface image from the top-down view; S3. Take an image of a known yellow mud area as a template, calculate the HSV histogram of the template and standardize it; S4. Traverse each corresponding area of the template and the sintering pallet surface image from the top-down view at a specific step size, perform histogram correlation comparison, and based on the correlation calculation result, combined with the morphological judgment conditions of yellow mud, judge whether there is yellow mud, so as to judge whether there is insufficient sintering ignition intensity; specifically including the following sub-steps: S4.

1. Establish a sliding window on the sintering pallet surface image from the top-down view with the same size as the template area, and traverse the entire surface area with a certain step size; S4.

2. During the sliding process, calculate the HSV histogram of the small surface area covered by the sliding window and standardize it, and calculate the correlation coefficient between it and the standardized HSV histogram of the template. The correlation coefficient calculation formula is as follows: Among them, k = 1, 2 are the labels of the histogram, and H k (I) represents the pixel value at pixel point I of the normalized HSV histogram H k , N is the total number of corresponding image pixel points, and I = 1, 2,..., N; If the correlation coefficient d(H 1 ,H 2 ) is greater than a pre-set correlation coefficient threshold γ, initially record and mark this area in the image, otherwise do nothing; S4.

3. Define the morphological judgment conditions of yellow mud. If the sliding windows with a correlation coefficient greater than the correlation coefficient threshold continuously exist in the vertical direction and the number is greater than the set value, that is, the number n of continuously marked areas in the vertical direction is greater than the pre-set sliding window number threshold μ, then it is judged that there is yellow mud here and there is insufficient sintering ignition intensity, otherwise it is judged that there is no yellow mud here and there is no insufficient sintering ignition intensity.

2. The method according to claim 1, characterized in that, step S2 specifically includes the following sub-steps: S2.

1. Represent the obtained sintering pallet surface area and the background area of the non-sintering pallet surface with white and black respectively to obtain a temporary picture; S2.

2. Extract the contour of the temporary picture, approximate the boundary of the largest contour, that is, the surface area, as a contour with four vertices, obtain the coordinates of the approximate four vertices of the surface area, and sort them in ascending order of their x-direction coordinates; S2.

3. Map the four sorted vertices on the original monitoring screen to the lower left corner, upper left corner, upper right corner, and lower right corner of the picture in turn, and perform perspective transformation on the picture to obtain the sintering pallet surface image from the top-down view.

3. The method according to claim 1, wherein, step S3 specifically includes the following steps: Take an image containing the yellow mud area in a top-down view as a template, convert the original image of the template into an image in the HSV color space, and then calculate and standardize the histograms of the three channels respectively; when standardizing, adopt linear normalization to scale the original histogram to a specified range, and the formula is as follows: where dst(p) is the standardized histogram, src(p) is the original histogram, α is the lower limit of the value after standardization, and β is the upper limit of the value after standardization.

4. A device for judging insufficient sintering ignition intensity based on machine vision, comprising a memory and one or more processors, and executable code is stored in the memory, wherein, when the processor executes the executable code, it is used to implement the method for judging insufficient sintering ignition intensity based on machine vision according to any one of claims 1-3.

5. A computer-readable storage medium, on which a program is stored, wherein, when the program is executed by the processor, it implements the method for judging insufficient sintering ignition intensity based on machine vision according to any one of claims 1-3.

Images