Steel flow landing point detection method and device based on deep learning, equipment and medium

Abstract

This invention provides a deep learning-based method, apparatus, device, and storage medium for detecting the landing point of a steel flow. The method includes acquiring historical and current images of a target steel converter; training a steel flow landing point detection model based on the historical images; inputting the current image into the trained model to obtain the detection result for the current image; extracting steel flow block images from the current image based on the label information as the coordinates of the steel flow; stitching together the extracted steel flow block images to obtain an initial steel flow image; extracting the steel flow contour from the initial image; obtaining the target abscissa of the steel flow based on the contour; and determining the landing point position of the steel flow based on the abscissa. By constructing a recognition model using deep learning on historical images, the method accurately identifies the landing point of the steel flow in the image, improving the accuracy of steel flow landing point recognition and providing effective data for subsequent processing.

Description

Technical Field

[0001] This application relates to the field of steel smelting technology, specifically to a method, apparatus, equipment, and medium for detecting the landing point of steel flow based on deep learning. Background Technology

[0002] In the converter steelmaking process, ensuring the precise pouring of molten steel into the ladle car below the furnace is an extremely critical operation. Improper handling can lead to a series of problems, such as molten steel leakage, equipment damage, reduced production efficiency, and even safety accidents. Traditional operating methods rely on workers wearing protective goggles near the converter to visually control the position of the ladle car to ensure that the molten steel flows precisely into the correct location.

[0003] However, this manual monitoring method demands a high level of experience and skill from workers and has many limitations. First, manual monitoring is susceptible to fatigue and negligence; prolonged work can lead to eye strain, affecting operational accuracy. Furthermore, this method requires significant manpower and time, reducing production efficiency. Therefore, traditional methods can no longer meet the demands of modern industrial production and require more advanced and intelligent technologies to replace them. Summary of the Invention

[0004] In view of the shortcomings of the prior art described above, the present invention provides a method, device, equipment and medium for detecting the landing point of steel flow based on deep learning, so as to solve the technical problem that the accuracy of detection results is low due to the influence of human subjectivity in the above-mentioned traditional methods.

[0005] This invention provides a deep learning-based method for detecting the landing point of a steel flow. The method includes: acquiring historical images of a target steel converter and a current image of the target steel converter; training a steel flow landing point detection model based on the historical images, and inputting the current image into the trained steel flow landing point detection model to obtain a detection result for the current image, the detection result including coordinate information and label information, the label information including at least the steel flow; extracting steel flow block images from the current image based on all coordinates of the steel flow as labeled, and stitching together all extracted steel flow block images to obtain an initial steel flow image; extracting the steel flow contour from the initial steel flow image, and obtaining the target abscissa of the steel flow based on the steel flow contour, and obtaining the landing point position of the steel flow based on the abscissa of the steel flow.

[0006] In one embodiment of the present invention, after obtaining the abscissa of the steel flow based on the steel flow contour, the method further includes: acquiring an initial ladle image, extracting the ladle contour of the initial ladle image, and obtaining the center coordinates of the ladle based on the ladle contour; comparing the abscissa of the steel flow and the center coordinates of the ladle, and determining the offset state of the steel flow based on the comparison result, so as to adjust the position of the ladle based on the offset state; wherein, the step of acquiring the initial ladle image includes: extracting the ladle block image in the current image based on the label information as the coordinates of the ladle, and stitching together all the extracted ladle block images to obtain the initial ladle image, wherein the label information also includes the ladle.

[0007] In one embodiment of the present invention, training a steel flow landing point detection model based on the historical image includes: determining a ladle region and a steel flow region in the historical image, wherein the ladle region is used to characterize the image of the ladle in the historical image, and the steel flow region is used to characterize the image of the steel flow in the historical image; labeling the ladle region and the steel flow region respectively based on preset image labels, wherein the preset image labels include ladle and steel flow; generating a training sample dataset based on the labeled historical image, and training an initial image recognition model based on the training sample dataset to obtain a trained steel flow landing point detection model.

[0008] In one embodiment of the present invention, extracting the steel flow contour of the initial steel flow image includes: extracting all contours in the initial steel flow image; expanding each pixel of each steel flow image block in the initial steel flow image outward to connect adjacent steel flow image blocks; shrinking each pixel in the expanded steel flow image inward to obtain the target steel flow direction, wherein adjacent steel flow blocks in the target steel flow direction are interconnected; extracting the contour of the target steel flow image, and determining the contour of the target steel flow image as the steel flow contour of the initial steel flow image.

[0009] In one embodiment of the present invention, after extracting all contours from the initial steel flow image, the method further includes: obtaining any complete contour information and determining the image region corresponding to the complete contour as a steel flow image block; calculating the image area of ​​the steel flow image block and comparing the calculated block area with a preset standard area; if the image area of ​​any steel flow image block is less than the preset standard area, then the steel flow image block corresponding to the image area is determined as an invalid block; if the image area of ​​any steel flow image block is greater than or equal to the preset standard area, then the steel flow image block corresponding to the image area is determined as a valid block, so as to obtain a target steel flow image based on the valid blocks.

[0010] In one embodiment of the present invention, obtaining the target abscissa of the steel flow based on the steel flow profile includes: collecting the abscissa information of all coordinate points in the steel flow profile; calculating the mean of all the abscissa information to obtain the abscissa mean, and determining the abscissa mean as the target abscissa of the steel flow.

[0011] In one embodiment of the present invention, comparing the abscissa of the steel flow with the center coordinate of the ladle, and determining the offset state of the steel flow based on the comparison result, includes: determining half of the long side of the smallest bounding rectangle of the ladle outline as the ladle radius, and determining the difference between the abscissa of the steel flow and the center coordinate of the ladle as the lateral deviation; calculating the deviation ratio between the lateral deviation and the ladle radius; when the deviation ratio is less than or equal to a preset deviation threshold, the offset state of the steel flow is considered to be a minor deviation, and no adjustment of the ladle position is required; when the deviation ratio is greater than the preset deviation threshold, the offset state of the steel flow is considered to be a severe deviation; if the deviation ratio is positive, the landing point of the steel flow is determined to be rightward relative to the ladle; if the deviation ratio is negative, the landing point of the steel flow is determined to be leftward relative to the ladle.

[0012] This invention provides a deep learning-based steel flow landing point detection device. The device includes: a data acquisition module for acquiring historical images of a target steel converter and a current image of the target steel converter; a deep learning module for training a steel flow landing point detection model based on the historical images and inputting the current image into the trained steel flow landing point detection model to obtain a detection result for the current image, the detection result including coordinate information and label information, the label information including at least steel flow; an image processing module for extracting steel flow block images from the current image based on all coordinates of steel flow as the label information, and stitching together all extracted steel flow block images to obtain an initial steel flow image; and a steel flow landing point determination module for extracting the steel flow contour of the steel flow image, obtaining the abscissa of the steel flow based on the steel flow contour, and obtaining the landing point position of the steel flow based on the abscissa of the steel flow.

[0013] The present invention provides an electronic device comprising: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the electronic device enables the steel flow landing point detection method based on deep learning as described above.

[0014] The present invention provides a computer-readable storage medium storing a computer program thereon, which, when executed by a computer processor, causes the computer to perform the deep learning-based steel flow landing point detection method described above.

[0015] The beneficial effects of this invention are as follows: This invention provides a deep learning-based method, apparatus, device, and storage medium for detecting the landing point of a steel flow. The method includes acquiring historical and current images of a target steel converter; training a steel flow landing point detection model based on the historical images; inputting the current image into the trained steel flow landing point detection model to obtain the detection result of the current image; extracting steel flow block images from the current image based on the label information as the coordinates of the steel flow; stitching together all extracted steel flow block images to obtain an initial steel flow image; extracting the steel flow contour from the initial steel flow image; obtaining the target abscissa of the steel flow based on the steel flow contour; and obtaining the landing point position of the steel flow based on the abscissa of the steel flow. By constructing a recognition model through deep learning of historical images, the method accurately identifies the landing point of the steel flow in the image, improving the accuracy of steel flow landing point recognition and providing effective data for subsequent processing.

[0016] Furthermore, the deep learning-based steel flow landing point detection method proposed in this application reduces manual intervention in the steel flow landing point detection process, improves production efficiency and safety, and further improves the traditional converter tapping process while ensuring the smooth flow of steel flow into the ladle car, making the production process more efficient and safer.

[0017] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:

[0019] Figure 1 This is a schematic diagram illustrating the implementation environment of a deep learning-based steel flow landing point detection method, as shown in an exemplary embodiment of this application.

[0020] Figure 2 This is a flowchart illustrating a deep learning-based steel flow point detection method as an exemplary embodiment of this application;

[0021] Figure 3 This is a schematic diagram illustrating the steps of a deep learning-based steel flow landing point detection method, as shown in an exemplary embodiment of this application.

[0022] Figure 4 This is a schematic diagram of any original image of the target steel converter shown in an exemplary embodiment of this application;

[0023] Figure 5 This is a schematic diagram of an annotated image of a target steel converter shown in an exemplary embodiment of this application;

[0024] Figure 6 This is a schematic diagram of an image after model segmentation and post-processing, illustrating an exemplary embodiment of this application;

[0025] Figure 7 This is an example diagram illustrating the offset of the steel flow landing point, as shown in an exemplary embodiment of this application;

[0026] Figure 8 This is an example diagram illustrating the offset of the steel flow landing point, as shown in another exemplary embodiment of this application;

[0027] Figure 9 This is a block diagram illustrating a deep learning-based steel flow point detection device, as shown in an exemplary embodiment of this application.

[0028] Figure 10 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application is shown. Detailed Implementation

[0029] The embodiments of the present invention will be described below with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention and not for limiting the scope of protection of the present invention.

[0030] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0031] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the invention. However, it will be apparent to those skilled in the art that embodiments of the invention may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the invention.

[0032] First, it's important to clarify that steel flow refers to the process of molten steel flowing from its source (such as an electric arc furnace or converter) and being formed through continuous casting or ingot casting. A ladle is a container used to hold and transport molten steel, typically made of refractory materials, capable of withstanding high temperatures and the impact of molten steel. The coordination between the steel flow and the ladle is a crucial step in the steelmaking process, involving the entire process from tapping to casting into ingots or billets. Precise control of the coordination between the steel flow and the ladle is essential to ensure stable and efficient production.

[0033] Figure 1 This is a schematic diagram illustrating the implementation environment of a deep learning-based steel flow landing point detection method, as shown in an exemplary embodiment of this application.

[0034] like Figure 1 As shown, the real-time environment of this deep learning-based steel flow landing point detection method includes a data acquisition device 101 and a computer device 102. The data acquisition device 101 is used to acquire historical and current images of the steel flow and ladle during the steel converter process, and sends the acquired image information to the computer device 102. The data acquisition device can be any device capable of capturing image information, such as an optical camera or thermal imager; this application does not impose any restrictions on this. The computer device 102 is used to perform deep learning based on the historical images acquired by the data acquisition device 101 to identify the steel flow and ladle in the current image, thereby obtaining the landing point position of the steel flow and the position information of the ladle in the current image. Based on the landing point position and the position information of the ladle, it determines the offset state of the steel flow and adjusts the position of the ladle based on this offset state to ensure that the steel flow accurately flows into the correct location. The computer device 102 can be at least one of a desktop graphics processing unit (GPU) computer, a GPU computing cluster, a neural network computer, etc., or it can be an intelligent processor integrated into the current vehicle; this application also does not impose any restrictions on this.

[0035] Figure 2 This is a flowchart illustrating a deep learning-based steel flow point detection method as an exemplary embodiment of this application.

[0036] like Figure 2 As shown, in an exemplary embodiment, the deep learning-based steel flow landing point detection method includes at least steps S210 to S240, which are described in detail below:

[0037] Step S210: Obtain historical images of the target steel converter and the current image of the target steel converter.

[0038] In one embodiment of this application, a high-resolution camera and sensor are preferably used to acquire images of the target steel converter. These images may include high-definition pictures of key parts such as the converter's appearance, furnace mouth, and tapping spout. The acquired images are then stored in a reliable storage device, such as a hard drive or cloud storage, ensuring the storage device has good stability and scalability to store a large amount of historical image data. Furthermore, the acquired images can be preprocessed, such as denoising, enhancement, and contrast adjustment, to improve image quality and recognizability. It should be noted that the image acquisition device and storage device mentioned in the above embodiments are merely illustrative examples. In practical applications, the image acquisition method, image storage device, and image preprocessing content can be adaptively adjusted based on actual needs, and this application does not impose any limitations on them.

[0039] Step S220: Train the steel flow landing point detection model based on historical images, and input the current image into the trained steel flow landing point detection model to obtain the detection result of the current image. The detection result includes coordinate information and label information, and the label information includes at least the steel flow.

[0040] In one embodiment of this application, training a steel flow landing point detection model based on historical images includes: determining a ladle region and a steel flow region in the historical images, wherein the ladle region represents the image of the ladle in the historical images, and the steel flow region represents the image of the steel flow in the historical images; labeling the ladle region and the steel flow region respectively based on preset image labels, wherein the preset image labels include ladle and steel flow; generating a training sample dataset based on the labeled historical images, and training an initial image recognition model based on the training sample dataset to obtain a trained steel flow landing point detection model.

[0041] In one specific embodiment of this application, a historical image dataset of the target steel converter is first obtained from a storage device. These historical images should include converter images at different time points and under different operating conditions. In the historical images, the ladle region and the steel flow region are identified through techniques such as threshold segmentation, edge detection, and morphological processing. The ladle region is used to characterize the position and shape of the ladle in the image, while the steel flow region is used to characterize the flow trajectory and state of the steel flow in the image. Then, based on preset ladle and steel flow image labels, the ladle region and the steel flow region are labeled. The labeling process can be performed using image labeling tools or manual labeling to ensure the accuracy and consistency of the labeling. Next, a training sample dataset is generated based on the labeled historical images. Each sample should include a historical image and the corresponding labeling information, namely the labels of the ladle region and the steel flow region. Deep learning techniques, such as convolutional neural networks (CNN), are used to train the model to identify and detect the ladle and steel flow regions in the historical images. Furthermore, during training, the model's parameters and structure can be continuously optimized iteratively to improve its accuracy and robustness. Different optimization algorithms and techniques, such as gradient descent, regularization, and ensemble learning, can also be used to improve model performance. Alternatively, independent test datasets can be used to test and evaluate the trained model, and further adjustments and optimizations can be made based on the test results. Finally, the trained steel flow landing point detection model is applied to a real-world production environment. By receiving real-time image data from the converter, the model can automatically detect the landing point of the steel flow and feed the results back to the control system or operators for automated monitoring and adjustment.

[0042] It should be noted that, through the above embodiments, historical images can be used to train the steel flow landing point detection model to improve the model's ability to identify and detect the state of steel flow and the accuracy of detection. This can be applied to the automated monitoring and management of the steel smelting process, thereby improving production efficiency and safety.

[0043] Step S230: Extract the steel flow block images in the current image based on the label information as the coordinates of all steel flow blocks, and stitch together all the extracted steel flow block images to obtain the initial steel flow image.

[0044] In one embodiment of this application, a high-definition camera and sensors are used to acquire real-time images of the target steel converter. Based on the coordinate information of the steel flow area in historical images, the corresponding steel flow area is located in the current image. Then, within the located steel flow area, image segmentation technology is used to divide the steel flow into several blocks. These blocks should be able to cover the entire flow trajectory of the steel flow and maintain the integrity and continuity of the blocks. Then, based on the segmented blocks, the image of each block is extracted from the current image, and all the extracted steel flow block images are stitched together in sequence to form an initial steel flow image. It should be noted that during the stitching process, attention should be paid to image matching and fusion to obtain a smooth and continuous steel flow image.

[0045] It should be noted that, through the above embodiments, a complete steel flow block image can be extracted from the current image based on the tag information, and then stitched together to obtain an initial steel flow image. This helps to improve the accuracy and real-time performance of steel flow status monitoring and provides strong support for the automated control of the steel smelting process.

[0046] In one embodiment of this application, extracting the steel flow contour of the initial steel flow image includes: extracting all contours in the initial steel flow image; expanding each pixel of each steel flow image block in the initial steel flow image outward to connect adjacent steel flow image blocks; shrinking each pixel in the expanded steel flow image inward to obtain the target steel flow direction, wherein adjacent steel flow blocks in the target steel flow direction are interconnected; extracting the contour of the target steel flow image, and determining the contour of the target steel flow image as the steel flow contour of the initial steel flow image.

[0047] In one specific embodiment of this application, image processing techniques, such as edge detection algorithms (e.g., the Canny edge detection algorithm), are first used to identify and extract the entire contour of the initial steel flow image. Then, by diffusing the color or grayscale value of each pixel or increasing the influence of neighboring pixels, each pixel of each steel flow image block in the initial steel flow image is expanded outward, so that adjacent steel flow image blocks are connected to each other to form a continuous steel flow image. Then, by reducing the color or grayscale value of each pixel, each pixel in the expanded steel flow image is shrunk inward to obtain the target steel flow direction and make adjacent steel flow image blocks interconnected. Then, the edge detection algorithm is used to extract the contour from the target steel flow image, and the extracted contour of the target steel flow image is determined as the steel flow contour of the initial steel flow image. These contours will be used for subsequent analysis and processing to understand the motion state, speed, trajectory, and other information of the steel flow.

[0048] It should be noted that, through the above embodiments, all contours in the initial steel flow image can be extracted and determined as the steel flow contour of the initial steel flow image, which helps to more accurately monitor and analyze the movement state of the steel flow and provide accurate data support for the automated control of the steel smelting process.

[0049] In one embodiment of this application, after extracting all contours from the initial steel flow image, the method further includes: obtaining any complete contour information and determining the image region corresponding to the complete contour as a steel flow image block; calculating the image area of ​​the steel flow image block and comparing the calculated block area with a preset standard area; if the image area of ​​any steel flow image block is less than the preset standard area, then the steel flow image block corresponding to the image area is determined as an invalid block; if the image area of ​​any steel flow image block is greater than or equal to the preset standard area, then the steel flow image block corresponding to the image area is determined as a valid block, so as to obtain the target steel flow image based on the valid blocks.

[0050] In one specific embodiment of this application, firstly, the complete contour information of any steel flow image block is extracted from the initial steel flow image using an edge detection algorithm; then, the image region corresponding to each complete contour is determined as a steel flow image block, and the image area of ​​each steel flow image block is calculated by calculating the number of pixels or the sum of the pixel areas within the block; next, the calculated block area is compared with a preset standard area. If the image area of ​​any steel flow image block is less than the preset standard area, the block is determined as an invalid block; if the image area of ​​any steel flow image block is greater than or equal to the preset standard area, the block is determined as a valid block. It should be noted that invalid blocks may contain small, insignificant, or falsely detected steel flow structures, while valid blocks represent the main and significant steel flow structures; finally, the valid blocks are spliced ​​or combined to form the target steel flow image. During splicing, attention should be paid to maintaining the continuity and smoothness between each valid block to obtain a complete steel flow trajectory and flow state. It should be noted that the preset standard area can be set according to actual needs, usually based on experience or historical data. This application does not impose any restrictions on its specific value or determination method.

[0051] Furthermore, it should be noted that, through the above embodiments, effective steel flow image blocks can be screened according to preset standard areas, and target steel flow images can be obtained based on these effective blocks. This helps to eliminate falsely detected or small steel flow structures, and improves the accuracy and reliability of steel flow status monitoring.

[0052] Step S240: Extract the steel flow contour of the initial steel flow image, and obtain the target abscissa of the steel flow based on the steel flow contour, and obtain the landing position of the steel flow based on the abscissa of the steel flow.

[0053] In one embodiment of this application, obtaining the target abscissa of the steel flow based on the steel flow profile includes: collecting the abscissa information of all coordinate points in the steel flow profile; calculating the mean of all abscissa information to obtain the abscissa mean, and determining the abscissa mean as the target abscissa of the steel flow.

[0054] In one specific embodiment of this application, firstly, the abscissa information of all coordinate points in the steel flow profile is collected. These coordinate points can be points on the steel flow profile extracted by an edge detection algorithm. Then, the abscissa information of all collected abscissa information is averaged using a simple arithmetic mean or weighted average method. The specific algorithm can be selected according to actual needs, and this application does not impose any restrictions on it. Finally, the calculated abscissa mean is used as the target abscissa of the steel flow, which represents the center position or average position of the steel flow in the image.

[0055] It should be noted that by applying the above target horizontal coordinate to the actual production monitoring system, the horizontal coordinate position of the steel flow can be monitored in real time to understand the movement status, speed and trajectory of the steel flow. At the same time, this information can be fed back to the upstream model for learning and optimization to improve the accuracy and stability of the model.

[0056] Furthermore, in one embodiment of this application, obtaining the center coordinates of the ladle based on its outline includes: constructing a minimum bounding rectangle associated with the ladle outline; obtaining the coordinates of the side length of the minimum bounding rectangle; calculating the length of the long side of the minimum bounding rectangle based on the coordinates of the side length; and determining half of the length of the long side as the radius of the ladle.

[0057] In one specific embodiment of this application, the outline of the ladle is first extracted from the initial ladle image, and a minimum bounding rectangle is found based on the extracted ladle outline, which can completely contain the outline of the ladle; then, the long side is determined in the found minimum bounding rectangle; finally, half of the long side is used as the radius of the ladle, which can reflect the size and shape of the ladle outline.

[0058] In one embodiment of this application, after obtaining the abscissa of the steel flow based on the steel flow profile, the method further includes: acquiring an initial ladle image, extracting the ladle profile from the initial ladle image, and obtaining the center coordinates of the ladle based on the ladle profile; comparing the abscissa of the steel flow with the center coordinates of the ladle, and determining the offset state of the steel flow based on the comparison result, so as to adjust the position of the ladle based on the offset state; wherein, the step of acquiring the initial ladle image includes: extracting the ladle block image from the current image based on the label information for all coordinates of the ladle, and stitching together all the extracted ladle block images to obtain the initial ladle image, and the label information also includes the ladle.

[0059] In one specific embodiment of this application, a high-definition camera and sensors are first used to acquire images of the ladle in real time, and the outline of the ladle is extracted from the initial ladle image. Then, based on the extracted ladle outline, the center coordinates of the ladle are calculated by averaging or centroid of all points on the outline. The calculated abscissa of the steel flow is compared with the center coordinates of the ladle. If the abscissa of the steel flow differs significantly from the center coordinates of the ladle, the steel flow is considered to be in an off-center state. Then, based on the comparison result, the off-center state is determined by setting a threshold or using other methods to determine whether the steel flow has deviated from the center of the ladle. The position of the ladle is then adjusted based on the off-center state. The specific adjustment method can be determined according to actual needs. For example, the ladle can be moved by controlling a robotic arm or conveyor belt to align it with the steel flow or restore it to the correct position. This application does not impose specific restrictions on the adjustment method.

[0060] It should be noted that, through the above embodiments, the ladle outline can be extracted based on the initial ladle image and the offset state of the steel flow can be determined to adjust the position of the ladle. This helps to improve the monitoring and management capabilities of the steel flow and ladle position during the steel smelting process, and promotes the automation and intelligence of the production process.

[0061] In one embodiment of this application, comparing the abscissa of the steel flow with the center coordinates of the ladle, and determining the offset state of the steel flow based on the comparison result, includes: determining half of the long side of the smallest bounding rectangle of the ladle outline as the ladle radius, and determining the difference between the abscissa of the steel flow and the abscissa of the ladle center coordinates as the lateral deviation; calculating the lateral deviation and the ladle radius to obtain the deviation ratio; when the deviation ratio is less than or equal to a preset deviation threshold, the offset state of the steel flow is considered to be a minor deviation, and no adjustment of the ladle position is required; when the deviation ratio is greater than the preset deviation threshold, the offset state of the steel flow is considered to be a severe deviation; if the deviation ratio is positive, it is determined that the landing point of the steel flow is deviated to the right relative to the ladle; if the deviation ratio is negative, it is determined that the landing point of the steel flow is deviated to the left relative to the ladle.

[0062] In one specific embodiment of this application, the outline of the ladle is first extracted from the initial ladle image. Based on the extracted ladle outline, a minimum bounding rectangle is found, and half of the long side of this minimum bounding rectangle is used as the radius of the ladle. Then, the difference between the horizontal coordinate of the steel flow and the center coordinate of the ladle is calculated. This difference is the lateral deviation. The lateral deviation is further calculated as a ratio to the ladle radius to obtain the deviation ratio, which is used to evaluate the position and degree of offset of the steel flow relative to the ladle. If the deviation ratio is less than or equal to a preset deviation threshold, the steel flow is considered to have a minor deviation, and no adjustment to the ladle's position is required. If the deviation ratio is greater than the preset deviation threshold, the steel flow is considered to have a serious deviation, and adjustment is required. The sign of the deviation ratio is used to determine whether the landing point of the steel flow is offset to the right or left relative to the ladle, providing directional guidance for adjustment. Finally, based on the judgment result, if adjustment is required, the ladle is moved by controlling a robotic arm or conveyor belt to align it with the steel flow or restore it to the correct position. Furthermore, after the ladle adjustment is completed, the adjusted ladle position can be applied to the actual production monitoring system. By monitoring the steel flow and ladle offset in real time, the adjustment process can be continuously optimized to improve the stability and efficiency of the production process. At the same time, this information can also be fed back to the upstream model for learning and optimization to improve the model's accuracy and stability.

[0063] It should be noted that, through the above embodiments, the ratio of the ladle radius to the distance from the steel flow landing point to the ladle center can be accurately calculated, thereby better assessing the state and position of the steel flow. This helps to improve the monitoring and management capabilities of the steel flow and ladle position during the steel smelting process, and promotes the automation and intelligence of the production process.

[0064] Figure 3 This is a schematic diagram illustrating the steps of a deep learning-based steel flow landing point detection method, as shown in an exemplary embodiment of this application. Figure 3 As shown, the deep learning-based steel flow landing point detection method includes the following steps:

[0065] Step S1: Train an instance segmentation model to segment and detect steel flow and steel ladle;

[0066] Step S2: Use the trained instance segmentation model to segment and detect the images captured in the furnace scene;

[0067] Step S3: The complete outline is obtained through post-processing using the area and location information of the steel flow and ladle.

[0068] Step S4: Calculate the x-coordinate of the landing point of the steel flow using the processed steel flow profile;

[0069] Step S5: Calculate the center coordinates of the ladle using the processed ladle profile;

[0070] Step S6: The degree of deviation of the steel flow landing point is classified by calculating the ratio of the radius of the ladle to the distance from the steel flow landing point to the center of the ladle.

[0071] Therefore, in one embodiment of this application, the specific execution steps of the deep learning-based steel flow landing point detection method are as follows:

[0072] Firstly, data is collected based on thermal imaging cameras, such as... Figure 4 The original image shown. It should be noted that, considering the special environmental factors such as smoke obscuring the view in the steel converter scene, in order to make the outlines of the steel flow and ladle clearer in the image for easier annotation and to reduce the difficulty of model recognition, a thermal imaging camera is usually used for image acquisition. However, the thermal imaging camera is only used as one image acquisition device in this embodiment, and this application does not impose any restrictions on the equipment and method of image acquisition. In addition, during the annotation process, the annotator should focus on distinguishing the outline of the steel flow, and the outline of the ladle should be the elliptical outline with obvious changes in brightness in the image. At the same time, attention should also be paid to the fact that the molten steel is brighter in the image due to its higher temperature, so as to avoid mistakenly annotating the molten steel in the ladle furnace as the outline of the ladle.

[0073] Then, based on prior knowledge, an instance segmentation method is used to segment and label the ladle images used to train the instance segmentation model, so that the ladle and the steel flow are labeled as two different categories, as shown in the figure. Figure 5 The annotated image is shown; and an instance segmentation model is trained based on the annotated image. The instance segmentation model can be one of the YOLO (You Only Look Once) series instance segmentation models, including but not limited to YOLOACT, YOLOv5 Seg, and YOLOv8 Seg. It should be noted that in this embodiment, an instance segmentation method is used for annotation, so only the steel flow and ladle need to be labeled, while the molten steel inside the furnace and the background are not labeled. In other specific embodiments, different methods can be used to annotate the original image based on the actual situation. This application does not impose any restrictions on the method of image annotation. In addition, the YOLO series models proposed in the above embodiments are only illustrative examples of one embodiment. From the perspective of technical principles, similar effects can be obtained by using other models. Therefore, this application does not impose specific restrictions on the type of instance segmentation model.

[0074] Next, the trained instance segmentation model is used to segment and detect the current image captured in the furnace scene to obtain the contours of the ladle and steel flow corresponding to the current image. The current image and the image after instance segmentation are shown below. Figure 6As shown. Given that image recognition may misidentify background elements as steel flow or ladles, resulting in incomplete outlines of the steel flow or ladles, further processing of the resulting images is necessary. For example: first, a contour extraction algorithm is used to obtain all contours in the segmented image; then, morphological dilation and erosion algorithms are used to eliminate small misidentified areas and remove gaps in larger areas; finally, a set area threshold and relative positional relationship are used to determine and confirm the correct identification of the complete outlines of the steel flow and ladles. It should be noted that the above image processing method is only an example. In practical applications, any other applicable image processing method can be used based on the actual situation, and this application does not impose any limitations on it.

[0075] Next, the abscissa of the steel flow landing point is calculated using the processed steel flow profile. This is done by calculating the average abscissa of all points in the complete steel flow profile, which is then used as the abscissa of the steel flow landing point. When calculating the center coordinates of the ladle using the processed ladle profile, the minimum bounding rectangle of the ladle profile is calculated, and the center coordinates of this bounding rectangle are used as the center coordinates of the ladle profile. Then, half the longer side of the minimum bounding rectangle of the ladle profile is used as the radius of the ladle. The ratio of the ladle radius to the distance from the steel flow landing point to the ladle center (i.e., the aforementioned deviation ratio) is calculated to determine the steel flow offset. The calculation method for the ratio of the steel flow landing point to the ladle center is as follows:

[0076] k = (lb) / (w / 2) Equation (1),

[0077] Where k is the ratio of the ladle radius to the distance from the ladle's landing point to its center, l is the x-coordinate of the ladle's landing point, b is the x-coordinate of the ladle's center, and w is the ladle radius.

[0078] Finally, based on the deviation ratio obtained above, the offset state of the steel flow based on the ladle is determined, thereby obtaining the adjustment strategy for generating the ladle or steel flow so that the steel flow can fall into the ladle more accurately.

[0079] Figure 7 This is an example diagram illustrating the offset of the steel flow landing point, as shown in an exemplary embodiment of this application; Figure 8 This is an example diagram illustrating the offset of the steel flow landing point, as shown in another exemplary embodiment of this application.

[0080] In one specific embodiment of this application, the detection result of the steel flow landing point of a current image is as follows: Figure 7As shown, the deviation ratio of the current steel flow based on the ladle is calculated to be greater than the preset threshold. Therefore, the current steel flow is considered to be seriously deviated from the ladle. The deviation ratio is positive, which means that the position of the steel flow based on the ladle is biased to the right. Therefore, the ladle needs to be moved to the right by an appropriate distance, or the position of the steel flow needs to be adjusted so that the position of the steel flow shifts to the left, so that the steel flow can flow into the ladle better.

[0081] In another specific embodiment of this application, the steel flow landing point detection result of a current image is as follows: Figure 8 As shown, the deviation ratio of the current steel flow based on the ladle is calculated to be greater than the preset threshold. Therefore, the current steel flow is considered to be seriously deviated from the ladle. The deviation ratio is also negative, which means that the position of the steel flow based on the ladle is to the left. Therefore, the ladle needs to be moved to the left by an appropriate distance, or the position of the steel flow needs to be adjusted so that the position of the steel flow is shifted to the right, so that the steel flow can flow into the ladle better.

[0082] Finally, it should be noted that the deep learning-based steel flow landing point detection method proposed in this application can identify the landing point of the steel flow through training and learning, and accurately calculate the degree of offset between the steel flow and the ladle. In addition, based on a large amount of training data, the deep learning model can continuously improve its prediction and judgment accuracy, thereby improving the accuracy of steel flow landing point determination. By acquiring images in real time and processing them with deep learning algorithms, the landing point of the steel flow can be quickly determined, and the degree of offset between the steel flow and the ladle can be detected in a timely manner, enabling timely adjustments during production and avoiding problems caused by offset. Through deep learning algorithms and image recognition technology, the determination of the steel flow landing point and the degree of offset can be completed automatically, reducing the need for manual intervention, reducing the possibility of human error, and improving the automation level of the production process. Furthermore, through deep learning and image recognition technology, intelligent management of the production process can also be achieved. By accurately monitoring and controlling the position of the steel flow and the ladle, production efficiency, product quality, and the stability of the entire production process can be effectively improved.

[0083] Figure 9 This is a block diagram illustrating a deep learning-based steel flow point detection device as an exemplary embodiment of this application. This device can be applied to... Figure 1 The implementation environment shown is illustrated. This device can also be applied to other exemplary implementation environments and specifically configured in other devices. This embodiment does not limit the implementation environment to which the device is applicable.

[0084] like Figure 9 As shown, the exemplary deep learning-based steel flow landing point detection device includes: a data acquisition module 910, a deep learning module 920, an image processing module 930, and a steel flow landing point determination module 940.

[0085] The system includes: a data acquisition module 910 for acquiring historical images of the target steel converter and current images of the target steel converter; a deep learning module 920 for training a steel flow landing point detection model based on historical images and inputting the current image into the trained steel flow landing point detection model to obtain the detection result of the current image, which includes coordinate information and label information, with the label information including at least the steel flow; an image processing module 930 for extracting steel flow block images from the current image based on the coordinates of the steel flow with the label information, and stitching together all the extracted steel flow block images to obtain an initial steel flow image; and a steel flow landing point determination module 940 for extracting the steel flow contour of the steel flow image, obtaining the abscissa of the steel flow based on the steel flow contour, and obtaining the landing point position of the steel flow based on the abscissa of the steel flow.

[0086] It should be noted that the deep learning-based steel flow landing point detection device and the deep learning-based steel flow landing point detection method provided in the above embodiments belong to the same concept. The specific operation methods of each module and unit have been described in detail in the method embodiments and will not be repeated here. In practical applications, the deep learning-based steel flow landing point detection device provided in the above embodiments can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. This is not a limitation here.

[0087] Embodiments of this application also provide an electronic device, including: one or more processors; and a storage device for storing one or more programs, which, when executed by the one or more processors, cause the electronic device to implement the deep learning-based steel flow landing point detection method provided in the above embodiments.

[0088] Figure 10 A schematic diagram of a computer system suitable for implementing the embodiments of this application is shown. It should be noted that... Figure 10 The computer system 1000 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0089] like Figure 10As shown, the computer system 1000 includes a Central Processing Unit (CPU) 1001, which can perform various appropriate actions and processes based on programs stored in Read-Only Memory (ROM) 1002 or programs loaded from storage portion 1008 into Random Access Memory (RAM) 1003, such as performing the methods described in the above embodiments. The RAM 1003 also stores various programs and data required for system operation. The CPU 1001, ROM 1002, and RAM 1003 are interconnected via a bus 1004. An Input / Output (I / O) interface 1005 is also connected to the bus 1004.

[0090] The following components are connected to I / O interface 1005: an input section 1006 including a keyboard, mouse, etc.; an output section 1007 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to I / O interface 1005 as needed. Removable media 1011, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 1010 as needed so that computer programs read from them can be installed into storage section 1008 as needed.

[0091] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 1009, and / or installed from removable medium 1011. When the computer program is executed by central processing unit (CPU) 1001, it performs various functions defined in the system of this application.

[0092] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0093] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0094] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0095] Another aspect of this application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a computer's processor, causes the computer to perform the deep learning-based steel flow landing point detection method as described above. This computer-readable storage medium may be included in the electronic device described in the above embodiments, or it may exist independently and not assembled into the electronic device.

[0096] Another aspect of this application provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the deep learning-based steel flow landing point detection method provided in the various embodiments above.

[0097] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. A deep learning-based steel flow landing point detection method, characterized in that, The method includes: Acquire historical images of the target steel converter, as well as the current image of the target steel converter; The steel flow landing point detection model is trained based on the historical images, and the current image is input into the trained steel flow landing point detection model to obtain the detection result of the current image. The detection result includes coordinate information and label information, and the label information includes at least the steel flow. Based on the tag information, the steel flow block images in the current image are extracted, and all the extracted steel flow block images are stitched together to obtain the initial steel flow image; Extract the steel flow contour from the initial steel flow image, and obtain the target abscissa of the steel flow based on the steel flow contour, so as to obtain the landing position of the steel flow based on the target abscissa of the steel flow. The process of obtaining the target abscissa of the steel flow based on the steel flow profile includes: collecting the abscissa information of all coordinate points in the steel flow profile, calculating the mean of all the abscissa information to obtain the abscissa mean, and determining the abscissa mean as the target abscissa of the steel flow. After obtaining the target abscissa of the steel flow, the method further includes: acquiring an initial ladle image, extracting the ladle outline from the initial ladle image, and obtaining the center coordinates of the ladle based on the ladle outline; comparing the abscissa of the steel flow with the center coordinates of the ladle, and determining the offset state of the steel flow based on the comparison result, so as to adjust the position of the ladle based on the offset state; The process of comparing the abscissa of the steel flow with the center coordinates of the ladle and determining the offset state of the steel flow based on the comparison result includes: determining half of the long side of the smallest bounding rectangle of the ladle outline as the ladle radius, and determining the difference between the abscissa of the steel flow and the center coordinates of the ladle as the lateral deviation; calculating the deviation ratio between the lateral deviation and the ladle radius; when the deviation ratio is less than or equal to a preset deviation threshold, the steel flow is considered to have a minor deviation and no adjustment to the ladle position is required; when the deviation ratio is greater than the preset deviation threshold, the steel flow is considered to have a severe deviation. If the deviation ratio is positive, the steel flow landing point is determined to be rightward relative to the ladle; if the deviation ratio is negative, the steel flow landing point is determined to be leftward relative to the ladle. 2.The deep learning-based steel flow landing point detection method of claim 1, wherein, The steps for obtaining the initial ladle image include: Based on the tag information, the steel ladle block image is extracted from the current image using all coordinates. The extracted steel ladle block images are then stitched together to obtain an initial steel ladle image. The tag information also includes the steel ladle. 3.The deep learning-based steel flow landing point detection method of claim 2, wherein, Training a steel flow point detection model based on the historical images includes: The ladle region and the steel flow region in the historical image are determined, wherein the ladle region is used to characterize the image of the ladle in the historical image, and the steel flow region is used to characterize the image of the steel flow in the historical image; The ladle area and the steel flow area are labeled based on preset image labels, wherein the preset image labels include ladle and steel flow; A training sample dataset is generated based on the labeled historical images, and the initial image recognition model is trained based on the training sample dataset to obtain the trained steel flow landing point detection model. 4.The deep learning-based steel flow landing point detection method of claim 2, wherein, Extracting the steel flow profile from the initial steel flow image includes: Extract all contours from the initial steel flow image; Each pixel of each steel flow image block in the initial steel flow image is expanded outward to connect adjacent steel flow image blocks. Each pixel in the expanded steel flow image is reduced inward to obtain the target steel flow direction, in which adjacent steel flow blocks are interconnected. Extract the contour of the target steel flow image and determine the contour of the target steel flow image as the steel flow contour of the initial steel flow image. 5.The deep learning-based steel flow landing point detection method of claim 4, wherein, After extracting all contours from the initial steel flow image, the process further includes: Obtain any complete contour information and determine the image region corresponding to the complete contour as a steel flow image block; Calculate the image area of ​​the steel flow image block and compare the calculated block area with a preset standard area; If the image area of ​​any steel flow image block is smaller than the preset standard area, then the steel flow image block corresponding to the image area is determined as an invalid block; If the image area of ​​any steel flow image block is greater than or equal to the preset standard area, then the steel flow image block corresponding to the image area is determined as a valid block, so as to obtain the target steel flow image based on the valid block. 6.A steel flow landing point detection device based on deep learning, characterized in that, The device includes: The data acquisition module is used to acquire historical images of the target steel converter, as well as the current image of the target steel converter; A deep learning module is used to train a steel flow landing point detection model based on the historical images, and input the current image into the trained steel flow landing point detection model to obtain the detection result of the current image. The detection result includes coordinate information and label information, and the label information includes at least the steel flow. The image processing module is used to extract the steel flow block images in the current image based on the coordinates of all steel flow based on the label information, and to stitch together all the extracted steel flow block images to obtain an initial steel flow image; A steel flow landing point determination module is used to extract the steel flow contour of the steel flow image and obtain the target abscissa of the steel flow based on the steel flow contour, so as to determine the landing point position of the steel flow based on the target abscissa of the steel flow; wherein, obtaining the target abscissa of the steel flow based on the steel flow contour includes: collecting the abscissa information of all coordinate points in the steel flow contour, calculating the mean of all the abscissa information to obtain the abscissa mean, and determining the abscissa mean as the target abscissa of the steel flow; after obtaining the target abscissa of the steel flow, the module further includes: acquiring an initial ladle image, extracting the ladle contour of the initial ladle image, and obtaining the center coordinate of the ladle based on the ladle contour; comparing the abscissa of the steel flow and the center coordinate of the ladle, and determining the offset state of the steel flow based on the comparison result, so as to determine the landing point position of the steel flow based on the offset. The state adjusts the position of the ladle; wherein, the abscissa of the steel flow and the center coordinate of the ladle are compared, and the offset state of the steel flow is determined based on the comparison result, including: determining half of the long side of the smallest bounding rectangle of the ladle outline as the ladle radius, and determining the difference between the abscissa of the steel flow and the abscissa of the center coordinate of the ladle as the lateral deviation; calculating the deviation ratio between the lateral deviation and the ladle radius; when the deviation ratio is less than or equal to a preset deviation threshold, the offset state of the steel flow is considered to be a minor deviation, and no adjustment of the ladle position is required; when the deviation ratio is greater than the preset deviation threshold, the offset state of the steel flow is considered to be a severe deviation. If the deviation ratio is positive, it is determined that the landing point of the steel flow is deviated to the right relative to the ladle; if the deviation ratio is negative, it is determined that the landing point of the steel flow is deviated to the left relative to the ladle.

7. An electronic device, comprising: The electronic device includes: One or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, cause the electronic device to implement the deep learning-based steel flow landing point detection method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, It stores a computer program, which, when executed by the computer's processor, causes the computer to perform the deep learning-based steel flow landing point detection method according to any one of claims 1 to 5.

Images