Billet inclination detection method, device and system and medium

Abstract

The embodiment of the invention provides a billet inclination detection method, device and system and a medium, and belongs to the technical field of industrial detection. The method comprises the following steps: acquiring a visible light image and a thermal imaging image of a steel billet running on a roller way, wherein the visible light image and the thermal imaging image comprise a steel billet full view and a roller way datum line; carrying out image fusion on the visible light image and the thermal imaging image; sequentially executing deep learning processing and polygon abstraction processing on the fused image so as to identify the lower bottom edge of the steel billet; and calculating the included angle between the lower bottom edge of the steel billet and the roller bed datum line to obtain the inclination angle of the steel billet. According to the embodiment of the invention, through image fusion, deep learning processing and polygon abstraction processing, the inclination angle of the steel billet is obtained in a low-cost, high-efficiency and real-time manner, and the applicability is high.

Description

Technical Field

[0001] The present invention relates to the technical field of industrial detection, and particularly relates to a method, device, system and medium for detecting the inclination of steel billets. Background Art

[0002] The iron and steel industry, as a basic industry of the national economy, plays a crucial role in the country's industrialization and modernization process. Steel billets are important intermediate products in the steel production process, and their quality and production efficiency directly affect the performance and finished product quality of the downstream rolling process. During the production process of steel billets, the steel billets need to be heated in a heating furnace and then transported to a descaling box through roller tables or tracks to remove the scale on the surface and improve the surface quality and mechanical properties of the steel.

[0003] During the process of transporting the steel billets from the heating furnace to the descaling box, the steel billets run at high speed on the roller tables or tracks in a high-temperature state (usually exceeding 1000 degrees Celsius). Due to the self-weight and high-temperature characteristics of the steel billets, higher requirements are put forward for the smoothness and wear resistance of the roller tables and tracks. However, in actual production, due to the long-term use of the roller tables and tracks, problems such as wear, aging, and deformation may occur, resulting in phenomena such as inclination and deviation of the steel billets during operation.

[0004] When the steel billets run obliquely on the tracks, it may cause them to fail to pass through the descaling box smoothly or even get stuck at the entrance of the descaling box, resulting in the stagnation of the production line. This not only affects the production efficiency and causes economic losses, but also may damage the equipment and increase the maintenance cost. In addition, the inclination of the steel billets may also trigger safety accidents and endanger the safety of the operators. Therefore, how to detect the inclination of the steel billets in real time and accurately and take corrective measures in a timely manner has become an urgent technical problem in the steel production process.

[0005] The traditional solution mainly involves regularly replacing the roller tables and rollers to maintain the normal operation of the equipment. However, this method not only consumes a large amount of funds and manpower, but also cannot solve the problem of steel billet inclination in real time, has hysteresis, and is difficult to respond to sudden steel billet inclination situations during the production process in a timely manner, which does not conform to the current development direction of automation and intelligentization in steel production.

[0006] In response to this, some steel mills have adopted a steel billet inclination detection scheme based on the combination of mechanical limit and physical detection. Although it solves the problem of steel billet inclination to a certain extent, it has at least the following deficiencies:

[0007] 1. High cost: The manufacturing, installation and maintenance costs of mechanical limit devices and physical sensors are relatively high. The limit devices need to use high-strength and high-temperature-resistant materials, increasing the material cost. Physical contact sensors have a short lifespan in high-temperature and dusty environments and need to be replaced frequently.

[0008] 2. Low detection accuracy: Physical contact sensors can only detect obvious tilting or offset of the billet, unable to accurately measure the tilting angle, and cannot detect slight tilting in a timely manner, which may lead to missed detections.

[0009] 3. Slow response speed: The response speed of mechanical and physical contact detections is limited by mechanical movement, unable to detect the tilting of the billet in real time and quickly, which may lead to delayed alarms and the inability to take corrective measures in a timely manner.

[0010] 4. Frequent wear and maintenance: Under high-temperature and high-speed conditions, the billet comes into frequent contact with mechanical limit devices and sensors, resulting in severe wear of the equipment and the need for frequent maintenance and replacement. This not only increases the maintenance cost but may also cause production stoppages due to equipment failures.

[0011] 5. Complex installation and debugging: In harsh environments such as high temperature and dust, the installation and debugging of mechanical devices and sensors are difficult and require professional technicians, increasing the labor cost and time cost.

[0012] 6. Safety hazards: Mechanical limit devices and physical sensors may fail due to wear or malfunctions, unable to provide effective tilting protection and detection, posing safety hazards. In addition, operators need to perform manual interventions in high-temperature and dangerous environments, which may lead to personal safety risks.

[0013] 7. Impact on production efficiency: Frequent equipment maintenance and manual interventions may increase the downtime of the production line, affecting production efficiency and economic benefits.

[0014] 8. Poor environmental adaptability: In harsh steel mill environments such as high temperature, dust, and vibration, the performance of mechanical devices and sensors is easily affected, with poor reliability and stability, which may lead to false alarms or missed detections.

[0015] Therefore, due to its high cost, low accuracy, slow response speed, complex maintenance, etc., this solution is difficult to meet the requirements of modern steel production for high efficiency, low cost, real-time performance, and safety.

[0016] In addition, there is another billet tilting detection solution in the existing technology, that is: using a laser rangefinder and a 3D scanner to obtain the three-dimensional shape and position attitude of the billet in real time, so as to judge whether the billet is tilted. Although this solution has advantages in detection accuracy by using advanced laser ranging and 3D imaging technologies, it still has the following deficiencies:

[0017] 1. Complex equipment and high cost: High-precision laser rangefinders and 3D scanners are expensive, and the overall investment cost of the system is high. In addition, the installation and debugging of the equipment require professional technicians, increasing the labor cost.

[0018] 2. Poor environmental adaptability: The steel production environment is harsh, with problems such as high temperature, dust, and strong light interference. Laser equipment is sensitive to environmental conditions. Dust and high temperature can affect the transmission and reception of laser signals, resulting in a decrease in measurement accuracy and even equipment failure.

[0019] 3. Difficult maintenance: Laser equipment requires high maintenance requirements and needs to be cleaned, calibrated, and overhauled regularly. The high-temperature and dust environment accelerate the aging and damage of the equipment, increasing the maintenance cost and workload.

[0020] 4. Complex data processing and poor real-time performance: A large amount of point cloud data generated by three-dimensional scanning requires complex algorithms for processing, with a large amount of calculation and a long processing time. It is difficult to meet the real-time requirements of high-speed production lines, which may lead to delays in detection and alarm.

[0021] 5. Low system reliability: In a harsh production environment, it is difficult to ensure the stability and reliability of laser equipment. Problems such as loss of measurement data and increased errors may occur, affecting the reliable operation of the system.

[0022] 6. Limited installation space: Laser rangefinders and three-dimensional scanning devices are relatively large in volume and require a certain amount of space for installation. On a production line with a compact layout, there may be a lack of sufficient installation positions, affecting the arrangement of equipment and the measurement range.

[0023] 7. Safety risks: High-power laser equipment may cause harm to the eyes and skin of operators, and additional safety protection measures need to be taken, increasing the complexity of safety management.

[0024] 8. Difficult system integration: Integrating laser ranging and three-dimensional imaging systems into existing production control systems involves the compatibility of software and hardware and the adaptation of communication protocols, increasing the complexity of system integration.

[0025] Therefore, in view of the technical problem that the billet cannot pass through the descaling box or even gets stuck at the entrance of the descaling box due to inclination when running on the roller table, and considering the deficiencies of the existing billet inclination detection schemes, the present application proposes a new billet inclination detection scheme. Summary of the Invention

[0026] The purpose of the embodiments of the present application is to provide a billet inclination detection method, device, system, and medium, which are used to at least partially solve the above technical problems.

[0027] To achieve the above object, an embodiment of the present application provides a method for detecting the inclination of a steel billet, including: obtaining a visible light image and a thermal imaging image of the steel billet running on a roller table, where the visible light image and the thermal imaging image include the full view of the steel billet and the roller table reference line; performing image fusion on the visible light image and the thermal imaging image; for the fused image, sequentially performing deep learning processing and polygon abstraction processing to identify the lower bottom edge of the steel billet; and calculating the angle between the lower bottom edge of the steel billet and the roller table reference line to obtain the steel billet inclination angle.

[0028] Optionally, the YOLOv8 algorithm, Mask R-CNN algorithm, SSD algorithm, and / or DETR algorithm are used to perform the deep learning processing on the fused image to obtain a steel billet contour model.

[0029] Optionally, for the steel billet contour model obtained after performing the deep learning processing, the Douglas-Peucker algorithm and / or the least squares method are used to perform the polygon abstraction processing to obtain a polygon showing the edge features of the steel billet.

[0030] Optionally, before calculating the angle between the lower bottom edge of the steel billet and the roller table reference line, the method for detecting the inclination of the steel billet further includes: performing parameter calibration and perspective transformation on the cameras that capture the visible light image and the thermal imaging image so that the calculation is performed in the world coordinate system.

[0031] Optionally, for the fused image, the method for detecting the inclination of the steel billet further includes identifying the lower bottom edge of the steel billet by any one or more of the following: using the Canny edge detection algorithm to detect the edge of the steel billet from the fused image and performing line fitting through the Hough transform to obtain the lower bottom edge of the steel billet; performing template matching on the fused image with a pre-established steel billet template to identify the lower bottom edge of the steel billet; and performing morphological operations on the fused image to identify the lower bottom edge of the steel billet.

[0032] Optionally, after obtaining the steel billet inclination angle, the method for detecting the inclination of the steel billet further includes: when the steel billet inclination angle exceeds a set inclination angle threshold, performing an alarm operation and / or a correction operation.

[0033] Optionally, after obtaining the inclination angle of the billet, the billet inclination detection method further includes comparing the billet inclination angle with one or more of the following to determine the calculation accuracy of the inclination angle: the billet inclination angle detected by a non-contact ranging sensor, where the non-contact ranging sensor includes a depth camera, a laser scanner, a millimeter-wave radar sensor, and / or an ultrasonic sensor; the billet inclination angle detected by an inertial sensor mounted on the billet; the billet inclination angle detected by an inclination sensor mounted on the roller table.

[0034] On the other hand, an embodiment of the present application further provides a billet inclination detection device, including: a memory configured to store instructions; and a processor configured to call the instructions from the memory and capable of implementing the billet inclination detection method according to any of the above when executing the instructions.

[0035] On the other hand, an embodiment of the present application further provides a billet inclination detection system, including: an image acquisition device including an industrial camera and a thermal imaging camera adapted to be installed at the billet position, respectively used to capture a visible light image and a thermal imaging image of the billet running on the roller table, where the visible light image and the thermal imaging image include the full view of the billet and the roller table reference line; and any of the above billet inclination detection devices, used to perform image fusion and image processing on the visible light image and the thermal imaging image captured by the image acquisition device to identify the lower bottom edge of the billet and calculate the included angle between the lower bottom edge of the billet and the roller table reference line to obtain the billet inclination angle.

[0036] Optionally, the billet inclination detection system further includes any one or more of the following linked to the billet inclination detection device: an alarm device for performing an alarm operation when the billet inclination angle exceeds a set inclination angle threshold; a communication module for notifying the production line control system to adjust the roller table speed or attitude to correct the billet position when the billet inclination angle exceeds the inclination angle threshold; a database module for storing the obtained billet inclination angle and / or billet running data; and a display device for real-time displaying the obtained billet inclination angle and / or billet running data.

[0037] On the other hand, an embodiment of the present application further provides a machine-readable storage medium, on which instructions are stored, and the instructions are used to make a machine execute any of the above billet inclination detection methods.

[0038] Through the above technical solutions, the embodiment of the present application obtains the billet inclination angle in a low-cost, high-efficiency, and real-time manner through image fusion, deep learning processing, and polygon abstraction processing, and has strong applicability.

[0039] Other features and advantages of the embodiments of the present application will be described in detail in the subsequent specific implementation part. Description of the Drawings

[0040] The drawings are used to provide a further understanding of the embodiments of the present application, and constitute a part of the specification. Together with the following detailed implementation manners, they are used to explain the embodiments of the present application, but do not constitute a limitation to the embodiments of the present application. In the drawings:

[0041] Figure 1 is a schematic flow chart of the billet inclination detection method according to the embodiment of the present application;

[0042] Figure 2 is a schematic diagram of multi-source image fusion and detection of an example according to the embodiment of the present application;

[0043] Figure 3 is a schematic diagram of polygon abstraction and bottom edge recognition of an example according to the embodiment of the present application;

[0044] Figure 4 is a schematic diagram of perspective transformation and inclination angle calculation of an example according to the embodiment of the present application;

[0045] Figure 5 is a schematic structural diagram of the billet inclination detection device according to the embodiment of the present application; and

[0046] Figure 6 is a schematic structural diagram of the billet inclination detection system according to the embodiment of the present application.

[0047] Description of the Reference Numerals

[0048] 100 Image acquisition device 200 Billet inclination detection device

[0049] 300 Alarm device 400 Communication module

[0050] 500 Database module 600 Display device

[0051] 110 Industrial camera 120 Thermal imaging camera Detailed Implementation Manner

[0052] The following details the specific implementation manners of the embodiments of the present application with reference to the drawings. It should be understood that the specific implementation manners described herein are only used to illustrate and explain the embodiments of the present application, and are not used to limit the embodiments of the present application.

[0053] It should be noted that the acquisition, transmission, storage, use, processing, etc. of data in the technical solution of the present application all comply with the relevant regulations of national laws and regulations. In the embodiments of the present application, some industry-existing solutions such as certain software, components, models, etc. may be mentioned. They should be regarded as exemplary, and their purpose is only to illustrate the feasibility in the implementation of the technical solution of the present application, but it does not mean that the applicant has already or necessarily used this solution.

[0054] Before introducing the embodiments of the present application, some terms to be involved are explained herein first so that those skilled in the art can better understand the embodiments of the present application. It should be noted that these explanations should also fall within the protection scope of the embodiments of the present application.

[0055] 1. Instance segmentation: A computer vision task that combines object detection and semantic segmentation, capable of distinguishing different individuals of the same category at the pixel level and obtaining the precise contours of each object. Optimization of the instance segmentation algorithm is to adjust and improve the network structure and parameters of the deep learning instance segmentation algorithm for specific application scenarios to improve the performance of the model.

[0056] 2. Polygon abstraction: A curve simplification algorithm used to simplify complex curves or contours into polygons composed of a small number of vertices, retaining the main features of the original shape and reducing the data volume.

[0057] 3. Perspective transformation: Using the internal and external parameters of the camera to perform coordinate transformation on the image, mapping the two-dimensional image captured by the camera to the actual three-dimensional space to eliminate the influence of perspective distortion.

[0058] 4. Homography matrix: In computer vision, a 3×3 matrix used to describe the point correspondence relationship between two planes, applied to the perspective transformation and correction of images.

[0059] 5. Object detection: A technology for automatically identifying and locating objects of interest in images or videos, usually outputting the position (such as bounding box) and class label of the object.

[0060] 6. High-temperature industrial camera: An industrial camera designed specifically for high-temperature environments, with characteristics such as high temperature resistance, dust resistance, and vibration resistance, capable of operating stably in high-temperature industrial environments such as steel.

[0061] 7. Thermal imaging camera: A camera that uses the infrared radiation emitted by objects to generate thermal images, capable of obtaining clear image information under poor visible light conditions or in high-temperature environments.

[0062] 8. Camera calibration: The process of determining the internal parameters (such as focal length, distortion coefficient) and external parameters (such as position, attitude) of the camera, used to correct image distortion and perform precise spatial measurements.

[0063] 9. Production line control system: A system used to monitor and control the operating states of various devices on the production line, realizing automated production, interlocking control, and exception handling.

[0064] 10. Multi-source image fusion: The process of fusing image data from different sensors (such as visible light cameras and thermal imaging cameras), comprehensively using various information to improve the accuracy and robustness of object detection.

[0065] Figure 1 is a schematic flowchart of the billet inclination detection method according to an embodiment of the present application. As Figure 1 shown, this billet inclination detection method is, for example, executed by a controller, and may at least include the following steps S100 - S400.

[0066] Step S100, obtain visible light images and thermal imaging images of the billet running on the roller table.

[0067] Among them, the visible light images and the thermal imaging images include the overall view of the billet and the roller table reference line.

[0068] For example, install a high - temperature industrial camera at key positions on the roller table where the billet runs to capture visible light images of the billet running on the roller table. The high - temperature industrial camera has the characteristics of high temperature resistance, dust resistance, and vibration resistance, and is suitable for the harsh environment of the steel mill. In addition, capture thermal imaging images of the billet through a thermal imaging camera. The application of this thermal imaging camera helps to solve the problem of the decline in the quality of visible light images in high - temperature environments. The high - temperature industrial camera and the thermal imaging camera need to be installed at appropriate heights and angles to ensure that corresponding images including the overall view of the billet and the roller table reference line can be captured. The controller obtains the corresponding visible light images and thermal imaging images from the corresponding cameras for image fusion in subsequent steps.

[0069] Step S200, perform image fusion on the visible light images and the thermal imaging images.

[0070] For example, first perform pre - processing such as denoising, enhancing contrast, and correcting distortion on the obtained visible light images and thermal imaging images to improve the image quality, and then use image fusion algorithms such as Alpha fusion, pyramid fusion, Poisson fusion, and frequency - domain fusion to fuse the visible light images and the thermal imaging images to obtain a fused image, enhancing the robustness of target detection.

[0071] Step S300, for the fused image, sequentially perform deep - learning processing and polygon abstraction processing to identify the lower bottom edge of the billet.

[0072] Among them, the purpose of deep - learning processing is essentially to perform target detection and instance segmentation on the fused image. In a preferred embodiment, the YOLOv8 algorithm (including the improved YOLOv8 algorithm), Mask R - CNN algorithm, SSD algorithm, and / or DETR algorithm are used to perform the deep - learning processing on the fused image to obtain a billet contour model.

[0073] Among them, YOLOv8 (You Only Look Once version 8) is a real-time object detection algorithm and one of the YOLO series of algorithms. It can complete object detection and instance segmentation in a single neural network, featuring high precision and high speed, and is suitable for real-time object detection tasks.

[0074] Mask R-CNN (Mask Region-Based Convolutional Neural Network) is a deep learning algorithm for object detection and instance segmentation. It generates high-quality segmentation masks while detecting objects and can accurately segment the boundaries of objects.

[0075] SSD (Single Shot MultiBox Detector) is a single-stage object detection algorithm with high detection speed and good detection accuracy. It predicts the object location and category in a single neural network and is suitable for real-time applications.

[0076] DETR (DEtection TRansformer) is an object detection model based on the Transformer architecture. It uses the attention mechanism to achieve end-to-end object detection and can maintain high detection accuracy in complex backgrounds.

[0077] Figure 2 It is a schematic diagram of multi-source image fusion and detection in an example of an embodiment of this application. In this example, the YOLOv8 algorithm is used for object detection. It is trained on a large amount of billet image data, including different angles, lighting, and environmental conditions, etc., to enhance the generalization ability of the model, and finally a billet contour model is obtained. That is, the network structure and training strategy of the YOLOv8 algorithm are optimized according to the billet characteristics, improving the accuracy and real-time performance of object detection and instance segmentation.

[0078] Furthermore, for the billet contour model obtained after performing the deep learning processing, in a preferred embodiment, the Douglas-Peucker algorithm and / or the least squares method are used to perform the polygon abstraction processing to obtain a polygon showing the edge features of the billet.

[0079] Figure 3Schematic diagram of polygon abstraction and bottom edge recognition of an example of an embodiment of the present application. The blue line is the roller path reference line, and the yellow edge line of the billet near the roller path is the lower bottom edge. In this example, the Douglas-Peucker algorithm is used to abstract the billet contour into a polygon and identify the lower bottom edge of the billet. Using the Douglas-Peucker algorithm, combined with the geometric features of the billet, the contour points of the instance segmentation are simplified and abstracted into a polygon, and then the algorithm tolerance parameters are set according to the standard shape of the billet to ensure that the polygon can accurately reflect the edge features of the billet. Finally, the lower bottom edge of the billet is identified from the polygon as the reference edge.

[0080] Among them, both the Douglas-Peucker algorithm and the least squares method are curve fitting algorithms, and the embodiments of the present application are not limited to the two. Moreover, in other examples, the lower bottom edge of the billet can also be identified by any one or more of the following in the embodiments of the present application:

[0081] 1. Use the Canny edge detection algorithm to detect the edge of the billet from the fused image, and perform line fitting through the Hough transform to obtain the lower bottom edge of the billet.

[0082] 2. Perform template matching on the fused image with a pre-established billet template to identify the lower bottom edge of the billet.

[0083] 3. Perform morphological operations on the fused image to identify the lower bottom edge of the billet. Among them, morphological operations such as erosion and dilation.

[0084] The lower bottom edge of the billet identified by these methods can also be used for subsequent calculation of the inclination angle of the billet.

[0085] Step S400, calculate the included angle between the lower bottom edge of the billet and the roller path reference line to obtain the inclination angle of the billet.

[0086] In a preferred embodiment, before calculating the included angle between the lower bottom edge of the billet and the roller path reference line, the cameras capturing the visible light image and the thermal imaging image are also calibrated for parameters and perspective transformed so that the calculation is performed in the world coordinate system. Among them, the included angle between the lower bottom edge of the billet and the roller path reference line can be directly calculated by conventional mathematical methods.

[0087] Figure 4It is a schematic diagram of perspective transformation and tilt angle calculation for an example of an embodiment of the present application. The blue line is the reference line of the roller table, and the yellow edge line of the billet close to the roller table is the lower base. In this example, first, camera calibration is performed to obtain internal and external parameters and calculate the homography matrix; then, perspective transformation is executed, that is, the image coordinate system is converted to the world coordinate system using the homography matrix to eliminate the influence of the camera's perspective. It should be noted that camera parameter calibration and perspective transformation are well-known image processing solutions in the art and will not be elaborated here. Additionally, in other examples, a perspective correction network based on deep learning can also be used to replace the traditional homography matrix calculation to automatically correct image distortion.

[0088] The above steps S100 - S400 obtain the tilt angle of the billet in a low-cost, high-efficiency, and real-time manner through image fusion, deep learning processing, and polygon abstraction processing, with strong applicability. Moreover, by real-time monitoring and measuring the tilt angle of the billet during operation, it can be determined whether it deviates from the normal posture to ensure the safety and smoothness of the production process.

[0089] After obtaining the tilt angle of the billet, the billet tilt detection method of the preferred embodiment of the present application may further include the following step S500.

[0090] Step S500, when the tilt angle of the billet exceeds the set tilt angle threshold, perform an alarm operation and / or a correction operation.

[0091] Among them, when the tilt angle of the billet exceeds the set tilt angle threshold, it is determined as tilt abnormality. The corresponding alarm operation may include triggering an audible and visual alarm to alert on-site personnel; the correction operation may include linking with the production line control system to automatically adjust the speed or posture of the roller table to correct the position of the billet; or transmitting the tilt abnormality information to the central control room through a remote operation interface so that the central control room can monitor the running state of the billet in real time and make adjustment decisions.

[0092] In another preferred embodiment, after obtaining the tilt angle of the billet, the billet tilt detection method may further include comparing the tilt angle of the billet with one or more of the following to determine the accuracy of tilt angle calculation: the tilt angle of the billet detected by a non-contact ranging sensor, where the non-contact ranging sensor includes a depth camera, a laser scanner, a millimeter-wave radar sensor, and / or an ultrasonic sensor; the tilt angle of the billet detected by an inertial sensor installed on the billet; the tilt angle of the billet detected by an inclination sensor installed on the roller table. For example, it may include comparison with the detection results of the following devices:

[0093] 1. Depth camera detection: Use a depth camera (such as a structured light or ToF camera) to obtain the depth information of the billet, construct 3D point cloud data, and directly calculate the spatial attitude and tilt angle of the billet.

[0094] 2. Laser scanner: Install a laser scanner above or on the side of the billet to obtain the cross-sectional shape of the billet, and judge the tilt by analyzing the cross-sectional contour. In addition, through the laser scanner, image recognition technology can be combined with laser ranging. Using the distance information provided by the laser, the size deviation in the image can be corrected to improve the accuracy of tilt calculation.

[0095] 3. Millimeter-wave radar sensor: Utilize the anti-interference ability of millimeter-wave radar in high-temperature and dusty environments to detect the position and tilt of the billet.

[0096] 4. High-temperature resistant inertial sensor (IMU) installed on the billet: Directly measure the tilt angle of the billet, and transmit the data wirelessly to the monitoring system.

[0097] 5. Tilt sensor installed on the roller table: Detect the tilt change of the roller table and indirectly infer the tilt of the billet.

[0098] 6. Ultrasonic sensor: On the basis of thermal imaging detection, add an ultrasonic sensor to detect the distance change between the billet and the roller table to judge the tilt.

[0099] In addition, in the example, a high-performance embedded computing device can be used to execute the above steps to improve the calculation efficiency.

[0100] In summary, the embodiments of the present application achieve low-cost, high-precision, and real-time detection of the tilt angle of the billet. Combining the above examples, it has at least the following advantages:

[0101] 1) Improve detection accuracy: Multi-source image fusion and the YOLOv8 algorithm improve the accuracy of billet detection and instance segmentation, ensuring accurate tilt angle calculation.

[0102] 2) Enhance real-time performance: The application of optimized algorithms and high-performance computing devices enables real-time processing of high-resolution images, rapid calculation of the billet tilt angle, timely detection of abnormalities, and meeting the real-time requirements of the production line.

[0103] 3) Reduce costs and maintenance difficulties: Compared with expensive and difficult-to-maintain laser ranging and mechanical detection equipment, the equipment used in the embodiments of the present application has lower costs, strong durability, and reduces maintenance frequency and costs.

[0104] 4) Strong adaptability: Adopt high-temperature industrial cameras and thermal imaging technology, etc., which can adapt to harsh environments such as high temperature and dust, and work stably and reliably.

[0105] 5) Improve safety: Automatic detection and alarm reduce the dependence on manual monitoring, lower the work intensity and safety hazards of personnel in high-temperature and high-risk environments.

[0106] 6) Promote production automation: It can be linked with the production line control system to achieve automatic tilt correction, improving the automation level and production efficiency of the production line.

[0107] 7) Enhance data value: It can store and analyze the billet operation data, providing valuable references for production management and equipment maintenance, helping to optimize the production process and extend the equipment life.

[0108] Through these advantages, the embodiments of the present application not only solve the technical problem of "the billet tilts during operation on the roller table, resulting in being unable to pass through the descaling box or even getting stuck at the entrance of the descaling box", but also provide a practical and reliable billet tilt detection solution, which is of great significance for improving the safety and efficiency of iron and steel production and helps the iron and steel production to develop towards automation and intelligentization.

[0109] Figure 5 It is a schematic structural diagram of the billet tilt detection device of the embodiments of the present application. As Figure 5 shown, the billet tilt detection device includes: a memory configured to store instructions; and a processor configured to call the instructions from the memory and be able to implement the billet tilt detection method of the above embodiments when executing the instructions.

[0110] Among them, the billet tilt detection device can be a controller, and the controller can be integrated in the production line control system, or configured in a computer in the central control room, or integrated in devices such as a server, PC, PAD, mobile phone, etc.

[0111] The processor contains a kernel, and the kernel retrieves the corresponding program unit from the memory. One or more kernels can be set, and the corresponding billet tilt detection method is executed by adjusting the kernel parameters.

[0112] The memory may include non-permanent memory in a computer-readable medium, in the form of random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash memory (flash RAM), and the memory includes at least one storage chip.

[0113] For more implementation details and effects of the billet tilt detection device, refer to the billet tilt detection device of the above embodiments, and no further elaboration will be made here.

[0114] Figure 6 It is a schematic structural diagram of the billet tilt detection system of the embodiments of the present application. As Figure 6As shown in the figure, the billet inclination detection system includes: an image acquisition device 100, which includes an industrial camera 110 and a thermal imaging camera 120 adapted to be installed at the billet position, and is respectively used to capture the visible light image and the thermal imaging image of the billet running on the roller table. The visible light image and the thermal imaging image include the overall view of the billet and the roller table reference line; and the billet inclination detection device 200 of the above embodiment, which is used to perform image fusion and image processing on the visible light image and the thermal imaging image captured by the image acquisition device, so as to identify the lower bottom edge of the billet, and calculate the included angle between the lower bottom edge of the billet and the roller table reference line to obtain the billet inclination angle.

[0115] Among them, the industrial camera 110 is preferably a high-temperature industrial camera.

[0116] Among them, for more implementation details of the billet inclination detection device 200, reference can be made to the foregoing embodiments, and details will not be elaborated here.

[0117] In a preferred embodiment, the billet inclination detection system further includes any one or more of the following that are linked with the billet inclination detection device 200: an alarm device 300, which is used to perform an alarm operation when the billet inclination angle exceeds a set inclination angle threshold; a communication module 400, which is used to notify the production line control system to adjust the roller table speed or posture to correct the billet position when the billet inclination angle exceeds the inclination angle threshold; a database module 500, which is used to store the obtained billet inclination angle and / or billet operation data; and a display device 600, which is used to display the obtained billet inclination angle and / or billet operation data in real time.

[0118] Among them, the alarm device 300 is, for example, an audible and visual alarm or other prompt alarm, which is used to remind the operator or the production line control system to perform abnormal processing; the communication module 400 is, for example, a communication interface circuit connected to the production line control system in communication, including a wired communication interface circuit and a wireless communication interface circuit; the database module 500 stores the billet inclination angle and related operation data, etc., for subsequent analysis; the display device 600 is, for example, a central control room display screen, which displays information such as the billet operation state and inclination angle in real time, and provides a friendly user interface.

[0119] Each part of the billet inclination detection system of the embodiment of the present application adopts a modular design, which is convenient for integration with the existing production line control system, and has good scalability and maintainability. Moreover, the integration of the billet inclination detection system of the embodiment of the present application and the production line control system can realize data sharing and functional linkage between the two, and when an abnormality is detected, it can automatically trigger the production line control system to perform correction.

[0120] An embodiment of the present application provides a machine-readable storage medium, on which instructions are stored, and the instructions are used to cause a machine to execute the billet inclination detection method of the above embodiment.

[0121] The present application also provides a computer program product, which is suitable for performing steps of initializing the above billet inclination detection method when executed on a data processing device.

[0122] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0123] The present application is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, and the combination of processes and / or blocks in the flowchart and / or block diagram can also be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for implementing the specified functions in one process Figure 1 one process or multiple processes and / or blocks Figure 1 or multiple blocks.

[0124] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device, and the instruction device implements the specified functions in one process Figure 1 one process or multiple processes and / or blocks Figure 1 or multiple blocks.

[0125] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the specified functions in one process Figure 1 one process or multiple processes and / or blocks Figure 1 or multiple blocks.

[0126] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0127] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0128] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0129] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A method for detecting inclination of a steel billet, characterized in that: include: Acquire a visible light image and a thermal imaging image of a steel billet running on a roller table, wherein the visible light image and the thermal imaging image include a full view of the steel billet and a reference line of the roller table; Performing image fusion on the visible light image and the thermal imaging image; For the fused image, deep learning processing and polygon abstraction processing are performed in sequence to identify the bottom edge of the billet; as well as The angle between the lower bottom edge of the billet and the roller reference line is calculated to obtain the billet inclination angle.

2. The method for detecting the inclination of a steel billet according to claim 1, characterized in that: The YOLOv8 algorithm, the Mask R-CNN algorithm, the SSD algorithm and / or the DETR algorithm are used to perform the deep learning processing on the fused image to obtain a steel billet contour model.

3. The method for detecting the inclination of a steel billet according to claim 1, characterized in that: For the steel billet contour model obtained after executing the deep learning process, the polygon abstraction process is performed using the Douglas-Peucker algorithm and / or the least squares method to obtain polygons showing the edge features of the steel billet.

4. The method for detecting the inclination of a steel billet according to claim 1, characterized in that: Before calculating the angle between the lower bottom edge of the steel billet and the roller reference line, the steel billet inclination detection method further includes: The camera that captures the visible light image and the thermal imaging image is calibrated and perspective transformed so that the calculation is performed in a world coordinate system.

5. The method for detecting the inclination of a steel billet according to claim 1, characterized in that: With respect to the fused image, the billet tilt detection method further includes identifying the lower bottom edge of the billet by any one or more of the following: Using the Canny edge detection algorithm to detect the edge of the steel billet from the fused image, and performing straight line fitting through Hough transform to obtain the bottom edge of the steel billet; Performing template matching on the fused image and a pre-established steel billet template to identify the lower bottom edge of the steel billet; as well as A morphological operation is performed on the fused image to identify the lower bottom edge of the steel billet.

6. The method for detecting the inclination of a steel billet according to any one of claims 1 to 5, characterized in that: After obtaining the billet inclination angle, the billet inclination detection method further comprises: When the inclination angle of the steel billet exceeds a set inclination angle threshold, an alarm operation and / or a corrective operation is performed.

7. The method for detecting the inclination of a steel billet according to any one of claims 1 to 5, characterized in that: After obtaining the billet inclination angle, the billet inclination detection method further comprises comparing the billet inclination angle with one or more of the following to determine the calculation accuracy of the inclination angle: The inclination angle of the steel billet is detected by a non-contact distance measurement sensor, wherein the non-contact distance measurement sensor comprises a depth camera, a laser scanner, a millimeter wave radar sensor and / or an ultrasonic sensor; The billet tilt angle detected by an inertial sensor mounted on the billet; The inclination angle of the billet is detected by the inclination sensor installed on the roller table.

8. A billet tilt detection device, characterized in that: include: a memory configured to store instructions; as well as A processor is configured to call the instructions from the memory and implement the billet tilt detection method according to any one of claims 1 to 7 when executing the instructions.

9. A billet tilt detection system, characterized in that: include: An image acquisition device, including an industrial camera and a thermal imaging camera adapted to be installed at the position of the steel billet, and used to capture a visible light image and a thermal imaging image of the steel billet running on the roller, respectively, wherein the visible light image and the thermal imaging image include the overall view of the steel billet and the roller reference line; as well as The billet inclination detection device described in claim 8 is used to perform image fusion and image processing on the visible light image and the thermal imaging image captured by the image acquisition device to identify the lower bottom edge of the billet, and calculate the angle between the lower bottom edge of the billet and the roller reference line to obtain the billet inclination angle.

10. The billet tilt detection system according to claim 9, characterized in that: The billet tilt detection system further includes any one or more of the following linked with the billet tilt detection device: An alarm device, used for executing an alarm operation when the inclination angle of the steel billet exceeds a set inclination angle threshold; A communication module, used for notifying the production line control system to adjust the roller speed or posture to correct the position of the billet when the billet inclination angle exceeds the inclination angle threshold; A database module, used for storing the obtained billet inclination angle and / or billet operation data; as well as The display device is used to display the obtained billet inclination angle and / or billet operation data in real time.

11. A machine-readable storage medium, characterized in that: The machine-readable storage medium stores instructions for enabling a machine to execute the billet tilt detection method according to any one of claims 1 to 7.

Images