Automatic steel rolling steel plate queue real-time tracking and measuring system based on photogrammetry

Abstract

The invention discloses an automatic steel rolling steel plate queue real-time tracking and measuring system based on photogrammetry, and relates to the technical field of machine vision and photogrammetry. The CMOS industrial camera set is electrically connected with the industrial computer set, the industrial computer set is in communication connection with the control terminal through the optical fiber ring network, and the CMOS industrial camera set comprises a plate shape measuring camera and a queue camera. The industrial computer unit comprises a plate shape measuring industrial computer, a queue industrial computer corresponds to the plate shape measuring camera and the queue camera respectively, and the CMOS industrial camera unit and the industrial computer unit are both installed on the top of a plant 20 meters above a roller way between a middle spraying system behind a blooming mill and a finishing mill. The industrial computer unit is internally provided with an intelligent visual processing system, a high-resolution CMOS camera is installed on the top of a plant, large target surface distortion calibration and an intelligent algorithm are combined, the system achieves full-field seamless coverage, and the risk that an existing thermal detection device is damaged due to the single-point discontinuous detection defect on a monitoring steel plate movement roller way and the environment high temperature is avoided.

Description

Technical Field

[0001] This invention relates to an automated real-time tracking and measurement system for steel plate queues in rolling mills based on photogrammetry, and relates to the fields of machine vision and photogrammetry. Background Technology

[0002] In hot rolling production lines for medium and heavy steel plates, dynamic tracking technology for steel plate queues mainly relies on high-temperature thermal sensors (thermal detection devices). Existing technologies detect the position of steel plates using thermal detection devices, but this has significant drawbacks: First, thermal detection devices are single-point detection methods, which cannot achieve seamless coverage of the entire field, resulting in blind spots and discontinuities in steel plate position tracking, affecting the continuity of automatic rolling. Second, thermal detection devices are installed on the side of the roller conveyor close to the steel plates, and the high-temperature environment (800-1000℃) can easily damage the devices, increasing maintenance costs and downtime risks. Third, this method cannot accurately measure the length and width of the steel plates, and can only roughly locate their positions. When the steel plates are connected end to end, they are easily misidentified as single steel plates, causing queue identification errors. These drawbacks limit the promotion of automatic rolling technology. Therefore, a real-time tracking and measurement system for automatic rolling steel plate queues based on photogrammetry is proposed to solve the problems existing in the current technology. Summary of the Invention

[0003] The purpose of this invention is to address the deficiencies or shortcomings of existing technologies by providing an automatic real-time tracking and measurement system for steel plate queues based on photogrammetry. By installing a high-resolution CMOS camera on the top of the factory building and combining it with large target distortion calibration and intelligent algorithms, the system achieves seamless full-field coverage, avoiding the single-point detection defects and high-temperature damage risks of existing thermal detection devices.

[0004] To achieve the above objectives, the present invention adopts the following technical solution: it includes a CMOS industrial camera group, an industrial computer group, and a control terminal. The CMOS industrial camera group and the industrial computer group are electrically connected. The industrial computer group is communicatively connected to the control terminal through a fiber optic ring network. The CMOS industrial camera group includes a plate shape measurement camera and a queue camera. The industrial computer group includes a plate shape measurement industrial computer and a queue industrial computer, which correspond to the plate shape measurement camera and the queue camera, respectively. Both the CMOS industrial camera group and the industrial computer group are installed on the top of the factory building at a height of 20 meters directly above the roller conveyor between the intermediate spray system after the primary rolling mill and the finishing rolling mill. An intelligent vision processing system is installed in the industrial computer group.

[0005] Furthermore, the intelligent vision processing system establishes a measurement plane coordinate system at the entrance of the finishing mill, with the length direction of the roller conveyor as the positive x-axis, and includes a steel plate micro-tracking module, a steel plate geometric dimension and shape measurement module, a data planning and communication module, and an equipment monitoring module.

[0006] Furthermore, the steel plate micro-tracking module identifies the position of the steel plate in real time through a multi-level exposure method, wherein the CMOS industrial camera group transmits images to the industrial computer group through the GigE Vision protocol.

[0007] Furthermore, the steel plate geometry and shape measurement module is based on large target distortion model calibration technology. It uses a customized calibration ruler (with circular markers spaced 0.5 meters apart) to calculate the three-dimensional coordinates of the steel plate. These three-dimensional coordinates are solved by the projection matrix M and satisfy the following equations:

[0008] Solve for M using the least squares method:

[0009]

[0010] Furthermore, the equipment monitoring module uses a temperature sensor to adjust the cooling unit of the protective enclosure in real time to maintain the internal temperature below 60°C.

[0011] Furthermore, the large target distortion model calibration technology precisely tailors the camera's field of view size based on the actual length and width of the steel plate running roller conveyor. It involves laying a calibration ruler with a dense array of precise calibration points on the roller conveyor and rationally dividing the calibration field into sections. Two-dimensional linear interpolation is performed within each section, effectively eliminating the difficulty of correcting large field-of-view distortion and significantly improving distortion correction accuracy.

[0012] Furthermore, the multi-stage exposure method employs an n-stage parameter combination E gm (e i g i m i The system takes turns acquiring image groups and transmits them in real time through the data planning and communication module.

[0013] Furthermore, the steel plate micro-tracking module executes a billet head-to-tail connection differentiation algorithm based on continuous images, and decomposes the steel plate when Li-1,k (k=1~m) is satisfied.

[0014] Furthermore, the steel plate micro-tracking module integrates roughing mill data and decomposes the steel plate when Di,k (k = 1 ~ m) is satisfied.

[0015] Furthermore, the measurement plane coordinate system has the origin o at the entrance of the finishing mill, the length direction of the roller table as the positive x-axis, and the width direction as the positive y-axis.

[0016] After adopting the above technical solution, the beneficial effects of the present invention are as follows: by installing a high-resolution CMOS camera on the top of the factory building, combined with large target distortion calibration and intelligent algorithms, the system achieves seamless full-field coverage, avoiding the single-point detection defects and high-temperature damage risks of existing thermal detection devices. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram of the layout of the CMOS industrial camera group and the industrial computer group in this invention;

[0019] Figure 2 This is a schematic diagram of the structural framework of the present invention;

[0020] Figure 3 This is a schematic diagram of the layout for the calibration of the length in this invention;

[0021] Figure 4 This is a schematic diagram illustrating the principle of the multi-stage exposure method in this invention;

[0022] Figure 5 This is a schematic diagram of slab tracking measurement in this invention;

[0023] Figure 6 This is a schematic diagram illustrating the principle of distinguishing and disassembling steel billets by connecting the head and tail based on continuous images in this invention;

[0024] Figure 7 This is a schematic diagram illustrating the principle of the method for distinguishing the head and tail of steel billets based on roughing information in this invention. Detailed Implementation

[0025] See Figures 1-7As shown, the technical solution adopted in this specific embodiment is as follows: It includes a CMOS industrial camera group, an industrial computer group, and a control terminal. The CMOS industrial camera group and the industrial computer group are electrically connected. The industrial computer group is communicatively connected to the control terminal through a fiber optic ring network. The control terminal is a terminal room, which houses a main control / data analysis industrial computer. The entire measurement process can be visualized and observed through a display screen, and global control can be performed. The CMOS industrial camera group includes a shape measurement camera and a queue camera. The industrial computer group includes a shape measurement industrial computer and a queue industrial computer, which correspond to the shape measurement camera and the queue camera, respectively. Both the CMOS industrial camera group and the industrial computer group are installed in the primary rolling mill. The CMOS industrial camera group and industrial computer group are both housed in a protective enclosure on the top of the factory building, 20 meters above the roller conveyor between the intermediate spray system after the primary mill and the finishing mill. The industrial computer group is equipped with an intelligent vision processing system. In this embodiment, taking a 42m×3m roller conveyor as an example, the actual length of the roller conveyor is 42m, and a 45m calibration ruler is set. For ease of description, the length of the calibration ruler is included in the length of the roller conveyor. To avoid high-temperature radiation and solve the problem of equipment interference, the CMOS industrial camera group and industrial computer group are set on the top of the factory building, 20 meters above the roller conveyor between the intermediate spray system after the primary mill and the finishing mill. The field of view covers the 42m×3m roller conveyor, which can effectively solve the problem of blind spots of thermal inspection devices.

[0026] The industrial computer group and the control terminal use fiber optic signal transmission. The 10Gbps bandwidth ensures real-time transmission of 20 million pixels at 30fps, ensuring the timeliness of data signal transmission and guaranteeing the accuracy of detection.

[0027] In this embodiment, an intelligent vision processing system is installed within the industrial computer group. Based on photogrammetry principles and the actual conditions of the medium plate mill, it utilizes the visual imaging and photogrammetry principles of a high-resolution CCD digital camera (CMOS industrial camera group) to achieve continuous, blind-angle-free, and precise position tracking of multiple steel plates. The position signals are then sent to the control system, enabling high-rhythm, fully automatic oscillation of the steel plates awaiting heating. Simultaneously, it completes the measurement of the geometric dimensions and shape of the steel plates during the finishing mill rolling process. Specifically:

[0028] The intelligent vision processing system establishes a measurement plane coordinate system at the entrance of the finishing mill, with the length of the roller conveyor as the positive x-axis. The starting point of the CMOS industrial camera group is also at the entrance of the finishing mill. Images of the position, shape, and size of the steel plates on the roller conveyor are captured and transmitted to the industrial computer group. Two sets of CMOS industrial cameras capture images from different positions and within different ranges, covering the detection / measurement area, thus enabling tracking and measurement of the steel plates. Specifically, the plate shape measurement camera starts at the entrance of the finishing mill, covering a length of 30 meters, mainly capturing data on single or multiple steel plates. The queue camera covers the entire roller conveyor, capturing the queue status of all steel plates on the roller conveyor. In the capturing process, the position of the steel plates is first tracked to determine the position of the steel plates. The system identifies the plate position, including the head and tail positions, and also recognizes the billet outline. All lines are composed of discrete pixels. To accurately measure the length and width of rolled and queued steel plates, high-precision calibration of the system's two CMOS industrial cameras is necessary. To improve calibration accuracy, this system employs a large-target distortion model calibration technique. First, a 45-meter-long rigid calibration ruler with circular markers spaced 0.5 meters apart is laid flat on the roller conveyor surface at 0.5-meter intervals to form a dot matrix. Then, two CMOS industrial cameras acquire images, and the camera field of view is precisely trimmed according to the actual length and width of the steel plate running roller conveyor. By laying a calibration ruler with a dense, precise calibration dot matrix on the roller conveyor and rationally dividing the calibration field, two-dimensional linear interpolation is performed within each segmented small interval. This effectively overcomes the low correction accuracy of large-field-of-view distortion and greatly improves distortion correction accuracy.

[0029] By solving for the two-dimensional linear interpolation grid of each segmented interval, the coordinates of the steel plate in the camera coordinate system are determined. Each coordinate point corresponds to a spatial point on the two-dimensional linear interpolation grid between the segments, thus its coordinates in the world coordinate system are known. This allows the coordinates of the steel plate in the world coordinate system to be obtained through matrix equations. Therefore, after capturing an image of the steel plate aligned with the calibration points, the coordinates of any point on the steel plate can be derived from the calibration points combined with the coordinate system of the CMOS industrial camera group itself.

[0030] The above calibration and coordinate point calculation are prerequisites for steel plate tracking measurement. After calibration, the equipment can be turned on to carry out actual measurement.

[0031] More specifically, the intelligent vision processing system includes a steel plate micro-tracking module, a steel plate geometry and shape measurement module, a data planning and communication module, and an equipment monitoring module. The steel plate micro-tracking module tracks the position of the steel plate, the steel plate geometry and shape measurement module collects the tracking data and determines the size and shape of the steel plate, the data planning and communication module classifies the data and formulates measurement strategies, and distributes the strategy to each execution terminal to realize tracking measurement, and the equipment monitoring module monitors the operating status of the camera group and computer group, mainly by setting temperature sensors to monitor the equipment temperature, and the test and control module in the control system tracks the monitoring situation and alarms when abnormal temperature occurs.

[0032] More specifically, the steel plate micro-tracking module identifies the position of the steel plate in real time through a multi-level exposure method, wherein the CMOS industrial camera group uses GigE... The Vision protocol transmits images to an industrial computer group. First, the position of the steel plate is tracked. After calibration, the queue cameras can identify the steel plate and locate its position (head and tail). A measurement plane coordinate system o-xy is established on the roller conveyor for the queue cameras. This coordinate system has its origin o at the mill entrance, with the length direction of the roller conveyor as the positive x-axis, the width direction as the positive y-axis, and the width direction of the rolled plate as the upward y-axis. The distance between adjacent calibration points is w, and the distance between calibration rulers in the width direction is h. Therefore, the coordinates of the i-th and j-th calibration points are (xi = w0 + w*i, yi = h*3 - j - h0) (i = 0 to m, j = 0, n). The positions of the calibration points are precisely located, and the calibration rulers are also precisely measured. The calibration rulers cover the entire roller conveyor in the width direction and the entire measurement field of view in the length direction according to measurement and tracking needs. A region of interest (ROI) is set on the camera, which helps reduce the amount of data transmitted and improve the image acquisition frame rate. Then, the calibration images of the camera are acquired, and the pixel position coordinates Pi,j(ci,rj) of each calibration point are identified and located. The measurement plane coordinate system position Qi,j(xi,yj) of each calibration point is paired with the image pixel position Pi,j(ci,rj), which constitutes the mapping relationship between the calibration point and the image pixel position to the measurement plane coordinate system position.

[0033] To establish the mapping relationship between any pixel in the image and its position in the measurement plane coordinate system, we can find the four nearest calibration points to that pixel. Let the pixel coordinates of the four nearest calibration points to pixel p(c,r) be Pi,j(ci,rj), Pi+1,j(ci+1,rj), Pi,j+1(ci,rj+1), and Pi,j+1(ci+1,rj+1), and their measurement coordinate system coordinates be Qi,j(xi,yj), Qi+1,j(xi+1,yj), Qi,j+1(x,yj+1), and Qi+1,j+1(xi+1,yj+1). Since points P and Q are planar points, there exists a homography matrix H3×3 such that:

[0034] Q′=HP′ (P′ and Q′ are the homogeneous coordinates of points P and Q respectively). Since P′ and Q′ are known, the position q(x,y) of the measurement coordinate system of the point can be obtained by q=Hp. Then, the true size and position of the outer contour line of the billet can be obtained by using the collected pixel coordinates through this calculation result.

[0035] For each steel billet, its minimum and maximum x-axis values ​​are extracted to obtain the billet's head and tail positions. The detected billets are numbered 1, 2, ..., n from smallest to largest position, and a multi-level exposure method is used to acquire billet images.

[0036] The multi-level exposure method uses n-level parameter combinations E gm (e i g i m i The system acquires image groups in rotation and transmits them in real time through the data planning and communication module. The temperature range of the steel billet is from over 800 degrees Celsius to over 1000 degrees Celsius. In order to detect the steel billet from the image, a multi-level exposure method is adopted. The exposure time (e), gain (g), and gamma (m) parameters of n levels are combined into an exposure parameter Egm, where Egmi(ei,gi,mi) (i=0~n-1). Each time the exposure parameter Egmi is set, one frame of image is acquired; then the next level of exposure parameter Egmi+1mod n is set, and another frame of image is acquired. That is, the camera images are acquired in rotation using n levels of exposure parameters. In this way, images with low exposure parameters are easy to detect high-temperature steel billets, while images with high exposure parameters are easy to detect steel billets with lower temperatures.

[0037] Each time, the n consecutive exposure images from low to high are regarded as a group, and the presence of steel billets is detected in each of these n images. If the same steel billet is detected in the low exposure image, for example, steel billet 1, 1′ and 1″ are the same steel billet detected in multiple images, then steel billet 1′ and steel billet 1″ detected at the same position in the high exposure image will no longer be counted.

[0038] More specifically, the steel plate geometry and shape measurement module is based on large target distortion model calibration technology. It uses a customized calibration ruler (with circular markers spaced 0.5 meters apart) to calculate the three-dimensional coordinates of the steel plate. These three-dimensional coordinates are solved through the projection matrix M and satisfy the following equations:

[0039] Solve for M using the least squares method:

[0040]

[0041] In this embodiment, the steel plate geometry and shape measurement module also uses the large target surface distortion model calibration technology. Since the calibration points have already been used, Z... w The constraint can be set to 0, where u and v are pixel coordinates. The values ​​are then substituted into the projection matrix equation for Z. w =0, and solve the system of linear equations to output the spatial coordinates (X) of the corner point of the steel plate. w Y w This allows for the acquisition of high-precision steel plate geometry.

[0042] More specifically, the equipment monitoring module uses a temperature sensor to adjust the cooling unit of the protective enclosure in real time to maintain the internal temperature below 60°C.

[0043] More specifically, the large target distortion model calibration technology precisely cuts the camera's field of view size according to the actual length and width of the steel plate running roller. By laying a calibration plate with a dense array of precise calibration points on the roller, and by reasonably dividing the calibration field into small sections, two-dimensional linear interpolation is performed in each small section. This effectively eliminates the difficulty of correcting large field of view distortion and greatly improves the accuracy of distortion correction.

[0044] More specifically, the steel plate micro-tracking module executes a continuous image-based algorithm to distinguish between connected steel billets. When Li-1,k (k = 1 to m) is satisfied, the steel plate is decomposed. When multiple steel billets move on the roller conveyor, they often connect end to end and stick together. This situation will cause the image to detect one steel billet. This can be decomposed into two pieces by analyzing continuous images, i-1 and i-1 consecutive images. In the i-1 group of images, two steel billets can be correctly identified, while in the i-1 group of images, because the two pieces are connected end to end, they are identified as one steel billet. When this occurs, the length of billet 1 in group i is approximately equal to the sum of the lengths of billet 1 and billet 2 in group (i-1). Based on this condition, it can be decomposed into two billets. Let the image in group (i-1) detect a total of m billets, each with a length of Li-1,k (k = 1 to m); and the image in group i detect a total of n billets, each with a length of Li,j (j = 1 to n). The judgment and decomposition method is as follows:

[0045] start:

[0046] forj = 1, 2, ..., n

[0047] fork = m, m-1, ..., j+1

[0048] If Li,j is approximately equal to s=jkLi-1,s

[0049] The j-th billet in the i-th image group is decomposed into mj billets according to the length ratio of the j+1 to m billets in the (i-1)-th image group. After the decomposition is completed, return to the "start" position to execute.

[0050] The above decomposition method can solve the problem of multiple steel billets being connected end to end.

[0051] More specifically, the steel plate micro-tracking module integrates roughing mill data. When Di,k (k = 1 to m) is satisfied, the steel plate is decomposed. In this embodiment, a method for distinguishing the connected ends of the steel billets based on roughing mill information is also adopted. When the steel billet enters the camera's field of view, it is already connected end-to-end, so other information is needed for decomposition. This can utilize the steel billet data from the roughing mill. It is known that during the detection of the i-th image group, the lengths of the queued steel billets are Di,k (k = 1 to m). The judgment and decomposition method is as follows:

[0052] start:

[0053] forj = 1, 2, ..., n

[0054] fork = m, m-1, ..., j+1

[0055] If Li,j is approximately equal to s=jkDi,s

[0056] The j-th billet in the i-th image group is decomposed into mj billets based on the length ratio of the j+1 to m billets from the roughing mill at the time of the i-th image group. After the decomposition is completed, return to the "start" position to execute.

[0057] The above method can solve the problem of multiple steel billets being connected end-to-end as soon as they enter the queuing camera's field of view.

[0058] The above description is only used to illustrate the technical solution of the present invention and is not intended to limit it. Any other modifications or equivalent substitutions made by those skilled in the art to the technical solution of the present invention, as long as they do not depart from the spirit and scope of the technical solution of the present invention, should be covered within the scope of the claims of the present invention.

Claims

1. A real-time tracking and measurement system for automated steel plate queues based on photogrammetry, characterized in that: It includes a CMOS industrial camera group, an industrial computer group, and a control terminal. The CMOS industrial camera group and the industrial computer group are electrically connected. The industrial computer group is connected to the control terminal through a fiber optic ring network. The CMOS industrial camera group includes a plate shape measurement camera and a queue camera. The industrial computer group includes a plate shape measurement industrial computer and a queue industrial computer, which correspond to the plate shape measurement camera and the queue camera, respectively. Both the CMOS industrial camera group and the industrial computer group are installed on the top of the factory building, 20 meters above the roller conveyor between the intermediate spray system after the primary rolling mill and the finishing mill. The industrial computer group is equipped with an intelligent vision processing system.

2. The automatic steel plate queue real-time tracking and measurement system based on photogrammetry according to claim 1, characterized in that: The intelligent vision processing system establishes a measurement plane coordinate system at the entrance of the finishing mill, with the length direction of the roller conveyor as the positive x-axis. It includes a steel plate micro-tracking module, a steel plate geometric dimension and shape measurement module, a data planning and communication module, and an equipment monitoring module.

3. The automatic steel plate queue real-time tracking and measurement system based on photogrammetry according to claim 2, characterized in that: The steel plate micro-tracking module identifies the position of the steel plate in real time through a multi-level exposure method, wherein the CMOS industrial camera group transmits images to the industrial computer group through the GigE Vision protocol.

4. The automatic steel plate queue real-time tracking and measurement system based on photogrammetry according to claim 2, characterized in that: The steel plate geometry and shape measurement module is based on large target distortion model calibration technology. It uses a customized calibration ruler (with circular markers spaced 0.5 meters apart) to calculate the three-dimensional coordinates of the steel plate. These three-dimensional coordinates are solved by the projection matrix M and satisfy the following equations: Solve for M using the least squares method:

5. The automatic steel plate queue real-time tracking and measurement system based on photogrammetry according to claim 2, characterized in that: The equipment monitoring module uses a temperature sensor to adjust the cooling unit of the protective enclosure in real time to maintain the internal temperature below 60°C.

6. The automatic steel plate queue real-time tracking and measurement system based on photogrammetry according to claim 4, characterized in that: The large target distortion model calibration technology is based on the actual length and width of the steel plate running roller to precisely cut the camera's field of view size. By laying a calibration plate with a dense array of precise calibration points on the roller, and by reasonably dividing the calibration field into small sections, two-dimensional linear interpolation is performed in each small section, thereby effectively eliminating the difficulty of correcting large field of view distortion.

7. The automatic steel plate queue real-time tracking and measurement system based on photogrammetry according to claim 3, characterized in that: The multi-level exposure method described above uses an n-level parameter combination E gm (e i ,g i ,m i The system takes turns acquiring image groups and transmits them in real time through the data planning and communication module.

8. The automatic steel plate queue real-time tracking and measurement system based on photogrammetry according to claim 2, characterized in that: The steel plate micro-tracking module executes a billet head-to-tail connection differentiation algorithm based on continuous images, and decomposes the steel plate when Li-1,k (k=1~m) is satisfied.

9. The automatic steel plate queue real-time tracking and measurement system based on photogrammetry according to claim 8, characterized in that: The steel plate micro-tracking module integrates roughing mill data and decomposes the steel plate when Di,k (k=1~m) is satisfied.

10. A real-time tracking and measurement system for automatic rolling mill steel plate queues based on photogrammetry according to claim 2, characterized in that: The measurement plane coordinate system has the origin o at the entrance of the finishing mill, the length direction of the roller table as the positive x-axis, and the width direction as the positive y-axis.

Images