Strip coil side guide plate spark detection method based on image recognition

Abstract

The application discloses a strip steel coiling time side guide plate spark detection method based on image recognition, which comprises the following steps: 1, an image collector (8) is arranged above the side guide plate, the image collector faces the strip steel (2) and the side guide plate, and is connected with a control system through a controller (7); 2, the image collector sends images to the controller, the controller converts the images into pixel images, and initializes a time domain median background model; 3, whether the image serial number is continuous is determined, if yes, step 4 is executed, and if not, the step 1 is returned; 4, a spark connected domain is extracted, and the actual width of the spark is calculated according to the spark connected domain; 5, whether the time domain median background model is updated is determined, if yes, step 6 is executed, and if not, step 7 is executed; 6, the time domain median background model is updated; 7, a spark recognition result is output; 8, whether the spark detection program is exited is determined, if yes, the detection is ended, and if not, the step 1 is returned. The application can recognize the spark condition between the side guide plate and the strip steel through image collection.

Description

TECHNICAL FIELD

[0001] The present application relates to a hot strip production control method, in particular to a strip side guide plate spark detection method based on image recognition during coiling. BACKGROUND

[0002] In the process of hot continuous rolling and coiling of the strip, the control of the side guide plate is directly related to the edge quality and the coil shape quality of the strip. The control system of the prior art mainly controls the opening degree of the side guide plate by means of short stroke preset or pressure feedback control. Please refer to the attached drawings. Figure 1 Taking the first coiler 51 as an example, the control process of the first side guide plate 31 is as follows: (1) after the head of the strip 2 exits the finishing mill F3 rack 12, the opening degree of the first side guide plate 31 is controlled to be W1; (2) after the head of the strip 2 reaches the laser detector 6, the opening degree of the first side guide plate 31 is controlled to be W2; (3) after the head of the strip 2 reaches the first coiling pinch roll 41, the opening degree of the first side guide plate 31 is controlled to be W3; (4) when the tail of the strip 2 exits the finishing mill F1 rack 11, the opening degree of the first side guide plate 31 is controlled to be W4; (5) when the tail of the strip 2 exits the finishing mill F7 rack 13, the opening degree of the first side guide plate 31 is controlled to be W5; (6) when the tail of the strip 2 reaches the front X meters of the side guide plate 31, the opening degree of the first side guide plate 31 is controlled to be W6.

[0003] In the whole control process, the operator needs to monitor the spark size of the side guide plate in real time through the monitor. If the opening degree of the side guide plate is controlled too small, the spark between the side guide plate and the strip is very large, which on the one hand is easy to cause serious edge damage of the strip, even to cause the strip to be stuck, and on the other hand will cause local wear of the side guide plate, shorten its service period. If the opening degree of the side guide plate is controlled too large, it will cause problems such as coil tower shape and layering. The root cause of the above problems is that after the coiling pinch roll bites the strip, it is difficult to ensure that the pressure applied by the side guide plate to both sides of the strip is symmetrical and moderate in size, and in the subsequent coiling process, the strip can always run along the front center line.

[0004] Chinese patent application CN201410695817 discloses a method for establishing a mathematical model of the optimal control pressure of the hot rolling coiler side guide plate, including the following steps: 1) input and output parameters of the mathematical model; 2) generation process of the mathematical model of the hot rolling coiler side guide plate control pressure; 3) coiling calculation structure model; 4) mathematical model of the lateral offset of the strip in the coiling process; 5) mathematical model of the coiler side guide plate pressure. This method adopts different control pressures for different specifications of the strip, but cannot know whether sparks are generated between the strip and the side guide plate during coiling and the size of the sparks, which is not conducive to the control of the production quality and the coiling quality of the strip. SUMMARY

[0005] The purpose of the present application is to provide a kind of based on image recognition's side guide plate's spark detection method when strip steel is coiled, can be identified by image acquisition the spark situation between side guide plate and strip steel.

[0006] The present application is realized as follows:

[0007] A kind of based on image recognition's side guide plate's spark detection method when strip steel is coiled, comprising the following steps:

[0008] Step 1: image collector is set up above each pair of side guide plate, and image collector is set to a pair of side guide plate and strip steel;The output end of image collector is connected with the input end of controller respectively, and the output end of controller is connected to control system;

[0009] Step 2: image collector sends the image collected to controller, and controller converts image into pixel graph according to proportion, and initializes time domain median background model;

[0010] Step 3: controller verifies whether image serial number is continuous, if yes, then execute step 4, if not, then write frame loss digital signal to controller's analog-digital conversion card, and return to step 1;

[0011] Step 4: extract spark connected domain on pixel graph, and calculate the actual width of spark according to spark connected domain;

[0012] Step 5: controller judges whether to update time domain median background model, if yes, then execute step 6, if not, then execute step 7;

[0013] Step 6: update time domain median background model;

[0014] Step 7: controller outputs spark recognition result, and writes the image collected by image collector into video;

[0015] Step 8: judge whether to exit spark detection, if yes, then end detection, if not, then return to step 1 and continue to detect spark.

[0016] In the step 1, the vertical distance between image collector and roller is 2-5m, the horizontal distance between image collector and the upstream end of side guide plate parallel section is 2-10m, and the length of side guide plate parallel section is 4-8m.

[0017] In the step 1, controller judges the working state of all image collectors, including the following steps:

[0018] Step 1.1: controller reads the working signal of each image collector, and judges whether it is in working state according to the working signal of each image collector, if all image collectors are in non-working state, then execute step 1.2, if one of image collectors is in working state, then execute step 1.3.

[0019] Step 1.2: unload the time domain median background model;

[0020] Step 1.3: the controller determines whether the image collector in the working state is disconnected from the network, if yes, step 1.4 is executed, if not, step 1.5 is executed;

[0021] Step 1.4: the controller writes a network disconnection digital signal, and the image collector re-connects to the network;

[0022] Step 1.5: the controller displays the image collected by the image collector in the working state in real time through the interface;

[0023] Step 1.6: the controller reads the strip temperature signal, adjusts the working parameters of the image collector according to the strip temperature signal, and returns to step 1.1.

[0024] The step 2 comprises the following steps:

[0025] Step 2.1: a world coordinate system is established based on the width and center distance of the roller, and the corner points of the side guide plate region to be detected are selected in the image, and the four corner points form a quadrilateral detection region image;

[0026] Step 2.2: the length and width of the overhead pixel map are set, and the ratio between the pixel map and the detection region image is calculated;

[0027] Step 2.3: the homography matrix of image transformation into the pixel map is calculated, and the image is converted into the pixel map according to the homography matrix;

[0028] Step 2.4: the median of the gray values of the continuous three frames of pixel maps is used as the background to establish a time domain median background model.

[0029] The step 4 comprises the following steps:

[0030] Step 4.1: the background in the pixel map is removed by the time domain median background model to obtain a foreground image, and the foreground image is thresholded to obtain a binary image;

[0031] Step 4.2: a maximum entropy threshold is set, and the binary image is locally thresholded by the maximum entropy threshold to obtain a binary image with a spark region;

[0032] Step 4.3: the geometric features of the spark region are calculated to form a spark connected domain;

[0033] Step 4.4: the actual width of the spark is calculated according to the ratio and the spark connected domain.

[0034] The method for calculating the spark connected domain comprises:

[0035] Step 4.3.1: apply a memory space as a stack, scan the binary image with spark area, and find the pixel point I(u, v) = 1;

[0036] Step 4.3.2: mark the pixel point found in step 4.3.1 as X, and stack all adjacent pixel points of the pixel point;

[0037] Step 4.3.3: pop the top pixel point of the stack, also mark the top pixel point as X, and stack all adjacent pixel points of the top pixel point;

[0038] Step 4.3.4: repeat step 4.3.3 until the stack is empty;

[0039] Step 4.3.5: traverse the image, repeat steps 4.3.1-4.3.4 until all "hole" pixels in the spark area are marked, forming a spark connected domain.

[0040] The step 6 comprises the following steps:

[0041] Step 6.1: set the image buffer size to A frames of images, the sampling interval to D frames of images, and the update frequency to F frames of images;

[0042] Step 6.2: the image collector sends the collected images to the controller in real time, and the controller buffers the received images to form an image sequence with a size of A;

[0043] Step 6.3: the controller deletes the earliest D frames of images in the image sequence in time sequence every D frames of images, and adds the newly obtained D frames of images to the tail of the buffered sequence;

[0044] Step 6.4: update the time domain median background model every F frames of images.

[0045] In step 6.4, the update method of the time domain median background model is: calculate the median of a coordinate point in A frames of images in the buffered image sequence as the gray value of the coordinate point in the time domain median background model, and traverse the A frames of images in the buffered image sequence coordinate by coordinate to complete the update of the time domain median background model.

[0046] In step 7, the controller sets a video capacity threshold, and if the actual video capacity reaches the video capacity threshold, a new video is created.

[0047] In step 7, the spark recognition result includes the number of working side sparks, the number of transmission side sparks, the fault state, the actual width of the working side spark, and the actual width of the transmission side spark.

[0048] Compared with the prior art, the present application has the following beneficial effects:

[0049] 1、The present application is equipped with an image collector, which can collect the images of a pair of side guides and a strip, and process the images through a controller to display the spark connected domain, for accurate identification of the spark state, without the need for manual monitoring during the strip coiling process, improving the automation level of spark monitoring, and facilitating timely, effective and accurate adjustment of the control mode of the side guide, thereby ensuring the coiling quality of the strip.

[0050] 2、The present application is equipped with a controller, which can capture images in real time during the movement of the strip and identify sparks, and output the number of sparks on the working side and the transmission side of the side guide, and accurately calculate the actual size of the sparks through a pixel image, which is beneficial to the judgment of the spark degree and provides reference data for the automatic adjustment of the side guide, thereby minimizing the wear of the side guide and improving the edge quality of the strip.

[0051] The present application is based on the real-time collection of images of the side guide and the strip during the strip coiling process by an image collector, and accurately identifies the number and size of sparks between the side guide and the strip through images, providing reliable data for the automatic adjustment control of the side guide, which is beneficial to improving the coiling quality and edge quality of the strip, reducing the wear of the side guide, and greatly reducing the work load of manual monitoring. BRIEF DESCRIPTION OF DRAWINGS

[0052] Figure 1 is the front view of a hot continuous rolling coiling production line in the prior art;

[0053] Figure 2 is the front view of a hot continuous rolling coiling production line in the spark detection method of the side guide during the coiling of the strip based on image recognition of the present application;

[0054] Figure 3 is the flowchart of the spark detection method of the side guide during the coiling of the strip based on image recognition of the present application;

[0055] Figure 4 is the flowchart of step 1 in the spark detection method of the side guide during the coiling of the strip based on image recognition of the present application;

[0056] Figure 5 is the flowchart of step 2 in the spark detection method of the side guide during the coiling of the strip based on image recognition of the present application;

[0057] Figure 6 is the time domain median background model graph of initialization in embodiment 1 of the spark detection method of the side guide during the coiling of the strip based on image recognition of the present application;

[0058] Figure 7 is the foreground image in embodiment 1 of the spark detection method of the side guide during the coiling of the strip based on image recognition of the present application;

[0059] Figure 8It is the binary image in the spark detection method for side guide plate during strip coiling based on image recognition embodiment 1 of the application;

[0060] Figure 9 It is the binary image with spark area in the spark detection method for side guide plate during strip coiling based on image recognition embodiment 1 of the application;

[0061] Figure 10 It is the pixel image of spark connected domain in the spark detection method for side guide plate during strip coiling based on image recognition embodiment 1 of the application;

[0062] Figure 11 It is the spark number schematic diagram in the spark detection method for side guide plate during strip coiling based on image recognition embodiment 1 of the application; wherein, the straight line is the transmission side spark number, and the dotted line is the working side spark number;

[0063] Figure 12 It is the spark actual width schematic diagram in the spark detection method for side guide plate during strip coiling based on image recognition embodiment 1 of the application; wherein, the straight line is the transmission side spark actual width, and the dotted line is the working side spark actual width.

[0064] In the figure, 11 is a finishing F1 rack, 12 is a finishing F3 rack, 13 is a finishing F7 rack, 2 is a strip, 31 is a first side guide plate, 32 is a second side guide plate, 41 is a first coiling pinch roll, 42 is a second coiling pinch roll, 51 is a first coiler, 52 is a second coiler, 6 is a laser detector, 7 is a controller, and 8 is an image collector. DETAILED DESCRIPTION

[0065] The application will be further described below in combination with the drawings and specific embodiments.

[0066] Please refer to the accompanying Figure 3 A spark detection method for side guide plate during strip coiling based on image recognition, comprising the following steps:

[0067] Please refer to the accompanying Figure 2 Step 1: An image collector 8 is arranged above the upstream of each pair of side guide plates along the movement direction of the strip 2, and the image collector 8 faces a pair of side guide plates and the strip 2, for collecting the image of the pair of side guide plates; the output end of the image collector 8 is connected with the input end of the controller 7 respectively, the image collector 8 sends the collected image to the controller 7, and the output end of the controller 7 is connected to the control system (not shown in the figure). Preferably, the controller 7 can adopt an industrial PLC (Programmable Logic Controller, i.e. programmable logic controller)

[0068] A pair of side guides are arranged on both sides of the roller way and contact the edge of the strip 2; among the pair of side guides, the side guide on one side of the strip 2 is a working side guide, and the side guide on the other side of the strip 2 is a transmission side guide. The side guide is composed of a slanting section and a parallel section, the slanting section is connected with the parallel section, and the slanting section forms a funnel-shaped opening structure at one end of the pair of side guides. The strip 2 enters between the pair of side guides from the funnel-shaped opening structure through the roller way, and then is sent to the coiler by the coiling pinch roll after passing through the parallel section.

[0069] If two sets of strip coiling equipment are arranged on the hot continuous rolling coiling production line, i.e. a pair of first side guides 31, a first coiling pinch roll 41 and a first coiler 51 as one set of strip coiling equipment, and a pair of second side guides 32, a second coiling pinch roll 42 and a second coiler 52 as another set of strip coiling equipment, the image collector 8 is arranged above the upstream of the pair of first side guides 31 and the pair of second side guides 32, and all the image collectors 8 are connected to the controller 7 for sending the collected images to the controller 7, and the controller 7 outputs the results to the control system after spark recognition processing of the images, for controlling the opening degree of the side guide.

[0070] The vertical distance between the image collector 8 and the roller way is 2-5 m, the horizontal distance between the image collector 8 and the upstream end of the parallel section of the side guide (the end of the parallel section of the side guide through which the strip 2 passes first is defined as the upstream end according to the movement direction of the strip 2) is 2-10 m, and the length of the parallel section of the side guide is 4-8 m, so that the shooting field of view of the image collector 8 can cover the entire domain of the pair of side guides. The appropriate image collector 8 can be selected according to the image collection requirement, and the image collector 8 is preferably a CCD (Charge-coupled Device, i.e. Charge-coupled Device) camera. The corresponding speed of the CCD camera is ≤100 ms, and the image collection speed of the CCD camera is 25-150 frames / s. The CCD camera is connected to the optical transmitter through twisted pair, the optical transmitter is connected to the optical receiver through optical fiber, and the optical receiver is connected to the controller 7 through twisted pair, so as to realize the transmission of the image.

[0071] Preferably, in order to avoid the influence of water mist on the shooting clarity of the CCD camera to the greatest extent, the roller way in the shooting area of the CCD camera is adopted as an internal water-cooled roller way.

[0072] Please refer to the accompanying drawings Figure 4 In the step 1, the controller 7 judges the working state of all the image collectors 8, including the following steps:

[0073] Step 1.1: The controller 7 reads the working signal of each image collector 8 through an analog-to-digital conversion card, and determines whether each image collector 8 is in a working state according to the working signal of each image collector 8. If all image collectors 8 are in a non-working state, step 1.2 is performed, and if one of the image collectors 8 is in a working state, step 1.3 is performed. The analog-to-digital conversion card can use an NI card, and the production company of the analog-to-digital conversion card is national instruments, so it is called an NI card.

[0074] Step 1.2: Unload the time domain median background model.

[0075] The image collector 8 may be interrupted due to factors such as vibration and temperature. Since the spark detection method of the present application is in a cyclic working mode during the coiling process of the strip steel, it is necessary to judge the network state of the image collector 8 to prevent the image collector 8 from being in a working state and being unable to determine the working state due to network interruption, and to prevent the image collector 8 from being in a working state and collecting images due to the existence of the time domain median background model. At the same time, the original time domain median background model needs to be unloaded, as the time domain median background model established for different batches and specifications of strip steel may be different. Unloading the time domain median background model sets all pixel gray values in the time domain median background model to 0, and then re-establishes the time domain median background model according to the image collected by the image collector 8 in a working state, so as to avoid misjudgment of sparks.

[0076] Step 1.3: The controller 7 determines whether the image collector 8 in a working state is interrupted, and if so, step 1.4 is performed, and if not, step 1.5 is performed.

[0077] Step 1.4: The controller 7 writes a network interruption digital signal (00001000, i.e. hardware failure) to the analog-to-digital conversion card, and the image collector 8 re-connects to the network through a network interruption restart module.

[0078] Step 1.5: The controller 7 displays the image collected by the image collector 8 in a working state in real time through an interface.

[0079] Step 1.6: The controller 7 reads the strip steel temperature signal through the analog-to-digital conversion card, and adjusts the working parameters of the image collector 8 according to the strip steel temperature signal, and returns to step 1.1.

[0080] The operation of the image collector 8 is in a cyclic mode, and if the main program of the spark detection does not exit, it returns to step 1.1 after adjusting the working parameters of the image collector 8. The working state check of the image collector 8 can be realized by a sub-thread, and is synchronized with the main program to improve the running efficiency of the entire flow program.

[0081] Preferably, the temperature of the center of the strip steel 2 can be measured by a temperature measuring instrument, and the temperature signal can be sent to an analog-digital conversion card. The working parameters of the image collector 8 can be adjusted according to the type and characteristics of the equipment, for example, when the image collector 8 is a CCD camera and the temperature of the center of the strip steel 2 measured by the temperature measuring instrument is in the range of 300-900℃, the exposure time of the CCD camera is adjusted to be 36000-2000 milliseconds.

[0082] To meet the real-time requirement, the video storage adopts a multi-thread mode. The CCD camera has a single producer and multiple consumer read-write model, and a cache space pre-opened for storing images. The producer index and the consumer index respectively point to the image to be added to the cache next frame and the image to be taken out from the cache next frame. The working thread of the CCD camera will sequentially add images to the cache space as the producer according to the producer index, and multiple consumers will take images from the cache space according to the consumer index and write them into the video. When the producer thread finds that the producer index is equal to the consumer index minus one, it is considered that the cache space is full, and the writing into the cache will be stopped, waiting for the consumer to end. When the consumer thread finds that the producer index is equal to the consumer index, it is considered that the cache space is empty, and the taking out of images will be stopped, waiting for the producer to end.

[0083] The controller 7 is provided with multiple signal channels, including: an analog input channel Ai0 for transmitting the temperature signal of the strip steel; an analog input channel Ai1 for transmitting the working signal of the 1# camera (i.e. the CCD camera facing the first side guide plate 31); an analog input channel Ai2 for transmitting the working signal of the 2# camera (i.e. the CCD camera facing the second side guide plate 32); an analog output channel Ao1 for outputting the actual maximum width of the driving side spark; an analog output channel Ao2 for outputting the actual maximum width of the working side spark; and a digital output channel Port0 for outputting the number of sparks and abnormal signals.

[0084] The range of the analog input signal and the analog output signal is 1-5V. The corresponding relationship between the actual maximum width of the spark (mm) and the output analog signal (V) is: output signal = actual maximum width of the spark / 25 + 1, and the output signal is 5V when the output signal > 5V. The output signal > 3V is the working state, and the output signal ≤ 3V is the non-working state.

[0085] The output digital signal is 8 bits, the first to third bits are the number of driving side sparks, the fourth bit is the hardware fault state bit, the fifth to seventh bits are the number of working side sparks, and the eighth bit is the packet loss fault state bit. Since the maximum value in the binary digital signal of the first to third bits and the fifth to seventh bits is 111, the maximum value of the number of driving side sparks and the number of working side sparks is 7, and when the number of sparks is greater than 7, it is taken as 7. The fourth bit is 1 when there is a hardware fault, and the eighth bit is 1 when there is a packet loss fault.

[0086] When the input values of the analog input channels Ai1 and Ai2 are both less than 3V, it indicates that there is no steel sheet curling. At this time, no camera is in working state, the output values of the analog output channels Ao1 and Ao2 are both 1V, and the digital signal output value is 00000000.

[0087] When the input value of the analog input channel Ai0 is the steel sheet temperature signal, the input value of the analog input channel Ai1 is greater than 5V, and the input value of the analog input channel Ai2 is less than 3V, it indicates that the 1# CCD camera above the first side guide plate 31 is in working state. At this time, the output values of the analog output channels Ao1 and Ao2 are the detection values XXX, and the digital output channel output value is 0XXX0XXX.

[0088] When the input value of the analog input channel Ai0 is the steel sheet temperature signal, the input value of the analog input channel Ai2 is greater than 5V, and the input value of the analog input channel Ai1 is less than 3V, it indicates that the 2# CCD camera above the second side guide plate 32 is in working state. At this time, the output values of the analog output channels Ao1 and Ao2 are the detection values XXX, and the digital output channel output value is 0XXX0XXX.

[0089] When the CCD camera is disconnected, the output values of the analog input channels and the analog output channels are arbitrary, and the output value of the digital output channel is 00001000.

[0090] Step 2: The image collector 8 sends the collected image to the controller 7, and the controller 7 converts the image into a pixel graph according to the proportion and initializes the time domain median background model.

[0091] After the on-site environment is calibrated, the image collected by the image collector 8 can be projected as a top view of the production line coiling area through the homography transformation of the image. In the world coordinate system, the top view is approximately parallel to the actual plane of the production line. After the calibration is completed, the pixel length in the pixel graph is in a fixed proportion to the actual length, and according to the proportion and the pixel length in the image, the actual width of the spark can be calculated.

[0092] Please refer to the accompanying drawings Figure 5 , and the step 2 comprises the following steps:

[0093] Step 2.1: Establish a world coordinate system with the width and center distance of the roller as the reference, select the corner points of the side guide plate area to be detected in the image, and the four corner points form a quadrilateral detection area image.

[0094] Step 2.2: Set the length and width of the top view pixel graph, and calculate the proportion between the pixel graph and the detection area image.

[0095] Step 2.3: Calculate the homography matrix of image transformation into pixel map, and convert the image into pixel map according to the homography matrix.

[0096] The calculation method of the homography matrix is:

[0097] In the image collected by the image collector 8, the correspondence between the three-dimensional world object and the pixel on the final image can be obtained through the transformation of the coordinate system, and the actual measurement of the object on the two-dimensional image can be obtained. The transformation process involves four coordinate systems: the world coordinate system, the collector coordinate system, the image coordinate system, and the pixel coordinate system. The world coordinate system O W -X W Y W Z W , which corresponds to the coordinates of the real world. The world coordinate system is obtained by translation and rotation to the collector coordinate system O C -X C Y C Z C , the collector coordinate system is obtained by the imaging model to the image coordinate system o-xy, and the image coordinate system is transformed to the pixel coordinate system uv by translation and scaling.

[0098] Assume that the rotation angle of the world coordinate system around the Z axis to the collector coordinate system is α. For a point P(X W ,Y W ,Z W ) in the world coordinate system, let the coordinates of the point in the collector coordinate system be P C (X C ,Y C ,Z C ), and the rotation relationship between the world coordinate system and the collector coordinate system around the Z axis is:

[0099]

[0100] where the 3x3 matrix is the rotation matrix of the world coordinate system around the Z axis, denoted as r1. Similarly, the rotation matrix of the world coordinate system around the X axis can be obtained, denoted as r2, and the rotation matrix of the world coordinate system around the Y axis can be obtained, denoted as r3. Then the actual rotation matrix of the world coordinate system is R=r1r2r3.

[0101] The coordinate offset of the world coordinate system along the X axis is denoted as t1, the coordinate offset of the world coordinate system along the Y axis is denoted as t2, and the coordinate offset of the world coordinate system along the Z axis is denoted as t3. Then the transformation formula of the world coordinate system to the collector coordinate system is:

[0102]

[0103] Let get the augmented matrix:

[0104]

[0105] The transformation of the collector coordinate system to the image coordinate system can be achieved based on a pinhole imaging model. The image coordinate system is a two-dimensional plane, and the coordinate transformation satisfies the principle of similar triangles. Let the coordinates of the image coordinate system be P e (X e ,Y e ,1), and the focal length of the camera be f. The relationship of the transformation of the collector coordinate system to the image coordinate system is as follows:

[0106]

[0107] The image coordinate system and the pixel coordinate system are in the same plane, and thus the difference between them lies in the position of the origin and the unit. Usually, the upper left corner of the image coordinate system is taken as the origin of the pixel coordinate system, and the unit of the pixel coordinate system is a pixel. The coordinates of the pixel coordinate system are P pixel (u,v,1). Therefore, the conversion relationship of the conversion of the image coordinate system to the pixel coordinate system is as follows:

[0108]

[0109] where dx and dy represent the width and height of each pixel point, respectively, and u0 and v0 represent the origin of the pixel coordinate system, respectively. In combination with equations (2) to (5), the complete transformation relationship of the conversion of the world coordinate system to the pixel coordinate system can be obtained as follows:

[0110]

[0111] Perspective transformation refers to a three-dimensional coordinate transformation that makes the original view plane rotate by a certain angle to a new view plane around the trace line (perspective axis) according to the perspective rotation law by using the collinear condition of the perspective center, the image point and the target point.

[0112] According to the perspective rotation law, the correspondence between the transformation matrix in the complete transformation relationship and the coordinates in the world coordinate system is as follows:

[0113]

[0114] The 3x3 matrix in equation (7) is a homography matrix. When a 33 is normalized to 1, there are a total of 8 unknown variables in the equation group, and four groups of mapping points are needed to determine the mapping relationship.

[0115] Step 2.4: Initialize the time domain median background model: that is, the median of the gray values of three consecutive pixel images is taken as the background to establish the time domain median background model.

[0116] Step 3: The controller 7 verifies whether the image sequence number is continuous, if yes, step 4 is performed, if not, a frame loss digital signal (10000000) is written to the analog-to-digital conversion card of the controller 7, and step 1 is returned. The verification of the image sequence number can be realized by a sub-thread, and is synchronized with the main program to improve the running efficiency of the whole flow program.

[0117] When the image collector 8 communication frame loss or time domain median background model has not been initialized, the analog input channel and the analog output channel are both arbitrary values, and the digital output channel outputs a value of 10000000.

[0118] An API (Application Programming Interface) of the image collector 8 can be used to assign a sequence number to each frame of image, and the sequence numbers of the continuous images are continuous and sequentially increased.

[0119] Step 4: The spark connected domain on the pixel graph is extracted, and the actual width of the spark is calculated according to the spark connected domain.

[0120] The step 4 includes the following steps:

[0121] Step 4.1: The background in the pixel graph is removed by the time domain median background model to obtain a foreground image, and the foreground image is subjected to thresholding processing to obtain a binary image.

[0122] Since the spark connected domain is contained in the foreground image, it needs to be extracted. The pixel graph is subtracted from the time domain median background model, so that the background is removed to eliminate the interference light such as sunlight, metal reflection and the like in the pixel graph. The contrast of the pixels in the bright and dark parts of the foreground image is obvious, but there is still a large amount of noise in the dark part, which can be eliminated by thresholding processing.

[0123] Preferably, a threshold parameter Th (1≤Th≤254) is set, the gray value of the pixel point on the pixel graph greater than or equal to the threshold parameter Th is set to 255, and the gray value of the pixel point less than the threshold parameter Th is set to 0 to obtain a binary image. The threshold parameter Th is used for consistent processing of the pixel graph.

[0124] Step 4.2: A maximum entropy threshold is set, and the binary image with the spark region is obtained by local thresholding of the binary image by the maximum entropy threshold. Preferably, 1≤maximum entropy threshold≤254.

[0125] The maximum entropy threshold is used for local threshold optimization of the pixel region with large bright-dark difference. Entropy is an important concept in information theory, commonly used in the field of data compression, and is a statistical measurement method used to determine the amount of information contained in a random data source.

[0126] The method of local thresholding is:

[0127] The binary image contains N pixel points I, i.e. contains N information, and the value of each pixel point I is independently obtained from different gray values g in a limited range [0, K]. The calculation formula of entropy in the binary image is:

[0128]

[0129] Where p(g) is the occurrence probability of the gray value g∈[0, K], i.e. the prior probability of g, which can be obtained by approximating the cumulative probability distribution of multiple images.

[0130] Let the segmentation threshold (i.e. the maximum entropy threshold) be t, and the sum of the probability distribution is:

[0131] Let Q be the distribution of i∈{0, 1, …, t}, and B be the distribution of i∈{t+1, …, K}, The specific form of the probability distribution is:

[0132]

[0133]

[0134] The entropy of the probability distribution of formula (9) is:

[0135]

[0136] The entropy of the probability distribution of formula (10) is:

[0137]

[0138] The segmentation threshold t is:

[0139] From the naked eye, the high-light region obtained after thresholding and local thresholding is a closed and connected spark region, but since the spark is not a tightly connected entity, there may be a few pixel points in the spark region that are below the threshold parameter Th and are set to zero. These pixel points will become "holes" in the spark region after thresholding, which may interfere with the spark size discrimination in the subsequent steps.

[0140] The "hole" pixels in the spark region can be filled by morphological operations. The principle of morphological operation is: based on morphological structure elements, the image can be analyzed to simplify the image data, maintain their basic shape characteristics, and remove irrelevant structures. According to the needs, morphological closing operation is performed on the image to remove "holes" and connect the surrounding areas to form a connected domain.

[0141] Step 4.3: Calculate the geometric features of the spark region to form a spark connected domain.

[0142] The method for calculating the spark connected domain includes:

[0143] Step 4.3.1: Apply a memory space as a stack, scan the binary image with the spark region, and find the pixel point I(u, v) = 1.

[0144] Step 4.3.2: Mark the pixel point found in step 4.3.1 as X, and stack all adjacent pixel points of the pixel point.

[0145] Step 4.3.3: Pop the top pixel point of the stack, also mark the top pixel point as X, and stack all adjacent pixel points of the top pixel point.

[0146] Step 4.3.4: Repeat step 4.3.3 until the stack is empty.

[0147] Step 4.3.5: Traverse the image, repeat steps 4.3.1-4.3.4 until all "hole" pixels in the spark region are marked, forming a spark connected domain.

[0148] In the spark connected domain, the difference between the maximum horizontal coordinate and the minimum horizontal coordinate of the pixel is the width of the spark connected domain, and the difference between the maximum vertical coordinate and the minimum vertical coordinate of the pixel is the length of the spark connected domain.

[0149] Step 4.4: Calculate the actual width of the spark according to the proportion and the spark connected domain. Since the spark will elongate as the strip steel advances, the width of the spark can accurately reflect the situation of the spark.

[0150] The calculation of the actual width of the spark can be implemented by a sub-thread and synchronized with the main program to improve the running efficiency of the entire flow program. During the process of calling the sub-thread to calculate the actual width of the spark, since the image collector 8 cyclically collects images, the controller 7 only captures one image at a time to avoid sub-thread blocking and calculation errors.

[0151] Step 5: The controller 7 judges whether to update the time domain median background model, if yes, step 6 is executed, if not, step 7 is executed.

[0152] Step 6: Update the time domain median background model.

[0153] The updating method of the time domain median background model is:

[0154] Step 6.1: Set the image buffer size to A frames of images, the sampling interval to D frames of images, and the update frequency to F frames of images, wherein the value range of A, D, and F can be adjusted according to the performance of the image collector 8.

[0155] Step 6.2: The image collector 8 sends the collected images to the controller 7 in real time, and the controller 7 caches the received images to form an image sequence with a size of A.

[0156] Step 6.3: The controller 7 deletes the earliest D-frame image in the image sequence in chronological order and adds the newly obtained D-frame image to the tail of the cached sequence every time it obtains D-frame images.

[0157] Step 6.4: Update the time-domain median background model every time F-frame images are obtained. The update method of the time-domain median background model is as follows: calculate the median of a coordinate point in A-frame images in the cached image sequence as the gray value of the coordinate point in the time-domain median background model, and traverse the A-frame images in the cached image sequence coordinate by coordinate to complete the update of the time-domain median background model.

[0158] The update of the time-domain median background model can be implemented by a sub-thread and synchronized with the main program to improve the running efficiency of the entire flow program.

[0159] Step 7: The controller 7 outputs the spark recognition result and writes the images collected by the image collector 8 into a video.

[0160] In the step 7, the controller 7 sets a video capacity threshold, and if the actual capacity of the video reaches the video capacity threshold, a new video is created. The writing of images into a video or the creation of a new video can be implemented by a sub-thread and synchronized with the main program to improve the running efficiency of the entire flow program.

[0161] The spark recognition result includes the number of working-side sparks, the number of transmission-side sparks, the fault state, the actual width of the working-side sparks, and the actual width of the transmission-side sparks,

[0162] Preferably, according to the actual width of the sparks, the degree of the sparks can be classified by a deep learning classification algorithm, and no spark is 0, and the sparks are classified into 1-M according to the severity, M∈[4, 10], which facilitates more accurate control of the side guide plate.

[0163] Step 8: Determine whether to exit the spark detection, if yes, end the detection, and if no, return to step 1 to continue the detection of sparks. The main program code of the spark detection has a Boolean quantity stopflag, and if the user clicks the close button on the interface or an abnormal error occurs, the stopflag is set to 1, and the program exits.

[0164] Embodiment 1: The present application is applied to the first side guide plate 31 of the hot continuous rolling production line winding area for spark detection.

[0165] Please refer to the attached Figure 3, step 1: set the image collector 8 above the upstream of the pair of first side guides 31 along the movement direction of the strip steel 2, and connect the control system of the first side guides 31 through the controller 7, as shown in FIG. 8. Figure 2 The image collector 8 adopts a CCD industrial camera with a model of BASLER avA1900-50GM, and the corresponding speed of the CCD camera head is ≤50 ms, and the controller 7 adopts an industrial PLC. The vertical distance between the CCD industrial camera and the roller is 4.18 m, the horizontal distance between the CCD industrial camera and the upstream end of the parallel section of the first side guides 31 is 8 m, and the length of the parallel section of the first side guides 31 is 6 m.

[0166] After the CCD industrial camera is connected with the industrial PLC, the camera parameters (including gain, exposure time, image format, etc.) can be set and saved through the application program Pylon Viewer, the CCD industrial camera can be called through the C++ SDK provided by BASLER officially, and the image captured by the CCD industrial camera is written into the MPEG-4 video through the encoding and decoding library provided by BASLER. The working state of the CCD industrial camera is checked, the Ai1 input signal of the industrial PLC is U1=5V>3V, the CCD industrial camera is in working state, and the CCD industrial camera is normally connected with the network. The Ai0 input signal of the industrial PLC is 300℃, and the exposure time of the CCD industrial camera is adjusted to 36000 ms.

[0167] Step 2: the image collector 8 sends the collected image to the controller 7, the controller 7 converts the image into a pixel graph according to the proportion, and initializes the time domain median background model, as shown in FIG. 9. Figure 6

[0168] In this embodiment, the standard length of the production line roller is 1580 mm, and the center distance is 360 mm. The world coordinate system is established based on the width of the roller and the center distance, the length of the detection area image is 4320 mm, and the width is 1580 mm. The length of the pixel graph is 2160 pixels, the width is 790 pixels, and the proportion between the pixel graph and the detection area image is 1:2.

[0169] The homography matrix of the image transformation into the pixel graph is calculated according to the coordinate system transformation, and the image is converted into the pixel graph according to the homography matrix.

[0170] Step 3: the controller 7 verifies the continuity of the image serial number.

[0171] Step 4: extract the spark connected domain on the pixel graph, and calculate the actual width of the spark according to the spark connected domain.

[0172] The background in the pixel graph is removed through the time domain median background model to obtain the foreground image, as shown in FIG. 10. Figure 7 ​The threshold value parameter Th=139 is set to threshold the foreground image to obtain a binary image, as shown in Fig. 6. Figure 8

[0173] The maximum entropy threshold value 5.545 is set, and the binary image is locally thresholded by the maximum entropy threshold value to obtain a binary image with a spark region, as shown in Fig. 7. Figure 9

[0174] The geometric features of the spark region are calculated to form a spark connected domain, as shown in Fig. 8. Figure 10 As shown in Fig. 9, the white part is the spark connected domain, the maximum width of the spark connected domain on the transmission side is 19.835 pixels, and the maximum width of the spark connected domain on the working side is 21.455 pixels, so the actual width of the spark on the transmission side is 39.67 mm and the actual width of the spark on the working side is 42.91 mm according to the ratio 1:2. Figure 10

[0175] Step 5: The industrial PLC judges the renewal time domain median background model.

[0176] Step 6: Renewal of the time domain median background model: the image buffer size is set to A=2 frames of images, the sampling interval is set to D=2 frames of images, and the update frequency is set to F=6 frames of images. Every F frames of images, the time domain median background model is updated: the median of a coordinate point in A=2 frames of images in the buffer image sequence is calculated as the gray value of the coordinate point in the time domain median background model, and the A=2 frames of images in the buffer image sequence are traversed coordinate by coordinate to complete the update of the time domain median background model.

[0177] Step 7: The industrial PLC outputs the spark recognition result 00010111, and writes the image collected by the image collector 8 into the video.

[0178] Step 8: The Boolean quantity stopflag in the main program is 0, and the step 1 is returned to continue detecting the spark.

[0179] After the coiling of the strip steel 2 is completed, the number of sparks displayed by the industrial PLC during the movement of the entire strip steel 2 is shown in Fig. 10, and the actual width of the spark is shown in Fig. 11. Figure 11 Figure 12

[0180] The above is only a preferred embodiment of the present application, and is not used to limit the protection scope of the present application, so any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.​​​​​

Claims

1. A method for detecting sparks of side guide plate during coiling of strip steel based on image recognition, characterized in that: It comprises the following steps: Step 1: image collectors (8) are arranged above each pair of side guides, and the image collectors (8) face a pair of side guides and the strip steel (2); the output ends of the image collectors (8) are connected with the input ends of the controller (7), and the output end of the controller (7) is connected to a control system; Step 2: the image collector (8) sends the collected image to the controller (7), and the controller (7) converts the image into a pixel image according to the proportion and initializes a time domain median background model; The step 2 comprises the following steps: Step 2.1: a world coordinate system is established based on the width and center distance of the roller way, and the corner points of the side guide area to be detected are selected in the image, and the four corner points form a quadrilateral detection area image; Step 2.2: the length and width of the overhead pixel image are set, and the proportion between the pixel image and the detection area image is calculated; Step 2.3: the homography matrix of the image converted into the pixel image is calculated, and the image is converted into the pixel image according to the homography matrix; Step 2.4: the median of the gray values of the continuous three frames of pixel images is used as the background to establish the time domain median background model; Step 3: the controller (7) verifies whether the image serial number is continuous, if yes, step 4 is executed, if not, a frame loss digital signal is written to the analog-digital conversion card of the controller (7), and step 1 is returned; Step 4: the spark connected domain on the pixel image is extracted, and the actual width of the spark is calculated according to the spark connected domain; The step 4 comprises the following steps: Step 4.1: the background in the pixel image is removed by the time domain median background model to obtain a foreground image, and the foreground image is thresholded to obtain a binary image; Step 4.2: a maximum entropy threshold is set, and the binary image is locally thresholded by the maximum entropy threshold to obtain a binary image with a spark area; Step 4.3: the geometric features of the spark area are calculated to form a spark connected domain; Step 4.4: the actual width of the spark is calculated according to the proportion and the spark connected domain; The method for calculating the spark connected domain comprises: Step 4.3.1: a memory space is applied as a stack, the binary image with the spark area is scanned, and a pixel point I(u,v)=1 is found; Step 4.3.2: the pixel point found in step 4.3.1 is marked as X, and all adjacent pixel points of the pixel point are stacked; Step 4.3.3: the top pixel point of the stack is popped, the top pixel point is also marked as X, and all adjacent pixel points of the top pixel point are stacked; Step 4.3.4: step 4.3.3 is repeated until the stack is empty; Step 4.3.5: the image is traversed, steps 4.3.1-4.3.4 are repeated until all "hole” pixels in the spark area are marked, and a spark connected domain is formed; Step 5: the controller (7) judges whether to update the time domain median background model, if yes, step 6 is executed, if not, step 7 is executed; Step 6: the time domain median background model is updated; Step 7: the controller (7) outputs the spark recognition result, and writes the image collected by the image collector (8) into a video; The spark recognition result includes the number of working side sparks, the number of transmission side sparks, a fault state, an actual width of the working side spark, and an actual width of the transmission side spark. Step 8: judging whether to exit the spark detection, if yes, ending the detection, if no, returning to step 1 to continue the detection of the spark.

2. The image recognition-based spark detection method for a side guide during coiling of a steel strip according to claim 1, characterized in that: In step 1, the vertical distance between the image collector (8) and the roller is 2-5 m, the horizontal distance between the image collector (8) and the upstream end of the parallel section of the side guide plate is 2-10 m, and the length of the parallel section of the side guide plate is 4-8 m.

3. The image recognition-based spark detection method for a side guide during coiling of a steel strip according to claim 1, characterized in that In step 1, the controller (7) judges the working state of all the image collectors (8), including the following steps: Step 1.1: the controller (7) reads the working signal of each image collector (8), and judges whether each image collector (8) is in the working state according to the working signal, if all the image collectors (8) are in the non-working state, step 1.2 is executed, if one of the image collectors (8) is in the working state, step 1.3 is executed; Step 1.2: unloading the time domain median background model; Step 1.3: the controller (7) judges whether the image collector (8) in the working state is disconnected from the network, if yes, step 1.4 is executed, if no, step 1.5 is executed; Step 1.4: the controller (7) writes a disconnection digital signal, and the image collector (8) reconnects to the network; Step 1.5: the controller (7) displays the picture collected by the image collector (8) in the working state in real time through the interface; Step 1.6: the controller (7) reads the strip temperature signal, and adjusts the working parameters of the image collector (8) according to the strip temperature signal, and returns to step 1.

1.

4. The image recognition-based spark detection method for a side guide during coiling of a steel strip according to claim 1, characterized in that: Step 6 includes the following steps: Step 6.1: setting the image cache size as A frames of images, the sampling interval as D frames of images, and the update frequency as F frames of images; Step 6.2: the image collector (8) sends the collected images to the controller (7) in real time, and the controller (7) caches the received images to form an image sequence with a size of A; Step 6.3: the controller (7) deletes the earliest D frames of images in the image sequence in chronological order every time D frames of images are obtained, and then adds the newly obtained D frames of images to the tail of the cache sequence; Step 6.4: updating the time domain median background model every F frames of images.

5. The image recognition based spark detection method for side guide of a strip coiler as claimed in claim 4, wherein: In step 6.4, the update method of the time domain median background model is that the median of a coordinate point in the A frames of images in the cache image sequence is calculated as the gray value of the coordinate point in the time domain median background model, and the cache image sequence is traversed coordinate by coordinate to complete the update of the time domain median background model.

6. The image recognition based spark detection method for side guide of a strip coiler as claimed in claim 1, wherein: In step 7, the controller (7) sets a video capacity threshold, and if the actual video capacity reaches the video capacity threshold, a new video is created.

Images