Segmented picture fusion method for continuous defects of low-feature wide and thick steel plate

Abstract

The invention belongs to the technical field of wide and thick steel plate defect recognition, and relates to a low-feature wide and thick steel plate continuous defect segmented picture fusion method, which comprises the following steps of: determining data needing to be collected, classifying the data, calculating the movement length of a steel plate before the next fast door, screening defect pictures with repeated parts, and carrying out duplicate removal processing on the pictures to obtain a segmented picture fusion result. And in combination with the picture position and the position of the defect on the picture, mapping the defect to the specific position of the steel plate, and identifying and combining the defects according to the coordinate of each defect on the steel plate coordinate system. The method is completely combined with actual production, depends on steel plate speed and shutter speed, combines steel plate surface defect data, and converts defects shot by a high-speed camera and identified by an intelligent algorithm to a specific steel plate position through the steps of steel plate picture screening and duplicate removal, defect coordinate conversion and defect screening, so that steel plate surface defect positioning is realized; manual checking is reduced, and steel plate quality inspection efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of wide and thick steel plate defect recognition, and in particular to a method for fusing segmented images of continuous defects of low-feature wide and thick steel plates. Background Art

[0002] In recent years, the judgment of steel plate appearance quality has become a topic of widespread concern. Among them, the characteristics and number of steel plate surface defects are important bases for judging the grade of steel plate surface defects and the appearance quality of steel plates. The development of steel plate surface defect recognition algorithms provides support for the judgment of steel plate appearance quality. However, for the various types of surface defects on thick and wide steel plates, the difficulty in identifying defect characteristics, the high overlap rate of images caused by the rapid movement of steel plates on the production line and high-speed camera shooting (a certain steel plate surface defect will appear in multiple images or across images), simple image splicing methods and single surface defect recognition algorithms will cause repeated defect judgments, which cannot meet the needs of steel plate surface defect recognition and grading and online judgment of the appearance quality of finished steel plates.

[0003] Currently, there are several feature detection, image stitching, and fusion algorithms. For example, patent CN106735029A uses samples to train a neural network to identify image features, patent CN119339200A uses a deep learning-based image fusion method, and SIFT-based feature detection algorithms use OpenCV's stitching method to perform panoramic stitching after feature detection. These algorithms can, to a certain extent, stitch multiple images with distinct features together. However, for the continuous segmented images captured by a high-speed camera during the movement of the steel plate, the overlapping boundary features between the two images are not obvious, making it impossible for the feature recognition algorithm to achieve accurate recognition and, therefore, unable to achieve the fusion and stitching of the continuous segmented images of the steel plate. Summary of the Invention

[0004] The purpose of the present invention is to provide a method for fusing segmented images of continuous defects of low-feature wide and thick steel plates. Based on a single steel plate surface recognition algorithm and data such as the steel plate movement speed and the shutter speed of a high-speed camera, the method realizes the fusion of continuous segmented images of low-feature wide and thick steel plate defects, providing a data basis for the judgment of steel plate surface defects.

[0005] The technical solution adopted by the present invention to solve the technical problem is: a method for fusing segmented images of continuous defects of low-feature wide and thick steel plates, comprising the following steps:

[0006] (1) Determine the basic data to be collected:

[0007] a. Fixed high-speed camera number, shutter speed, and number of images taken by a fixed high-speed camera;

[0008] b. Steel plate speed;

[0009] c. Image length, height and coordinate information on the steel plate coordinate system;

[0010] d. The coordinates of the defect marking box in the image coordinate system;

[0011] (2) Image filtering and deduplication:

[0012] According to the shutter speed and the plate speed, calculate the movement length V of the plate before the next shutter p / V c , thereby calculating the length of the overlapping part of the two pictures and the number of repeated pictures; calculate W cm ×V c / V p =k+δ, where k=|W cm ×V c / V p |, δ<1, k is the number of images with repeated parts, δ is the proportion of repeated parts in a certain image;

[0013] (3) Defect coordinate transformation:

[0014] Combine the image position and the position of the defect on the image to map the defect to the specific position of the steel plate; first, determine the relative relationship between the image coordinate system and the steel plate coordinate system through the image coordinates and the camera number, and then determine the coordinates of the defect on the steel plate coordinate system through the relationship between the defect coordinates and the image coordinate system, combined with image screening. The coordinate conversion formula between the defect image coordinate system and the steel plate coordinate system is: (X, Y) = (W cm +V p / V c ×Jy pi / W px ×W cm ,H cm ×(I+1)-x pi / H px ×H cm );

[0015] (4) Defect merging:

[0016] After screening the images, the defects are identified and merged based on the coordinates of each defect on the steel plate coordinate system;

[0017] Set thresholds α and β. If two defects belong to the same type and the coordinates between the tail midpoint (X1, Y1) of the previous defect identification box and the head midpoint (X2, Y2) of the next defect identification box are less than (α, β), they are considered to be the same defect.

[0018] Furthermore, the basic data to be collected in step (1) include:

[0019] Shutter speed Vc / times / second;

[0020] Steel plate speed V p / times / second: on-site collection;

[0021] Image coordinate I: the coordinate of the image on the Y axis of the steel plate (0,1...);

[0022] Image coordinate J: the coordinate of the image on the X-axis of the steel plate (0,1...);

[0023] Image width W px / pixel: 1080px;

[0024] Image width W cm / meter: 0.25 meters;

[0025] Image height H px / pixel: 1920px;

[0026] Image height H cm / meter: 0.45 meters;

[0027] Pixel point x coordinate x pi / Pixel: The picture is used as the reference system;

[0028] Pixel point y coordinate y pi / Pixel: The picture is used as the reference system;

[0029] Steel plate coordinate X i / m: X of the steel plate after y mapping of a certain point in the image;

[0030] Steel plate coordinate Y i / m: The y of the steel plate after the x-mapping of a certain point in the image.

[0031] Furthermore, the values ​​of α and β in step (4) are calculated by taking a portion of the samples, and the distribution of the coordinate difference (ΔX, ΔY) between the midpoint of the head of the latter defect and the tail of the former defect of two adjacent defects of the same type in the samples is statistically analyzed, and its mean (μ1, μ2) and standard deviation (σ1, σ2) are calculated;

[0032] Set the threshold (k is an adjustable coefficient, such as k = 2 to cover 95% of positive samples):

[0033] α=μ1+k×σ1, β=μ2+k×σ2;

[0034] At the same time, according to the maximum allowable length L of this type of defect in the rules:

[0035] α max =L / 2; β max =L / 2;

[0036] Make sure α and β do not exceed this value to avoid merging across defects.

[0037] The present invention has the following beneficial effects: The method for fusing segmented images of continuous defects of low-feature wide and thick steel plates provided by the present invention does not require detection and identification of the overlapping boundary features of two images. It only needs to take the steel plate as the object and realize the fusion of continuous segmented images of low-feature steel plate defects based on the recognition algorithm of single steel plate surface defects and data such as steel plate movement speed and high-speed camera shutter speed. The present invention is fully integrated with production practice, relying on steel plate speed and shutter speed, combined with steel plate surface defect data, through steel plate image screening and deduplication, defect coordinate conversion and defect screening steps, the defects captured by high-speed cameras and identified by intelligent algorithms are converted to specific steel plate positions, thereby realizing the positioning of steel plate surface defects, reducing manual inspection, and improving steel plate quality inspection efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 This is a flow chart of the method for fusing segmented images of continuous defects in low-feature wide and thick steel plates of the present invention.

[0039] Figure 2 It is a schematic diagram of the defect image coordinate system and the steel plate coordinate system of the present invention. DETAILED DESCRIPTION

[0040] The following are specific embodiments of the present invention to further describe the technical solution of the present invention, but the scope of protection of the present invention is not limited to these embodiments. Any changes or equivalent substitutions that do not deviate from the concept of the present invention are included in the scope of protection of the present invention.

[0041] The process of the continuous defect segmentation image fusion method for low-feature wide and thick steel plates is as follows Figure 1 As shown in the figure, taking a steel plant's 4300mm thick plate production line as an example, a 10.9m long and 2.89m wide thick plate passes through a surface inspection instrument with 9 cameras. The shutter speed is 100 times / second and the speed of the steel plate on the roller is 2m / second.

[0042] 1. Determine the basic data to be collected:

[0043] a. Fixed high-speed camera number, shutter speed, and number of images taken by a fixed high-speed camera;

[0044] b. Steel plate speed;

[0045] c. Image length, height and coordinate information on the steel plate coordinate system;

[0046] d. The coordinates of the defect marking box in the image coordinate system;

[0047] e. Basic information of steel plate such as length and width.

[0048] The basic data to be collected mainly include:

[0049] Shutter speed V c / times / second;

[0050] Steel plate speed V p / times / second: on-site collection;

[0051] Image coordinate I: the coordinate of the image on the Y axis of the steel plate (0,1...);

[0052] Image coordinate J: the coordinate of the image on the X-axis of the steel plate (0,1...);

[0053] Image width W px / pixel: 1080px;

[0054] Image width W cm / meter: 0.25 meters;

[0055] Image height H px / pixel: 1920px;

[0056] Image height H cm / meter: 0.45 meters;

[0057] Pixel point x coordinate x pi / Pixel: The picture is used as the reference system;

[0058] Pixel point y coordinate y pi / Pixel: The picture is used as the reference system;

[0059] Steel plate coordinate X i / m: X of the steel plate after y mapping of a certain point in the image;

[0060] Steel plate coordinate Y i / m: The y of the steel plate after the x-mapping of a certain point in the image.

[0061] The above data comes from the first level, meter inspection system, MES (Manufacturing Execution System) or requires manual input. Interfaces are established with these systems to achieve data collection with a minimum time interval of 1 second.

[0062] 2. Image filtering and deduplication:

[0063] Fixed high-speed cameras will capture multiple images of surface defects on the same steel plate. When the steel plate grade is determined based on the number of defects, these images will be counted repeatedly, so they need to be screened and deduplicated.

[0064] According to the shutter speed and the plate speed, calculate the movement length V of the plate before the next shutter p / V c, and thus calculate the length of the overlapping part of the two pictures and the number of repeated pictures. Calculate W cm ×V c / V p =k+δ, where k=|W cm ×V c / V p |, δ<1, k is the number of images with repeated parts, and δ is the ratio of repeated parts in a certain image. In other words, a defect will appear repeatedly in k+1 images. Screening out the middle k-1 images can effectively reduce the total number of defects in the steel plate images without affecting the characteristics of individual defects.

[0065] Assume that the image sequence is list p init =[p0,p1,p2,...,p k ,p k+1 ], the filtered picture sequence is listp result =[]. The pseudo code for image filtering and duplicate removal is:

[0066]

[0067] For example, if the current shutter speed is 100 times / second and the steel plate speed is 2 meters / second, the movement length of the steel plate before the next shutter is calculated to be 0.02 meters. At the same time, k+δ=12.5, that is, the number of pictures with repeated parts is 12, and the distance between the k+1th picture and the first picture is the picture width W. cm If the value of is 0.5, the 11 repeated images in the middle are discarded, and only the first and n(k+1)th images are retained.

[0068] 3. Defect coordinate transformation:

[0069] Since the coordinate system of the defect on the image has a different origin from the coordinate system of the image on the steel plate, it is necessary to combine the image position with the position of the defect on the image to map the defect to the specific position of the steel plate. The relative relationship between the image coordinate system and the steel plate coordinate system can be determined by the image coordinates and the camera number. Then, the coordinates of the defect on the steel plate coordinate system can be determined by the relationship between the defect coordinates and the image coordinate system, combined with image screening. The schematic diagram of the defect image coordinate system and the steel plate coordinate system is shown below. Figure 2 The coordinate transformation formula is:

[0070] (X,Y)=(W cm +V p / V c ×Jy pi / W px ×W cm ,H cm ×(I+1)-x pi / Hpx ×H cm )

[0071] Specifically: According to the relationship between the image coordinate system and the steel plate coordinate system, the position (X, Y) of the defect on the steel plate is calculated. pi ,y pi ) divided by the image resolution H px ×W px The length and width ratio coordinates of the defect position on the image can be obtained; then these coordinates are multiplied by the actual height H of the image. cm and width W cm , get the actual position coordinates of the defect on the picture (x pi / H px ×H cm ,y pi / W px ×W cm ); The X-axis coordinate of the image origin in the steel plate coordinate system is the image width W cm Add the length of movement before each shutter click and multiply it by the coordinate J of the image on the X-axis of the steel plate; the Y-axis coordinate of the image origin in the steel plate coordinate system is the image height multiplied by the coordinate I of the image on the Y-axis of the steel plate plus 1; swap the x and y coordinates of the position of the defect on the image to keep them consistent with the coordinate axis of the steel plate; finally, use the coordinate value of the image origin in the steel plate coordinate system to process the position coordinate of the defect on the image and obtain the real position coordinate of the defect on the steel plate. That is, (X,Y)=(W cm +V p / V c ×Jy pi / W px ×W cm ,H cm ×(I+1)-x pi / H px ×H cm ).

[0072] For example: if the current shutter speed is 100 times / second, the steel plate speed is 2 meters / second, the position coordinates of a defect on the image are (300, 200), and the coordinates of the image on the steel plate are (189, 2), then the actual position coordinates of the defect on the image are (0.073125, 0.046296), and the coordinates of the image origin on the steel plate are (4.03, 1.35). Swap the x and y coordinates of the defect on the image to keep them consistent with the steel plate coordinate axes. Finally, use the coordinates of the image origin on the steel plate coordinate system to subtract the defect position coordinates on the image, and the actual position coordinates of the defect on the steel plate are (3.983704, 1.276875).

[0073] 4. Defect merging:

[0074] After screening the images, a defect may be divided into two or more parts, and the defects are identified and merged based on the coordinates of each defect in the steel plate coordinate system.

[0075] Thresholds α and β are set. If two defects belong to the same type and the coordinates between the tail midpoint (X1, Y1) of the previous defect identification frame and the head midpoint (X2, Y2) of the next defect identification frame are less than (α, β), they are considered to be the same defect. At this time, the length of the defect on the x-axis is the maximum X value of the next defect identification frame minus the minimum X value of the previous defect identification frame, and the length of the defect on the y-axis is the maximum Y value of the two defect identification frames minus the minimum Y value of the two defect identification frames.

[0076] The values ​​of α and β are calculated by taking some samples. The distribution of the coordinate difference (ΔX, ΔY) between the midpoint of the head of the latter defect and the tail of the former defect of two adjacent defects of the same type in the sample is statistically analyzed, and its mean (μ1, μ2) and standard deviation (σ1, σ2) are calculated.

[0077] Set the threshold (k is an adjustable coefficient, such as k = 2 to cover 95% of positive samples):

[0078] α=μ1+k×σ1, β=μ2+k×σ2;

[0079] At the same time, according to the maximum allowable length L of this type of defect in the rules:

[0080] α max =L / 2; β max =L / 2;

[0081] Make sure α and β do not exceed this value to avoid merging across defects.

[0082] For example: Set the threshold to (0.00001, 0.01). There are two scratch defects on the steel plate. The tail midpoint of the former defect identification frame is (4.122150, 1.667525), the length on the x-axis is 0.08m, and the length on the y-axis is 0.035m. The head midpoint of the latter defect identification frame is (4.122150, 1.668525), the length on the x-axis is 0.04m, and the length on the y-axis is 0.03m. The two defects are merged into one defect. At this time, the length of the scratch defect on the x-axis is 0.012m, and the length on the y-axis is 0.0335m.

[0083] The present invention is not limited to the above-mentioned embodiments. Anyone should be aware that any structural changes made under the guidance of the present invention, and any technical solutions that are the same or similar to those of the present invention, fall within the scope of protection of the present invention.

[0084] The technology, shape, and structure not described in detail in the present invention are all well-known technologies.

Claims

1. A method for fusion of segmented images of continuous defects of low-feature wide and thick steel plates, characterized by: The following steps are involved: (1) Determine the basic data to be collected: a. Fixed high-speed camera number, shutter speed, and number of images taken by a fixed high-speed camera; b. Steel plate speed; c. Image length, height and coordinate information on the steel plate coordinate system; d. The coordinates of the defect marking box in the image coordinate system; (2) Image filtering and deduplication: According to the shutter speed and the plate speed, calculate the movement length V of the plate before the next shutter p / V c , thereby calculating the length of the overlapping part of the two pictures and the number of repeated pictures; calculate W cm ×V c / V p =k+δ, where k=|W cm ×V c / V p |, δ<1, k is the number of images with repeated parts, δ is the proportion of repeated parts in a certain image; (3) Defect coordinate transformation: Combine the image position and the position of the defect on the image to map the defect to the specific position of the steel plate; first, determine the relative relationship between the image coordinate system and the steel plate coordinate system through the image coordinates and the camera number, and then determine the coordinates of the defect on the steel plate coordinate system through the relationship between the defect coordinates and the image coordinate system, combined with image screening. The coordinate conversion formula between the defect image coordinate system and the steel plate coordinate system is: (X, Y) = (W cm +V p / V c ×Jy pi / W px ×W cm ,H cm ×(I+1)-x pi / H px ×H cm ); (4) Defect merging: After screening the images, the defects are identified and merged based on the coordinates of each defect on the steel plate coordinate system; Set thresholds α and β. If two defects belong to the same type and the coordinates between the tail midpoint (X1, Y1) of the previous defect identification box and the head midpoint (X2, Y2) of the next defect identification box are less than (α, β), they are considered to be the same defect.

2. The method for fusion of segmented images of continuous defects of low-feature wide and thick steel plates according to claim 1, characterized in that: The basic data to be collected in step (1) include: Shutter speed V c / times / second; Steel plate speed V p / times / second: on-site collection; Image coordinate I: the coordinate of the image on the Y axis of the steel plate (0,1...); Image coordinate J: the coordinate of the image on the X-axis of the steel plate (0,1...); Image width W px / pixel: 1080px; Image width W cm / rice: 0.25 m; Image height H px / pixel: 1920px; Image height H cm / rice: 0.45 m; Pixel point x coordinate x pi / Pixel: The picture is used as the reference system; Pixel point y coordinate y pi / Pixel: The picture is used as the reference system; Steel plate coordinate X i / m: X of the steel plate after y mapping of a certain point in the image; Steel plate coordinate Y i / m: The y of the steel plate after the x-mapping of a certain point in the image.

3. The method for fusion of segmented images of continuous defects of low-feature wide and thick steel plates according to claim 1, characterized in that: The values ​​of α and β in step (4) are obtained by taking a portion of the samples and calculating the distribution of the coordinate difference (ΔX, ΔY) between the midpoint of the head of the latter defect and the midpoint of the tail of the former defect between two adjacent defects of the same type in the samples, and calculating the mean (μ1, μ2) and standard deviation (σ1, σ2); Set the threshold (k is an adjustable coefficient, such as k = 2 to cover 95% of positive samples): α=μ1+k×σ1, β=μ2+k×σ2; At the same time, according to the maximum allowable length L of this type of defect in the rules: a max =L / 2;β max =L / 2; Make sure α and β do not exceed this value to avoid merging across defects.

Images