Steel piling detection method and system based on computer vision

Abstract

The invention provides a steel piling detection method and system based on computer vision, and the method comprises the steps: 1, arranging a camera, guaranteeing that the camera can capture a to-be-detected bar, carrying out the static frame extraction of an input high-speed image video sequence, and obtaining a bar image needing to be detected; step 2, reading the first N images in the image stream as positioning images, and then performing preprocessing to generate a positioning template file; step 3, reading the (N + 1) th image of the image stream and all images after the image stream, then performing preprocessing, and extracting color features in the target area; and step 4, loading a positioning template file corresponding to the camera, comparing the extracted color features in the target area with the color features in the positioning template file, and judging whether a steel piling phenomenon occurs or not. The method is applied to steel heaping detection, the real-time performance of steel heaping detection in production is guaranteed, the accuracy of steel heaping detection is greatly improved, and meanwhile the personal safety of workers is guaranteed.

Description

Technical Field

[0001] The present invention relates to the field of computer vision and digital image technology, and in particular to a steel pile detection method and system based on computer vision. Background Art

[0002] With the rapid progress of society in the 21st century, industry has also ushered in rapid development. As the backbone of industry, steel has achieved a certain degree of automated production in the modern steel industry, and society has higher requirements for steel quality. Taking products as an example, from large steel structure support columns, high-speed EMU wheel axles, to "hand-torn steel" with a thickness of only 0.015 mm, and "pen tip steel" with processing precision thinner than a hair, more varieties, stronger performance, and a wider market have become the new label of the steel industry. Safe production is the basic guarantee for the development of the steel industry and an important condition for improving economic benefits. Due to human factors, aging equipment and untimely maintenance, technical defects, lack of safety facilities and personal safety protection products, and lack of emergency plans in the face of major emergencies, all constitute the reasons for the safety hazards in steel production, which seriously hinder the development of the steel industry. In the process of bar production and manufacturing, steel pile-up occurs due to the presence of foreign matter in the rolling groove or guide groove, uneven bar temperature, too small roller friction, unqualified roller ring material, inadequate roller ring cooling, and excessive bar size. The phenomenon of steel piling not only affects the quality and production efficiency of steel, but also causes serious property losses and casualties. In recent years, computer vision technology has developed rapidly, and various detection technologies based on computer vision have been widely used and developed in many practical application systems. Computer vision uses industrial cameras and computers to replace humans, and uses computers to identify and track targets without human intervention. In general, it is to use machines instead of human eyes to process and analyze images. It has the characteristics of high efficiency, reliability, continuity, flexibility, etc. At this stage, most of the steel piling detection is done through on-site manual detection, which not only requires manpower costs, but also poses a great challenge to the personal safety of the inspectors. In addition, traditional manual detection can no longer meet the needs of factory automation upgrades. These situations make the use of scientific and technological methods to detect the phenomenon of steel piling an inevitable development trend.

[0003] Patent application document CN111340027A discloses a method, system, electronic device and medium for identifying steel piles, wherein the method comprises: collecting image information of rolling tracks and setting regions of interest; obtaining detection targets through multidimensional data processing, wherein the multidimensional data processing comprises one of the following: HVS recognition and binarization recognition; extracting the outer contour of the detection target and obtaining parameter information of the outer contour, wherein the parameter information comprises one of the following: width, height and area; and determining whether steel piles occur through the parameter information. However, the patent cannot completely solve the existing technical problems, nor can it meet the requirements of the present invention.

[0004] In view of the problems that traditional manual naked eye detection is difficult, inefficient, risky, unreliable, and greatly affected by subjectivity, the present invention proposes a steel pile detection method based on computer vision. Summary of the invention

[0005] In view of the defects in the prior art, the object of the present invention is to provide a steel pile detection method and system based on computer vision.

[0006] The method for detecting steel piles based on computer vision provided by the present invention comprises:

[0007] Step 1: Arrange the camera to ensure that the camera can capture the rod to be inspected, extract the still frame of the input high-speed image video sequence, and obtain the rod image that needs to be inspected;

[0008] Step 2: Read the first N images in the image stream as positioning images, and then perform preprocessing to generate a positioning template file, where N is a natural number greater than or equal to 1;

[0009] Step 3: Read the N+1th image and all subsequent images in the image stream, and then perform preprocessing to extract the color features in the target area;

[0010] Step 4: Load the positioning template file of the corresponding camera, compare the color features in the extracted target area with the color features in the positioning template file, and determine whether steel pile occurs; if steel pile occurs, trigger an alarm message to the control center, and keep the steel pile photos in the abnormal situation folder; otherwise, save the production photos periodically.

[0011] Preferably, the step 1 comprises: collecting video streams in the rod conveying area; using TCP communication mode to transmit the video stream content to the NVR; taking a preset frame number I for the rod conveying video, extracting images every I frame number, and selecting a preset interval frame number according to the conveying speed of the rod to convert the input video stream into an image stream;

[0012] The step 2 includes: reading the first N images in the image stream as positioning images, the positioning images include images without steel and images with steel, and the steel images in the selected positioning template file need to cover the positions of all rods under safe conditions; performing preprocessing operations on the images, including morphological processing to eliminate noise in the target area, and image binarization to segment the target area, wherein the image needs to be binarized twice, and the white and red colors of the rods are used as thresholds for binarization respectively; accumulating the results of the binarization of all images, and saving the final accumulated results in a DAT file, and also saving them in different files according to the type of color and the camera number.

[0013] Preferably, the step 3 comprises:

[0014] Step 3.1: Binarize the image to be detected, and set the color threshold according to the color of the helmet;

[0015] Step 3.2: Perform contour detection on the binary image, save all contour points in the contours vector, and then obtain the circumscribed rectangle of the contour through the API interface RotatedRect() provided by OpenCV;

[0016] Step 3.3: Determine whether the outline belongs to a safety helmet based on the outline size and the horizontal and vertical spans of the outline. If it belongs to a safety helmet, it means that a worker is repairing the equipment. The steel pile inspection is directly determined to be safe.

[0017] Step 3.4: Binarize the image to be inspected again, remove pixels that are not in the preset color range, and set the color threshold according to the color of the rod to obtain the binary images IW and IR.

[0018] Preferably, step 4 comprises:

[0019] Step 4.1: Load the positioning template file, read the binary data in the file, convert the data into MAT format, and save them as FW and FR respectively;

[0020] Step 4.2: Read the binary images IW and IR, traverse the target area of ​​image IR, when the pixel value at the position (x, y) in image IR is 255, and the pixel value at the same position in image FR is not 255, then the red pixel danger count value unsafeR is increased by 1; when the pixel value at the position (x, y) in image FR is 255, the red pixel safety count value safeR is increased by 1, and the expression is:

[0021]

[0022] Among them, IR (x,y) is the pixel value of image IR at (x,y), FR (x,y)is the pixel value of image FR at (x, y);

[0023] Step 4.3: Traverse the image FW and count the number of positions safeW with pixel values ​​of 255 in the target area;

[0024] Step 4.4: Calculate the insecurity rate, the expression is:

[0025] rate=unsafeR / safeR (2)

[0026] Step 4.5: If safeW is less than the white threshold, it means that the color of the bar is more inclined to red. When the unsafe rate rate is greater than the preset threshold, it indicates that steel piling occurs in the image to be detected, otherwise it indicates that there is no steel piling in the image to be detected; if safeW is not less than the white threshold, it means that the color of the bar is more inclined to white. When the unsafe count value unsafeR is greater than the preset unsafe threshold, it indicates that there is steel piling in the image to be detected, otherwise it indicates that there is no steel piling in the image to be detected.

[0027] Preferably, the image binarization process includes: setting thresholds T1 and T2, (x,y) Perform segmentation and segment image g (x,y) for:

[0028]

[0029] The morphological operation of binary image adopts morphological opening operation, which includes corrosion and expansion. First, corrosion operation is performed to eliminate pseudo defects. The expression is:

[0030]

[0031] Among them, A is the binary image, B is the structural element, x and y represent the position coordinates, is the corrosion operation symbol, B (x,y) Represents a binary image at coordinate (x, y);

[0032] Then the dilation operation is performed to fill the holes in the target area to eliminate the small particle noise in the target area. The expression is:

[0033]

[0034] in, is the expansion operation symbol;

[0035] Finally, the binarized and morphologically processed images are added together and saturation operation is performed. If the sum of the pixel values ​​at the same position of the two images is greater than 255, the pixel value in the result image is set to 255; the pixel value expression is:

[0036]

[0037] Among them, P and Q are the pixel values ​​after binarization and morphological processing for supervised detection of steel pile events, respectively. xy and Q xy is the pixel value at (x,y) of the image after binarization and morphological processing.

[0038] The computer vision-based steel pile detection system provided by the present invention includes:

[0039] Module M1: extract still frames from the input high-speed video sequence to obtain the rod image that needs to be detected and processed;

[0040] Module M2: read the first N images in the image stream as positioning images, and then perform preprocessing to generate a positioning template file, where N is a natural number greater than or equal to 1;

[0041] Module M3: reads the N+1th image and all subsequent images in the image stream, and then performs preprocessing to extract the color features in the target area;

[0042] Module M4: Load the positioning template file of the corresponding camera, compare the color features in the extracted target area with the color features in the positioning template file, and determine whether steel pile occurs; if steel pile occurs, trigger an alarm message to the control center, and keep the steel pile photos in the abnormal situation folder; otherwise, save the production photos periodically.

[0043] Preferably, the module M1 comprises: collecting video streams in the rod conveying area; using TCP communication mode to transmit the video stream content to the NVR; taking a preset frame number I for the rod conveying video, extracting images every I frames, and selecting a preset interval frame number according to the conveying speed of the rod to convert the input video stream into an image stream;

[0044] The module M2 includes: reading the first N images in the image stream as positioning images, the positioning images include those without steel and with steel, and the steel images in the selected positioning template file need to cover the positions of all rods under safe conditions; performing preprocessing operations on the images, including morphological processing to eliminate noise in the target area, and image binarization to segment the target area, wherein the image needs to be binarized twice, and the white and red colors of the rods are used as thresholds for binarization respectively; accumulating the results of the binarization of all images, and saving the final accumulated results in a DAT file, and also saving them in different files according to the type of color and the camera number.

[0045] Preferably, the module M3 comprises:

[0046] Module M3.1: Binarize the image to be detected, and set the color threshold according to the color of the helmet;

[0047] Module M3.2: Perform contour detection on the binary image, save all contour points in the contours vector, and then obtain the circumscribed rectangle of the contour through the API interface RotatedRect() provided by OpenCV;

[0048] Module M3.3: Determine whether the outline belongs to a safety helmet based on the outline size and the horizontal and vertical spans of the outline. If it belongs to a safety helmet, it means that a worker is repairing the equipment, and the steel pile inspection is directly determined to be safe;

[0049] Module M3.4: Binarize the image to be inspected again, remove pixels that are not in the preset color range, and set the color threshold according to the color of the rod to obtain the binary images IW and IR.

[0050] Preferably, the module M4 comprises:

[0051] Module M4.1: Load the positioning template file, read the binary data in the file, convert the data into MAT format, and save them as FW and FR respectively;

[0052] Module M4.2: Read the binary images IW and IR, traverse the target area of ​​image IR, when the pixel value at the position (x, y) in image IR is 255, and the pixel value at the same position in image FR is not 255, the red pixel danger count value unsafeR is increased by 1; when the pixel value at the position (x, y) in image FR is 255, the red pixel safety count value safeR is increased by 1, and the expression is:

[0053]

[0054] Among them, IR (x,y) is the pixel value of image IR at (x,y), FR (x,y) is the pixel value of image FR at (x, y);

[0055] Module M4.3: traverse the image FW and count the number of positions safeW with a pixel value of 255 in the target area;

[0056] Module M4.4: Calculate the unsafe rate, the expression is:

[0057] rate=unsafeR / safeR (2)

[0058] Module M4.5: If safeW is less than the white threshold, it means that the color of the bar is more inclined to red. When the unsafe rate rate is greater than the preset threshold, it indicates that steel pile phenomenon occurs in the image to be detected, otherwise it indicates that there is no steel pile phenomenon in the image to be detected; if safeW is not less than the white threshold, it means that the color of the bar is more inclined to white. When the unsafe count value unsafeR is greater than the preset unsafe threshold, it indicates that there is steel pile phenomenon in the image to be detected, otherwise it indicates that there is no steel pile phenomenon in the image to be detected.

[0059] Preferably, the image binarization process includes: setting thresholds T1 and T2, (x,y) Perform segmentation and segment image g (x,y) for:

[0060]

[0061] The morphological operation of binary image adopts morphological opening operation, which includes corrosion and expansion. First, corrosion operation is performed to eliminate pseudo defects. The expression is:

[0062]

[0063] Among them, A is the binary image, B is the structural element, x and y represent the position coordinates, is the corrosion operation symbol, B (x,y) Represents a binary image at coordinate (x, y);

[0064] Then the dilation operation is performed to fill the holes in the target area to eliminate the small particle noise in the target area. The expression is:

[0065]

[0066] in, is the expansion operation symbol;

[0067] Finally, the binarized and morphologically processed images are added together and saturation operation is performed. If the sum of the pixel values ​​at the same position of the two images is greater than 255, the pixel value in the result image is set to 255; the pixel value expression is:

[0068]

[0069] Among them, P and Q are the pixel values ​​after binarization and morphological processing for supervised detection of steel pile events, respectively. xy and Q xy is the pixel value at (x,y) of the image after binarization and morphological processing.

[0070] Compared with the prior art, the present invention has the following beneficial effects:

[0071] (1) Compared with traditional human eye detection, the present invention greatly shortens the detection time, improves the detection efficiency, and overcomes the problems of inconsistent detection results and high false detection rate in different production environments;

[0072] (2) The present invention comprehensively considers the interference caused by environmental factors, solves the visual inspection problem in complex environments through pre-processing noise reduction and contour search, and ensures the personal safety of workers;

[0073] (3) The present invention utilizes and optimizes algorithms in computer vision to accurately detect the steel pile phenomenon. The method is simple, flexible to implement, and highly practical. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0075] Figure 1 The present invention is a flow chart of a method for detecting steel piles based on computer vision;

[0076] Figure 2 It is the steel-free drawing in the application scenario;

[0077] Figure 3 There are steel drawings for application scenarios;

[0078] Figure 4 This is the false detection image due to the helmet in the application scenario;

[0079] Figure 5 This is a flowchart for detecting a safety helmet in the present invention;

[0080] Figure 6 Generate a white steel pile example image for the application scenario;

[0081] Figure 7 An example diagram of red steel pile is generated for the application scenario. DETAILED DESCRIPTION

[0082] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several changes and improvements can also be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0083] Example 1

[0084] like Figure 1 As shown, the present invention provides a method for detecting steel piles based on computer vision, and the specific steps are as follows:

[0085] Step 1: Arrange the camera to ensure that the camera can capture the rod to be inspected, extract the still frame from the input high-speed video sequence, and obtain the rod image that needs to be inspected;

[0086] Step 2: Read the first N images in the image stream as positioning images, preprocess the images, and generate a positioning template file, where N is a natural number greater than or equal to 1;

[0087] Step 3: Read the N+1th image and all subsequent images in the image stream, preprocess the images, and extract the color features in the target area;

[0088] Step 4: Load the positioning template file of the corresponding camera, compare the color features extracted in step (3) with the color features in the positioning template file, and determine whether steel pile occurs. If steel pile occurs, an alarm message is triggered and sent to the control center, and the steel pile photo is retained in the abnormal situation folder; if production is normal, the production photos are periodically saved.

[0089] In step 1, the extraction of the image of the rod to be detected is achieved through the following steps: the first step is to collect the video stream in the rod transmission area; the second step is to use the TCP communication mode to transmit the video stream content to the NVR; the third step is to take the frame number I of the transmission video of the rod, extract the image every I frame number, and according to the transmission speed of the rod, select the preset interval frame number to convert the input video stream into an image stream.

[0090] In step 2, the generation of the positioning template file is achieved by the following steps: the first step is to read the first N images in the image stream as positioning images, such as Figure 2 and Figure 3 As shown, the positioning image includes two types of images: one without steel and one with steel (the one without steel means that there is no bar at the steel-walking position during the normal operation of the machine, and the one with steel means that there is a bar at the steel-walking position during the normal operation of the machine). The steel image in the selected positioning template file needs to cover all possible positions of the bar under safe conditions, and the width of the bar needs to be diverse; the second step is to preprocess the image, which includes morphological processing to eliminate noise in the target area and image binarization to segment the target area, wherein the image needs to be binarized twice, with the white and red colors of the bar as thresholds for binarization respectively; the third step is to accumulate the results of the binarization of all images, and save the final accumulated result in a DAT file, and also save it in different files according to the type of color and camera number.

[0091] In step 3, the specific process of preprocessing the image to be detected is as follows:

[0092] Step 1: Binarize the image to be tested, and set the color threshold according to the color of the helmet. Since the colors of helmets on different production lines are different, the color of helmets on different production lines is extracted and the color threshold is set. Binarization refers to the process of setting the grayscale value of the pixel on the image to 0 or 255, making the entire image appear in obvious black and white.

[0093] Step 2: Perform contour detection on the binary image in step (1), save all contour points in the contours vector, and then obtain the circumscribed rectangle of the contour through the API interface RotatedRect() provided by OpenCV;

[0094] Step 3: Determine whether the outline belongs to a safety helmet based on the outline size and the horizontal and vertical spans of the outline. If it belongs to a safety helmet, it means that a worker is repairing the equipment, and the steel pile inspection is directly determined to be safe;

[0095] like Figure 4 The color features of the helmet and the bar are similar, mainly containing red, which makes it easy to detect the occurrence of steel piling when a red helmet appears in the image. By observing the features of the helmet and the bar, although the two are similar in color, their contour shapes are different. The contour shape of the bar is similar to a rod, while the contour shape of the helmet is similar to a regular square. The method of detecting the helmet is as follows Figure 5 As shown, the specific steps are as follows:

[0096] Use OpenCV's API interface findContours() to obtain the contour of the binary image;

[0097] Calculate the size of the contour and the number of contour points;

[0098] Determine whether the number and size of the contours are within the specified threshold. If the number of contour points is too large or too small, and the contour size does not meet the specified threshold, it is determined that there is no safety helmet; otherwise, there is a safety helmet.

[0099] Step 4: Binarize the image to be inspected again to remove pixels that are not within the specific color range. The result of the processing is usually a one-dimensional binary image. The color threshold is set according to the color of the rod, and the result is saved as IW and IR.

[0100] In step 4, the specific steps of comparing the color feature to be detected with the color feature in the positioning template file are as follows:

[0101] Step 1: Load the positioning template file, read the binary data in the file, convert the data into MAT format, and save them as FW and FR respectively;

[0102] Step 2: Read the binary images IW and IR, traverse the target area of ​​image IR, and the rod will only be transferred within this range under safe conditions. When the pixel value at the position (x, y) in image IR is 255, and the pixel value at the same position in image FR is also not 255, the red pixel danger count value unsafeR is increased by 1; when the pixel value at the position (x, y) in image FR is 255, the red pixel safety count value safeR is increased by 1. The expression is:

[0103]

[0104] Among them, IR (x,y) is the pixel value of image IR at (x,y), FR (x,y) is the pixel value of image FR at (x, y);

[0105] Step 3: traverse the image FW and count the number of positions safeW where the pixel value is 255 in the target area;

[0106] Step 4: Calculate the insecurity rate, the expression is:

[0107] rate=unsafeR / safeR (2)

[0108] Step 5: If safeW is less than the white threshold, it means the color of the bar is more red, e.g. Figure 6 As shown in , when the unsafe rate rate is greater than a certain threshold, the steel pile phenomenon appears in the image to be detected, otherwise the steel pile phenomenon does not appear in the image to be detected; otherwise, it means that the color of the bar is more inclined to white, such as Figure 7 As shown, when the unsafe count value unsafeR is greater than a certain unsafe threshold, steel piling phenomenon appears in the image to be detected, otherwise, steel piling phenomenon does not appear in the image to be detected.

[0109] Image binary segmentation, select appropriate thresholds T1 and T2, and segment the original image f (x,y) Perform segmentation and segment image g (x,y) It can be expressed as:

[0110]

[0111] After that, the binary image is subjected to morphological operation, and the morphological opening operation is adopted. The opening operation includes corrosion and expansion. First, the corrosion operation is performed to eliminate pseudo defects. The expression is:

[0112]

[0113] Where A is the binary image, B is the structural element, x and y represent the position coordinates, is the corrosion operation symbol, B (x,y)Represents a binary image at coordinate (x, y).

[0114] Then the dilation operation is performed to fill some holes in the target area to eliminate the small particle noise in the target area. The expression is:

[0115]

[0116] Where A is the binary image after the corrosion operation, B is the structural element, x and y represent the position coordinates, is the symbol for the expansion operation.

[0117] Finally, the binarized and morphologically processed images are added together and a saturation operation is performed. That is, if the sum of the pixel values ​​at the same position of the two images is greater than 255, the pixel value in the result image is set to 255; otherwise, the pixel value is expressed as follows:

[0118]

[0119] Where P and Q are the pixel values ​​after binarization and morphological processing for supervised detection of steel pile events, respectively. xy and Q xy is the pixel value at (x,y) of the image after binarization and morphological processing.

[0120] In summary, the present invention uses computer vision to supervise and detect steel piling events, overcoming the limitations of manual detection to improve the level of automated production in factory manufacturing of prefabricated components; compared with existing detection methods, this method has the characteristics of fast speed and high accuracy.

[0121] Example 2

[0122] The present invention also provides a steel pile detection system based on computer vision. The steel pile detection system based on computer vision can be implemented by executing the process steps of the steel pile detection method based on computer vision, that is, those skilled in the art can understand the steel pile detection method based on computer vision as a preferred implementation of the steel pile detection system based on computer vision.

[0123] The computer vision-based steel pile detection system provided by the present invention includes: module M1: extracting still frames from an input high-speed image video sequence to obtain a rod image that needs to be detected and processed; module M2: reading the first N images in the image stream as positioning images, and then preprocessing to generate a positioning template file, wherein N is a natural number greater than or equal to 1; module M3: reading the N+1th image and all subsequent images in the image stream, and then preprocessing to extract color features in a target area; module M4: loading the positioning template file of the corresponding camera, comparing the extracted color features in the target area with the color features in the positioning template file, and determining whether steel pile occurs; if steel pile occurs, triggering an alarm message to be sent to a control center, and retaining photos of the steel pile in an abnormal situation folder; otherwise, periodically saving production photos.

[0124] The module M1 includes: collecting video streams in the rod conveying area; using TCP communication mode to transmit the video stream content to the NVR; taking a preset frame number I for the rod conveying video, extracting images every I frame number, and selecting a preset interval frame number according to the conveying speed of the rod to convert the input video stream into an image stream;

[0125] The module M2 includes: reading the first N images in the image stream as positioning images, the positioning images include those without steel and with steel, and the steel images in the selected positioning template file need to cover the positions of all rods under safe conditions; performing preprocessing operations on the images, including morphological processing to eliminate noise in the target area, and image binarization to segment the target area, wherein the image needs to be binarized twice, and the white and red colors of the rods are used as thresholds for binarization respectively; accumulating the results of the binarization of all images, and saving the final accumulated results in a DAT file, and also saving them in different files according to the type of color and the camera number.

[0126] The module M3 includes: module M3.1: binarizing the image to be detected, and setting the color threshold according to the color of the helmet; module M3.2: performing contour detection on the binary-processed graphics, saving all contour points in the contours vector, and then obtaining the circumscribed rectangle of the contour through the API interface RotatedRect() provided by OpenCV; module M3.3: judging whether the contour belongs to a helmet according to the contour size and the horizontal and vertical spans of the contour. If it belongs to a helmet, it means that a worker is repairing the equipment, and the steel pile detection is directly judged to be safe; module M3.4: binarizing the image to be detected again, removing pixels that are not in the preset color range, and setting the color threshold according to the color of the rod to obtain the binary images IW and IR.

[0127] The module M4 includes: module M4.1: loading the positioning template file, reading the binary data in the file, converting the data into MAT format, and saving them as FW and FR respectively; module M4.2: reading the binary images IW and IR, traversing the target area of ​​the image IR, when the pixel value at the position (x, y) in the image IR is 255, and the pixel value at the same position in the image FR is not 255, the red pixel point danger count value unsafeR is increased by 1; when the pixel value at the position (x, y) in the image FR is 255, the red pixel point safety count value safeR is increased by 1, and the expression is:

[0128]

[0129] Among them, IR (x,y) is the pixel value of the image IR at (x,y), FR (x,y) is the pixel value of image FR at (x, y);

[0130] Module M4.3: traverse the image FW and count the number of positions safeW with a pixel value of 255 in the target area;

[0131] Module M4.4: Calculate the unsafe rate, the expression is:

[0132] rate=unsafeR / safeR (2)

[0133] Module M4.5: If safeW is less than the white threshold, it means that the color of the bar is more inclined to red. When the unsafe rate rate is greater than the preset threshold, it indicates that steel pile phenomenon occurs in the image to be detected, otherwise it indicates that there is no steel pile phenomenon in the image to be detected; if safeW is not less than the white threshold, it means that the color of the bar is more inclined to white. When the unsafe count value unsafeR is greater than the preset unsafe threshold, it indicates that there is steel pile phenomenon in the image to be detected, otherwise it indicates that there is no steel pile phenomenon in the image to be detected.

[0134] The image binarization process includes: setting thresholds T1 and T2, (x,y) Perform segmentation and segment image g (x,y) for:

[0135]

[0136] The morphological operation of binary image adopts morphological opening operation, which includes corrosion and expansion. First, corrosion operation is performed to eliminate pseudo defects. The expression is:

[0137]

[0138] Among them, A is the binary image, B is the structural element, x and y represent the position coordinates, is the corrosion operation symbol, B (x,y) Represents a binary image at coordinate (x, y);

[0139] Then the dilation operation is performed to fill the holes in the target area to eliminate the small particle noise in the target area. The expression is:

[0140]

[0141] in, is the expansion operation symbol;

[0142] Finally, the binarized and morphologically processed images are added together and saturation operation is performed. If the sum of the pixel values ​​at the same position of the two images is greater than 255, the pixel value in the result image is set to 255; the pixel value expression is:

[0143]

[0144] Among them, P and Q are the pixel values ​​after binarization and morphological processing for supervised detection of steel pile events, respectively. xy and Q xy is the pixel value at (x,y) of the image after binarization and morphological processing.

[0145] Those skilled in the art know that, in addition to implementing the system, device and its various modules provided by the present invention in a purely computer-readable program code, it is entirely possible to implement the same program in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers and embedded microcontrollers by logically programming the method steps. Therefore, the system, device and its various modules provided by the present invention can be considered as a hardware component, and the modules included therein for implementing various programs can also be considered as structures within the hardware component; the modules for implementing various functions can also be considered as both software programs for implementing the method and structures within the hardware component.

[0146] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essence of the present invention. In the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.

Claims

1. A method for detecting steel piles based on computer vision, characterized in that: include: Step 1: Arrange the camera to ensure that the camera can capture the rod to be inspected, extract the still frame of the input high-speed image video sequence, and obtain the rod image that needs to be inspected; Step 2: Read the first N images in the image stream as positioning images, and then perform preprocessing to generate a positioning template file, where N is a natural number greater than or equal to 1; Step 3: Read the N+1th image and all subsequent images in the image stream, and then perform preprocessing to extract the color features in the target area; Step 4: Load the positioning template file of the corresponding camera, compare the color features in the extracted target area with the color features in the positioning template file, and determine whether steel piling occurs; If steel pile occurs, an alarm message will be triggered and sent to the control center, and photos of the steel pile will be kept in the abnormal situation folder; Otherwise, save production photos periodically.

2. The method for detecting steel piles based on computer vision according to claim 1, characterized in that: The step 1 comprises: collecting a video stream in the rod conveying area; using the TCP communication mode to transmit the video stream content to the NVR; taking a preset frame number I for the rod conveying video, extracting an image every I frame number, and selecting a preset interval frame number according to the conveying speed of the rod to convert the input video stream into an image stream; The step 2 includes: reading the first N images in the image stream as positioning images, the positioning images include images without steel and images with steel, and the steel images in the selected positioning template file need to cover the positions of all rods under safe conditions; performing preprocessing operations on the images, including morphological processing to eliminate noise in the target area, and image binarization to segment the target area, wherein the image needs to be binarized twice, and the white and red colors of the rods are used as thresholds for binarization respectively; accumulating the results of the binarization of all images, and saving the final accumulated results in a DAT file, and also saving them in different files according to the type of color and the camera number.

3. The method for detecting steel piles based on computer vision according to claim 2, characterized in that: The step 3 comprises: Step 3.1: Binarize the image to be detected, and set the color threshold according to the color of the helmet; Step 3.2: Perform contour detection on the binary image, save all contour points in the contours vector, and then obtain the circumscribed rectangle of the contour through the API interface RotatedRect() provided by OpenCV; Step 3.3: Determine whether the outline belongs to a safety helmet based on the outline size and the horizontal and vertical spans of the outline. If it belongs to a safety helmet, it means that a worker is repairing the equipment. The steel pile inspection is directly determined to be safe. Step 3.4: Binarize the image to be inspected again, remove pixels that are not in the preset color range, and set the color threshold according to the color of the rod to obtain the binary images IW and IR.

4. The method for detecting steel piles based on computer vision according to claim 3, characterized in that: The step 4 comprises: Step 4.1: Load the positioning template file, read the binary data in the file, convert the data into MAT format, and save them as FW and FR respectively; Step 4.2: Read the binary images IW and IR, traverse the target area of ​​image IR, when the pixel value at the position (x, y) in image IR is 255, and the pixel value at the same position in image FR is not 255, then the red pixel danger count value unsafeR is increased by 1; when the pixel value at the position (x, y) in image FR is 255, the red pixel safety count value safeR is increased by 1, and the expression is: Among them, IR (x,y) is the pixel value of image IR at (x,y), FR (x,y) is the pixel value of image FR at (x, y); Step 4.3: Traverse the image FW and count the number of positions safeW with pixel values ​​of 255 in the target area; Step 4.4: Calculate the insecurity rate, the expression is: rate=unsafeR / safeR (2) Step 4.5: If safeW is less than the white threshold, it means that the color of the bar is more inclined to red. When the unsafe rate rate is greater than the preset threshold, it indicates that steel piling occurs in the image to be detected, otherwise it indicates that there is no steel piling in the image to be detected; if safeW is not less than the white threshold, it means that the color of the bar is more inclined to white. When the unsafe count value unsafeR is greater than the preset unsafe threshold, it indicates that there is steel piling in the image to be detected, otherwise it indicates that there is no steel piling in the image to be detected.

5. The method for detecting steel piles based on computer vision according to claim 4, characterized in that: The image binarization process includes: setting thresholds T1 and T2, (x,y) Perform segmentation and segment image g (x,y) for: The morphological operation of binary image adopts morphological opening operation, which includes corrosion and expansion. First, corrosion operation is performed to eliminate pseudo defects. The expression is: Among them, A is the binary image, B is the structural element, x and y represent the position coordinates, is the corrosion operation symbol, B (x,y) Represents a binary image at coordinate (x, y); Then the dilation operation is performed to fill the holes in the target area to eliminate the small particle noise in the target area. The expression is: Among them, ⊕ is the expansion operation symbol; Finally, the binarized and morphologically processed images are added together and saturation operation is performed. If the sum of the pixel values ​​at the same position of the two images is greater than 255, the pixel value in the result image is set to 255; the pixel value expression is: Among them, P and Q are the pixel values ​​after binarization and morphological processing for supervised detection of steel pile events, respectively. xy and Q xy is the pixel value at (x,y) of the image after binarization and morphological processing.

6. A steel pile detection system based on computer vision, characterized in that: include: Module M1: extract still frames from the input high-speed video sequence to obtain the rod image that needs to be detected and processed; Module M2: read the first N images in the image stream as positioning images, and then perform preprocessing to generate a positioning template file, where N is a natural number greater than or equal to 1; Module M3: reads the N+1th image and all subsequent images in the image stream, and then performs preprocessing to extract the color features in the target area; Module M4: Load the positioning template file of the corresponding camera, compare the color features in the extracted target area with the color features in the positioning template file, and determine whether steel piling occurs; If steel pile occurs, an alarm message will be triggered and sent to the control center, and photos of the steel pile will be kept in the abnormal situation folder; Otherwise, save production photos periodically.

7. The computer vision-based steel pile detection system according to claim 6, characterized in that: The module M1 includes: collecting video streams in the rod conveying area; using TCP communication mode to transmit the video stream content to the NVR; taking a preset frame number I for the rod conveying video, extracting images every I frame number, and selecting a preset interval frame number according to the conveying speed of the rod to convert the input video stream into an image stream; The module M2 includes: reading the first N images in the image stream as positioning images, the positioning images include those without steel and with steel, and the steel images in the selected positioning template file need to cover the positions of all rods under safe conditions; performing preprocessing operations on the images, including morphological processing to eliminate noise in the target area, and image binarization to segment the target area, wherein the image needs to be binarized twice, and the white and red colors of the rods are used as thresholds for binarization respectively; accumulating the results of the binarization of all images, and saving the final accumulated results in a DAT file, and also saving them in different files according to the type of color and the camera number.

8. The computer vision-based steel pile detection system according to claim 7, characterized in that: The module M3 comprises: Module M3.1: Binarize the image to be detected, and set the color threshold according to the color of the helmet; Module M3.2: Perform contour detection on the binary image, save all contour points in the contours vector, and then obtain the circumscribed rectangle of the contour through the API interface RotatedRect() provided by OpenCV; Module M3.3: Determine whether the outline belongs to a safety helmet based on the outline size and the horizontal and vertical spans of the outline. If it belongs to a safety helmet, it means that a worker is repairing the equipment, and the steel pile inspection is directly determined to be safe; Module M3.4: Binarize the image to be inspected again, remove pixels that are not in the preset color range, and set the color threshold according to the color of the rod to obtain the binary images IW and IR.

9. The computer vision-based steel pile detection system according to claim 8, characterized in that: The module M4 comprises: Module M4.1: Load the positioning template file, read the binary data in the file, convert the data into MAT format, and save them as FW and FR respectively; Module M4.2: Read the binary images IW and IR, traverse the target area of ​​image IR, when the pixel value at the position (x, y) in image IR is 255, and the pixel value at the same position in image FR is not 255, the red pixel danger count value unsafeR is increased by 1; when the pixel value at the position (x, y) in image FR is 255, the red pixel safety count value safeR is increased by 1, and the expression is: Among them, IR (x,y) is the pixel value of image IR at (x,y), FR (x,y) is the pixel value of image FR at (x, y); Module M4.3: traverse the image FW and count the number of positions safeW with a pixel value of 255 in the target area; Module M4.4: Calculate the unsafe rate, the expression is: rate=unsafeR / safeR (2) Module M4.5: If safeW is less than the white threshold, it means that the color of the bar is more inclined to red. When the unsafe rate rate is greater than the preset threshold, it indicates that steel pile phenomenon occurs in the image to be detected, otherwise it indicates that there is no steel pile phenomenon in the image to be detected; if safeW is not less than the white threshold, it means that the color of the bar is more inclined to white. When the unsafe count value unsafeR is greater than the preset unsafe threshold, it indicates that there is steel pile phenomenon in the image to be detected, otherwise it indicates that there is no steel pile phenomenon in the image to be detected.

10. The computer vision-based steel pile detection system according to claim 9, characterized in that: The image binarization process includes: setting thresholds T1 and T2, (x,y) Perform segmentation and segment image g (x,y) for: The morphological operation of binary image adopts morphological opening operation, which includes corrosion and expansion. First, corrosion operation is performed to eliminate pseudo defects. The expression is: Among them, A is the binary image, B is the structural element, x and y represent the position coordinates, is the corrosion operation symbol, B (x,y) Represents a binary image at coordinate (x, y); Then the dilation operation is performed to fill the holes in the target area to eliminate the small particle noise in the target area. The expression is: Among them, ⊕ is the expansion operation symbol; Finally, the binarized and morphologically processed images are added together and saturation operation is performed. If the sum of the pixel values ​​at the same position of the two images is greater than 255, the pixel value in the result image is set to 255; the pixel value expression is: Among them, P and Q are the pixel values ​​after binarization and morphological processing for supervised detection of steel pile events, respectively. xy and Q xy is the pixel value at (x,y) of the image after binarization and morphological processing.

Images