Surface defect detection system and method based on image feature extraction

Abstract

The invention discloses a surface defect detection system and method based on image feature extraction. The method comprises a surface extraction stage and a defect detection stage. A surface extraction stage: acquiring industrial field video stream data, and detecting and extracting a to-be-detected surface area; and a defect detection stage: carrying out defect detection on the image in the to-be-detected surface range by using an image feature extraction method, and determining a to-be-repaired range according to a defect position. According to the invention, defect detection can be carried out on the to-be-detected surface in the monitoring video, and the defect detection accuracy in an industrial environment is improved; the method is suitable for an efficient extraction detection process, and can reduce the calculation time in a scene where the position of industrial equipment is relatively fixed.

Description

Technical Field

[0001] The present invention belongs to the technical field of defect detection, and in particular relates to a surface defect detection system and method based on image feature extraction. Background Art

[0002] At present, surface defect detection, as a key quality control method, is widely used in the manufacturing and maintenance of transformers, switch cabinets, conductive rods and other equipment in the power grid field. In these fields, the quality of the equipment surface is directly related to the safety of the power system and the reliability of operation. Therefore, it is particularly important to detect defects on the equipment surface. Traditional surface defect detection methods mainly rely on manual detection and simple physical detection methods. These methods have problems such as low efficiency, poor accuracy, and strong subjectivity, and are difficult to meet the needs of modern power industry production and maintenance.

[0003] In addition, since there may be tiny cracks, pits, scratches and other defects on the surface of power grid equipment, these minor defects are often difficult to be accurately identified and recorded manually during rapid production and maintenance; and long-term manual inspection work can easily lead to fatigue of inspectors, further reducing the efficiency and accuracy of inspection. Another common traditional method is physical detection. Although it improves the automation level and accuracy of detection to a certain extent, it usually requires special detection equipment and complex operating procedures, which increases the cost of detection and the difficulty of operation. It has high requirements for the operating environment and is difficult to fully meet in the actual power production and maintenance environment.

[0004] Based on the above considerations, it is necessary to propose a surface defect detection system and method based on image feature extraction, which combines the camera to obtain and detect the pixel-level information of the surface to be detected contained in the monitoring video stream data, and realizes real-time surface defect detection function on computing devices with limited computing power. Summary of the invention

[0005] In view of the above-mentioned deficiencies of the prior art, an object of the present invention is to provide a surface defect detection system and method based on image feature extraction, so as to solve the problem in the prior art that lightweight equipment cannot be used to perform real-time surface defect detection in an industrial environment.

[0006] To achieve the above object, the technical solution adopted by the present invention is as follows:

[0007] A surface defect detection system based on image feature extraction of the present invention comprises: a remote detector, a server and a client;

[0008] A remote detector is used to obtain industrial site video stream data in real time, detect and extract the surface area to be detected in the video stream data, perform defect detection on the image within the surface area to be detected, determine the range to be repaired according to the defect location, and send the repair range result data corresponding to the video stream to the server;

[0009] The server is used to obtain the result data sent by the remote detector in real time and store it;

[0010] The client is used to obtain result data from the server in real time and query historical result data.

[0011] Further, the remote detector includes: a camera, a processor and a communication interface;

[0012] A camera for collecting video stream data of an industrial site containing a surface to be inspected;

[0013] A processor, using an image feature extraction method to perform defect detection on an image within the range of the surface to be detected, and determining a range to be repaired according to the defect location;

[0014] The communication interface is used to send the repair range result data corresponding to the video stream to the server.

[0015] A surface defect detection method based on image feature extraction of the present invention is based on the above system and includes a surface extraction stage and a defect detection stage;

[0016] Surface extraction stage: Obtain industrial site video stream data, detect and extract the surface area to be detected;

[0017] Defect detection stage: Use image feature extraction methods to perform defect detection on images within the range of the surface to be inspected, and determine the scope that needs to be repaired based on the defect location.

[0018] Furthermore, the specific steps of the surface extraction stage are:

[0019] (11) The industrial site video stream data collected in real time is continuously divided into sampling segments S with adaptively changing lengths using the interval sampling method, and this is used as a processing cycle;

[0020] (12) For each sampling segment S, a pixel feature method based on phased processing is used to selectively extract the coordinates of the surface area to be detected in the video, H = ((x 1 ,y 1 ),(x 2 ,y 2 ),(x 3 ,y 3 ),(x 4 ,y4 )) and maps it to a new width x 2 -x 1 , Gao Wei In the frame of the surface area to be detected.

[0021] Furthermore, the interval sampling method in step (11) is specifically:

[0022] (111) When continuously acquiring video stream data, start with 1, record the incoming video frame index, and start with video frame number 2, and use every n+1 frames as a sampling segment S, where n increases linearly from 0;

[0023] (112) For each sampling segment S, it is divided into a retention period S p and extraction period S d , for each video frame F in the sampling segment S, every n+1 frames is taken as a sampling interval; the last frame F in the sampling interval is taken as d Add extraction period S d , the first n frames F in each sampling interval p1 To F pn Add retention period S p ; Except for the first frame of the video stream data, each sampling segment is executed in sequence according to the retention period-extraction period.

[0024] Furthermore, the pixel feature method for phased processing in step (12) is specifically as follows:

[0025] (121) Extraction period S d Contains a video frame F d , an adaptive strategy is used to extract the surface area to be detected. The extraction stages used in an extraction period are divided into: range determination and extraction mapping; the extraction process included in a complete extraction period is divided into: range determination-extraction mapping process D 1 , Extraction mapping process D 2 , and Extraction Mapping-Scope Determination-Extraction Mapping Process D 3 ; When the video frame index is 1, the range determination-extraction mapping process D is used 1 ; Except for the first frame, if the retention period requires a range reset, the range determination-extraction mapping process D is used 1 ; Otherwise, use the extraction mapping process D 2 , make a judgment during the process, if the range needs to be reset, then use the extraction mapping-range determination-extraction mapping process D 3 ; After the range is reset, n starts to increase linearly from 0 again;

[0026] (122) Retention period S p Contains n video frames Fp1 to F pn , determine the coordinates H of the surface area to be detected obtained by applying the range determination of the previous sampling segment to the current sampling segment video frame. If the determination result is that the pixel values ​​on both sides of at least one of the two range determination lines are significantly different, it means that the boundary coordinates of the surface to be detected in the sampling segment have changed, and the range needs to be reset during the extraction period of the sampling segment; wherein, each time the range is reset, the length of the sampling segment is reset, n=0, and continues to increase linearly.

[0027] Furthermore, the specific steps of determining the range in step (121) are:

[0028] (1211) The input image is converted into a grayscale image. The grayscale value calculation formula is as follows:

[0029] I gray (x,y)=0.299I r (x,y)+0.587I g (x,y)+0.114I b (x,y)

[0030] Among them, I r (x,y),I g (x,y),I b (x, y) are the pixel values ​​of the red, green and blue channels of the image respectively; I gray (x, y) is the pixel value of the grayscale image. The grayscale image is processed by histogram equalization to enhance the contrast of the image and obtain the histogram equalization image pixel value I equalized (x, y) and its corresponding video frame F′ d , the calculation formula is as follows:

[0031]

[0032] Among them, I gray.max and I gray.min are the maximum and minimum pixel values ​​of the grayscale image respectively;

[0033] (1212) Using the local threshold method to analyze the above video frame F′ d Perform edge detection to improve detection accuracy as follows:

[0034]

[0035] Among them, T(x,y) is the local threshold, μ(x,y) is the average gray value around the pixel (x,y), σ(x,y) is the standard deviation, k is the adjustment parameter, R is the dynamic range constant, and the gradient value of the pixel is calculated as follows:

[0036] G x(x,y)=I equalized (x+1,y)-I equalized (x-1,y)

[0037] G y (x,y)=I equalized (x,y+1)-I equalized (x,y-1)

[0038]

[0039] Among them, G x (x,y) is the gradient value of the pixel point (x,y) in the x direction, G y (x,y) is the gradient value of the pixel point (x,y) in the y direction, G(x,y) is the total gradient value of the pixel point (x,y), when G(x,y) exceeds T(x,y), the pixel point is retained as an edge point;

[0040] (1213) Performing Hough line detection on the detected edge image, extracting and screening out the two longest straight lines, extending the two screened straight lines to the edge of the image, and obtaining the coordinates of the surface area to be detected.

[0041] Furthermore, the specific steps of extracting the mapping in step (121) are:

[0042] The coordinates of the surface area to be detected enclosed by the two straight lines obtained above are H, H = ((x 1 ,y 1 ),(x 2 ,y 2 ),(x 3 ,y 3 ),(x 4 ,y 4 )), and use perspective transformation to map to the new video frame F″ d The coordinates of H′ are in the range of ((0,0), (w,0), (0,h), (w,h)), where w = x 2 -x 1 , Adopting extraction and mapping process D 2 When the coordinates H of the surface area to be detected are the same as the video frame result in the previous sampling segment, the extraction mapping-range determination-extraction mapping process D is adopted. 3 When , the coordinate H of the surface area to be detected is the result of the range determination in the current sampling segment.

[0043] Furthermore, the specific steps of the defect detection stage are:

[0044] (21) Based on the new video frame F″ of the surface area to be detected d, linear defect detection and polygonal defect detection are performed in turn, and the repair range is calculated.

[0045] Furthermore, the linear defect detection method in step (21) is specifically as follows:

[0046] (211) Calculate the new video frame F″ d The sum of the grayscale values ​​of each row is as follows:

[0047]

[0048] Among them, row gray (i) represents the new video frame F″ d The sum of the grayscale values ​​of the i-th row in d (i,j) represents the new video frame F″ d The gray value of the jth pixel in the i-th row; take window as the sliding window size, and calculate the sum of the gray values ​​in the sliding window as follows:

[0049]

[0050] Among them, window gray (i) represents the sum of the grayscale values ​​in the i-th sliding window, and calculates the mean of all sliding windows;

[0051] (212) Calculate the difference between the sum of the grayscale values ​​of two adjacent sliding windows, and find the index with the smallest product of the adjacent differences: diff(i) = (window gray (i)-window gray (i+1))*(window gray (i-1)-window gray (i))

[0052] Wherein, diff(i) represents the product of the grayscale value difference between the current sliding window i and the previous and next sliding windows;

[0053] The linear defect range that needs to be repaired is E 1 , E 1 =((0,index),(w,index),(0,index+window),(w,index+window)).

[0054] Furthermore, the polygon defect detection method in step (21) is specifically as follows:

[0055] The new video frame F″ d Each pixel value in the grayscale image is binarized with reference to the preset binarization threshold μ, as follows:

[0056]

[0057] Among them, I d (x,y) is the new video frame F″ d The pixel value, I b (x, y) is the pixel value of the image after binarization;

[0058] The binary image is corroded and expanded, and its connected regions are marked. Specifically, for each pixel in the image, if the pixel value is foreground and is not marked, a breadth-first search is used to search for all connected foreground pixels starting from the pixel, marking it as a new connected region until all pixels are checked. The pixel coordinate set R of the i-th connected region is obtained as follows:

[0059] R={(x i ,y i )|(x i ,y i ) are pixel coordinates within the connected region}

[0060] According to the pixel coordinate set R, the coordinates of the four vertices of the bounding rectangle of the polygon defect are calculated as follows:

[0061] x left =min(x i )

[0062] x right =max(x i )

[0063] y up =min(y i )

[0064] y down =max(y i )

[0065] Among them, x left is the horizontal coordinate of the leftmost pixel in the polygon defect range, x right is the horizontal coordinate of the rightmost pixel in the polygon defect range, y up y is the vertical coordinate of the top pixel in the polygon defect range, down The vertical coordinate of the bottom pixel of the polygon defect range;

[0066] The polygon defect range E that needs to be repaired 2 ,as follows:

[0067] E 2 =((x left ,y up ),(x right ,y up ),(xleft ,y down ),(x right ,y down )).

[0068] Beneficial effects of the present invention:

[0069] 1. The present invention can perform defect detection on the surface to be detected in the monitoring video, and improve the accuracy of defect detection in the industrial environment;

[0070] 2. The present invention uses an interval sampling method to divide the continuous monitoring video analysis task into a retention period and an extraction period. While ensuring the detection accuracy, the computing resources can be reasonably utilized, saving the overall computing overhead;

[0071] 3. The present invention uses a phased processing method, which can be applied to efficient extraction and detection processes, and can reduce calculation time in scenarios where the location of industrial equipment is relatively fixed;

[0072] 4. Based on the processor, the present invention only needs to install a monocular camera to run a real-time surface defect detection system, and the cost is very low. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figure 1 is a system architecture diagram of the present invention;

[0074] Figure 2 is a flow chart of the method of the present invention;

[0075] Figure 3 It is a schematic diagram of the phased processing in the present invention. DETAILED DESCRIPTION

[0076] In order to facilitate the understanding of those skilled in the art, the present invention is further described below in conjunction with embodiments and drawings. The contents mentioned in the implementation modes are not intended to limit the present invention.

[0077] Reference Figure 1 As shown, a surface defect detection system based on image feature extraction of the present invention comprises: a remote detector, a server and a client;

[0078] A remote detector is used to obtain industrial site video stream data in real time, detect and extract the surface area to be detected in the video stream data, perform defect detection on the image within the surface area to be detected, determine the range to be repaired according to the defect location, and send the repair range result data corresponding to the video stream to the server;

[0079] The server is used to obtain the result data sent by the remote detector in real time and store it;

[0080] The client is used to obtain result data from the server in real time and query historical result data.

[0081] Wherein, the remote detector comprises: a camera, a processor and a communication interface;

[0082] A camera for collecting video stream data of an industrial site containing a surface to be inspected;

[0083] A processor, using an image feature extraction method to perform defect detection on an image within the range of the surface to be detected, and determining a range to be repaired according to the defect location;

[0084] The communication interface is used to send the repair range result data corresponding to the video stream to the server.

[0085] Reference Figure 2 , Figure 3 As shown, a surface defect detection method based on image feature extraction of the present invention is based on the above system and includes a surface extraction stage and a defect detection stage;

[0086] Surface extraction stage: Obtain industrial site video stream data, detect and extract the surface area to be detected;

[0087] Wherein, the specific steps of the surface extraction stage are:

[0088] (11) The industrial site video stream data collected in real time is continuously divided into sampling segments S with adaptively changing lengths using the interval sampling method, and this is used as a processing cycle;

[0089] (12) For each sampling segment S, a pixel feature method based on phased processing is used to selectively extract the coordinates of the surface area to be detected in the video, H = ((x 1 ,y 1 ),(x 2 ,y 2 ),(x 3 ,y 3 ),(x 4 ,y 4 )) and maps it to a new width x 2 -x 1 , Gao Wei In the frame of the surface area to be detected.

[0090] Specifically, the interval sampling method in step (11) is as follows:

[0091] (111) When continuously acquiring video stream data, start with 1, record the incoming video frame index, and start with video frame number 2, and use every n+1 frames as a sampling segment S, where n increases linearly from 0;

[0092] (112) For each sampling segment S, it is divided into a retention period S p and extraction period S d , for each video frame F in the sampling segment S, every n+1 frames is taken as a sampling interval; the last frame F in the sampling interval is taken as d Add extraction period S d , the first n frames F in each sampling interval p1 To F pn Add retention period S p ; Except for the first frame of the video stream data, each sampling segment is executed in sequence according to the retention period-extraction period.

[0093] Reference Figure 3 As shown, the pixel feature method for phased processing in step (12) is specifically as follows:

[0094] (121) Extraction period S d Contains a video frame F d , an adaptive strategy is used to extract the surface area to be detected. The extraction stages used in an extraction period are divided into: range determination and extraction mapping; the extraction process included in a complete extraction period is divided into: range determination-extraction mapping process D 1 , Extraction mapping process D 2 , and Extraction Mapping-Scope Determination-Extraction Mapping Process D 3 ; When the video frame index is 1, the range determination-extraction mapping process D is used 1 ; Except for the first frame, if the retention period requires a range reset, the range determination-extraction mapping process D is used 1 ; Otherwise, use the extraction mapping process D 2 , make a judgment during the process, if the range needs to be reset, then use the extraction mapping-range determination-extraction mapping process D 3 ; After the range is reset, n starts to increase linearly from 0 again;

[0095] (122) Retention period S p Contains n video frames F p1 to F pn , determine the coordinates H of the surface area to be detected obtained by applying the range determination of the previous sampling segment to the current sampling segment video frame. If the determination result is that the pixel values ​​on both sides of at least one of the two range determination lines are significantly different, it means that the boundary coordinates of the surface to be detected in the sampling segment have changed, and the range needs to be reset during the extraction period of the sampling segment; wherein, each time the range is reset, the length of the sampling segment is reset, n=0, and continues to increase linearly.

[0096] Specifically, the specific steps of determining the range in step (121) are:

[0097] (1211) The input image is converted into a grayscale image. The grayscale value calculation formula is as follows:

[0098] I gray (x,y)=0.299Ir(x,y)+0.587I g (x,y)+0.114I b (x,y)

[0099] Among them, I r (x,y),I g (x,y),I b (x, y) are the pixel values ​​of the red, green and blue channels of the image respectively; I gray (x, y) is the pixel value of the grayscale image. The grayscale image is processed by histogram equalization to enhance the contrast of the image and obtain the histogram equalization image pixel value I equalized (x, y) and its corresponding video frame F′ d , the calculation formula is as follows:

[0100]

[0101] Among them, I gray.max and I gray.min are the maximum and minimum pixel values ​​of the grayscale image respectively;

[0102] (1212) Using the local threshold method to analyze the above video frame F′ d Perform edge detection to improve detection accuracy as follows:

[0103]

[0104] Among them, T(x,y) is the local threshold, μ(x,y) is the average gray value around the pixel (x,y), σ(x,y) is the standard deviation, k is the adjustment parameter, R is the dynamic range constant, and the gradient value of the pixel is calculated as follows:

[0105] G x (x,y)=I equalized (x+1,y)-I equalized (x-1,y)

[0106] G y (x,y)=I equalized (x,y+1)-I equalized (x,y-1)

[0107]

[0108] Among them, G x(x,y) is the gradient value of the pixel point (x,y) in the x direction, G y (x,y) is the gradient value of the pixel point (x,y) in the y direction, G(x,y) is the total gradient value of the pixel point (x,y), when G(x,y) exceeds T(x,y), the pixel point is retained as an edge point;

[0109] (1213) Performing Hough line detection on the detected edge image, extracting and screening out the two longest straight lines, extending the two screened straight lines to the edge of the image, and obtaining the coordinates of the surface area to be detected.

[0110] Specifically, the specific steps of extracting the mapping in step (121) are:

[0111] The coordinates of the surface area to be detected enclosed by the two straight lines obtained above are H, H = ((x 1 ,y 1 ),(x 2 ,y 2 ),(x 3 ,y 3 ),(x 4 ,y 4 )), and use perspective transformation to map to the new video frame F″ d The coordinates of H′ are in the range of ((0,0), (w,0), (0,h), (w,h)), where w = x 2 -x 1 , Adopting extraction and mapping process D 2 When the coordinates H of the surface area to be detected are the same as the video frame result in the previous sampling segment, the extraction mapping-range determination-extraction mapping process D is adopted. 3 When , the coordinate H of the surface area to be detected is the result of the range determination in the current sampling segment.

[0112] Defect detection stage: Use image feature extraction methods to perform defect detection on images within the range of the surface to be inspected, and determine the range that needs to be repaired based on the defect location;

[0113] The specific steps of the defect detection stage are:

[0114] (21) Based on the new video frame F″ of the surface area to be detected d , linear defect detection and polygonal defect detection are performed in turn, and the repair range is calculated.

[0115] Specifically, the linear defect detection method in step (21) is as follows:

[0116] (211) Calculate the new video frame F″ d The sum of the grayscale values ​​of each row is as follows:

[0117]

[0118] Among them, row gray (i) represents the new video frame F″ d The sum of the grayscale values ​​of the i-th row in d (i,j) represents the new video frame F″ d The gray value of the jth pixel in the i-th row; take window as the sliding window size, and calculate the sum of the gray values ​​in the sliding window as follows:

[0119]

[0120] Among them, window gray (i) represents the sum of the grayscale values ​​in the i-th sliding window, and calculates the mean of all sliding windows;

[0121] (212) Calculate the difference between the sum of the grayscale values ​​of two adjacent sliding windows, and find the index with the smallest product of the adjacent differences: diff(i) = (window gray (i)-window gray (i+1))*(window gray (i-1)-window gray (i)) where diff(i) represents the product of the grayscale value difference between the current sliding window i and the previous and next sliding windows;

[0122] The linear defect range that needs to be repaired is E 1 , E 1 =((0,index),(w,index),(0,index+window),(w,index+window)).

[0123] Specifically, the polygon defect detection method in step (21) is as follows:

[0124] The new video frame F″ d Each pixel value in the grayscale image is binarized with reference to the preset binarization threshold μ, as follows:

[0125]

[0126] Among them, I d (x,y) is the new video frame F″ d The pixel value, I b (x, y) is the pixel value of the image after binarization;

[0127] The binary image is corroded and expanded, and its connected regions are marked. Specifically, for each pixel in the image, if the pixel value is foreground and is not marked, a breadth-first search is used to search for all connected foreground pixels starting from the pixel, marking it as a new connected region until all pixels are checked. The pixel coordinate set R of the i-th connected region is obtained as follows:

[0128] R={(x i ,y i )|(x i ,y i ) are pixel coordinates within the connected region}

[0129] According to the pixel coordinate set R, the coordinates of the four vertices of the bounding rectangle of the polygon defect are calculated as follows:

[0130] x left =min(x i )

[0131] x right =max(x i )

[0132] y up =min(y i )

[0133] y down =max(y i )

[0134] Among them, x left is the horizontal coordinate of the leftmost pixel in the polygon defect range, x right is the horizontal coordinate of the rightmost pixel in the polygon defect range, y up y is the vertical coordinate of the top pixel in the polygon defect range, down The vertical coordinate of the bottom pixel of the polygon defect range;

[0135] The polygon defect range E that needs to be repaired 2 ,as follows:

[0136] E 2 =((x left ,y up ),(x right ,y up ),(x left ,y down ),(x right ,y down )).

[0137] The present invention has many specific application paths. The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements can be made without departing from the principle of the present invention. These improvements should also be regarded as the protection scope of the present invention.

Claims

1. A surface defect detection system based on image feature extraction, characterized in that: include: Remote detectors, servers, and clients; A remote detector is used to obtain industrial site video stream data in real time, detect and extract the surface area to be detected in the video stream data, perform defect detection on the image within the surface area to be detected, determine the range to be repaired according to the defect location, and send the repair range result data corresponding to the video stream to the server; The server is used to obtain the result data sent by the remote detector in real time and store it; The client is used to obtain result data from the server in real time and query historical result data.

2. The surface defect detection system based on image feature extraction according to claim 1 is characterized in that: The remote detector includes: a camera, a processor and a communication interface; A camera for collecting video stream data of an industrial site containing a surface to be inspected; A processor, using an image feature extraction method to perform defect detection on an image within the range of the surface to be detected, and determining a range to be repaired according to the defect location; The communication interface is used to send the repair range result data corresponding to the video stream to the server.

3. A surface defect detection method based on image feature extraction, based on the system according to any one of claims 1-2, characterized in that: The method includes a surface extraction stage and a defect detection stage; Surface extraction stage: Obtain industrial site video stream data, detect and extract the surface area to be detected; Defect detection stage: Use image feature extraction methods to perform defect detection on images within the range of the surface to be inspected, and determine the scope that needs to be repaired based on the defect location.

4. The surface defect detection method based on image feature extraction according to claim 3 is characterized in that: The specific steps of the surface extraction stage are: (11) The industrial site video stream data collected in real time is continuously divided into sampling segments S with adaptively changing lengths using the interval sampling method, and this is used as a processing cycle; (12) For each sampling segment S, a pixel feature method based on phased processing is used to selectively extract the coordinates of the surface area to be detected in the video, H = ((x1, y1), (x2, y2), (x3, y3), (x4, y4)), and map it to a new area with a width of x2-x1 and a height of In the frame of the surface area to be detected.

5. The surface defect detection method based on image feature extraction according to claim 4 is characterized in that: The interval sampling method in step (11) is specifically: (111) When continuously acquiring video stream data, start with 1, record the incoming video frame index, and start with video frame number 2, and use every n+1 frames as a sampling segment S, where n increases linearly from 0; (112) For each sampling segment S, it is divided into a retention period S p and extraction period S d , for each video frame F in the sampling segment S, every n+1 frames is taken as a sampling interval; the last frame F in the sampling interval is taken as d Add extraction period S d , the first n frames F in each sampling interval p1 To F pn Add retention period S p ; Except for the first frame of the video stream data, each sampling segment is executed in sequence according to the retention period-extraction period.

6. The surface defect detection method based on image feature extraction according to claim 4 is characterized in that: The pixel feature method of the staged processing in step (12) is specifically: (121) Extraction period S d Contains a video frame F d , an adaptive strategy is used to extract the surface area to be detected. The extraction stages used in an extraction period are divided into: range determination and extraction mapping; the extraction processes included in a complete extraction period are divided into: range determination-extraction mapping process D1, extraction mapping process D2, and extraction mapping-range determination-extraction mapping process D3; When the video frame index is 1, the range determination-extraction mapping process D1 is used; except for the first frame, if the retention period requires a range reset, the range determination-extraction mapping process D1 is used; otherwise, the extraction mapping process D2 is used, and a judgment is made during the process. If a range reset is required, the extraction mapping-range determination-extraction mapping process D3 is used; after the range reset occurs, n starts to increase linearly from 0 again; (122) Retention period S p Contains n video frames F p1 to F pn , determine the coordinates H of the surface area to be detected obtained by applying the range determination of the previous sampling segment to the current sampling segment video frame. If the determination result is that the pixel values ​​on both sides of at least one of the two range determination lines are significantly different, it means that the boundary coordinates of the surface to be detected in the sampling segment have changed, and the range needs to be reset during the extraction period of the sampling segment; wherein, each time the range is reset, the length of the sampling segment is reset, n=0, and continues to increase linearly.

7. The surface defect detection method based on image feature extraction according to claim 6 is characterized in that: The specific steps of determining the range in step (121) are: (1211) The input image is converted into a grayscale image. The grayscale value calculation formula is as follows: I gray (x,y)=0.299I r (x,y)+0.587I g (x,y)+0.114I b (x,y) Among them, I r (x,y),I g (x,y),I b (x, y) are the pixel values ​​of the red, green and blue channels of the image respectively; I gray (x, y) is the pixel value of the grayscale image. The grayscale image is processed by histogram equalization to enhance the contrast of the image and obtain the histogram equalization image pixel value I equalized (x, y) and its corresponding video frame F′ d , the calculation formula is as follows: Among them, I gray.max and I gray.min are the maximum and minimum pixel values ​​of the grayscale image respectively; (1212) Using the local threshold method to analyze the above video frame F′ d Perform edge detection to improve detection accuracy as follows: Among them, T(x,y) is the local threshold, μ(x,y) is the average gray value around the pixel (x,y), σ(x,y) is the standard deviation, k is the adjustment parameter, R is the dynamic range constant, and the gradient value of the pixel is calculated as follows: G x (x,y)=I equalized (x+1,y)-I equalized (x-1,y) G y (x,y)=I equalized (x,y+1)-I equalized (x,y-1) Among them, G x (x,y) is the gradient value of the pixel point (x,y) in the x direction, G y (x,y) is the gradient value of the pixel point (x,y) in the y direction, G(x,y) is the total gradient value of the pixel point (x,y), when G(x,y) exceeds T(x,y), the pixel point is retained as an edge point; (1213) Performing Hough line detection on the detected edge image, extracting and screening out the two longest straight lines, extending the two screened straight lines to the edge of the image, and obtaining the coordinates of the surface area to be detected.

8. The surface defect detection method based on image feature extraction according to claim 7 is characterized in that: The specific steps of extracting the mapping in step (121) are: The coordinates of the surface area to be detected enclosed by the two straight lines, H = ((x1, y1), (x2, y2), (x3, y3), (x4, y4)), are mapped to the new video frame F" using perspective transformation d The coordinates of H′ are in the range of ((0,0), (w,0), (0,h), (w,h)), where w = x2-x1, When the extraction mapping process D2 is adopted, the coordinate H of the surface area to be detected is the same as the video frame result in the previous sampling segment; when the extraction mapping-range determination-extraction mapping process D3 is adopted, the coordinate H of the surface area to be detected is the result of range determination in the current sampling segment.

9. The surface defect detection method based on image feature extraction according to claim 8 is characterized in that: The specific steps of the defect detection stage are: (21) Based on the new video frame F″ of the surface area to be detected d , linear defect detection and polygonal defect detection are performed in sequence, and the repair range is calculated; The linear defect detection method in step (21) is specifically as follows: (211) Calculate the new video frame F″ d The sum of the grayscale values ​​of each row is as follows: Among them, row gray (i) represents the new video frame F″ d The sum of the grayscale values ​​of the i-th row in d (i,j) represents the new video frame F″ d The gray value of the jth pixel in the i-th row; take window as the sliding window size, and calculate the sum of the gray values ​​in the sliding window as follows: Among them, window gray (i) represents the sum of the grayscale values ​​in the i-th sliding window, and calculates the mean of all sliding windows; (212) Calculate the difference between the sum of the grayscale values ​​of two adjacent sliding windows, and find the index with the smallest product of the adjacent differences: diff(i)=(window gray (i)-window gray (i+1))*(window gray (i-1)-window gray (i)) Wherein, diff(i) represents the product of the grayscale value difference between the current sliding window i and the previous and next sliding windows; The linear defect range that needs to be repaired is E1, E1 = ((0, index), (w, index), (0, index + window), (w, index + window)).

10. The surface defect detection method based on image feature extraction according to claim 9, characterized in that: The polygon defect detection method in step (21) is specifically as follows: The new video frame F″ d Each pixel value in the grayscale image is binarized with reference to the preset binarization threshold μ, as follows: Among them, I d (x,y) is the new video frame F″ d The pixel value, I b (x, y) is the pixel value of the image after binarization; The binary image is corroded and expanded, and its connected regions are marked. Specifically, for each pixel in the image, if the pixel value is foreground and is not marked, a breadth-first search is used to search for all connected foreground pixels starting from the pixel, marking it as a new connected region until all pixels are checked. The pixel coordinate set R of the i-th connected region is obtained as follows: R={(x i ,y i )|(x i ,y i ) are pixel coordinates within the connected region} According to the pixel coordinate set R, the coordinates of the four vertices of the bounding rectangle of the polygon defect are calculated as follows: x left =min(x i ) x right =max(x i ) and up =min(and i ) and down =max(y i ) Among them, x left is the horizontal coordinate of the leftmost pixel in the polygon defect range, x right is the horizontal coordinate of the rightmost pixel in the polygon defect range, y up y is the vertical coordinate of the top pixel in the polygon defect range, down The vertical coordinate of the bottom pixel of the polygon defect range; The polygon defect range E2 that needs to be repaired is as follows: E2=((x left ,y up ),(x right ,y up ),(x left ,y down ),(x right ,y down ))。

Images