A method and device for detecting splash sparks based on background model

Abstract

The present invention relates to a method and device for detecting splash sparks based on a background model, wherein the method comprises: step S1: calibrating the detection scene on site to obtain transformation parameters for converting an input image into a plane viewing angle; step S2: establishing a background model or background mask image of the detection scene; step S3: extracting a foreground part based on the background model or background mask image; step S4: extracting a spark connected domain from the extracted foreground part to solve the spark geometric information; step S5: updating the background model according to the background mask image and the foreground extraction result. Compared with the prior art, the present invention has the advantages of good real-time performance, high accuracy, and no need for manual data annotation.

Description

Technical Field

[0001] The invention relates to the field of splash spark detection, and in particular to a splash spark detection method and device based on a background model. Background Art

[0002] In many industrial production lines for manufacturing, cutting and curling of metal workpieces, due to the friction exerted on the raw materials by processing equipment such as cutting tools and fixtures and the high-speed movement of the conveyor belt, flying sparks will be generated at the connection between the metal material and the processing equipment. Generally, the density or size of flying sparks is related to factors such as the size of the force, the size of the contact area at the material connection, and the roughness of the contact surface at the connection. Since the above factors are correlated with flying sparks, the size of flying sparks can also be used as one of the reference factors for the processing quality of the workpiece. For example, in the process of strip curling, applying clamping force to the strip will generate flying sparks on both sides of it. The density or size of the sparks generated is positively correlated with the actual clamping force. By accurately measuring the size of the sparks, automatic feedback control of the strip curling process can be achieved.

[0003] Existing spark detection technology is mainly based on machine vision theory. For example, Chinese patent CN112749650A discloses a method for spark detection based on machine vision, which trains a deep learning model by annotating spark image data sets to achieve effective spark detection. This method has the advantage of high accuracy.

[0004] However, compared with the electric spark caused by short circuit, the splashing spark has the following characteristics: first, the retention time is short; second, the spark will disperse and move at high speed after leaving the contact surface. The real-time performance of the above-mentioned deep learning detection method and the accuracy of small object detection cannot meet the detection requirements. In addition, there is also the disadvantage of requiring manual annotation of a certain number of spark image data sets, and the high annotation and time cost. Summary of the invention

[0005] The purpose of the present invention is to provide a splash spark detection method and device based on background model.

[0006] The purpose of the present invention can be achieved by the following technical solutions:

[0007] A splash spark detection method based on a background model, comprising:

[0008] Step S1: calibrate the detection scene on site to obtain transformation parameters for converting the input image into a plane view;

[0009] Step S2: Establishing a background model or background mask image of the detection scene;

[0010] Step S3: extracting the foreground part based on the background model or the background mask image;

[0011] Step S4: extracting the spark connected domain of the extracted foreground part to obtain the spark geometry information;

[0012] Step S5: Update the background model according to the background mask image and the current foreground extraction result.

[0013] The transformation parameters include a pixel-to-actual length ratio R and a homography matrix H.

[0014] The step S1 specifically includes:

[0015] Step S11: measuring the actual length L and width W of the rectangular area to be detected;

[0016] Step S12: Determine the length l and width w of the converted image in pixels:

[0017] l=L*R

[0018] w=W*R

[0019] Step S13: Use the DLT algorithm to solve the homography matrix H for converting the image perspective into a vertical top-down perspective. The on-site calibration process is completed, and the pixel-to-actual length ratio R and the homography matrix H are obtained.

[0020] The background model is a set of pixel values ​​or a set of probability distributions having the same size as the input image after conversion to a plane perspective;

[0021] The background mask image is a single-channel image output after binary classification of the input image converted into a plane perspective according to the background model, the pixel value of the background coordinate point is 0, and the pixel value of the foreground coordinate point is 1.

[0022] The step S3 specifically includes:

[0023] Step S31: transforming the input image to be detected into a plane perspective;

[0024] Step S32: Determine whether there is a background mask image, if yes, execute step S33, otherwise, execute step S35;

[0025] Step S33: Convert the input image into a plane view according to the background mask image processing:

[0026]

[0027] Where: I′(x) is the pixel value of pixel point x in the input image after conversion to a plane perspective, I(x) is the pixel value of pixel point x in the input image before conversion to a plane perspective, I Mask(x) is the pixel value of pixel x in the background mask image;

[0028] Step S34: extracting all pixels whose pixel values ​​are not 0 in the input image after being converted into a plane perspective to form a foreground part;

[0029] Step S35: calculating the statistical distance between the pixel value or probability distribution of each pixel point in the input image converted into a plane perspective and each pixel point in the background model;

[0030] Step S36: determining whether the statistical distance of each pixel is greater than a preset background threshold, if so, the pixel is regarded as a foreground pixel, otherwise the pixel is regarded as a background pixel;

[0031] Step S37: extract all foreground pixels as the foreground part.

[0032] The process of obtaining the background mask image includes:

[0033] Step S21: Match the image of the current frame with the pixels of the same coordinates of the previous K consecutive historical frame images one by one. When the statistical distance between the pixel values ​​of the two matching parties is less than a preset threshold, it is considered that the match is successful;

[0034] Step S22: Count the number of times each pixel passes the matching. If it is greater than a preset confidence number, the pixel is regarded as a background pixel, otherwise it is regarded as a foreground pixel.

[0035] The step S4 comprises:

[0036] Step S41: setting a fixed threshold T0 to perform fixed threshold binarization processing on the foreground part;

[0037] Step S42: using a maximum entropy threshold algorithm to solve the maximum entropy threshold of the foreground part, and performing binarization processing on the foreground part based on the maximum entropy threshold;

[0038] Step S43: Calculate the intersection of the image binarized by the fixed threshold and the image binarized by the maximum entropy threshold as the binarization result;

[0039] Step S44: performing a morphological closing operation on the binarization result to obtain a morphological processing result;

[0040] Step S45: in the morphological processing result, all connected domains with pixel values ​​of 1 are obtained by seed filling algorithm as spark regions;

[0041] Step S46: Calculate the actual value of the spark geometry information based on the ratio R of the pixel to the actual length and the spark area.

[0042] The step S5 specifically includes:

[0043] Step S51: selecting a foreground pixel point of the background mask image;

[0044] Step S52: randomly generate a random number N between [0,1]. When the random number N is greater than the preset update probability P, step S53 is executed. Otherwise, no update is performed.

[0045] Step S53: updating the pixel values ​​or probability distribution of the pixel points of the foreground pixels in the same coordinates of the background model according to the pixel values ​​of the pixel points of the foreground part detected this time;

[0046] Step S54: determine whether there are any unprocessed foreground pixels. If yes, select another foreground pixel and execute step S52. Otherwise, end the update.

[0047] The plane perspective is a top-down perspective.

[0048] A splash spark detection device based on a background model comprises a processor, a memory, and a program. When the processor executes the program, the above method is implemented.

[0049] Compared with the prior art, the present invention has the following beneficial effects:

[0050] 1. No need to manually label data, reducing initial labor costs;

[0051] 2. Good real-time performance and significantly improved processing speed;

[0052] 3. The background model is updated frame by frame in real time, and is adaptive to the detection environment and has strong robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 It is a flow chart of the splash spark detection method based on the background model in the present invention;

[0054] Figure 2 The original image with the boundary points marked in the embodiment of the present invention;

[0055] Figure 3 is a top view after homography transformation in an embodiment of the present invention;

[0056] Figure 4 is a schematic diagram of a “historical frame” in an embodiment of the present invention;

[0057] Figure 5 is a schematic diagram of a “current frame” in an embodiment of the present invention;

[0058] Figure 6 is a schematic diagram of a background mask image in an embodiment of the present invention;

[0059] Figure 7 Schematic diagram of spark connected domain extraction results in an embodiment of the present invention. DETAILED DESCRIPTION

[0060] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0061] A method for detecting flying sparks based on a background model, the method being in the form of a computer program executable by a computer system, such as Figure 1 As shown, including:

[0062] Step S1: calibrate the detection scene on site to obtain transformation parameters for converting the input image into a plane perspective. The plane perspective is a bird's-eye view. The transformation parameters include a pixel-to-actual length ratio R and a homography matrix H, specifically including:

[0063] Step S11: measuring the actual length L and width W of the rectangular area to be detected;

[0064] Step S12: Determine the length l and width w of the converted image in pixels:

[0065] l=L*R

[0066] w=W*R

[0067] Step S13: Use the DLT algorithm to solve the homography matrix H for converting the image perspective into a vertical top-down perspective. The on-site calibration process is completed, and the pixel-to-actual length ratio R and the homography matrix H are obtained.

[0068] Specifically, first, a rectangular area to be detected of the flying spark is selected, the actual length and width of the rectangular area are measured by a measuring tool, and the four boundary points of the area are selected in the image and the pixel coordinates are recorded, such as Figure 2 As shown; secondly, determine the length and width of the top view according to the requirements, and the length and width of the top view must be in a fixed ratio R with the actual length and width; finally, the DLT algorithm is used to calculate the homography matrix H that converts the image perspective into a vertical top view to complete the calibration process. The top view of the original image after perspective transformation based on the calibrated homography matrix is ​​as follows Figure 3 shown.

[0069] Step S2: Establish a background model or background mask image of the detection scene; the process of obtaining the background mask image includes: Step S21: Match the image of the current frame with the pixels of the same coordinates of the previous K consecutive historical frame images one by one. When the statistical distance between the pixel values ​​of the two matching parties is less than a preset threshold, the match is considered to be successful; Step S22: Count the number of times each pixel is matched. If it is greater than a preset confidence number, the pixel is used as a background pixel, otherwise it is used as a foreground pixel.

[0070] The appropriate background modeling algorithm should be selected according to the actual engineering environment. In this embodiment, the nearest neighbor node (KNN) background modeling algorithm is used as an example to introduce it. The algorithm traverses all pixel points of the input image (current frame) and matches them with the pixel points with the same coordinates in the previous K consecutive images (historical frames). When the statistical distance between the pixel values ​​of the two matching parties is less than the preset threshold, the match is considered to be successful. When the number of matches is greater than the confidence number, the pixel point is considered to be "background" and the corresponding pixel point in the background mask image is set to 0; otherwise, it is judged to be "foreground" and the pixel value is set to 1. In this embodiment, a total of 7 frames of continuous single-channel images are selected for background modeling. Figure 4 This is a schematic diagram of one of the "historical frames". Figure 5 is a schematic diagram of the “current frame”. The output background mask image is as follows: Figure 6 shown.

[0071] Step S3: extracting the foreground part based on a background model or a background mask image, where the background model is a set of pixel values ​​or a set of probability distributions having the same size as the input image after conversion to a plane perspective;

[0072] The background mask image is a single-channel image output after binary classification of the input image converted to a plane perspective according to the background model. The pixel value of the background coordinate point is 0, and the pixel value of the foreground coordinate point is 1.

[0073] Step S3 specifically includes:

[0074] Step S31: transforming the input image to be detected into a plane perspective;

[0075] Step S32: Determine whether there is a background mask image, if yes, execute step S33, otherwise, execute step S35;

[0076] Step S33: Convert the input image into a plane view according to the background mask image processing:

[0077]

[0078] Where: I′(x) is the pixel value of pixel point x in the input image after conversion to a plane perspective, I(x) is the pixel value of pixel point x in the input image before conversion to a plane perspective, IMask (x) is the pixel value of pixel x in the background mask image;

[0079] Step S34: extracting all pixels whose pixel values ​​are not 0 in the input image after being converted into a plane perspective to form a foreground part;

[0080] Step S35: calculating the statistical distance between the pixel value or probability distribution of each pixel point in the input image converted into a plane perspective and each pixel point in the background model;

[0081] Among them, if the background model is a set of pixel values, step S35 is to calculate the statistical distance between the pixel values ​​of each pixel point in the input image after conversion to a plane perspective and each pixel point in the background model; if the background model is a probability distribution set, based on the probability distribution, calculate the probability that each pixel point in the input image after conversion to a plane perspective is a foreground pixel point.

[0082] Step S36: Determine whether the statistical distance of each pixel is greater than a preset background threshold. If so, the pixel is regarded as a foreground pixel, otherwise the pixel is regarded as a background pixel. If the background model is a set of pixel values, the preset background threshold is the upper limit of the statistical distance, and pixels exceeding the upper limit are regarded as foreground pixels. If the background model is a set of probability distributions, the preset background threshold is the upper limit of the probability, and pixels exceeding the upper limit are regarded as foreground pixels.

[0083] Step S37: extract all foreground pixels as the foreground part.

[0084] Step S4: extracting the spark connected domain of the extracted foreground part to obtain spark geometry information. Step S4 includes:

[0085] Step S41: setting a fixed threshold T0 to perform fixed threshold binarization processing on the foreground part to obtain a fixed threshold binary image I0;

[0086] Step S42: using a maximum entropy threshold algorithm to solve the maximum entropy threshold of the foreground part, and performing binarization processing on the foreground part based on the maximum entropy threshold to obtain a maximum entropy threshold binary image I1;

[0087] Step S43: Calculate the intersection of the image binarized by the fixed threshold and the image binarized by the maximum entropy threshold, and use it as the binarization result I B =I0∩I1;

[0088] Step S44: Binarization result I B Perform morphological closing operation to obtain morphological processing result I m ;

[0089] Step S45: Morphological processing result I mIn the above, all connected domains with pixel values ​​of 1 are obtained as spark regions through the seed filling algorithm;

[0090] Step S46: Calculate the actual value of the spark geometry information based on the ratio R of the pixel to the actual length and the spark area.

[0091] Step S5: updating the background model according to the background mask image and the foreground extraction result, specifically including:

[0092] Step S51: selecting a foreground pixel point of the background mask image;

[0093] Step S52: randomly generate a random number N between [0,1]. When the random number N is greater than the preset update probability P, step S53 is executed. Otherwise, no update is performed.

[0094] Step S53: updating the pixel values ​​or probability distribution of the pixel points of the foreground pixels in the same coordinates of the background model according to the pixel values ​​of the pixel points of the foreground part detected this time;

[0095] Step S54: determine whether there are any unprocessed foreground pixels. If yes, select another foreground pixel and execute step S52. Otherwise, end the update.

Claims

1. A splash spark detection method based on background model, characterized in that: include: Step S1: calibrate the detection scene on site to obtain transformation parameters for converting the input image into a plane view; Step S2: Establishing a background model or background mask image of the detection scene; Step S3: extracting the foreground part based on the background model or the background mask image; Step S4: extracting the spark connected domain of the extracted foreground part to solve the spark geometric information; Step S5: updating the background model according to the background mask image and the foreground extraction result; The transformation parameters include a pixel-to-actual length ratio R and a homography matrix H; The step S1 specifically includes: Step S11: measuring the actual length L and width W of the rectangular area to be detected; Step S12: Determine the length l and width w of the converted image in pixels: l=L*R w=W*R Step S13: using the DLT algorithm to solve the homography matrix H of converting the image perspective into a vertical top-down perspective, the on-site calibration process is completed, and the pixel-to-actual length ratio R and the homography matrix H are obtained; The background model is a set of pixel values ​​or a set of probability distributions having the same size as the input image after conversion to a plane perspective; The background mask image is a single-channel image output after binary classification of the input image converted to a plane perspective according to the background model, the pixel value of the background coordinate point is 0, and the pixel value of the foreground coordinate point is 1; The step S3 specifically includes: Step S31: transforming the input image to be detected into a plane perspective; Step S32: Determine whether there is a background mask image, if yes, execute step S33, otherwise, execute step S35; Step S33: Convert the input image into a plane view according to the background mask image processing: Where: I′(x) is the pixel value of pixel point x in the input image after conversion to a plane perspective, I(x) is the pixel value of pixel point x in the input image before conversion to a plane perspective, I Mask (x) is the pixel value of pixel x in the background mask image; Step S34: extracting all pixels whose pixel values ​​are not 0 in the input image after being converted into a plane perspective to form a foreground part; Step S35: calculating the statistical distance between the pixel value or probability distribution of each pixel point in the input image converted into a plane perspective and each pixel point in the background model; Step S36: determining whether the statistical distance of each pixel is greater than a preset background threshold, if so, the pixel is regarded as a foreground pixel, otherwise the pixel is regarded as a background pixel; Step S37: extracting all foreground pixels as foreground parts; The step S4 comprises: Step S41: setting a fixed threshold T0 to perform fixed threshold binarization processing on the foreground part; Step S42: using a maximum entropy threshold algorithm to solve the maximum entropy threshold of the foreground part, and performing binarization processing on the foreground part based on the maximum entropy threshold; Step S43: Calculate the intersection of the image binarized by the fixed threshold and the image binarized by the maximum entropy threshold as the binarization result; Step S44: performing a morphological closing operation on the binarization result to obtain a morphological processing result; Step S45: in the morphological processing result, all connected domains with pixel values ​​of 1 are obtained by seed filling algorithm as spark regions; Step S46: Calculate the actual value of the spark geometry information based on the ratio R of the pixel to the actual length and the spark area.

2. The method for detecting splash sparks based on a background model according to claim 1, characterized in that: The process of obtaining the background mask image includes: Step S21: Match the image of the current frame with the pixels of the same coordinates of the previous K consecutive historical frame images one by one. When the statistical distance between the pixel values ​​of the two matching parties is less than a preset threshold, it is considered that the match is successful; Step S22: Count the number of times each pixel is matched. If the number is greater than a preset confidence number, the pixel is regarded as a background pixel; otherwise, it is regarded as a foreground pixel.

3. The method for detecting splash sparks based on a background model according to claim 1, characterized in that: The step S5 specifically includes: Step S51: selecting a foreground pixel point of the background mask image; Step S52: randomly generate a random number N between [0,1]. When the random number N is greater than the preset update probability P, step S53 is executed. Otherwise, no update is performed. Step S53: updating the pixel values ​​or probability distribution of the foreground pixels in the same coordinates of the background model according to the pixel values ​​of the foreground pixels detected this time; Step S54: determine whether there are any unprocessed foreground pixels. If yes, select another foreground pixel and execute step S52. Otherwise, end the update.

4. The method for detecting splash sparks based on a background model according to claim 1, characterized in that: The plane perspective is a top-down perspective.

5. A splash spark detection device based on a background model, comprising a processor, a memory, and a program, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 4 is implemented.

Images