Nuclear fuel pool foreign object falling detection method, electronic device and storage medium

Abstract

The present invention discloses a method for detecting foreign object falling in a nuclear fuel pool, an electronic device and a storage medium, belonging to the field of machine vision and target detection. The present invention uses a high-resolution industrial camera, combines the advantages of the background difference method and the inter-frame difference method, combines the characteristics of static and dynamic detection, and realizes the detection of foreign object targets in the nuclear fuel pool through multi-angle and multi-level detection methods, preventing foreign objects from causing harm to equipment or personnel and ensuring the safe operation of nuclear power plants; the position of the moving target is determined by comparing the distance between the selected box and the ground truth box, and the area of the moving target region is determined by combining the CIoU intersection over union ratio through the weighted average method. Combining the two algorithms can comprehensively analyze the motion state of the target object, improve the comprehensiveness and reliability of detection, reduce the false detection rate and the missed detection rate; at the same time, the detection efficiency is improved, the detection result of the foreign object target can be fed back in real time, and the requirement for computing power is low, and the overall has strong applicability and high sensitivity.

Description

Technical Field

[0001] The present invention belongs to the field of machine vision and target detection, and relates to a foreign object falling detection method, an electronic device and a storage medium for a nuclear fuel pool. Background Art

[0002] Nuclear power plants are important energy production facilities. However, there are usually a large number of complex equipment operations and personnel activities inside nuclear power plants. If a tiny foreign object falls into the nuclear fuel pool during the operation of the nuclear power plant or the operation of the staff, it will not only cause potential pollution, but may also lead to equipment damage, production interruption and even safety accidents, bringing irreversible losses. Therefore, developing a reliable foreign object falling detection system and a real-time detection method is crucial for the production operation, daily safety monitoring and management of nuclear power plants.

[0003] For foreign object detection in industrial scenarios, the traditional manual inspection method has certain subjective human factors, and is inefficient, time-consuming and laborious. Conventional motion target detection methods based on image processing usually have problems such as missed detection and false detection, and the detection accuracy is not high. In recent years, with the development of computer vision and artificial intelligence technologies, foreign object falling detection systems based on image recognition and deep learning have gradually developed and become a research hotspot. However, deep learning algorithms have high requirements for hardware computing power, and due to the high resolution of industrial cameras and the large size of the output video, the computational complexity is high, the iteration process is long, and the operation time cost is large. Considering the above factors, designing a multi-threaded motion target detection algorithm based on background difference method and continuous frame difference method can analyze the video captured by industrial cameras inside nuclear power plants, detect in real time whether there are foreign objects falling above the nuclear fuel pool, and issue an alarm in time, so as to improve the detection efficiency and accuracy of foreign object falling. Summary of the Invention

[0004] In order to further improve the foreign object detection accuracy and detection speed in the nuclear power plant scenario, the present invention provides a foreign object falling detection method for a nuclear fuel pool. This method uses an industrial camera in the nuclear power plant scenario to capture and record real-time images of the nuclear fuel pool, and analyzes the video captured by the high-resolution industrial camera, and can accurately detect in real time millimeter-sized tiny foreign objects falling above the fuel pool, so as to ensure the safe production and operation efficiency of the nuclear power plant and reduce potential production and pollution risks.

[0005] In order to achieve the above invention purpose, the technical solution adopted by the present invention is:

[0006] A foreign object falling detection method for a nuclear fuel pool, comprising the following steps:

[0007] Read the video captured by the industrial camera and preprocess the video;

[0008] The background difference method and the continuous frame difference method are respectively used to frame the moving target area in the current frame image;

[0009] For each frame of image, compare the distances between the moving target areas framed by the background difference method and the continuous frame difference method and the real object respectively, and use the center point coordinates of the moving target area closer to the real object as the center position of the output selection box; use the weighted average method to calculate the area of the output selection box; calculate the width and height of the output selection box based on the aspect ratio of the real box;

[0010] Judge whether the output selection box and its adjacent surrounding selection boxes contain the same moving target, and perform region merging based on the overlap ratio between the selection boxes containing the same moving target to determine the final target area and the center position;

[0011] Calculate the height of the moving target based on the final target area and mark it at the corresponding position in the video.

[0012] According to a foreign object falling detection method in some embodiments of the present application, in order to prevent subsequent false detection due to redundant noise points and at the same time meet the input requirements of the detection algorithm, the video captured by the industrial camera is read and preprocessed, which specifically includes:

[0013] Denoise the original video data by means of wavelet transform, decompose the video into frame images and convert each frame image into a grayscale image. The conversion method is:

[0014] V gray = 0.3V R + 0.59V G + 0.11V B (1)

[0015] where V gray represents the grayscale value, and V R , V G , V B represent the component sizes of the image in the red, green, and blue channels respectively;

[0016] Use the two-dimensional Haar wavelet to decompose the grayscale image into four wavelet coefficients, namely the approximation component A and three detail components: the horizontal detail component h, the vertical detail component V, and the diagonal detail component D. The calculation process is:

[0017]

[0018] where a, b, c, and d represent the pixel values of each matrix after dividing the grayscale image into 2×2 pixel matrices;

[0019] After the first-level decomposition, wavelet transform processing is performed on the approximate component A again to obtain the next-level approximate component, horizontal detail component, vertical detail component, and diagonal detail component; then the decomposition is repeated for the next-level approximate component; finally, the four wavelet coefficients obtained from the third-level decomposition are thresholded using the soft thresholding method. The four wavelet coefficients are compared with the set threshold respectively, and the wavelet coefficients less than the threshold in each region are set to zero, and the wavelet coefficients greater than or equal to the threshold are set to the difference between the coefficient value and the threshold, so as to remove noise; the formula for setting the threshold is:

[0020]

[0021] where MAD represents the median value of the absolute values of the wavelet coefficients after three-level decomposition of the grayscale image, 0.6745 represents the adjustment coefficient of the standard variance of Gaussian noise, and N is the size of the wavelet decomposition region;

[0022] The processed four wavelet coefficients are reconstructed into a new grayscale image using inverse wavelet transform, and the calculation formula is:

[0023]

[0024] where A′3, H′3, V′3, and D′3 represent the approximate component, horizontal detail component, vertical detail distribution component, and diagonal detail component obtained after three-level wavelet decomposition and thresholding respectively.

[0025] According to a foreign object falling detection method in some embodiments of the present application, the background difference method is used to compare the current frame image with the selected reference background, and the moving target area in the current frame image is framed, specifically including:

[0026] The current video is decomposed frame by frame, and the first frame of the video is used as the reference background, then the current frame image can be expressed as:

[0027] F i (x i ,y i ,g i )=b(x i ,y i ,g i )+t i (x i ,y i ,g i )+n i (x i ,y i ,g i ) (11)

[0028] where F i (x i ,y i ,gi ) represents the i-th frame of video image, where i = 1, 2, 3... n; b(x i , y i , g i ) represents the reference background part in the video image; t i (x i , y i , g i ) represents the moving target part in the i-th frame of video image; n i (x i , y i , g i ) represents the noise part in the i-th frame of video image; x i and y i respectively represent the horizontal and vertical coordinate values of the pixel point in the i-th frame of video image; g i represents the gray value corresponding to this pixel point;

[0029] To obtain the moving target part, calculate the difference image between the current frame image F i (x i , y i , g i ) and the reference background image, that is:

[0030] d i (x i , y i , g i ) = F i (x i , y i , g i ) - b(x i , y i , g i ) + n i (x i , y i , g i ) (12)

[0031] where, d i (x i , y i , g i ) represents the difference image jointly composed of the moving target area and the noise part in the i-th frame of image;

[0032] Perform histogram equalization processing on the difference image d i (x i , y i , g i ), where the histogram equalization is calculated according to formula (13):

[0033]

[0034] Among them, s k represents the pixel point of the k-th gray level of the differential image, u represents the sum of all pixels in the differential image, and u m represents the number of pixels of the current gray level, and L represents the number of gray levels;

[0035] Finally, the threshold function is used to perform binarization to obtain the binarized video frame image G i (x i , y i , g i ), where the formula of the threshold function is:

[0036]

[0037] Among them, β, η, and θ are all non-negative real numbers. θ represents the set threshold, β represents the set gray value exceeding the threshold, and η represents the set gray value not exceeding the threshold;

[0038] The binarized video frame image G i (x i , y i , g i ) is subjected to morphological filtering processing by methods such as image erosion or image dilation to obtain the differential image P i (x i , y i , g i );

[0039] Perform region connectivity analysis on the differential image P i (x i , y i , g i ). If the area S1 of a certain connected region Q i (x i , y i , g i ) in P i (x i , y i , g i ) is not less than the set threshold T1, then this region is identified as the moving target region t i (x i , y i , g i ), that is:

[0040]

[0041] Finally, the minimum bounding rectangle of this region is divided as the bounding region of the moving target.

[0042] According to a method for detecting foreign object falling in a nuclear fuel pool in some embodiments of the present application, the continuous frame difference method is used to compare the current frame image with the previous frame image, and the moving target area in the current frame image is framed out, which specifically includes:

[0043] Decompose the current video frame by frame, calculate the difference between the current frame image and the previous frame image to obtain the corresponding difference image, and the calculation method is as follows:

[0044] c j (x j ,y j ,g j )=F j (x j ,y j ,g j )-F j-1 (x j ,y j ,g j )+n j (x j ,y j ,g j ) (16)

[0045] Wherein, F j (x j ,y j ,g j ) represents the j-th frame video image, j = 2, 3, 4... n; F j-1 (x j ,y j ,g j ) represents the corresponding (j - 1)-th frame image; c j (x j ,y j ,g j ) represents the difference image of the continuous frame difference method;

[0046] Perform histogram equalization processing on the difference image c j (x j ,y j ,g j ) and use the threshold function to achieve binarization to obtain the binarized video frame image G j (x j , y j,g j ) with the calculation method referring to formula (14);

[0047] The obtained binarized video frame image G j (x j ,y j ,g jPerform morphological filtering processing by methods such as image erosion or image dilation to obtain the difference image P j (x j ,y j ,g j );

[0048] Perform region connectivity analysis on the difference image P j (x j ,y j ,g j ). If the area S2 of a certain connected region Q j (x j ,y j ,g j ) is not less than the set threshold T2, then this region is identified as the moving target region t j (x j ,y j ,g j ):

[0049]

[0050] Finally, divide the minimum bounding rectangle of this region as the bounding region of the moving target.

[0051] According to a foreign object falling detection method in some embodiments of the present application, the weighted average method is to calculate the Complete-IoU (CIoU) of the area between the bounding regions of the moving targets obtained by the background difference method and the inter-frame difference method and the real target region respectively, assign corresponding weights to the output results of each method, and then calculate the detection result by averaging; the specific steps are as follows:

[0052] Calculate the distances between the center points of all the bounding regions of the moving targets at the same position by the background difference method and the inter-frame difference method and the center point of the real target region respectively, and take the smaller distance as the center point of the output selected box;

[0053] Calculate the CIoU of the area between the bounding regions of the moving targets obtained by the background difference method and the inter-frame difference method and the real target region respectively; the CIoU and its related calculation formulas are:

[0054]

[0055] Among them, IoU represents the intersection over union; A represents the predicted box, that is, the bounding region of the moving target; B represents the real box; S |A∩B| and S |A∪B| represent the area of A intersection B and the area of A union B respectively; ρ 2(A, B) represents the square of the Euclidean distance between the geometric center points of the predicted box A and the ground truth box B; c represents the diagonal length of the smallest bounding box that can simultaneously contain the current predicted box and the ground truth box; w and h represent the width and height respectively; α represents the weight coefficient; v represents the similarity of the width-to-height ratio between the predicted box and the ground truth box;

[0056] For CIoU based on background difference method and inter-frame difference method, the weighted average method is used to calculate the area of the selected box to be output, and the calculation formula is as follows:

[0057]

[0058] Let the width-to-height ratio of the selected box to be output be equal to that of the ground truth box, and the width and height of the selected box to be output are respectively:

[0059]

[0060] Among them, S′ represents the area of the selected box to be output; w A ′ represents the width of the selected box to be output; h A ′ represents the height of the selected box to be output; w B represents the width of the ground truth box; h B represents the height of the ground truth box; q represents the total number of selected boxes selected by the background difference method and the inter-frame difference method; p represents a selected box at a specific position selected by the background difference method or the inter-frame difference method, CIoU p represents the CIoU of the selected box p, S p ′ represents the area of the selected box p.

[0061] According to a method for detecting foreign object falling in a nuclear fuel pool in some embodiments of the present application, the determining the target final area and the center position specifically includes:

[0062] Taking the selected box to be output as the main selected box, calculate the CIoU between the main selected box and its surrounding adjacent selected boxes. If the CIoU is less than the set threshold, it means that there is no same moving target between the main selected box and the surrounding adjacent selected boxes, and this main selected box is output as the detection result; if the CIoU is greater than the threshold, it proves that there is the same moving target between the main selected box and the surrounding adjacent selected boxes, and proceed to the next step;

[0063] Compare the area between the main selected box and its surrounding adjacent selected boxes, and take the selected box with the largest area as the new main selected box. The calculation formula is as follows:

[0064]

[0065] Among them, S′ represents the area of the main selected box at a certain position, r represents the remaining adjacent selected boxes at this position, o represents the number of the remaining adjacent selected boxes at this position, S r* represent the corresponding areas of the remaining adjacent selected boxes;

[0066] Calculate the overlap ratio between the new main selected box and the remaining adjacent selected boxes. If it is greater than the set threshold or the center point distance is less than the specified threshold, then fuse the remaining adjacent selected boxes into the main selected box and update the center point coordinates and the minimum bounding rectangle area of the main selected box; if neither condition is met, delete the corresponding adjacent selected box; among them, the calculation formula for the center point coordinates of the fused selected box is as follows:

[0067]

[0068] where x r and y r respectively represent the abscissa and ordinate of the center point of the fused selected box; (x r1 , y r1 ), (x r2 , y r2 ) represent the coordinates of the two selected boxes before fusion; w r1 , w r2 represent the widths of the two selected boxes before fusion; h r1 , h r2 represent the heights of the two selected boxes before fusion.

[0069] According to a foreign object falling detection method in some embodiments of the present application, the calculating the height of the moving target specifically includes:

[0070] Restore the video frame image from a grayscale image to a color image, and the calculation method is:

[0071]

[0072] where V gray represents the grayscale value of a certain pixel point in the frame image, and V R , V G , V B respectively represent the component values of the red, green, and blue channels after conversion of this pixel point;

[0073] According to the position of the current industrial camera and the height information of the calibration points in the picture, calculate the height of the center point of the current moving target area and display it in the real-time picture to achieve the calculation of the height of the moving target; its calculation formula is:

[0074]

[0075] where h z represents the height of the zth moving target in the current frame image, y rzrepresents the vertical coordinate value of the center point of the bounding box of the z-th moving target calculated. m2 and m1 respectively represent the actual heights of two calibration points in the actual scene, and R h represents the number of vertical pixels between two calibration points in the video frame.

[0076] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and capable of running on the processor; when the processor executes the computer program, the electronic device executes the method described in any one of the above embodiments.

[0077] The present invention also provides a storage medium, the storage medium includes a computer program, when the computer program runs on an electronic device, the electronic device executes the method described in any one of the above embodiments.

[0078] The beneficial effects of the present invention are as follows: The present invention utilizes a high-resolution industrial camera and a supporting lens, combines the advantages of the background difference method and the inter-frame difference method, combines the characteristics of static and dynamic detection, and realizes the detection of foreign object targets in the nuclear fuel pool through multi-angle and multi-level detection methods, preventing foreign objects from causing harm to equipment or personnel and ensuring the safe operation of nuclear power plants; The background difference method can effectively extract moving target objects under a static background, while the inter-frame difference method can capture the movement trajectory of the target object. The two determine the position of the moving target by comparing the distance between the selected box and the true box, and determine the area of the moving target region by combining the CIoU intersection over union ratio through the weighted average method. Combining the two algorithms can comprehensively analyze the movement state of the target object, improve the comprehensiveness and reliability of detection, reduce the false detection rate and the missed detection rate; At the same time, the multi-threaded calculation method is used to parallelly implement the operations of the background difference method and the inter-frame difference method, and partial parameters can also be adjusted and optimized according to other requirements; Compared with other target detection methods, it saves processing time and improves the operation efficiency, can real-time feedback the detection results of foreign object targets, has low requirements for computing power, can achieve a faster detection speed, and has strong applicability and high sensitivity as a whole. In addition, the present invention can also be used in similar scenarios in other industrial fields and provide reference for the safety monitoring and management of related industries. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] Figure 1 is a flowchart of the method of the present invention.

[0080] Figure 2 is a flowchart of the wavelet denoising method adopted by the present invention.

[0081] Figure 3It is a diagram of the selected result of the moving target area. Among them, (a) is the reference background during the test, (b) is the background image containing the falling object, (c) is the segmented image of the falling object, and (d) is the output result image containing the selected box after detecting the falling object.

[0082] Figure 4 It is a schematic diagram for calculating the height of foreign objects.

[0083] Figure 5 It is a diagram of the detection output result. Specific implementation manner

[0084] In order to make the purpose, technical solution and advantages of the present invention clearer and more understandable, the following will describe the specific implementation manner of the present invention in detail with reference to the accompanying drawings.

[0085] As Figure 1 shown, this embodiment provides a method for detecting foreign object falling in a nuclear fuel pool, including the following steps:

[0086] Step 1: Read the video and preprocess the video to prevent misdetection due to redundant noise points in the subsequent process, and at the same time meet the input requirements of the detection algorithm. The preprocessing method is as Figure 2 shown, and specifically includes the following sub-steps:

[0087] Step 1.1: Denoise the original video data by using wavelet transform, decompose the video into frame images, and convert each frame image into a grayscale image. The conversion method is calculated according to formula (1):

[0088] V gray = 0.3V R + 0.59V G + 0.11V B (1)

[0089] where V gray represents the grayscale value, and V R , V G , V B represent the component sizes of the image in the red, green, and blue channels respectively;

[0090] Step 1.2: Use two-dimensional Haar wavelet to decompose the grayscale image into four wavelet coefficients, namely the approximation component A and three detail components: horizontal detail component H, vertical detail component V, and diagonal detail component D. The calculation process is as follows:

[0091]

[0092]

[0093] Among them, a, b, c, and d respectively represent the pixel values of each matrix after dividing the grayscale image into 2×2 pixel matrices;

[0094] After the first-level decomposition, wavelet transform processing is performed again on the approximate component A to obtain the next-level approximate component, horizontal detail component, vertical detail distribution component, and diagonal detail component; then the decomposition is repeated for the next-level approximate component; finally, the four wavelet coefficients obtained from the third-level decomposition are thresholded using the soft thresholding method. The four wavelet coefficients are respectively compared with a set threshold. The wavelet coefficients smaller than the threshold in each block area are set to zero, and the wavelet coefficients greater than or equal to the threshold are set to the difference between the coefficient value and the threshold, thereby removing noise; the formula for setting the threshold is:

[0095]

[0096] Among them, MAD represents the median value of the absolute values of the wavelet coefficients after three-level decomposition of the grayscale image, 0.6745 represents the adjustment coefficient of the Gaussian noise standard deviation, and N is the size of the wavelet decomposition area;

[0097] Step 1.3, use the inverse wavelet transform to reconstruct the processed four wavelet coefficients into a new grayscale image. The calculation formula is:

[0098]

[0099] Among them, A′3, h′3, v′3, and D′3 respectively represent the approximate component, horizontal detail component, vertical detail component, and diagonal detail component obtained after three-level wavelet decomposition and thresholding processing.

[0100] Step 2, create a thread pool containing 4 threads. The 4 threads respectively execute reading video frames, detecting the target area using the background difference method, detecting the target area using the inter-frame difference method, and outputting the annotated video. The video preprocessed in Step 1 is sent frame by frame into the detection queue starting from the first frame.

[0101] Step 3, use the background difference method to compare the current frame image with the selected reference background, and frame out the moving target area in the current image. Specifically, it includes the following sub-steps:

[0102] Step 3.1, decompose the current video frame by frame, and use the first frame of the video as the reference background. Then the current frame image can be expressed as:

[0103] F i (x i ,y i ,g i )=b(x i ,y i ,g i )+t i (xi , y i , g i ) + n i (x i , y i , g i ) (11)

[0104] Among them, F i (x i , y i , g i ) represents the i-th frame of video image, i = 1, 2, 3... n; b(x i , y i , g i ) represents the reference background part in the video image; t i (x i , y i , g i ) represents the moving target part in the i-th frame of video image; n i (x i , y i , g i ) represents the noise part in the i-th frame of video image; x i and y i respectively represent the horizontal and vertical coordinate values of the pixel point in the i-th frame of video image; g i represents the gray value corresponding to this pixel point;

[0105] To obtain the moving target part, it is necessary to calculate the difference image between the current frame image F i (x i , y i , g i ) and the reference background image, that is:

[0106] d i (x i , y i , g i ) = F i (x i , y i , g i ) - b(x i , y i , g i ) + n i (x i , y i , g i ) (12)

[0107] Among them, d i (x i , y i , g i) represents the difference image jointly composed of the moving target area and the noise part;

[0108] Step 3.2: Perform histogram equalization on the difference image d i (x i ,y i ,g i ), where the histogram equalization is calculated according to formula (13):

[0109]

[0110] where s k represents the pixel point at the k-th gray level of the difference image, u represents the sum of all pixels in the difference image, u m represents the number of pixels at the current gray level, and L represents the number of gray levels;

[0111] Finally, use the threshold function to achieve binarization, obtaining the binarized video frame image G i (x i ,y i ,g i ), where the formula of the threshold function is:

[0112]

[0113] where β, η, and θ are all non-negative real numbers, θ represents the threshold, β represents the set gray value exceeding the threshold, and η represents the set gray value not exceeding the threshold;

[0114] Step 3.3: Perform morphological filtering on the above-processed binarized video frame image G i (x i ,y i ,g i ) by methods such as image erosion or image dilation to obtain the difference image P i (x i ,y i ,g i ); The image erosion is similar to image dilation. Both traverse the entire image through a 3×3 pixel matrix. During the traversal, the pixel points in the image are coincided with the center point of the matrix. Among them, image erosion updates the pixel value of the coincided image point to the minimum pixel value among the remaining pixels covered by the 3×3 pixel matrix, while image dilation updates the pixel value of the coincided image point to the maximum pixel value among the remaining pixels covered by the 3×3 pixel matrix, thereby removing discrete noise points in the image and enhancing the edges or connection parts in the image;

[0115] Step 3.4: Perform operations on the difference image P i (x i ,yi , g i ) Perform regional connectivity analysis. If P i (x i , y i , g i ) has a connected region Q i (x i , y i , g j ) with an area S1 not less than the set threshold T1, then this region is recognized as the moving target region t i (x i , y i , g i ), that is:

[0116]

[0117] Finally, divide the smallest bounding rectangle of this region as the bounding area of the moving target.

[0118] Step 4: Synchronously create and execute a video processing thread with Step 3. Use the continuous frame difference method to compare the current frame image with the previous frame image, and also select the moving target region. Specifically, it includes the following sub-steps:

[0119] Step 4.1: Decompose the current video frame by frame, calculate the difference between the current frame image and the previous frame image to obtain the corresponding difference image. The calculation method is:

[0120] c j (x j , y j , g j ) = F j (x j , y j , g j ) - F j-1 (x j , y j , g j ) + n j (x j , y j , g j ) (16)

[0121] Among them, F j (x j , y j , g j ) represents the j-th frame of the video image, j = 2, 3, 4... n; F j-1 (x j , y j , g j ) represents the corresponding (j - 1)-th frame image, c j (xj , y j , g j ) represents the difference image of the consecutive frame difference method;

[0122] Step 4.2, perform histogram equalization on the difference image c j (x j , y j , g j ), and finally use the threshold function to achieve binarization to obtain the binary video frame image G j (x j , y j , g j );

[0123] Step 4.3, perform morphological filtering on the binary video frame image G j (x j , y j , g j ) by methods such as image erosion or image dilation to obtain the difference image P j (x j , y j , g j ); The processing method refers to Step 3.3;

[0124] Step 4.4, perform region connectivity analysis on the difference image P j (x i , y i , g i ). If the area S2 of a certain connected region Q j (x j , y j , g j ) is not less than the set threshold T2, then this region is identified as the moving target region t j (x j , y j , g j ):

[0125]

[0126] Finally, divide the minimum bounding rectangle of this region as the bounding area of the moving target.

[0127] Step 5. Compare the distances between the bounding areas of the moving targets selected from the difference images obtained in the last frame of each of the two video processing threads in Step 3 and Step 4 and the real object respectively. Take the center point coordinates of the moving target region closer to the real object as the center position of the output selected box; calculate the area of the output selected box using the weighted average method; calculate the width and height of the output selected box based on the aspect ratio of the real box; The detection result is as Figure 3As shown in the figure, it specifically includes the following sub-steps:

[0128] Calculate the distances between the center points of the selected regions of all moving objects obtained by the background difference method and the inter-frame difference method and the center point of the real target region at the same position respectively, and take the smaller distance as the center point of the output selected box;

[0129] Calculate the CIoU of the areas between the selected regions of moving objects obtained by the background difference method and the inter-frame difference method and the real target region respectively; The CIoU and its related calculation formulas are:

[0130]

[0131]

[0132] Among them, IoU represents the intersection over union; A represents the predicted box, that is, the selected region of the moving object; B represents the real box; S |A∩B| and S |A∪B| represent the area of A intersecting B and the area of A union B respectively; ρ 2 (A, B) represents the square of the Euclidean distance between the geometric center points of the predicted box A and the real box B; c represents the diagonal length of the smallest bounding box that can contain the current predicted box and the real box at the same time; w and h represent the width and height respectively; α represents the weight coefficient; v represents the similarity of the aspect ratio between the predicted box and the real box;

[0133] Based on the CIoU of the background difference method and the inter-frame difference method, calculate the area of the output selected box by the weighted average method, and the calculation formula is as follows:

[0134]

[0135] Make the aspect ratio of the output selected box equal to the aspect ratio of the real box, and the width and height of the output selected box are respectively:

[0136]

[0137] Among them, S′ represents the area of the output selected box; w A ′ represents the width of the output selected box; h A ′ represents the height of the output selected box; w B represents the width of the real box; hB 表 represents the height of the real box; q represents the total number of selected boxes selected by the background difference method and the inter-frame difference method; p represents a specific selected box selected by the background difference method or the inter-frame difference method, CIoU p represents the CIoU of the selected box p, S p ′ represents the area of the selected box p.

[0138] Step 6: Determine whether the output selection box and its adjacent selection boxes around it contain the same moving target. Based on the overlapping ratio between the selection boxes containing the same moving target, perform region merging to determine the final target region and the center position, preventing misdetection. Specifically, it includes the following sub-steps:

[0139] Step 6.1: Take the output selection box as the main selection box, calculate the CIoU between the main selection box and its adjacent selection boxes around it. If the CIoU is less than the set threshold, it means that the main selection box and the adjacent selection boxes around it do not contain the same moving target, and output this main selection box as the detection result; if the CIoU is greater than the threshold, it proves that the main selection box and the adjacent selection boxes around it contain the same moving target, and proceed to the next step;

[0140] Step 6.2: Compare the area of the main selection box with the area of its adjacent selection boxes around it, and take the selection box with the largest area as the new main selection box; the calculation formula is as follows:

[0141]

[0142] where S′ represents the area of the main selection box at a certain position, r represents the other adjacent selection boxes at this position, o represents the number of the other adjacent selection boxes at this position, and S r * represents the corresponding area of the other adjacent selection boxes;

[0143] Step 6.3: Determine whether the overlapping ratio between the new main selection box and the other adjacent selection boxes is greater than the set threshold or the center point distance is less than the specified threshold. If neither is satisfied, delete the corresponding adjacent selection box; if so, fuse the other adjacent selection boxes into the main selection box and update the center point coordinates and the minimum circumscribed rectangle area of the main selection box. The calculation formula for the center point coordinates of the fused selection box is as follows:

[0144]

[0145] where x r and y r respectively represent the abscissa and ordinate of the center point of the fused selection box; (x r1 , y r1 ), (x r2 , y r2 ) represent the coordinates of the two selection boxes before fusion; w r1 , w r2 represent the widths of the two selection boxes before fusion; h r1 , h r2 represent the heights of the two selection boxes before fusion.

[0146] Step 7: Calculate the height of the moving target finally determined in Step 6 and mark it at the corresponding position in the video. Specifically, it includes the following sub-steps:

[0147] Step 7.1: Restore the video frame image from a grayscale image. The calculation method is as follows:

[0148]

[0149]

[0150] where V gray represents the grayscale value of a certain pixel point in the frame image, and V R , V G , and V B respectively represent the component values of the red, green, and blue channels after conversion of this pixel point;

[0151] Step 7.2: Calculate the height of the center point of the current moving target area based on the position of the current industrial camera and the height information of the calibration points in the picture, and display it in the real-time picture to achieve the height calculation of the moving target. The calculation formula is as follows:

[0152]

[0153] where h z represents the height of the z-th moving target in the current frame image, y rz represents the ordinate value of the center point of the bounding box area of the z-th moving target calculated in Step 6, m2 and m1 respectively represent the actual heights of two calibration points in the actual scene, and R h represents the number of vertical pixels between two calibration points in the video frame picture.

[0154] As Figure 4 shown, it is known that the height of the black tile part is about 15 cm. Then, the top of the black tile and the corresponding vertical ground position can be used as reference height marking points. The height from the top of the black tile to the ground in the image, that is, the length of the red line segment, is about 96 pixel points. Therefore, an object with an actual height of 1 cm occupies about 6.4 pixel points in the image. Therefore, the relative height of the foreign object is calculated proportionally according to the number of pixel points of the foreign object from the ground in the video frame image. The output result of the final video frame image is as Figure 5 shown. The position of the center point of the bounding box area of the foreign object in the image is about 87 pixel points from the ground. Then, the height of the current foreign object can be calculated to be about 13.72 cm.

[0155] According to the above steps, the present invention is experimentally compared with the object detection method based on Scale-Invariant Feature Transform (SIFT), the object detection method based on Haar feature cascade classifier, the object detection method based on Histogram of Oriented Gradients (HOG), and the object tracking method based on optical flow method on the video and image data of foreign object falling collected by the present invention. The experimental results are shown in Table 1:

[0156] Table 1 Comparison of Detection Performance of Different Methods for Foreign Object Drop in Nuclear Fuel Pool

[0157]

[0158]

[0159] In summary, the present invention can timely detect the situation of foreign object drop in the relevant area of the nuclear fuel pool inside the nuclear power plant, prevent foreign objects from causing harm to equipment or personnel, and ensure the safe operation of the nuclear power plant; it also improves work efficiency, reduces the waste of human resources, and has low requirements for hardware computing power, enabling a relatively fast detection speed; it also improves the detection accuracy. By using high-resolution industrial cameras and supporting lenses, it can accurately capture and detect the position of foreign object drop and the situation of moving objects, avoiding missed detection and false detection; moreover, it reduces the maintenance cost and repair expenses, and avoids damage or failure of some related equipment.

[0160] The above-described embodiments are all preferred embodiments of the present invention and do not impose other forms of limitation on the present invention. Any person skilled in the relevant art may make changes or imitations using the above content. However, any changes made to the above embodiments based on the essence of the method of the present invention without departing from the content of the present invention still fall within the protection scope of the present invention.

Claims

1. A method for detecting the fall of foreign objects in a nuclear fuel pool, characterized in that, The method includes: Reading the video captured by an industrial camera and preprocessing the video; Using the background difference method and the consecutive frame difference method respectively to frame the moving target area in the current frame image; For each frame image, compare the distances between the moving target areas framed by the background difference method and the consecutive frame difference method and the real object respectively, and take the center point coordinates of the moving target area closer to the real object as the center position of the output selected box; calculate the area of the output selected box using the weighted average method; calculate the width and height of the output selected box based on the aspect ratio of the real box; the specific process is as follows: Calculate the distances between the center points of all the moving target framed areas at the same position by the background difference method and the frame difference method and the center point of the real target area respectively, and take the one with the smaller distance as the center point of the output selected box; Calculate the CIoU of the areas between the moving target framed areas obtained by the background difference method and the frame difference method and the real target area respectively; Based on the CIoU of the background difference method and the frame difference method, calculate the area of the output selected box using the weighted average method, and the calculation formula is as follows: Make the aspect ratio of the output selected box equal to the aspect ratio of the real box, and the width and height of the output selected box are respectively: Among them, S′ represents the area of the output selected box; w A ′ represents the width of the output selected box; h A ′ represents the height of the output selected box; w B represents the width of the ground truth box; h B represents the height of the ground truth box; q represents the total number of selected boxes framed by the background difference method and the inter-frame difference method; p represents a selected box at a specific position framed by the background difference method or the inter-frame difference method, and CIoU p represents the CIoU of the selected box p, and S p ′ represents the area of the selected box p; Judge whether the output selected box and its adjacent selected boxes around it contain the same moving target, and perform region merging based on the overlapping ratio between the selected boxes containing the same moving target to determine the final target area and the center position; Calculate the height of the moving target based on the final target area and mark it at the corresponding position in the video.

2. The foreign object drop detection method for a nuclear fuel pool according to claim 1, characterized in that The step of reading the video captured by the industrial camera and preprocessing the video specifically includes: Denoise the original video data by means of wavelet transform, decompose the video into frame images and convert each frame image into a grayscale image; Use two-dimensional Haar wavelet to decompose the grayscale image into four wavelet coefficients, namely the approximation component and three detail components: horizontal detail component, vertical detail component and diagonal detail component; After the first-level decomposition, perform wavelet transform processing on the approximation component again to obtain the next-level approximation component, horizontal detail component, vertical detail component and diagonal detail component; then repeat the decomposition on the next-level approximation component; finally, perform thresholding processing on the four wavelet coefficients of the third-level decomposition using the soft threshold method, compare the four wavelet coefficients with the set threshold respectively, set the wavelet coefficients less than the threshold to zero in each block area, and set the wavelet coefficients greater than or equal to the threshold to the difference between the coefficient and the threshold, so as to remove noise; the formula for setting the threshold is: where MAD represents the median value of the absolute values of the wavelet coefficients after three-level decomposition of the grayscale image, 0.6745 represents the adjustment coefficient of the standard variance of Gaussian noise, and N is the size of the wavelet decomposition area; Use inverse wavelet transform to reconstruct the processed four wavelet coefficients into a new grayscale image.

3. A method for detecting foreign object fall in a nuclear fuel pool according to claim 1, characterized in that, Use the background difference method to compare the current frame image with the selected reference background and frame the moving target area in the current frame image, specifically including: Decompose the current video frame by frame, and use the first frame of the video as the reference background, then the current frame image is expressed as: F i (x i ,y i ,g i ) = b(x i ,y i ,g i ) + t i (x i ,y i ,g i ) + n i (x i ,y i ,g i ) (5) Among them, F i (x i , y i , g i ) represents the i-th frame of video image, where i = 1, 2, 3... n; b(x i , y i , g i ) represents the reference background part in the video image; t i (x i , y i , g i ) represents the moving target part in the i-th frame of video image; n i (x i , y i , g i ) represents the noise part in the i-th frame of video image; x i and y i respectively represent the horizontal and vertical coordinate values of the pixel point in the i-th frame of video image; g i represents the gray value corresponding to this pixel point; Calculate the current frame image F i (x i , y i , g i ) and the difference image between the reference background image: d i (x i ,y i ,g i ) = F i (x i ,y i ,g i ) - b(x i ,y i ,g i ) + n i (x i ,y i ,g i ) (6) where d i (x i , y i , g i ) represents the differential image jointly composed of the moving target area and the noise part in the i-th frame image; For the differential image d i (x i ,y i ,g i ) perform histogram equalization processing, where the histogram equalization is calculated according to formula (7): where s k represents the pixel points of the k-th gray level of the differential image, u represents the sum of all pixels in the differential image, and u m represents the number of pixels of the current gray level, and L represents the number of gray levels; Finally, the threshold function is used to implement binarization to obtain the binarized video frame image G i (x i , y i , g i ), where the formula of the threshold function is: Among them, β, η, and θ are all non-negative real numbers. θ represents the set threshold, β represents the set gray value exceeding the threshold, and η represents the set gray value not exceeding the threshold; Perform morphological filtering on the binary video frame image G i (x i ,y i ,g i ) to obtain the difference image P i (x i ,y i ,g i ); For the differential image P i (x i ,y i ,g i ), perform region connectivity analysis. If the area S1 of a certain connected region Q i (x i ,y i ,g i ) in P i (x i ,y i ,g i ) is not less than the set threshold T1, then this region is recognized as a moving target region t i (x i ,y i ,g i ), that is: Finally, the minimum circumscribed rectangle of this area is divided as the bounding area of the moving target.

4. A method for detecting foreign object falling in a nuclear fuel pool according to claim 1, characterized in that, The current frame image is compared with the previous frame image by using the continuous frame difference method to frame the moving target area in the current frame image, which specifically includes: The current video is decomposed frame by frame, and the difference between the current frame image and the previous frame image is calculated to obtain the corresponding difference image. The calculation method is as follows: c j (x j ,y j ,g j ) = F j (x j ,y j ,g j ) - F j-1 (x j ,y j ,g j ) + n j (x j ,y j ,g j ) (10) Among them, F j (x j , y j , g j ) represents the j-th frame of video image, where j = 2, 3, 4... n; F j-1 (x j , y j , g j ) represents the corresponding (j - 1)-th frame image; c j (x j , y j , g j ) represents the difference image of the consecutive frame difference method; For the differential image c j (x j ,y j ,g j ) perform histogram equalization processing, and use the threshold function to achieve binarization to obtain the binary video frame image G j (x j ,y j ,g j ); The obtained binary video frame image G j (x j ,y j ,g j ) is subjected to morphological filtering to obtain a difference image P j (x j ,y j ,g j ); For the differential image P j (x j ,y j ,g j ), perform region connectivity analysis. If the area S2 of a certain connected region Q j (x j ,y j ,g j ) is not less than the set threshold T2, then identify this region as the moving target region t j (x j ,y j ,g j ): Finally, the minimum circumscribed rectangle of this area is divided as the bounding area of the moving target.

5. A method for detecting foreign object falling in a nuclear fuel pool according to claim 1, characterized in that The determination of the final area and the center position of the target specifically includes: Taking the output selected box as the main selected box, calculate the CIoU between the main selected box and its surrounding adjacent selected boxes. If the CIoU is less than the set threshold, it means that there is no same moving target between the main selected box and its surrounding adjacent selected boxes, and this main selected box is output as the detection result; if the CIoU is greater than the threshold, it proves that there is the same moving target between the main selected box and its surrounding adjacent selected boxes, and proceed to the next step; Compare the area between the main selected box and its surrounding adjacent selected boxes, and take the selected box with the largest area as the new main selected box. The calculation formula is as follows: Among them, S′ represents the area of the main selected box at a certain position, r represents the remaining adjacent selected boxes at that position, o represents the number of the remaining adjacent selected boxes at that position, and S r * represents the corresponding area of the remaining adjacent selected boxes; Calculate the overlap ratio between the new main selected box and the remaining adjacent selected boxes. If it is greater than the set threshold or the center point distance is less than the specified threshold, then fuse the remaining adjacent selected boxes into the main selected box and update the center point coordinates and the minimum circumscribed rectangle area of the main selected box; if neither is satisfied, delete the corresponding adjacent selected box; among them, the calculation formula for the center point coordinates of the fused selected box is as follows: Among them, x r and y r respectively represent the abscissa and ordinate of the center point of the selected box after fusion; (x r1 , y r1 ), (x r2 , y r2 ) represent the coordinates of the two selected boxes before fusion; w r1 , w r2 represent the widths of the two selected boxes before fusion; h r1 , h r2 represent the heights of the two selected boxes before fusion.

6. A method for detecting the fall of foreign objects in a nuclear fuel pool according to claim 5, characterized in that, The CIoU and its related calculation formulas are: Among them, IoU represents the intersection over union; A represents the predicted bounding box, that is, the selected area of the moving target box; B represents the ground truth bounding box; S |A∩B| and S |A∪B| represent the area of A intersecting B and the area of A union B respectively; ρ 2 (A, B) represents the square of the Euclidean distance between the geometric center points of the predicted bounding box A and the ground truth bounding box B; c represents the diagonal length of the smallest bounding box that can simultaneously contain the current predicted bounding box and the ground truth bounding box; w and h represent the width and height respectively; α represents the weight coefficient; v represents the similarity of the aspect ratio between the predicted bounding box and the ground truth bounding box.

7. A method for detecting the fall of foreign objects in a nuclear fuel pool according to claim 1, characterized in that, The calculation of the height of the moving target specifically includes: Restore the video frame image from a grayscale image to a color image; According to the position of the current industrial camera and the height information of the calibration points in the picture, calculate the height of the center point of the current moving target area and display it in the real-time picture; the calculation formula is: Among them, h z represents the height of the z-th moving object in the current frame image, and y rz represents the vertical coordinate value of the center point of the bounding box of the z-th moving object calculated. m2 and m1 respectively represent the actual heights of two calibration points in the actual scene, and R h represents the number of vertical pixels between two calibration points in the video frame.

8. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored on the memory and capable of running on the processor; when the processor executes the computer program, the electronic device executes the nuclear fuel pool foreign object falling detection method according to any one of claims 1-7.

9. A storage medium, characterized in that, The storage medium includes a computer program, and when the computer program runs on an electronic device, the electronic device executes the nuclear fuel pool foreign object falling detection method according to any one of claims 1-7.

Images