A floating body net-off intelligent detection method of a cold source intercepting net

Abstract

The application discloses a kind of floating body off-net intelligent detection methods of cold source interception net, comprising: the monitoring video data of target interception net is collected using camera, the key frame picture shown with floating body in video is extracted, as sample picture to form floating body sample set;The area containing floating body in sample picture is circled out by graphic annotation tool, forms annotation object frame, the picture in annotation object frame contains floating body and part background information of floating body, and annotation object frame is labeled, to obtain the annotation of each sample picture;Sample picture and its corresponding annotation are input to YOLO network model for training, to realize the automatic annotation of floating body area in sample picture, if the label result output by YOLO network model is floating body off-net, then send alarm to remind.The cold source interception net floating body off-net intelligent detection method provided by the application realizes the automatic monitoring of floating body off-net in all-day large range.

Description

Technical Field

[0001] This invention relates to the field of early warning technology for cold source interception nets, and in particular to an intelligent detection method for the detachment of floats from cold source interception nets. Background Technology

[0002] The safety of the water intake is crucial for nuclear power plants. The interception net, used to trap marine life, is directly related to the safety of the intake. Floats maintain the stability of the net; if a float detaches from the net, the net's load-bearing capacity weakens, affecting its stability and potentially threatening the safety of the intake. Currently, the main method is for staff to check the float's status via video surveillance. The drawback is that the monitoring is real-time, and with numerous cameras, comprehensive coverage by manpower is impossible.

[0003] In recent years, with the rapid development of deep learning technology, computer vision technology has been widely used in various fields. Among these, object detection is a relatively mature application, with YOLOv5 being a classic model widely adopted. It can accurately locate objects in images and determine their categories. Although traditional vision methods are no longer mainstream in many fields due to the development of deep learning methods, they still play an important role in many aspects, such as color filtering and affine transformations.

[0004] The above background information is provided only to assist in understanding the inventive concept and technical solution of this invention. It does not necessarily belong to the prior art of this patent application, nor does it necessarily provide technical teaching. In the absence of clear evidence that the above information was disclosed before the filing date of this patent application, the above background information should not be used to evaluate the novelty and inventiveness of this application. Summary of the Invention

[0005] To overcome the shortcomings of existing technologies, this invention provides an intelligent detection method for the detachment of floats in a cold source interception net. The specific technical solution is as follows:

[0006] A method for intelligent detection of float detachment from cold source interception net is provided, including the following steps: using a camera to collect monitoring video data of the target interception net, extracting key frame images showing floats in the video, and using them as sample images to form a float sample set;

[0007] The regions containing buoys in the sample images are circled using an image annotation tool to form annotation bounding boxes. The images within the annotation bounding boxes contain the buoys and some background information of the buoys. The annotation bounding boxes are then annotated to obtain the annotations for each sample image. The annotations are divided into two categories: one for buoys that have detached from the interception net and the other for buoys that have not detached from the interception net. The sample images and their corresponding annotations are then input into a YOLO network model for training to achieve automatic annotation of the buoy regions in the sample images.

[0008] The keyframe images from the current surveillance video data are processed and then input into the trained YOLO network model. If the label output by the YOLO network model is a buoy that has escaped the interception net, an alarm is issued.

[0009] Furthermore, the YOLO network model first identifies the buoy in the keyframe image to be detected and calculates the current position coordinates of the buoy. It then compares the current position coordinates of the buoy with the corresponding pre-stored original position coordinates. If the error between the two exceeds a preset range, it is determined that the buoy has gone off-net, and a corresponding mark is made, thereby issuing an alarm reminder. Otherwise, it is necessary to combine the area relationship between the buoy and its background image to further determine the state of the buoy.

[0010] Furthermore, determining the state of the pontoon by combining the area relationship between the pontoon and its background image includes the following steps:

[0011] Furthermore, determining the state of the pontoon by combining the area relationship between the pontoon and its background image includes the following steps:

[0012] The buoy image is cropped from the original captured image according to coordinates. Then, the buoy area in the cropped image is located to form a labeled object box. The area of ​​the region that meets the buoy color range condition is calculated as the buoy area. If it is larger than a preset threshold, it is judged that the net has been lost, and corresponding marking is performed, and then an alarm is issued.

[0013] Furthermore, images of the same buoy are captured from the original images taken at multiple nearby time points according to their coordinates. Then, the buoy area in the captured images is located to form a labeled object box. The area of ​​the region that meets the buoy color range condition is calculated as the buoy area, and the average value is taken. If the average value is greater than a preset threshold, it is determined that the net has been lost, and a corresponding mark is made, thereby issuing an alarm reminder.

[0014] Furthermore, the preset threshold is a times the average area of ​​the adjacent buoys that have not been detached from the net, where a is ∈ (1.1, 1.2).

[0015] Furthermore, the preset threshold is a times the average of the float areas of the same float when it was not previously detached from the net, where a is ∈ (1.1, 1.2).

[0016] Furthermore, if the area ratio of the float is greater than the preset threshold, the degree of area distortion is detected and it is determined whether it is normal. If it is determined to be abnormal, it indicates that the float is damaged and needs to be replaced.

[0017] Furthermore, in the YOLO network model, the input data is processed sequentially through a preprocessing module, a backbone network module, a neck network module, and a head network module before being output. The preprocessing module is used to normalize and standardize the image, the backbone network module is used to extract features from the image, the neck network is used to integrate and reprocess the extracted features at different stages, and the head network module is used to predict the location and label category of the target.

[0018] Furthermore, only one pontoon is displayed within the labeled object box, and the pixel count error within each labeled object box is less than 5%; the original position coordinates corresponding to the pontoon in the captured image are stored in advance, and the keyframe image, along with the original position coordinates associated with its pontoon, is output to the YOLO network model for processing.

[0019] Furthermore, the camera uses RGB color mode for shooting, the float is set to red, the interception net is set to green, and the color channels are adjusted to highlight the red in the image and weaken the display of other colors.

[0020] Compared with existing technologies, the present invention has the following advantages: it realizes automatic monitoring of the buoy netting detachment over a wide area throughout the day, which is cheaper, more stable, and more efficient than manual identification. Attached Figure Description

[0021] Figure 1 This is a flowchart illustrating the intelligent detection method for float detachment of a cold source interception net provided in an embodiment of the present invention;

[0022] Figure 2 This is a schematic diagram of the YOLO network model framework in the intelligent detection method for float detachment of cold source interception net provided in the embodiments of the present invention. Detailed Implementation

[0023] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0024] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion.

[0025] In one embodiment of the present invention, a method for intelligent detection of float detachment from a cold source interception net is provided, comprising the following steps:

[0026] The system uses cameras to collect surveillance video data of the target interception network, periodically takes pictures of the area around the interception network, and extracts key frame images of floats in the video as sample images to form a float sample set.

[0027] The regions containing pontoons in the sample images are circled using an image annotation tool to form annotation bounding boxes. Each annotation bounding box contains the pontoon and some background information about it. Only one pontoon is displayed within each box, and the pixel count error within each box is less than 5%. These boxes are then annotated to obtain the annotations for each sample image. The annotations are divided into two categories: pontoons that have detached from the net and pontoons that have not. The sample images and their corresponding annotations are then input into a YOLO network model for training to achieve automatic annotation of the pontoon regions in the sample images.

[0028] After processing the keyframe images in the current surveillance video data, they are input into the trained YOLO network model. If the label result output by the YOLO network model is a buoy that has detached from the interception net, an alarm is issued, which includes the location coordinates of the detached buoy. If the label result is a buoy that has not detached from the interception net, manual verification is required to reduce the workload.

[0029] The original position coordinates corresponding to the pontoon in the captured image are stored in advance. The keyframe image, along with its associated original position coordinates, is output to the YOLO network model for processing. See also Figure 1The YOLO network model first identifies the buoy in the keyframe image to be detected and calculates its current position coordinates. It then compares these coordinates with the corresponding pre-stored original position coordinates. If the error exceeds a preset range (e.g., the difference between the horizontal or vertical coordinates is greater than 3 meters), it is determined that the buoy has detached from the net, is marked accordingly, and an alarm is issued. Otherwise, the area relationship between the buoy and its background image is used to further determine the buoy's status. When the buoy is not detached from the net, the depth of its submersion is greater than when it is detached due to the pulling of the net. Different submersion depths result in different areas above the water surface. When the buoy is detached from the net, its area above the water surface is greater than when it is not detached. This principle is used to determine whether the buoy has detached from the net.

[0030] To further determine the state of the pontoon by combining the area relationship between the pontoon and its background image, the following steps are included:

[0031] The buoy image is cropped from the original captured image according to coordinates. The buoy region within the cropped image is then located, forming a bounding box. The area of ​​the region satisfying the buoy's color range condition is calculated; areas with the same or similar colors are used as the buoy area. If this area is larger than a preset threshold, it is determined that the buoy has detached from the net, is marked accordingly, and an alarm is issued. If the buoy area is larger than the preset threshold, area distortion is detected and its normality is determined. If it is determined to be abnormal, it indicates that the buoy is damaged and needs replacement. In a preferred embodiment, to avoid misjudgment caused by the buoy temporarily and significantly emerging from the water surface due to waves, it is necessary to consider the situation at multiple time points. Specifically, the image of the same buoy is cropped from the original captured images at multiple adjacent time points according to coordinates. The buoy region within the cropped image is then located, forming a bounding box. The area of ​​the region with the same color as the buoy is calculated as the buoy area, and the average value is taken. If the average value is larger than a preset threshold, it is determined that the buoy has detached from the net, is marked accordingly, and an alarm is issued.

[0032] The preset threshold includes, but is not limited to, the following setting methods:

[0033] (1) The shooting area gradually increases as the buoys move from far to near, which may lead to misjudgment. Therefore, it is more reasonable to refer to two or four adjacent buoys for area comparison. Therefore, the preset threshold is a times the average area of ​​the buoys corresponding to the adjacent buoys that have not been detached from the net, where a is ∈ (1.1, 1.2).

[0034] (2) The buoy is prone to rising and falling with the water surface, which can cause changes in the shooting area and lead to misjudgment. Therefore, the preset threshold is a times the average of the buoy area of ​​the same buoy when it was not removed from the net, where a is ∈ (1.1, 1.2).

[0035] (3) The preset threshold is an area threshold, and its calculation formula is as follows:

[0036]

[0037] This value is positively correlated with the y-axis coordinate of the center point of the buoy and the average area of ​​the buoy, where N is the number of buoys and α is an empirical parameter. min +y max ) / 2 is the y-coordinate of the current buoy center point, S i The area of ​​the i-th buoy is shown in the following comparison. The area of ​​the buoy must satisfy the following formula:

[0038] δ i-1 *S i-1 <δ i *S i <δ i+1 *S i+1

[0039] Where, δ i These are empirical values ​​related to the buoy spacing and the y-axis coordinate.

[0040] The intelligent detection method for pontoon detachment in cold source interception nets provided by this invention uses deep learning to detect whether a pontoon has detached from the net. It can cover more cameras based on hardware resources and can monitor continuously 24 / 7. Compared to manual identification, it is cheaper and more efficient. Furthermore, with more data collected, the data available for training becomes richer, resulting in better model robustness.

[0041] In one embodiment of the present invention, after processing the float sample set, a training set, a validation set, and a test set are randomly selected in proportion to train the YOLO network model. The YOLO network model uses the YOLOv5 detection model. The input data is processed sequentially through a preprocessing module, a backbone network module, a neck network module, and a head network module before output. The preprocessing module is used to normalize and standardize the images. The backbone network module is used to extract features from the images. The neck network is used to integrate and reprocess the extracted features at different stages. The head network module is used to predict the location and label category of the target.

[0042] To build a YOLOv5 network structure in PyTorch and load the training and validation sets into the YOLOv5 network, see [link / reference]. Figure 2The images undergo a series of preprocessing steps: the backbone network extracts image features, the neck network performs feature fusion, and the head network makes judgments to ultimately obtain the predicted bounding box's label and coordinate information. The predicted bounding box is compared with the ground truth label, and a loss function is used to calculate and backpropagate to the aforementioned networks, influencing the next iteration. After multiple iterations, the optimal weight data, i.e., the model file, is obtained. YOLOv5 is used to detect the images, obtaining the buoy's position coordinates and label category. Targets labeled as "detached buoys" trigger an alarm, while buoys classified as "on the interception net" are sent to the post-processing module for judgment using traditional visual methods.

[0043] The camera uses RGB color mode for shooting. RGB color mode is an industry color standard that obtains various colors by varying the red (R), green (G), and blue (B) color channels and superimposing them. RGB represents the colors of the red, green, and blue channels. The float is set to red, and the interception net is set to green. The color channels are adjusted to highlight the red in the image and weaken the display of other colors.

[0044] Specifically, taking a red pontoon as an example, the pontoon is cropped from the original image according to its coordinates. Then, the pontoon area in the cropped image is located, i.e., the prominent red area in the image is selected. To further highlight the red area, half the sum of the B and G channels is subtracted from the R channel. Areas with an R value greater than a set color threshold are the pontoon areas. Finally, the area of ​​the red area that meets the requirements is calculated, and an alarm is triggered for those red areas that are significantly larger than other pontoons. If the pontoon is covered by an interception net (green or other colored mesh), the color of the covered pontoon will change accordingly. Furthermore, the area of ​​the pontoons gradually increases from far to near, and comparing the area with two or four adjacent pontoons is more reasonable.

[0045] The intelligent detection method for pontoon detachment in cold source interception nets provided by this invention is based on deep learning and traditional vision technology. It detects and warns of anomalies in the pontoon trawl net; it obtains the pontoon position and classifies its state (whether the pontoon has detached from the net) using a YOLOv5 detection model; for pontoons that the model does not detect, traditional vision methods are used to determine if the pontoon is abnormal. When an anomaly occurs, an anomaly alarm message is sent to remind management personnel to handle and repair it promptly. This method is independently developed based on deep learning networks such as YOLOv5; test results show that the recall rate for pontoon trawl net identification is over 95%.

[0046] The above description is merely a preferred embodiment of the present invention and does not limit its patent scope. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, whether directly or indirectly applied to other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A method for detecting the intelligent floating of the cold source interception net, characterized in that, The method comprises the following steps: Collecting monitoring video data of the target intercepting net by using a camera, extracting key frame pictures showing buoys in the video as sample pictures to form a buoy sample set; Outlining the area containing the buoy in the sample pictures by using a graphic labeling tool to form a labeled object frame, the picture in the labeled object frame containing the buoy and part of the background information of the buoy, and labeling the labeled object frame to obtain the label of each sample picture, wherein the label is divided into two categories, one is a buoy that has detached from the intercepting net, and the other is a buoy that has not detached from the intercepting net; inputting the sample pictures and the corresponding labels into a YOLO network model for training to realize automatic labeling of the buoy area in the sample pictures; Processing the key frame pictures in the current monitoring video data and inputting them into the trained YOLO network model, and if the label result output by the YOLO network model is a buoy that has detached from the intercepting net, an alarm is issued; The YOLO network model first identifies the buoy in the key frame picture to be detected and calculates the current position coordinates of the buoy, compares the current position coordinates of the buoy with the corresponding pre-stored original position coordinates, and if the error between the two exceeds a preset range, it is judged that the net has been detached, corresponding marking is performed, and an alarm is issued, otherwise, the area relationship between the buoy and its background image is combined to further judge the state of the buoy.

2. The pontoon off-network intelligent detection method according to claim 1, characterized in that, Combining the area relationship between the buoy and its background image to further judge the state of the buoy comprises the following steps: The buoy image is cut off from the collected and photographed original picture according to the coordinates, and then the buoy area in the screenshot is located to form a labeled object frame, the area of the region meeting the buoy color range condition is calculated as the buoy area, and if it is greater than a preset threshold, it is judged that the net has been detached, corresponding marking is performed, and an alarm is issued.

3. The pontoon off-network intelligent detection method according to claim 1, characterized in that, The images of the same buoy are cut off from the collected and photographed original pictures at adjacent time points according to the coordinates, and then the buoy area in the screenshot is located to form a labeled object frame, the area of the region meeting the buoy color range condition is calculated as the buoy area, and the average value is taken, and if the average value is greater than a preset threshold, it is judged that the net has been detached, corresponding marking is performed, and an alarm is issued.

4. The pontoon off-network intelligent detection method according to claim 2 or 3, characterized in that, The preset threshold is a times the average value of the buoy areas corresponding to the adjacent buoys that have not detached from the net, and a is ∈(1.1, 1.2).

5. The pontoon off-network intelligent detection method according to claim 2 or 3, characterized in that, The preset threshold is a times the average value of the buoy areas corresponding to the same buoy at different times when the buoy has not detached from the net, and a is ∈(1.1, 1.2).

6. The pontoon off-network intelligent detection method according to claim 2, characterized in that, If the buoy area is greater than the preset threshold, the area distortion degree detection is performed and it is judged whether it is normal, and if it is judged to be abnormal, it indicates that the buoy has been damaged, prompting replacement.

7. The pontoon off-network intelligent detection method according to claim 1, characterized in that, In the YOLO network model, the input data is sequentially processed by a preprocessing module, a backbone network module, a neck network module and a head network module and then output, the preprocessing module is used for normalizing and standardizing the picture, the backbone network module is used for extracting features in the picture, the neck network is used for integrating and reprocessing the extracted features at different stages, and the head network module is used for predicting the position and label category of the target.

8. The pontoon off-network intelligent detection method according to claim 1, characterized in that, Only one buoy is displayed in each labeled object box, and the pixel point error in each labeled object box is less than 5%; the original position coordinates corresponding to the buoy in the photographed picture are stored in advance, and the key frame picture and the original position coordinates associated with the buoy thereof are output to the YOLO network model for processing.

9. The pontoon off-network intelligent detection method according to claim 1, characterized in that, The camera adopts an RGB color mode for shooting, the buoy is set to red, and the interception net is set to green, and the color channel is adjusted to highlight the red color in the image and weaken the display of other colors.

Images