A method and terminal for detecting fishing behavior in power transmission lines

Abstract

The present invention discloses a fishing behavior detection method and terminal for power transmission lines, the method comprising: training a semantic segmentation model and a target detection model respectively through historical images of the power transmission line area, obtaining a water segmentation model and a fishing detection model, realizing automatic segmentation of a target water map from a monitoring image and detecting targets related to fishing behavior, so that the model can effectively extract specific features in the current scene, and improve the accuracy of model segmentation or detection. The target water map is segmented according to the monitoring image, and a monitoring inspection point is constructed, and a pan-tilt camera device is called to perform fixed-point cruise, so that the pan-tilt camera device can focus on the water area for monitoring, improve monitoring efficiency, and simultaneously obtain the corresponding cruise point image, which can effectively avoid the problem that the image pixels taken by the camera are unclear and the target cannot be accurately identified, and improve the recall rate and accuracy of fishing behavior recognition under the power transmission line.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method and a terminal for detecting fishing behavior of a power transmission line. Background Art

[0002] With the research and development of visual deep learning models, automatic detection of fishing behavior in prohibited fishing sections based on images taken by surveillance cameras has become an important means of current river and lake management.

[0003] At present, there are many methods for automatically detecting fishing behavior based on images, but in the power transmission line scenario, the automatic detection methods of fishing behavior have certain technical defects. A fishing behavior detection method and system are disclosed in a patent document (patent publication number is CN116721459A), which first obtains the water area, then obtains the edge of the water area and the edge of the fishing rod through traditional images, combines the edges of the two to obtain the position of the fisherman, and then uses the behavior recognition module to determine whether there is fishing behavior. However, in the power transmission scenario, due to the height of the camera installation position, the collected image cannot robustly obtain the edge of the fishing rod using the traditional image algorithm, and it is easy to fail to locate the position of the person, which is easy to cause missed detection. A patent document (patent publication number CN116630890A) discloses a fishing behavior detection method and system by analyzing the fishing area, and identifies fishing behavior through target detection and other technologies. However, under the transmission line, the camera is usually installed on the transmission tower, the data shooting angle is a bird's-eye view, and the vertical distance from the ground is more than 5 meters. The target pixel ratio of people and fishing rods is often less than 30 pixels. At the same time, the amount of data on people and fishing rods in this scene is small, resulting in a low recall rate for the detection of pedestrians and fishing rods by this method, and it is unable to effectively distinguish whether there is fishing behavior by people. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a method and terminal for detecting fishing behavior for power transmission lines, so as to achieve effective detection of fishing behavior and improve detection accuracy based on the power transmission line scenario.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0006] A method for detecting fishing behavior in a power transmission line, comprising:

[0007] Obtain historical imagery of transmission line areas;

[0008] The preset semantic segmentation model and target detection model are trained respectively according to the historical images to obtain a water area segmentation model and a fishing detection model;

[0009] Acquiring a monitoring image of the power transmission line area;

[0010] Segment the monitoring image using the water area segmentation model to obtain a target water area map;

[0011] Constructing a monitoring cruise point according to the target water area map, and mobilizing a preset pan-tilt camera device to perform a fixed-point cruise according to the monitoring cruise point to obtain a cruise point image;

[0012] Performing target detection on the cruise point image according to the fishing detection model to obtain a target detection result, and mapping the target detection result to the monitoring image to obtain a fishing detection image;

[0013] Determine whether there is fishing behavior in the power transmission line area according to the fishing detection image.

[0014] In order to solve the above technical problems, another technical solution adopted by the present invention is:

[0015] A fishing behavior detection terminal for power transmission lines comprises a memory, a processor and a computer program stored in the memory and running on the processor. When the processor executes the computer program, each step of the above-mentioned fishing behavior detection method for power transmission lines is implemented.

[0016] The beneficial effect of the present invention is that the preset semantic segmentation model and target detection model are trained respectively by the historical images of the transmission line area, and the water area segmentation model and the fishing detection model are obtained, so as to realize the automatic segmentation of the target water area map from the monitoring image of the transmission line area and the detection of the target related to the fishing behavior, so that the corresponding model can effectively extract the specific features in the current scene, thereby improving the accuracy of the model segmentation or detection, thereby enhancing the accuracy of the subsequent fishing behavior detection. Because the camera is installed at a high position under the transmission line and the monitoring range is large, the target water area map is segmented according to the monitoring image of the transmission line area, and the monitoring inspection point is constructed based on the target water area map, and the pan-tilt camera device is called for fixed-point cruise, so that the pan-tilt camera device can focus on the water area for monitoring, improve the monitoring efficiency, and at the same time, the corresponding cruise point image is obtained in the form of cruise point, which can effectively avoid the problem that the image pixels taken by the camera in the transmission line scene are not clear and the target cannot be accurately identified and located, which greatly improves the recall rate and accuracy of fishing behavior recognition under the transmission line. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A flowchart of a method for detecting fishing behavior on a power transmission line provided by an embodiment of the present invention;

[0018] Figure 2 A schematic diagram of the division of a cruising area provided in an embodiment of the present invention;

[0019] Figure 3 A flowchart of a fishing behavior detection method provided by an embodiment of the present invention;

[0020] Figure 4 A schematic diagram of the result of open set target detection provided by an embodiment of the present invention;

[0021] Figure 5 A schematic diagram of a water area segmentation result provided by an embodiment of the present invention;

[0022] Figure 6 A schematic diagram of the structure of a fishing behavior detection terminal for power transmission lines provided by an embodiment of the present invention;

[0023] Description of labels:

[0024] 300. A fishing behavior detection terminal for a power transmission line; 301. A memory; 302. A processor. DETAILED DESCRIPTION

[0025] In order to explain the technical content, achieved objectives and effects of the present invention in detail, the following is an explanation in combination with the implementation modes and the accompanying drawings.

[0026] Please refer to Figure 1 , an embodiment of the present invention provides a method for detecting fishing behavior in a power transmission line, comprising:

[0027] Obtain historical imagery of transmission line areas;

[0028] The preset semantic segmentation model and target detection model are trained respectively according to the historical images to obtain a water area segmentation model and a fishing detection model;

[0029] Acquiring a monitoring image of the power transmission line area;

[0030] Segment the monitoring image using the water area segmentation model to obtain a target water area map;

[0031] Constructing a monitoring cruise point according to the target water area map, and mobilizing a preset pan-tilt camera device to perform a fixed-point cruise according to the monitoring cruise point to obtain a cruise point image;

[0032] Performing target detection on the cruise point image according to the fishing detection model to obtain a target detection result, and mapping the target detection result to the monitoring image to obtain a fishing detection image;

[0033] Determine whether there is fishing behavior in the power transmission line area according to the fishing detection image.

[0034] From the above description, it can be seen that the beneficial effect of the present invention is that: the preset semantic segmentation model and target detection model are trained respectively by the historical images of the transmission line area, and the water segmentation model and the fishing detection model are obtained, so as to realize the automatic segmentation of the target water map from the monitoring image of the transmission line area and the detection of the target related to the fishing behavior, so that the corresponding model can effectively extract the specific features in the current scene, thereby improving the accuracy of the model segmentation or detection, thereby enhancing the accuracy of the subsequent fishing behavior detection. Because the camera is installed at a high position under the transmission line and the monitoring range is large, the target water map is segmented according to the monitoring image of the transmission line area, and the monitoring inspection point is constructed based on the target water map, and the pan-tilt camera device is called for fixed-point cruise, so that the pan-tilt camera device can focus on the water area for monitoring, improve the monitoring efficiency, and obtain the corresponding cruise point image in the form of cruise point, which can effectively avoid the problem that the image pixels taken by the camera in the transmission line scene are not clear and the target cannot be accurately identified and located, which greatly improves the recall rate and accuracy of fishing behavior recognition under the transmission line.

[0035] Furthermore, the preset semantic segmentation model and target detection model are trained respectively according to the historical images to obtain the water segmentation model and the fishing detection model, including:

[0036] Annotating the water area of ​​the historical image by using a preset annotation model to obtain a water area dataset;

[0037] Using a preset annotation tool to annotate the personnel and fishing gear in the historical image, to obtain a fishing target data set;

[0038] Training a preset semantic segmentation model according to the water area data set to obtain a water area segmentation model;

[0039] A preset target detection model is trained according to the fishing target data set to obtain a fishing detection model.

[0040] From the above description, we can see that since the water area in the image is large, the water area in the historical image can be automatically annotated through the annotation model, thereby improving the annotation efficiency of the data set. For people and fishing gear with smaller pixel areas in the image, they are manually annotated through the annotation tool to avoid the problem of annotation errors in the annotation model. Different annotation methods are used for water areas, people, and fishing gear, which improves the annotation efficiency while ensuring the accuracy of the annotation, thereby ensuring the correctness of subsequent fishing behavior detection.

[0041] Furthermore, the water area of ​​the historical image is annotated by a preset annotation model to obtain a water area dataset, including:

[0042] Get the preset water area text information;

[0043] Performing open set target detection on the historical image according to the water area text information to obtain the coordinates of the water area annotation box;

[0044] Segmenting the historical image into a water area according to the coordinates of the water area annotation frame to obtain a water area image;

[0045] Data cleaning is performed based on the water area image to obtain a water area segmentation data set.

[0046] From the above description, it can be seen that the so-called open set target detection means that the target categories that can be detected are not limited to the training categories, so the corresponding target box can be output by inputting text information. Since the water areas in historical images can include various types, such as rivers, rivers or lakes, open set target detection is used to achieve unified extraction of different water areas in historical images to avoid omissions. At the same time, the water area of ​​the historical image is segmented based on the coordinates of the water area annotation box to obtain the precise location of the water area in the historical image. After data cleaning, the small and discontinuous water areas are filtered out to obtain the water area segmentation data set to avoid interference with the training of subsequent models and ensure the accuracy of the data set.

[0047] Furthermore, the target water area map is a binary segmentation map;

[0048] The constructing of the monitoring cruise point according to the target water area map comprises:

[0049] Searching for the contour of the water area according to the binary segmentation map to obtain the boundary of the water area;

[0050] Calculate the circumscribed rectangle of the water area boundary to obtain a cruising area;

[0051] Obtaining a ratio value of the binary segmentation map to the monitoring image;

[0052] Dividing the cruising area according to the ratio value to obtain a plurality of cruising sub-areas;

[0053] The center point of each cruise sub-area is determined, and the center point is marked as a monitored cruise point.

[0054] It can be seen from the above description that after segmenting the monitoring image, a target water area map is obtained, so that the pan-tilt camera device can focus on the water area for key monitoring. Since the water area in the image is large, and the pan-tilt camera device installed by the power transmission line is at a high position, the pixels of people and fishing gear in the monitoring image taken by the camera device account for a small proportion, and more effective features cannot be captured, resulting in poor model training effect. In this regard, the present invention constructs monitoring cruise points in the water area after searching for the water area boundary, and divides the original large water area monitoring range into multiple smaller monitoring areas, so that the pan-tilt camera device can perform fixed-point shooting according to the monitoring cruise points, thereby capturing clear images at each cruise point, avoiding the small features of people and fishing gear, improving the effect of model detection, and effectively avoiding the problems of missed detection and wrong detection.

[0055] Furthermore, the cruise area is divided according to the ratio value to obtain a plurality of cruise sub-areas including:

[0056] Determine whether the ratio value is greater than a preset threshold;

[0057] If yes, a preset first value is obtained, and the cruising area is evenly divided into a plurality of cruising sub-areas according to the first value;

[0058] Otherwise, obtaining a preset second value, and evenly dividing the cruising area into a plurality of cruising sub-areas according to the second value;

[0059] The first value is greater than the second value.

[0060] From the above description, it can be seen that when the pixel ratio of the target water area map in the monitoring image is large, it means that the water area monitored by the current monitoring image is large, so it is necessary to evenly divide the cruise area into more cruise sub-areas to ensure that all features in each cruise sub-area can be clearly captured by the PTZ camera device. When the pixel ratio of the target water area map in the monitoring image is small, it means that the water area monitored by the current monitoring image is small, so the cruise area does not need to be divided into too many cruise sub-areas, avoiding waste of resources of the PTZ camera device and improving cruise efficiency.

[0061] Further, the determining the center point of each of the cruise sub-areas includes:

[0062] Detecting whether there are water areas in the cruise sub-areas one by one;

[0063] If it exists, determining the center point of the cruise sub-area;

[0064] If not present, the cruising sub-area is marked as a non-cruising area.

[0065] From the above description, it can be seen that since the shape of the water area is an irregular figure, after the cruise area is divided into multiple cruise sub-areas, it cannot be guaranteed that there is water in each cruise sub-area. Therefore, after checking whether there is water in each cruise sub-area one by one, if there is water, a monitoring cruise point is set, if there is no water, no monitoring cruise point is set, so as to avoid invalid cruising of the pan-tilt camera device, improve cruise efficiency, and save data resources of the device.

[0066] Furthermore, the mobilizing of the preset pan-tilt camera device to perform fixed-point cruising according to the monitored cruising point to obtain the cruising point image includes:

[0067] Mobilizing a preset PTZ camera device to run to the monitoring cruise point;

[0068] Adjusting the shooting parameters of the pan-tilt camera device to obtain a real-time image of the monitored cruise point;

[0069] Determining whether the clarity of the real-time image reaches a preset standard value;

[0070] If yes, marking the real-time image as a cruise point image;

[0071] Otherwise, return to the step of adjusting the shooting parameters of the pan-tilt camera device to obtain the real-time image of the monitored cruise point.

[0072] From the above description, it can be seen that after the pan-tilt camera device is adjusted to run to the monitoring cruise point, the shooting parameters of the pan-tilt camera device are adjusted to obtain clear real-time images of the monitoring cruise point. If the clarity does not reach the preset standard value, the shooting parameters are adjusted to re-shoot, so that the pixel proportion and features of personnel and fishing gear in the image can be more obvious, avoiding the problem of small data features caused by the large shooting range of the pan-tilt camera device, and improving the recognition accuracy of the subsequent fishing detection model.

[0073] Furthermore, the objects of the target detection include people and fishing gear;

[0074] The performing target detection on the cruise point image according to the fishing detection model to obtain a target detection result, and mapping the target detection result to the monitoring image to obtain the fishing detection image comprises:

[0075] Performing target detection on the cruise point image according to the fishing detection model to obtain a person marking frame and a fishing gear marking frame;

[0076] The personnel marking frame and the fishing gear marking frame are mapped to the monitoring image according to the transformation relationship between the cruise point image and the monitoring image to obtain a fishing detection image.

[0077] From the above description, it can be seen that after obtaining the cruise point image, target detection is performed through the cruise point image to ensure that the fishing detection model can accurately extract the characteristics of people and fishing gear in the image, so as to accurately identify people and fishing gear, and then mark the identified people and fishing gear and uniformly map them to the monitoring image to obtain the fishing detection image. That is, the detection results of all cruise points are mapped to a monitoring image for unified judgment to improve the efficiency of data processing, and at the same time, it is conducive to judging the possible relationship between people and fishing gear in different cruise points, and ensuring the correctness of detection.

[0078] Further, determining whether there is fishing behavior in the power transmission line area according to the fishing detection image includes:

[0079] If a person mark frame is detected in the fishing detection image, determining whether the person mark frame is located in a preset fishing area;

[0080] If yes, determining whether the fishing gear marking frame exists in the fishing detection image;

[0081] If it exists, then calculate the intersection and union ratio of the personnel mark frame and the fishing gear mark frame;

[0082] If the intersection-and-combination ratio is greater than a preset standard ratio, fishing behavior exists in the power transmission line area.

[0083] From the above description, it can be seen that when there is a person mark frame in the fishing detection image, further fishing behavior judgment is performed. Since there may be pedestrians near the water area, it is necessary to further judge whether there is a fishing gear mark frame in the current area, and then use the intersection and union ratio between the two mark frames to confirm whether there is a relationship between the person and the fishing gear. If the intersection and union ratio is greater than the preset standard ratio, it means that the fishing gear may belong to the person, so it is determined that there is fishing behavior in the current transmission line area. In this way, the judgment process of fishing behavior is simplified and the detection efficiency is improved.

[0084] Please refer to Figure 6 Another embodiment of the present invention provides a fishing behavior detection terminal for power transmission lines, including a memory, a processor, and a computer program stored in the memory and running on the processor, and when the processor executes the computer program, each step of the above-mentioned fishing behavior detection method for power transmission lines is implemented.

[0085] From the above description, it can be seen that the beneficial effect of the present invention is that: the preset semantic segmentation model and target detection model are trained respectively by the historical images of the transmission line area, and the water segmentation model and the fishing detection model are obtained, so as to realize the automatic segmentation of the target water map from the monitoring image of the transmission line area and the detection of the target related to the fishing behavior, so that the corresponding model can effectively extract the specific features in the current scene, thereby improving the accuracy of the model segmentation or detection, thereby enhancing the accuracy of the subsequent fishing behavior detection. Because the camera is installed at a high position under the transmission line and the monitoring range is large, the target water map is segmented according to the monitoring image of the transmission line area, and the monitoring inspection point is constructed based on the target water map, and the pan-tilt camera device is called for fixed-point cruise, so that the pan-tilt camera device can focus on the water area for monitoring, improve the monitoring efficiency, and obtain the corresponding cruise point image in the form of cruise point, which can effectively avoid the problem that the image pixels taken by the camera in the transmission line scene are not clear and the target cannot be accurately identified and located, which greatly improves the recall rate and accuracy of fishing behavior recognition under the transmission line.

[0086] The present invention provides a method and terminal for detecting fishing behavior in a power transmission line, which can be used in a monitoring device under a power transmission line to effectively detect fishing behavior and improve detection accuracy. The method and terminal are described below through specific embodiments:

[0087] Please refer to Figures 1 to 3 , Embodiment 1 of the present invention is:

[0088] A method for detecting fishing behavior in a power transmission line, comprising:

[0089] S1. Obtain historical images of the transmission line area.

[0090] It should be noted that the historical images are surveillance images of fishing activities in the power transmission line area, that is, the historical images include water areas, personnel and fishing gear.

[0091] In some embodiments, the monitoring device is mounted on a tower of a power transmission line, and the power transmission line area is photographed from above through the monitoring device.

[0092] S2. According to the historical images, the preset semantic segmentation model and target detection model are trained respectively to obtain a water area segmentation model and a fishing detection model.

[0093] Specifically, the step S2 includes:

[0094] S21. Annotate the water area in the historical image using a preset annotation model to obtain a water area dataset.

[0095] S22. Labeling the personnel and fishing gear in the historical image using a preset labeling tool to obtain a fishing target data set.

[0096] S23. Training a preset semantic segmentation model according to the water area data set to obtain a water area segmentation model.

[0097] In some embodiments, the semantic segmentation model is an OCRNet semantic segmentation model. The OCRNet semantic segmentation model is iteratively trained according to the water area data set to obtain a plurality of initial water area segmentation models with different parameters; the mIoU value (mean Intersection of Union) of each initial water area segmentation model is calculated, and the optimal model in the initial water area segmentation model is determined as the final water area segmentation model according to the mIoU value.

[0098] S24. Train a preset target detection model according to the fishing target data set to obtain a fishing detection model.

[0099] In some embodiments, the target detection model is a YOLOv8 target detection model. The YOLOv8 target detection model is iteratively trained according to the fishing target data set to obtain a plurality of initial fishing detection models with different parameters; the mAP value (mean Average Precision) of each initial target detection model is calculated, and the optimal model among the initial fishing detection models is determined as the final fishing detection model according to the mAP value.

[0100] S3. Acquire a monitoring image of the power transmission line area.

[0101] S4. Segment the monitoring image using the water area segmentation model to obtain a target water area map.

[0102] In some embodiments, some noise with smaller area in the target water area map can be filtered out by corrosion expansion to reduce interference factors in the target water area map.

[0103] S5. Construct a monitoring cruise point according to the target water area map, and mobilize a preset pan-tilt camera device to perform fixed-point cruise according to the monitoring cruise point to obtain a cruise point image.

[0104] In an optional implementation, the target water area map is a binary segmentation map.

[0105] In the step S5: constructing a monitoring cruise point according to the target water area map includes:

[0106] S51, searching for water area contours according to the binary segmentation map to obtain water area boundaries.

[0107] S52: Calculate the circumscribed rectangle of the water area boundary to obtain a cruising area.

[0108] S53: Obtain a ratio value between the binary segmentation map and the monitoring image.

[0109] In some embodiments, the ratio of the pixel area of ​​the binary segmentation image to the pixel area of ​​the monitoring image may be calculated, that is, the pixel area ratio of the binary segmentation image in the monitoring image.

[0110] S54: Divide the cruising area according to the ratio value to obtain a plurality of cruising sub-areas.

[0111] S55: Determine the center point of each cruise sub-area, and mark the center point as a monitored cruise point.

[0112] like Figure 2 As shown, Figure 2 The black area in the middle is the water area. After the cruise area is divided, four cruise sub-areas are obtained, namely cruise sub-area (a), cruise sub-area (b), cruise sub-area (c) and cruise sub-area (d); among them, the monitoring cruise point corresponding to cruise sub-area (a) is Point1, the monitoring cruise point corresponding to cruise sub-area (b) is Point2, the monitoring cruise point corresponding to cruise sub-area (c) is Point3, and the monitoring cruise point corresponding to cruise sub-area (d) is Point4.

[0113] In the step S5, mobilizing the preset PTZ camera device to perform fixed-point cruising according to the monitored cruising point, and obtaining the cruising point image includes:

[0114] S56, mobilizing a preset PTZ camera device to run to the monitoring cruise point.

[0115] In some embodiments, the monitoring cruise points are provided with corresponding sequential numbers, and the pan-tilt camera device is mobilized to run to the monitoring cruise points with corresponding sequential numbers in sequence.

[0116] S57, adjusting the shooting parameters of the pan-tilt camera device to obtain a real-time image of the monitored cruise point.

[0117] In some embodiments, adjusting the shooting parameters of the pan-tilt camera device specifically includes performing operations such as zooming in and out on the pan-tilt camera device.

[0118] S58: Determine whether the clarity of the real-time image reaches a preset standard value.

[0119] S581: If yes, mark the real-time image as a cruise point image.

[0120] S582: Otherwise, return to the step of adjusting the shooting parameters of the pan-tilt camera device to obtain a real-time image of the monitored cruise point.

[0121] S6. Perform target detection on the cruise point image according to the fishing detection model to obtain a target detection result, and map the target detection result to the monitoring image to obtain a fishing detection image.

[0122] The objects of target detection include people and fishing gear.

[0123] Specifically, step S6 includes:

[0124] S61. Perform target detection on the cruise point image according to the fishing detection model to obtain a person marking frame and a fishing gear marking frame.

[0125] S62: Map the personnel mark frame and the fishing gear mark frame to the monitoring image according to the transformation relationship between the cruise point image and the monitoring image to obtain a fishing detection image.

[0126] S7. Determine whether there is fishing behavior in the power transmission line area according to the fishing detection image.

[0127] Specifically, Figure 3 As shown, step S7 includes:

[0128] S71. If a person mark frame is detected in the fishing detection image, determine whether the person mark frame is located in a preset fishing area.

[0129] In some embodiments, the preset fishing area can be determined by the water boundary of the target water map.

[0130] In some embodiments, if no person marking box is detected in the fishing detection image, the current fishing detection image is skipped and the next fishing detection image is detected.

[0131] In some embodiments, if the person marking box is not located in a preset fishing area, the current fishing detection image is skipped and the next fishing detection image is detected.

[0132] S72: If yes, determine whether the fishing gear marking frame exists in the fishing detection image.

[0133] In some embodiments, if there is no fishing gear marking box, the current fishing detection image is skipped and the next fishing detection image is detected.

[0134] S73. If it exists, calculate the intersection and union ratio of the personnel mark frame and the fishing gear mark frame.

[0135] S74: If the intersection-to-combination ratio is greater than a preset standard ratio, fishing behavior exists in the power transmission line area.

[0136] In some embodiments, the preset standard ratio is 0.

[0137] When all fishing detection images are detected, the pan-tilt camera device completes the fishing behavior detection work.

[0138] Please refer to Figures 4 to 5 , Embodiment 2 of the present invention is:

[0139] A method for detecting fishing behavior in a power transmission line, which differs from the first embodiment in that the implementation methods of step S21, step S54 and step S55 are limited.

[0140] In an optional implementation, the step S21 includes:

[0141] S211. Obtain preset water area text information.

[0142] In some embodiments, the water area text information includes text information describing the water area, such as rivers, flowing water, and lakes.

[0143] S212: performing open set target detection on the historical image according to the water area text information to obtain the coordinates of the water area annotation box.

[0144] In some embodiments, the open set target detection is performed on the historical image through the Grounding DINO model to obtain the coordinates of the water area annotation box as follows: Figure 4 As shown, the historical image includes a water area marking box A and a water area marking box B.

[0145] S213 , performing water area segmentation on the historical image according to the coordinates of the water area annotation frame to obtain a water area image.

[0146] In some embodiments, the SAM model is used to segment the historical image into water areas according to the coordinates of the water area annotation frame to obtain a water area image such as Figure 5 As shown, the water area image includes water area A and water area B.

[0147] S214: Perform data cleaning according to the water area image to obtain a water area segmentation data set.

[0148] In some embodiments, the purpose of data cleaning is to filter out small and discontinuous water areas in the water area image, such as Figure 5 The water area B marked in is the water area that needs to be filtered. It can be seen from the historical images that the PTZ camera device cannot clearly capture the boundary of water area B, so there is no need to add water area B to the water area segmentation dataset.

[0149] In an optional implementation, the step S54 includes:

[0150] S541: Determine whether the ratio value is greater than a preset threshold.

[0151] S542: If yes, a preset first value is obtained, and the cruising area is evenly divided into a plurality of cruising sub-areas according to the first value.

[0152] S543: Otherwise, obtain a preset second value, and divide the cruise area into a plurality of cruise sub-areas according to the second value. The first value is greater than the second value.

[0153] It should be noted that, dividing the cruising area into a plurality of cruising sub-areas on an average basis according to the first value or the second value means that the number of cruising sub-areas obtained by the average division is the same as the first value or the second value.

[0154] In some embodiments, the preset threshold is 1 / 3, the first value is 9, and the second value is 4. When the ratio is greater than 1 / 3, the cruising area is evenly divided into 9 cruising sub-areas; when the ratio is less than or equal to 1 / 3, the cruising area is evenly divided into 4 cruising sub-areas. In this way, when the water area is too large, the pan-tilt camera device cannot clearly capture the personnel and fishing gear on the entire water surface. After dividing the cruising sub-areas, the pan-tilt camera device can accurately monitor and shoot according to the cruising sub-areas, thereby obtaining clear monitoring images.

[0155] In some embodiments, each cruising area includes rectangular coordinates of a circumscribed rectangle, and each cruising area is evenly divided into a plurality of cruising sub-areas according to the rectangular coordinates.

[0156] In an optional implementation, the step S55 includes:

[0157] S551, detecting whether there are water areas in the cruise sub-areas one by one.

[0158] S552: If it exists, determine the center point of the cruise sub-area.

[0159] S553: ​​If not, the cruise sub-area is marked as a non-cruise area, that is, there is no need to set a monitoring cruise point in an area without water.

[0160] Please refer to Figure 6 , Embodiment 3 of the present invention is:

[0161] A fishing behavior detection terminal 300 for a power transmission line includes a memory 301, a processor 302, and a computer program stored in the memory 301 and running on the processor 302. When the processor 302 executes the computer program, each step of a fishing behavior detection method for a power transmission line in Embodiment 1 and Embodiment 2 is implemented.

[0162] In summary, the present invention provides a method and terminal for detecting fishing behavior in power transmission lines, which obtains historical images of the power transmission line area, annotates the water area, personnel and fishing gear in the historical images, and obtains a water area data set and a fishing target data set; the preset semantic segmentation model and target detection model are trained according to the corresponding data sets, respectively, to obtain a water area segmentation model and a fishing detection model, so as to automatically segment the target water area map from the monitoring image of the power transmission line area and detect the targets related to the fishing behavior, so that the corresponding model can effectively extract the specific features in the current scene, thereby improving the accuracy of the model segmentation or detection, thereby enhancing the accuracy of subsequent fishing behavior detection. Since the camera is installed at a high position under the power transmission line and the monitoring range is large, the target water area map is segmented according to the monitoring image of the power transmission line area, and a monitoring inspection point is constructed based on the target water area map, and the pan-tilt camera device is called for fixed-point cruising, so that the pan-tilt camera device can focus on the water area for monitoring, thereby improving the monitoring efficiency. At the same time, a larger monitored water area is divided into multiple smaller monitored water areas, and the corresponding cruise point images are obtained in the form of cruise points, so that the pan-tilt camera device can fully capture all target features of the water area at the cruise point, which can effectively avoid the problem of unclear pixels of images taken by the camera in the transmission line scene and the inability to accurately identify and locate the target, greatly improving the recall rate and accuracy of fishing behavior recognition under the transmission line.

[0163] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent transformations made using the contents of the present invention's specification and drawings, or directly or indirectly applied in related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for detecting fishing behavior in power transmission lines, characterized in that: include: Obtain historical imagery of transmission line areas; The preset semantic segmentation model and target detection model are trained respectively according to the historical images to obtain a water area segmentation model and a fishing detection model; Acquiring a monitoring image of the power transmission line area; Segment the monitoring image using the water area segmentation model to obtain a target water area map; Constructing a monitoring cruise point according to the target water area map, and mobilizing a preset pan-tilt camera device to perform a fixed-point cruise according to the monitoring cruise point to obtain a cruise point image; Performing target detection on the cruise point image according to the fishing detection model to obtain a target detection result, and mapping the target detection result to the monitoring image to obtain a fishing detection image; determining whether there is fishing behavior in the power transmission line area according to the fishing detection image; The target water area map is a binary segmentation map; The constructing of the monitoring cruise point according to the target water area map comprises: Searching for the contour of the water area according to the binary segmentation map to obtain the boundary of the water area; Calculate the circumscribed rectangle of the water area boundary to obtain a cruising area; Obtaining a ratio value of the binary segmentation map to the monitoring image; Dividing the cruising area according to the ratio value to obtain a plurality of cruising sub-areas; Determine the center point of each cruise sub-area, and mark the center point as a monitored cruise point; Determining the center point of each of the cruise sub-areas comprises: Detecting whether there are water areas in the cruise sub-areas one by one; If it exists, determining the center point of the cruise sub-area; If not, marking the cruise sub-area as a non-cruise area; The mobilizing of the preset pan-tilt camera device to perform fixed-point cruising according to the monitored cruising point to obtain the cruising point image includes: Mobilizing a preset PTZ camera device to run to the monitoring cruise point; Adjusting the shooting parameters of the pan-tilt camera device to obtain a real-time image of the monitored cruise point; Determining whether the clarity of the real-time image reaches a preset standard value; If yes, marking the real-time image as a cruise point image; Otherwise, return to the step of adjusting the shooting parameters of the pan-tilt camera device to obtain a real-time image of the monitored cruise point; The objects of target detection include people and fishing gear; The performing target detection on the cruise point image according to the fishing detection model to obtain a target detection result, and mapping the target detection result to the monitoring image to obtain the fishing detection image comprises: Performing target detection on the cruise point image according to the fishing detection model to obtain a person marking frame and a fishing gear marking frame; The personnel marking frame and the fishing gear marking frame are mapped to the monitoring image according to the transformation relationship between the cruise point image and the monitoring image to obtain a fishing detection image.

2. A method for detecting fishing behavior in power transmission lines according to claim 1, characterized in that: The method of training the preset semantic segmentation model and target detection model respectively according to the historical images to obtain the water segmentation model and the fishing detection model includes: Annotating the water area of ​​the historical image by using a preset annotation model to obtain a water area dataset; Using a preset annotation tool to annotate the personnel and fishing gear in the historical image, to obtain a fishing target data set; Training a preset semantic segmentation model according to the water area data set to obtain a water area segmentation model; A preset target detection model is trained according to the fishing target data set to obtain a fishing detection model.

3. A method for detecting fishing behavior in power transmission lines according to claim 2, characterized in that: The water area of ​​the historical image is annotated by a preset annotation model to obtain a water area dataset, including: Get the preset water area text information; Performing open set target detection on the historical image according to the water area text information to obtain the coordinates of the water area annotation box; Segmenting the historical image into a water area according to the coordinates of the water area annotation frame to obtain a water area image; Data cleaning is performed based on the water area image to obtain a water area segmentation data set.

4. A method for detecting fishing behavior in power transmission lines according to claim 1, characterized in that: The cruise area is divided according to the ratio value to obtain a plurality of cruise sub-areas including: Determine whether the ratio value is greater than a preset threshold; If yes, a preset first value is obtained, and the cruising area is evenly divided into a plurality of cruising sub-areas according to the first value; Otherwise, obtaining a preset second value, and evenly dividing the cruising area into a plurality of cruising sub-areas according to the second value; The first value is greater than the second value.

5. A method for detecting fishing behavior in power transmission lines according to claim 1, characterized in that: Determining whether fishing behavior exists in the power transmission line area according to the fishing detection image includes: If a person mark frame is detected in the fishing detection image, determining whether the person mark frame is located in a preset fishing area; If yes, determining whether the fishing gear marking frame exists in the fishing detection image; If it exists, then calculate the intersection and union ratio of the personnel mark frame and the fishing gear mark frame; If the intersection-and-combination ratio is greater than a preset standard ratio, fishing behavior exists in the power transmission line area.

6. A fishing behavior detection terminal for power transmission lines, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the processor implements the various steps in the method for detecting fishing behavior for power transmission lines as described in any one of claims 1-5.

Images