Fishing behavior detection method and device and nonvolatile storage medium

Abstract

The invention discloses a fishing behavior detection method and device and a nonvolatile storage medium. The method comprises the steps that image data of a target area is acquired, and the target area is a preset range area containing a power transmission line; performing portrait detection on the image data by using a first detection model, and determining first target image data when detecting that a portrait exists in the image data, the first detection model being used for detecting whether the portrait exists in the image data, and the first target image data being image data of which the portrait is identified through a detection frame in the image data; a second detection model is used for detecting the fishing behavior of the first target image data, the fishing behavior detection result is determined, and the second detection model is used for detecting whether the portrait in the first target image data has the fishing behavior or not. According to the method and the device, the technical problems of many detection risks, low precision and low efficiency in a fishing behavior detection method in the related technology are solved.

Description

Technical Field

[0001] The present application relates to the field of image processing, and more specifically, to a method, device and non-volatile storage medium for detecting fishing behavior. Background Art

[0002] Maintaining the safety of transmission lines is crucial to the stability of the power system. However, people fishing near transmission lines, especially fishing under transmission lines, pose a serious threat to the safe operation of transmission lines. Fishing rods and fishing lines may come into contact with digital display lines, causing power accidents, casualties and equipment damage. In order to ensure the safety of transmission lines, effective detection and early warning of fishing activities around transmission lines are required.

[0003] With the rapid progress in the field of image processing technology and the widespread application of camera equipment in recent years, image technology has also been applied to the monitoring field of various industries. The application of image processing methods to assist in daily disaster prevention, on-site exploration and maintenance on transmission lines can greatly reduce labor costs and accelerate the location and elimination efficiency of fault problems. There are roughly the following technical solutions for detecting fishing behavior near transmission lines in related technologies: manual methods, distributed sensors, and fiber optic sensing equipment. For manual methods, whether it is manual inspection or video monitoring, there are problems such as low inspection efficiency, long inspection cycle, and hidden dangers of human fatigue, and the labor cost demand is too high. Distributed sensors need to lay sensors over a large area and rely on batteries for power supply. Subsequent sensor maintenance and battery replacement will increase costs. Moreover, most transmission lines are in the wild, and data transmission relies on wireless signals. There are delays in the transmission and processing of information, poor real-time performance, and low recognition efficiency. The data collected and fed back by fiber optic sensing equipment are mostly in the form of vibration sound wave signals, which cannot well visualize the on-site conditions. Harsh environmental factors, such as strong winds and heavy rains, may interfere with the vibration signal, and the activity trajectories of wild animals may also cause optical fiber sensing equipment to misjudge abnormal behavior detection. The above fishing behavior detection methods all have the disadvantages of high detection risks, low accuracy, and low efficiency.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present application provide a method, device and non-volatile storage medium for detecting fishing behavior, so as to at least solve the technical problems in the related art that the methods for detecting fishing behavior have many detection risks, low accuracy and low efficiency.

[0006] According to one aspect of an embodiment of the present application, a method for detecting fishing behavior is provided, comprising: acquiring image data of a target area, wherein the target area is a preset range area including power transmission lines; performing portrait detection on the image data using a first detection model, and determining first target image data when a portrait is detected in the image data, wherein the first detection model is used to detect whether there is a portrait in the image data, and the first target image data is image data in which a portrait is identified by a detection box in the image data; performing fishing behavior detection on the first target image data using a second detection model, and determining a fishing behavior detection result, wherein the second detection model is used to detect whether there is fishing behavior in the portrait in the first target image data.

[0007] In some embodiments of the present application, determining the first target image data includes: using a first detection model to determine a first confidence level and a detection category corresponding to the image data, wherein the first confidence level is used to represent the detection credibility of the first detection model, and the detection category is used to represent the category to which the object in the image data belongs; based on the first confidence level and the detection category corresponding to the image data, determining the first target image data.

[0008] In some embodiments of the present application, first target image data is determined based on a first confidence level and a detection category corresponding to the image data, including: screening out first image data whose detection category is a portrait and whose first confidence level is greater than a preset threshold; using a detection frame to identify the position of the portrait in the first image data, and determining the first image data containing the detection frame as the first target image data.

[0009] In some embodiments of the present application, the image data and the first target image data have the same size. Before using the second detection model to detect fishing behavior on the image data, the method also includes: obtaining a first size corresponding to a detection box in the first target image data; enlarging the first size according to a preset ratio to obtain a second size; and cropping the first target image data according to the second size.

[0010] In some embodiments of the present application, the second detection model is trained in the following manner: acquiring target historical image data of the target area; determining historical label data corresponding to the target historical image data; inputting the target historical image data and historical label data into the original second detection model for training, and stopping the training when preset conditions are met to obtain the second detection model.

[0011] In some embodiments of the present application, obtaining target historical image data of a target area includes: obtaining historical image data of the target area; using a first detection model to identify portraits in the historical image data, and determining image data in the historical image data in which the portraits are identified by a detection box as first historical image data; and cropping the first historical image data to obtain target historical image data.

[0012] In some embodiments of the present application, the first detection model is deployed on the front-end device. When the first detection model detects the presence of a portrait in the image data, the first target image data is uploaded to the back-end server, and the second detection model is deployed in the back-end server.

[0013] According to another aspect of an embodiment of the present application, a device for detecting fishing behavior is also provided, including: an acquisition module, used to acquire image data of a target area, wherein the target area is a preset range area including power transmission lines; a first detection module, used to perform portrait detection on the image data using a first detection model, and when a portrait is detected in the image data, determine first target image data, wherein the first detection model is used to detect whether there is a portrait in the image data, and the first target image data is image data in which a portrait is identified by a detection box in the image data; a second detection module, used to perform fishing behavior detection on the first target image data using a second detection model, and determine a fishing behavior detection result, wherein the second detection model is used to detect whether there is fishing behavior in the portrait in the first target image data.

[0014] According to another aspect of an embodiment of the present application, a non-volatile storage medium is further provided, in which a program is stored, wherein when the program is running, the device where the non-volatile storage medium is located is controlled to execute the above-mentioned method for detecting fishing behavior.

[0015] According to another aspect of an embodiment of the present application, a computer program product is also provided, including computer instructions, which implement the above-mentioned method for detecting fishing behavior when executed by a processor.

[0016] In an embodiment of the present application, image data of a target area is acquired, wherein the target area is a preset range area including power transmission lines; a first detection model is used to perform portrait detection on the image data, and when a portrait is detected in the image data, first target image data is determined, wherein the first detection model is used to detect whether there is a portrait in the image data, and the first target image data is image data in which a portrait is identified by a detection frame in the image data; a second detection model is used to perform fishing behavior detection on the first target image data to determine a fishing behavior detection result, wherein the second detection model is used to detect whether the portrait in the first target image data involves fishing behavior. By first identifying the portrait in the image data and then identifying the fishing behavior in the image data with the portrait, the purpose of improving the accuracy of detection accuracy and detection efficiency is achieved, and unnecessary detection risks can be avoided through algorithm recognition, thereby solving the technical problems of many detection risks, low accuracy and low efficiency in the fishing behavior detection methods in the related arts. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0018] Figure 1 It is a hardware structure block diagram of a computer terminal for implementing a method for detecting fishing behavior according to an embodiment of the present application;

[0019] Figure 2 is a flow chart of a method for detecting fishing behavior according to an embodiment of the present application;

[0020] Figure 3 It is a structural schematic diagram of a fishing behavior detection device according to an embodiment of the present application. DETAILED DESCRIPTION

[0021] In order to make people in the technical field better understand the solution of this application, the technical solution in the embodiment of this application will be clearly and completely described below in conjunction with the drawings in the embodiment of this application. Obviously, the described embodiment is only a part of the embodiment of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technical people in this field without making creative work should fall within the scope of protection of this application.

[0022] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0023] In order to better understand the embodiments of the present application, the technical terms involved in the embodiments of the present application are explained as follows:

[0024] Single Shot MultiBox Detector (SSD): It is a deep learning model for object detection. It can predict the location and category of multiple objects at the same time during a single forward propagation of the image, so it is called "single detection". The MultiBox Detector part refers to the model's ability to predict multiple bounding boxes of different sizes and proportions to detect objects of different scales. The SSD model has been widely used in real-time object detection tasks due to its fast detection speed and high accuracy.

[0025] MobileNet: is a lightweight deep learning model mainly used for image classification and object detection. MobileNet uses Depthwise Separable Convolution, which can decompose the standard convolution operation into two smaller independent operations, thereby significantly reducing the amount of calculation and model parameters, making the model lightweight and efficient.

[0026] You Only Look Once version 5small model (YOLOv5s): It is a lightweight model in the fifth generation version of the You Only Look Once (YOLO) series of target detection models. YOLOv5s is designed for real-time target detection. While maintaining high accuracy, it has low requirements for computing resources and is suitable for deployment on edge devices and mobile devices. YOLOv5s is a version of the YOLO target detection model. It is designed for real-time object detection and is particularly suitable for edge devices with limited resources. Compared with other versions of YOLOv5 (such as YOLOv5m, YOLOv5l, YOLOv5x), it has lower computational complexity and faster detection speed while maintaining high detection accuracy, which makes it very suitable for deployment on embedded systems or mobile devices. It uses a single convolutional neural network to predict bounding boxes and class probabilities, but its network architecture is optimized to reduce the number of parameters and increase detection speed.

[0027] The technical solutions for detecting fishing behavior near power transmission lines in the related art are roughly as follows: human methods, distributed sensors, and optical fiber sensing equipment. For the human methods, whether it is manual inspection or video monitoring, there are problems such as low inspection efficiency, long inspection cycle, and hidden dangers of human fatigue, and the labor cost demand is too high. Distributed sensors need to be laid over a large area, and rely on batteries for power supply. Subsequent sensor maintenance and battery replacement will increase costs. Moreover, most power transmission lines are in the wild, and data transmission relies on wireless signals. There are delays in the transmission and processing of information, poor real-time performance, and low recognition efficiency. The data collected and fed back by optical fiber sensing equipment are mostly in the form of vibration sound wave signals, which cannot well visualize the on-site conditions. Encountering harsh environmental factors, such as strong winds and heavy rains, may interfere with the vibration signal, and the activity tracks of wild animals may also cause optical fiber sensing equipment to misjudge abnormal behavior detection. The above-mentioned fishing behavior detection methods all have the disadvantages of many detection risks, low accuracy, and low efficiency. In order to solve this problem, a relevant solution is provided in the embodiment of the present application, which is described in detail below.

[0028] According to an embodiment of the present application, an embodiment of a method for detecting fishing behavior is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0029] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 FIG. 1 shows a hardware structure block diagram of a computer terminal for implementing a method for detecting fishing behavior. Figure 1 As shown, the computer terminal 10 may include one or more (102a, 102b, ..., 102n are used to illustrate) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission module 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. A person of ordinary skill in the art can understand that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0030] It should be noted that the one or more processors 102 and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 10. As described in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0031] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the method for detecting fishing behavior in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, the above-mentioned method for detecting fishing behavior is realized. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0032] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0033] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 .

[0034] In the above operating environment, the present application embodiment provides a flow chart of a method for detecting fishing behavior, such as Figure 2 As shown, the method comprises the following steps:

[0035] Step S202, acquiring image data of a target area, wherein the target area is a preset range area including a transmission line, for example, the target area may be on or around a transmission line tower or within 100 meters of a transmission line tower.

[0036] The following is a specific embodiment:

[0037] A high-definition camera is installed on a transmission line tower or surrounding poles to continuously acquire image data (e.g., video data) around the transmission line, and pre-process the image data (e.g., denoising, contrast enhancement, etc.) to better identify fishing behavior (e.g., thin objects such as fishing rods near human figures).

[0038] For example, denoising and contrast enhancement can be performed by:

[0039] Denoising: Non-Local Means Denoising (NLMD): Considering that fishing rods are often in natural environments, such as water surfaces, grass and other backgrounds, these backgrounds contain a lot of textures and details. The non-local mean denoising method removes noise by finding non-local areas in the image that are similar to the target area, and uses the statistical characteristics of these similar areas to remove noise while maintaining fine texture and edge information, which is conducive to the clear presentation of the fishing rod lines. The specific implementation method is: For each pixel in the image, the NLMD method calculates its similarity with other pixels in the entire image. The similarity is usually determined based on the difference in grayscale or color values ​​of the two pixel neighborhoods. The size of the pixel neighborhood needs to be adjusted according to the actual situation. For details such as fishing rods, a smaller neighborhood needs to be set to maintain details; according to the similarity, a weight is assigned to the neighborhood of each pixel in the image. The weight reflects the degree of similarity between neighborhoods. The higher the similarity, the greater the weight. This weight calculation ensures that more information about similar areas is referenced when removing noise; for each pixel, the weighted average of its neighborhood and the neighborhood of other pixels is used to obtain the denoised pixel value. Finally, in order to verify the effectiveness of the NLMD denoising method, the clarity and detail preservation of the fishing rod lines before and after denoising can be evaluated by comparing the images before and after processing. The denoising effect can also be quantified by objective indicators such as peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM).

[0040] Or through the bilateral filter: The bilateral filter can keep the edges clear while smoothing the image, which is suitable for processing the situation where the image contains both noise and details that need to be preserved. By properly adjusting the spatial Gaussian weight and the grayscale Gaussian weight, the noise on the fishing rod can be effectively reduced without blurring its edges.

[0041] Enhance contrast: Histogram Equalization: In natural light, the fishing rod in the fishing scene may appear dim due to lighting problems. Histogram equalization technology enhances the image contrast by redistributing the intensity (grayscale or color) values ​​of the image, making the fishing rod more prominent. However, simple histogram equalization may cause the image to be overexposed, so using Adaptive Histogram Equalization (AHE) or Contrast Limited Adaptive Histogram Equalization (CLAHE) can better preserve image details.

[0042] The images captured by the front-end high-definition camera may blur the details of targets such as fishing rods due to changes in lighting (such as shadows, direct sunlight, cloudy days, etc.), affecting the accuracy of subsequent behavior analysis. To overcome this problem, the retina model algorithm (Retinex Algorithm, referred to as Retinex) can be applied in the image preprocessing stage: The Retinex algorithm is based on the characteristics of the human visual system and can effectively separate the illumination and reflection components from the image, thereby maintaining the natural contrast of the image under various lighting conditions. Multi-scale Retinex (MSR) or single-scale Retinex (SSR) variants are used. MSR is to apply the Retinex algorithm at different scales and then fuse the results to eliminate the effects of lighting changes in the image. This method ensures that the image maintains good contrast and clarity under various lighting conditions, especially when dealing with slender objects such as fishing rods taken at long distances. SSR applies the Retinex algorithm at a single scale. Compared with MSR, it is faster to calculate, but it may not be as effective as MSR under some extreme lighting conditions.

[0043] Denoising can reduce random noise in images, such as Gaussian noise, salt and pepper noise, etc. These noises will interfere with feature extraction and affect recognition accuracy, remove non-target information in images, and reduce the false alarm rate of subsequent detection algorithms. Enhancing contrast can make different areas in an image clearer, help highlight the target object, make features more obvious, and make the difference between the target and the background more obvious, which helps the algorithm to identify the target more accurately and reduce missed detections due to insufficient contrast, thereby improving overall processing efficiency.

[0044] Step S204: Use the first detection model to perform portrait detection on the image data, and determine first target image data when a portrait is detected in the image data.

[0045] In the technical solution provided in step S204, the first detection model is used to detect whether there is a portrait in the image data, and the first target image data is the image data in which the portrait is identified by a detection box in the image data. There are many ways to determine the first target image data. For example, the first target image data can be determined in the following ways: using the first detection model to determine the first confidence and detection category corresponding to the image data, wherein the first confidence is used to represent the detection credibility of the first detection model, and the detection category is used to represent the category to which the object in the image data belongs; based on the first confidence and detection category corresponding to the image data, the first target image data is determined. Through the first confidence and detection category screening, the accuracy of portrait detection can be further improved, false alarms can be reduced, and it is suitable for complex environment monitoring around power facilities.

[0046] In the above steps, based on the first confidence and detection category corresponding to the image data, determining the first target image data can be achieved in the following manner: screening out the first image data whose detection category is a portrait and whose first confidence is greater than a preset threshold; using a detection frame to identify the position of the portrait in the first image data, and determining the first image data containing the detection frame as the first target image data. This precise portrait positioning and screening technology that uses a detection frame to identify the position of the portrait in the first image data and determines the first image data containing the detection frame as the first target image data can ensure that only image data that truly contains a portrait is further processed, and is suitable for scenarios where high-precision monitoring of power facilities is performed.

[0047] The following is a specific embodiment:

[0048] Use the first detection model (e.g., MobileNet SSD object detection algorithm), where MobileNet SSD can provide fast and real-time target detection capabilities while ensuring a certain degree of accuracy. The algorithm has the advantage of being lightweight and suitable for portrait detection on resource-constrained devices. It combines MobileNet as a feature extractor and SSD as a detection head to quickly detect portrait targets in images.

[0049] The following are the steps for training the first detection model: read historical image data (e.g., video stream) from the acquisition device (e.g., camera) in real time, and collect each frame of the image to establish a fishing recognition image database for subsequent model training, testing and verification, and use the deep learning framework (e.g., TensorFlow, PyTorch, OpenCV framework) to use the initial first detection model as a pre-trained model, adjust the image size and perform binarization and normalization processing. For example, binarization and normalization can be performed in the following ways: Binarization converts the image into an image with only black and white pixel values, which is usually used to simplify image features and make it easier to analyze. In fishing behavior detection, binarization can be used to highlight the contrast in the image and help identify the contours of slender objects such as fishing rods and fishing lines: First, convert the color image into a grayscale image, because the grayscale image is easier to binarize, and set a grayscale threshold. Usually, this threshold needs to be adjusted according to the specific content and background of the image. The choice of threshold is the key. Too high a threshold may cause the loss of small features, while too low a threshold may introduce too much noise. The grayscale values ​​of all pixels in the image are compared with the threshold. Pixel values ​​above the threshold are set to white (usually 255), and pixel values ​​below the threshold are set to black (usually 0). Normalization is to adjust the pixel values ​​of the image to a specific range, usually between 0 and 1, so as to ensure that the data between different images are numerically comparable, which is helpful for model training and prediction. In fishing behavior detection, normalization can reduce the impact of external factors such as lighting changes on image analysis. Normalization usually includes the following steps: determine the range to be normalized. For image processing, it is usually to adjust the pixel value from the range of 0 to 255 to the range of 0 to 1. Find the maximum and minimum pixel values ​​in the image, or use the universal maximum value of 255 and the minimum value of 0 for calculation, and finally perform pixel normalization on each pixel value in the image based on the maximum and minimum pixel values ​​in the image. Normalization can be performed first to ensure that the numerical range of all image data is consistent, and then binarization can be performed to more clearly highlight the key features in the image. These two processing methods can be used separately or in combination.

[0050] The historical image data after image resizing, binarization and normalization is input into the pre-trained model for training, testing and reasoning verification to obtain the detection box, first confidence and detection category: According to the reasoning result, the detected "portrait" target is screened out (for example, the detection category ID is set to 1, indicating "portrait"), and an initial value of the first confidence threshold is set, such as 0.5 (i.e., the above preset threshold) to filter out low-confidence detections, that is, only when it is greater than this threshold can it be determined that there is a portrait in the picture. This threshold can be manually adjusted according to the performance of the model training, so as to obtain a better threshold. If a portrait is detected, the detection box is drawn on the original image (i.e., the above detection box is used to identify the position of the portrait in the first image data), and the detection results are visualized. In addition, in order to efficiently run MobileNet SSD on resource-constrained devices, the model can be optimized and converted to a lightweight format using TensorFlow Lite and deployed to mobile devices or embedded systems.

[0051] After obtaining the first detection model, when it is necessary to identify the image data, directly use the first monitoring model to identify the image data, and filter out the first image data whose detection category is portrait and whose first confidence is greater than a preset threshold (for example, 0.5); use the detection frame to identify the portrait position in the first image data, and determine the first image data containing the detection frame as the first target image data.

[0052] While completing the above work, multiple frames of first target image data containing portrait information are uploaded to the back-end server in the form of video, which facilitates the back-end to conduct specific analysis of specific portrait behaviors and improve the recognition rate of abnormal behaviors. The image data before this is stored on the front-end acquisition device, and the first detection model is deployed on the front-end acquisition device to reduce the pressure on the back-end server facing massive image data. The result of portrait detection is used as the condition for triggering the upload of images to the server, that is, only the first target image data is uploaded to the back-end server. When the first detection model detects the presence of a portrait in the image data, the first target image data is uploaded to the back-end server, and the second detection model is deployed in the back-end server. This front-end and back-end separation architecture can make full use of the computing resources of the front-end device for preliminary screening, reduce the burden on the back-end server, and improve the response speed and processing capacity of the overall system. It is suitable for real-time monitoring systems of large-scale power facilities.

[0053] Step S206: Use the second detection model to perform fishing behavior detection on the first target image data to determine a fishing behavior detection result.

[0054] In the technical solution provided in step S206, the second detection model (for example, the YOLOv5s deep learning model) is used to detect whether the human figure in the first target image data has fishing behavior. The image data and the first target image data have the same size. Before using the second detection model to detect fishing behavior on the image data, in order to obtain a more accurate and clear detection image, the first target image data needs to be cropped, for example: obtaining the first size corresponding to the detection frame in the first target image data; enlarging the first size according to a preset ratio (for example, 2-3 times the detection frame) to obtain a second size; and cropping the first target image data according to the second size. By adjusting the image size, the input of the second detection model can be optimized, and the accuracy of fishing behavior detection can be improved, which is particularly suitable for scenarios of long-distance monitoring of activities near power transmission lines.

[0055] In the technical solution provided in step S206, the second detection model is trained in the following manner: obtaining the target historical image data of the target area; determining the historical label data corresponding to the target historical image data; inputting the target historical image data and the historical label data into the original second detection model for training, and stopping the training when the preset conditions are met to obtain the second detection model. There are many ways to obtain the target historical image data of the target area, for example: obtaining the historical image data of the target area; using the first detection model to identify the human figure in the historical image data, and determining the image data of the human figure identified by the detection frame in the historical image data as the first historical image data; cropping the first historical image data to obtain the target historical image data. This model training method based on historical data can make the second detection model more adaptable to the environmental characteristics of a specific target area, improve the accuracy and robustness of fishing behavior detection, and is suitable for long-term safety monitoring of power facilities. By cropping the image data containing human figures, the interference of irrelevant information can be reduced, and the efficiency and accuracy of model training can be improved. It is suitable for environmental change monitoring and behavior analysis around power facilities.

[0056] The following is a specific embodiment of training the second detection model:

[0057] Obtain the target historical image data of the target area (the cropped historical image data), determine the historical label data of these target historical image data manually or using annotation software, the historical label data is specifically the location of the marked fishing-related items (e.g., fishing rods), divide the target historical image data appropriately and input it into the original second detection model for training and testing, and stop training when the preset conditions are met (e.g., the number of training iterations reaches the training threshold or the accuracy of the model's recognition result reaches the recognition threshold, e.g., 90%) to obtain the second detection model. In addition, the trained YOLOv5s model can also be quantized or converted into formats such as TensorRT and TensorFlow Lite, and its lightweightness can meet the needs of edge device deployment.

[0058] When the second detection model is used to detect fishing behavior on the first target image data, the cropped first target image data is input into the second detection model. The second detection model detects a fishing rod in the cropped image, and it can be considered that fishing behavior may have occurred. A more accurate judgment can be made based on the confidence of the detection, the number of fishing rods, and other information. A second preset threshold (for example, 0.5) is set. If the confidence of the detection is higher than the second preset threshold, it can be determined that there is a fishing rod in the area, and then the fishing behavior is determined.

[0059] The reason why it is a two-step detection of portrait detection and fishing behavior detection is that the convolutional neural network inside the second detection model (YOLOv5s model) will perform convolution operations on the image before making fishing behavior inferences, and extract multi-level features such as edges, textures, and shapes from it. In this process, the image will be proportionally reduced. If there is no first step of portrait detection and cropping, most of the content in the image will be directly reduced. For example: the general monitoring screen has a resolution of 1080X1920, and the input of the YOLOv5s model requires a resolution of 640X640. Therefore, the convolutional neural network inside YOLOv5s will reduce the image to a fixed size of 640x640. This reduction step is to meet the model input requirements, but it can also reduce the computational complexity. However, in the process of image reduction, thin small targets such as fishing rods and fishing lines will be reduced and pixel information will be lost. Especially in high-resolution images, the number of pixels of small targets is very small. After being reduced to 640x640, it is impossible to retain enough features, making it difficult for the model to detect accurately. The two-layer detection method (portrait detection and fishing behavior detection) in the embodiment of the present application performs a small reduction after the first effective detection and cropping, trying to retain the original image information and improve the recognition rate. At the same time, the two-layer detection can analyze the information of human posture in a more detailed manner, which is convenient for the expansion of the analysis function of the monitoring system.

[0060] Through the above steps S202-S206, the algorithm can be split, and the personnel detection algorithm can be transplanted to the front-end device to reduce the pressure of the back-end service. The image data is uploaded only when the personnel are detected. Through this two-step detection method, that is, the two-step detection method of using portrait detection and fishing behavior detection, the loss of small target information due to image reduction is reduced, and the detection accuracy is improved. By combining portrait detection and fishing behavior recognition, the accuracy and efficiency of detection are effectively improved. Preliminary portrait detection on the front-end device can quickly filter irrelevant images and reduce the processing pressure of the back-end server, while the second detection model focuses on the recognition of fishing behavior, ensuring accurate monitoring of fishing behavior near the transmission line. This method can not only detect potential threats to power facilities in a timely manner, but also effectively reduce the false alarm rate, providing a strong guarantee for the safe operation of power facilities. In addition, this method can also be applied to other scenarios that require monitoring of human behavior, such as security monitoring in public areas, and has broad application prospects and value. Through real-time monitoring and early warning, power accidents caused by improper behaviors such as fishing can be effectively prevented and reduced, protecting power facilities while also ensuring the safety of the public.

[0061] The present application also provides a device for detecting fishing behavior, a schematic diagram of the structure of the device is shown in FIG. Figure 3 As shown, it includes an acquisition module 302 , a first detection module 304 , and a second detection module 306 .

[0062] The acquisition module 302 is used to acquire image data of a target area, wherein the target area is a preset range area including the power transmission line.

[0063] The first detection module 304 is used to perform portrait detection on the image data using the first detection model, and determine the first target image data when the presence of the portrait is detected in the image data, wherein the first detection model is used to detect whether there is a portrait in the image data, and the first target image data is the image data in which the portrait is identified by a detection box in the image data.

[0064] The first detection module 304 is also used to use the first detection model to determine the first confidence and detection category corresponding to the image data, wherein the first confidence is used to represent the detection credibility of the first detection model, and the detection category is used to represent the category to which the object in the image data belongs; based on the first confidence and detection category corresponding to the image data, the first target image data is determined.

[0065] The first detection module 304 further selects first image data whose detection category is portrait and whose first confidence is greater than a preset threshold; uses a detection frame to identify the position of the portrait in the first image data, and determines the first image data containing the detection frame as the first target image data.

[0066] The second detection module 306 is used to perform fishing behavior detection on the first target image data using a second detection model to determine a fishing behavior detection result, wherein the second detection model is used to detect whether a human figure in the first target image data has fishing behavior.

[0067] The second detection module 306 is further configured to obtain a first size corresponding to the detection frame in the first target image data; enlarge the first size according to a preset ratio to obtain a second size; and crop the first target image data according to the second size.

[0068] The second detection module 306 is also used to obtain target historical image data of the target area; determine the historical label data corresponding to the target historical image data; input the target historical image data and the historical label data into the original second detection model for training, and stop training when the preset conditions are met to obtain the second detection model.

[0069] The second detection module 306 is also used to obtain historical image data of the target area; use the first detection model to identify portraits in the historical image data, and determine the image data in the historical image data where the portraits are identified by the detection box as the first historical image data; and crop the first historical image data to obtain the target historical image data.

[0070] It should be noted that the various modules in the above-mentioned fishing behavior detection device can be program modules (for example, a set of program instructions that implement a certain specific function) or hardware modules. For the latter, it can be expressed in the following forms, but is not limited to this: the expression form of each of the above-mentioned modules is a processor, or the functions of each of the above-mentioned modules are implemented by a processor.

[0071] The embodiment of the present application also provides a non-volatile storage medium, the non-volatile storage medium includes a stored program, wherein when the program is running, the device where the non-volatile storage medium is located is controlled to execute the above method for detecting fishing behavior. For example, image data of a target area is obtained, wherein the target area is a preset range area including power transmission lines; a first detection model is used to perform portrait detection on the image data, and when a portrait is detected in the image data, first target image data is determined, wherein the first detection model is used to detect whether a portrait exists in the image data, and the first target image data is image data in which a portrait is identified by a detection frame in the image data; a second detection model is used to perform fishing behavior detection on the first target image data, and a fishing behavior detection result is determined, wherein the second detection model is used to detect whether a portrait in the first target image data is engaged in fishing behavior.

[0072] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory is used to store a program, the processor is connected to the memory, and is used to run the program, wherein the above method for detecting fishing behavior is executed when the program is running. For example, image data of a target area is obtained, wherein the target area is a preset range area including power transmission lines; a first detection model is used to perform portrait detection on the image data, and when a portrait is detected in the image data, first target image data is determined, wherein the first detection model is used to detect whether a portrait exists in the image data, and the first target image data is image data in which a portrait is identified by a detection frame in the image data; a second detection model is used to perform fishing behavior detection on the first target image data, and a fishing behavior detection result is determined, wherein the second detection model is used to detect whether a portrait in the first target image data is engaged in fishing behavior.

[0073] According to another aspect of the embodiment of the present application, a computer program product is also provided, including a computer program, which implements the above method for detecting fishing behavior when executed by a processor. For example, image data of a target area is obtained, wherein the target area is a preset range area including power transmission lines; a first detection model is used to perform portrait detection on the image data, and when a portrait is detected in the image data, first target image data is determined, wherein the first detection model is used to detect whether there is a portrait in the image data, and the first target image data is image data in which a portrait is identified by a detection frame in the image data; a second detection model is used to perform fishing behavior detection on the first target image data, and a fishing behavior detection result is determined, wherein the second detection model is used to detect whether the portrait in the first target image data is engaged in fishing behavior.

[0074] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0075] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0076] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0077] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0078] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the relevant technology or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk, etc. Various media that can store program codes.

[0079] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technical personnel in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for detecting fishing behavior, characterized in that: include: Acquire image data of a target area, wherein the target area is a preset range area including a power transmission line; Performing portrait detection on the image data using a first detection model, and determining first target image data when a portrait is detected in the image data, wherein the first detection model is used to detect whether a portrait is in the image data, and the first target image data is image data in which a portrait is identified by a detection frame in the image data; A second detection model is used to perform fishing behavior detection on the first target image data to determine a fishing behavior detection result, wherein the second detection model is used to detect whether a human figure in the first target image data is engaging in fishing behavior.

2. The method according to claim 1, characterized in that Determining first target image data, including: Determining a first confidence and a detection category corresponding to the image data using the first detection model, wherein the first confidence is used to represent the detection credibility of the first detection model, and the detection category is used to represent the category to which the object in the image data belongs; The first target image data is determined based on a first confidence level and a detection category corresponding to the image data.

3. The method according to claim 2, characterized in that Determining the first target image data based on a first confidence level and a detection category corresponding to the image data includes: Filter out the first image data whose detection category is a portrait and whose first confidence level is greater than a preset threshold; A detection frame is used to identify a position of the portrait in the first image data, and the first image data including the detection frame is determined as the first target image data.

4. The method according to claim 1, characterized in that: The image data and the first target image data have the same size, and before using the second detection model to perform fishing behavior detection on the image data, the method further includes: Obtaining a first size corresponding to a detection frame in the first target image data; Enlarging the first size according to a preset ratio to obtain a second size; The first target image data is cropped according to the second size.

5. The method according to claim 1, characterized in that The second detection model is trained by the following method: Acquiring target historical image data of the target area; Determining historical label data corresponding to the target historical image data; The target historical image data and the historical label data are input into the original second detection model for training, and the training is stopped when a preset condition is met to obtain the second detection model.

6. The method according to claim 5, characterized in that Acquiring target historical image data of the target area includes: Acquiring historical image data of the target area; Using the first detection model to identify a human portrait in the historical image data, and determining image data in the historical image data where a human portrait is identified by a detection frame as first historical image data; The first historical image data is cropped to obtain the target historical image data.

7. The method according to claim 1, characterized in that The method also includes: the first detection model is deployed on a front-end device, and when the first detection model detects the presence of a human portrait in the image data, the first target image data is uploaded to a back-end server, and the second detection model is deployed in the back-end server.

8. A fishing behavior detection device, characterized in that: include: An acquisition module, used for acquiring image data of a target area, wherein the target area is a preset range area including a power transmission line; A first detection module, configured to perform portrait detection on the image data using a first detection model, and determine first target image data when a portrait is detected in the image data, wherein the first detection model is used to detect whether a portrait is in the image data, and the first target image data is image data in which a portrait is identified by a detection frame in the image data; A second detection module is used to perform fishing behavior detection on the first target image data using a second detection model to determine a fishing behavior detection result, wherein the second detection model is used to detect whether a human figure in the first target image data has fishing behavior.

9. A non-volatile storage medium, characterized in that: The non-volatile storage medium stores a program, wherein when the program is executed, the device where the non-volatile storage medium is located is controlled to execute the method for detecting fishing behavior as described in any one of claims 1 to 7.

10. A computer program product comprising computer instructions, characterized in that: When the computer instructions are executed by a processor, the method for detecting fishing behavior described in any one of claims 1 to 7 is implemented.

Images