Method for detecting phishing behavior under high-far monitoring angle, electronic device and storage medium

Abstract

The method for detecting fishing behavior under high monitoring angle of view, the electronic device and the storage medium, extract the circumscribed rectangular region of the water surface region in the image, traverse the original image using a sliding window, when the overlapping area of the sliding window and the circumscribed rectangle of the water surface region exceeds a preset threshold, the image region in the sliding window is cut and personnel detection is performed, the detected target detection frame is merged and redundant is removed, the personnel detection result is obtained, the distance between the personnel detection result and the edge contour of the water surface region is calculated, and if it exceeds the preset threshold, it is considered that the current personnel has fishing behavior and an alarm is given. The present application does not require human participation, solves the problems of difficult personnel detection under high monitoring angle of view, multiple application scenarios, time-consuming and laborious manual division of water surface region and inaccuracy, compared with the scheme of directly cutting the whole image using a sliding window for detection, the detection time is shortened by about 50%, and the accuracy of logical judgment of personnel fishing abnormal behavior is ensured.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image processing, and is a fishing behavior detection method under a high and far monitoring view angle, an electronic device and a storage medium. BACKGROUND

[0002] The surroundings of a transformer substation and a power transmission line are usually water bodies such as rivers, lakes and reservoirs, which are gathering places for fishing enthusiasts, so it is necessary to monitor the water surface area around the transformer substation. If people appear around the water area, it is necessary to judge whether the people have fishing behavior and give an early warning. However, the common camera position in these scenes is high and far, and the pixel ratio of the person in the image captured by the camera is extremely small, which belongs to a small target. Therefore, whether relying on manual identification of whether there is fishing abnormal behavior in the image or directly detecting the person in the image by a deep learning algorithm, the effect is extremely poor and is seriously disturbed by the background.

[0003] The prior art has some methods for detecting small targets under a high and far monitoring view angle: cutting the original image, detecting the cut sub-image and merging the detection results of the sub-image. Although this method increases the accuracy of detecting small targets under a high and far monitoring view angle and provides technical support for subsequent logical judgment of whether the person has fishing behavior, this method needs to detect each cut sub-image, which increases the additional calculation cost compared with directly detecting the original image. At the same time, since all areas in the image are covered, the output result has a lot of redundant information, which cannot effectively focus the attention of the algorithm on the water surface and its surrounding area that needs to be focused, thereby increasing the workload of manual verification and early warning. Therefore, it is necessary to screen unnecessary areas to avoid invalid early warning. The prior art can separate the water surface and its surrounding area by manually dividing the ROI area and cutting and detecting it, but due to the irregularity of the water surface contour, it is still a challenge to select an appropriate cutting position and size. If the ROI area is too large, it may introduce redundant information and interfere with subsequent processing; and if the cutting is too small, it may cause missed detection. In addition, this method needs to manually calibrate the ROI area for each scene, and the operation of dividing the ROI area lacks flexibility. If it needs to be re-divided, it will increase the additional workload. Therefore, it is necessary to develop a small target detection method based on images taken under a high and far monitoring view angle, which focuses on the water surface and its surrounding area and can flexibly adjust the ROI area according to specific needs, has a high accuracy, a low calculation cost and a fishing abnormal behavior detection means. SUMMARY

[0004] The problem to be solved by the present application is that the existing camera around the substation is high and far away, and the detection effect of the original image is very poor, which is not conducive to the subsequent logical judgment of fishing abnormal behavior based on the detected personnel in the image. Although the existing small target detection method improves the accuracy of small target detection by cropping the original image, detecting the cropped sub-image and merging the detection results of the sub-image, it needs to detect each cropped sub-image, which increases the additional calculation cost. At the same time, this method covers all regions in the image, and cannot effectively focus the attention of the algorithm on the water surface and its surrounding area. Although manual division of ROI region can achieve the purpose of screening unnecessary regions, it is difficult to select appropriate division position and size, and manual calibration of ROI region is time-consuming and laborious, and re-division will increase the additional workload. Under the premise that it is impossible to directly judge whether there is fishing abnormal behavior by detecting fishing rods and other tools, a fishing abnormal behavior detection method is needed, which focuses on the water surface and its surrounding area, can flexibly adjust the ROI region according to specific needs, and takes into account accuracy and low calculation cost.

[0005] The technical scheme of the present application is: a fishing behavior detection method under high monitoring angle, comprising the following steps:

[0006] Step 1: Construct a water surface region binary image dataset in different scenes, and obtain the network weight of the image semantic segmentation network by training;

[0007] Step 2: Construct a personnel dataset, and obtain the network weight of the target detection network with global perception ability by training;

[0008] Step 3: Obtain the image captured by the monitoring camera, take the upper left corner of the image as the coordinate origin, the horizontal direction as the x-axis, and the vertical direction as the y-axis, and establish a rectangular coordinate system;

[0009] Step 4: Detect the water surface region in the image using the image semantic segmentation network, obtain the edge contour of the water surface region and the set of edge point coordinates [(x0,x1,……x k ),(y0,u1,……y k )] constituting the water surface region, and obtain the maximum value (x water_min ,x water_max ) and (y water_min ,y water_max ) of the edge point coordinates, so as to deduce the maximum circumscribed rectangle of the water surface region edge contour, and save the left upper corner coordinate (x water_min ,y water_min ) and the right lower corner coordinate (x water_max ,y water_max ) of the circumscribed rectangle;

[0010] Step5: Traverse the original image and cut using the sliding window strategy, and calculate the overlapping area S of the original image region in the sliding window and the maximum circumscribed rectangle of the water surface region edge contour derived in Step4 before cutting overlap If S overlap is greater than the set threshold, it indicates that the region of the sliding window contains the water surface and the adjacent region that needs attention, and the original image region in the sliding window is cut and saved at this time, otherwise, the region in the sliding window is not adjacent to the water surface, which is a non-attention region, and the sliding window moves to the next region for judgment;

[0011] Step6: Use a target detection network with global perception ability to detect personnel in the subgraph region cut and saved from the original image in Step5, save the detection results of each subgraph, and obtain the target detection box of the personnel in the subgraph region by merging and eliminating redundancies of the detection results, and save the coordinates of the detection box:

[0012] [(x 0_min ,y 0_min ,x 0_max ,y 0_max ), (x 1_min ,y 1_min ,x 1_max ,y 1_max )…(x k_min ,y k_min ,x k_max ,y k_max )];

[0013] Step7: Calculate the local lowest center point coordinates representing each detected personnel through the detection box saved in Step6:

[0014] x k_minimum_center =(x k_min +x k_max ) / 2

[0015] y k_minimum_center =y k_max

[0016] Calculate the shortest distance w minimum from the local lowest center point (x k_minimum_center ,y k_minimum_center ) of each detected personnel to the water surface region edge contour obtained in Step4, if w minimum is less than or equal to the set threshold, it is considered that the target has suspected fishing behavior, and a warning is given and the target detection box is displayed in the original image, waiting for further verification by manual; If w minimum is greater than the set threshold, it is considered that the target does not have fishing behavior, and no warning and detection result display is performed;

[0017] Step 8: Obtain the image captured by the monitoring camera, and repeat the operations of Steps 3-7 to realize continuous detection of whether a person exists in the fishing abnormal behavior under the high monitoring view angle.

[0018] The application further provides an electronic device, which comprises a processor and a memory, and the memory stores at least one instruction or at least one program, the at least one instruction or at least one program is loaded and executed by the processor, and the high monitoring view angle fishing behavior detection method is realized.

[0019] The application further provides a computer readable storage medium, which stores at least one instruction or at least one program, and the at least one instruction or at least one program is executed to realize the high monitoring view angle fishing behavior detection method.

[0020] The application uses an improved Deeplabv3+ network to extract the edge contour of the water surface region in the image captured by the monitoring camera, and deduces the maximum circumscribed rectangle of the water surface region. On this basis, a sliding window is used to traverse the original image, and only when the overlapping area of the image region in the sliding window and the maximum circumscribed rectangle of the water surface region exceeds a preset threshold, it is indicated that the region in the sliding window is adjacent to the water surface, wherein if there is a person, there may be a fishing behavior, and only then the image region in the sliding window is divided and saved, a target detection network with global perception ability is used to detect the personnel in the divided and saved region, an improved iou calculation method minIOU is used to realize merging and redundancy elimination of the personnel target detection frame detected in each subgraph region, and finally distance calculation is performed on the final personnel detection result and the edge contour of the water surface region extracted in advance. If it exceeds the preset threshold, it is considered that the current person is suspected to have a fishing behavior, a warning is given, and the target detection frame is displayed in the original image, and further manual verification is waited. The application does not need human participation, solves the problems of difficult small target personnel detection under a high monitoring view angle, multiple application scenarios, complex scenarios, time-consuming and laborious manual division of the water surface region, and inaccuracy, and compared with direct result detection using a sliding window cutting, the detection time is shortened by about 50%, while the accuracy of the personnel fishing abnormal behavior logical judgment is ensured.

[0021] Compared with the prior art, the application has the following advantages:

[0022] (1) In the premise of a large number of scenes to be monitored and large differences in water surface areas in different scenes to be monitored, compared with manually dividing the ROI region, the water surface region can be adaptively segmented according to the scene shot by the camera, and the range of the water surface surrounding region to be monitored can be flexibly adjusted according to the segmentation result. Since the segmentation result directly affects the accuracy of subsequent processing, the network structure of Deeplabv3+ is improved to improve the feature extraction capability of the network, fully utilize the channel information and spatial information of the feature map, and ensure the accuracy of the segmentation of the water surface and the edge region.

[0023] (2) It is suitable for high remote monitoring view angle. By introducing an improved image semantic segmentation algorithm, the semantic information in the image is fully utilized to optimize the traditional segmentation of the original image and the small target detection method. Only the subgraph containing the water surface and the water surface surrounding region that needs to be focused on needs to be cut, saved and detected, which shortens the detection time by about 50%, greatly reduces the calculation cost, solves the difficulty of small target detection under high remote monitoring view angle, and avoids detecting the subgraph after cutting the whole image.

[0024] (3) In the premise that the proportion of personnel pixels in the image is small and it is impossible to determine whether there is fishing behavior through a fishing rod or the like, the application can flexibly configure detection standards according to specific scenes, has wide applicability to various monitoring scenes, ensures that the target in the detection frame output to the original image only contains a target suspected to have fishing behavior that needs to be further verified by a human, effectively filters invalid early warnings, and reduces the workload of subsequent manual verification. BRIEF DESCRIPTION OF DRAWINGS

[0025] Figure 1 The flowchart of the fishing behavior detection method under high remote monitoring view angle of the application.

[0026] Figure 2 The improved MobilNetv2 network structure of the application, (a) is the improvement of the inverted residual module with residual edges, and (b) is the improvement of the inverted residual module without using residual edges.

[0027] Figure 3 The improved Deeplabv3+ model structure diagram of the application.

[0028] Figure 4 The schematic diagram of the monitoring scene applied in the embodiment of the application.

[0029] Figure 5 The schematic diagram of extracting the edge contour of the water surface region in the scene by using the improved Deeplabv3+ network and deriving the maximum circumscribed rectangle of the water surface region.

[0030] Figure 6A schematic diagram of a subgraph containing a water surface and adjacent areas obtained by sliding cutting for an embodiment of the present application.

[0031] Figure 7 Final detection results for an embodiment of the present application. Specific implementation method

[0032] The present application proposes a fishing behavior detection method under a high monitoring visual angle, which adaptively extracts the edge contour of the water surface area according to the image captured by the camera and deduces the maximum circumscribed rectangle of the water surface area, cuts the subgraph containing the water surface and the surrounding area in the image by combining the sliding window and detects it, merges and removes the redundant detection frame of the personnel target in each subgraph area, and calculates the distance between the final personnel detection result and the edge contour of the water surface area extracted in advance, so as to better distinguish, display and warn whether the personnel in the high monitoring visual angle have fishing behavior.

[0033] The present application relates to deep learning, image semantic segmentation algorithm and target detection algorithm with global perception ability, and the overall steps are as shown in Figure 1 The specific implementation process is as follows.

[0034] Step 1: Construct a water surface area binary image dataset in different scenes, label the edge of the water surface area binary image in different scenes by drawing a point, make a training set and a test set, train the image semantic segmentation network, and obtain the network weight of the image semantic segmentation network.

[0035] As a preferred embodiment of the present application, the image semantic segmentation network adopts an improved Deeplabv3+, which includes improvement of the backbone network and improvement of the decoder part.

[0036] The following improvements are made to the Backbone part of the Deeplabv3+ network: As the Backbone of the DeepLabv3+ network, the MobileNetV2 network is stacked by an inverted residual module, as shown in Figure 2For the inverse residual module using residual edges, i.e., the inverse residual module with shortcut connection, the CBAM attention module is introduced after the depth separable convolution of the original branch; for the inverse residual module not using residual edges, three branches are expanded on the basis of the original branch to obtain four branches, and the four branches have different scales of receptive fields: among the three expanded branches, one branch uses 1x1 convolution, and the remaining two branches set the boundary padding and receptive field expansion factor of the depth separable convolution in the original branch structure to 2 or 3 at the same time, i.e., the boundary padding and receptive field expansion factor of the depth separable convolution of one branch are 2, and the boundary padding and receptive field expansion factor of the depth separable convolution of the other branch are 3; the four branches are integrated, and the different scale features extracted by different scale receptive fields are fused as the output of the inverse residual module not using residual edges. The features extracted by the improved Backbone are more rich in spatial scale than the features extracted by the original Backbone, which is beneficial to the subsequent pixel classification in the image.

[0037] In the Deeplabv3+ network structure, only one high-level feature and one low-level feature with the same size after processing are fused, the feature extraction of the fused feature map is insufficient, the edge details are few, the boundary information of the water surface area cannot be accurately segmented, and the decoder structure is simple, the low-level feature information is lacking, therefore, the decoder part of the Deeplabv3+ network is improved as follows: the Deeplabv3+ only inputs the feature map output by the first 4 layers of the MobileNetv2 network into the decoder as a shallow feature map, the two feature maps output by the first 6 layers and the first 14 layers of the MobileNetv2 network are adjusted in size after passing through the CBAM attention module, and then are spliced with the feature map output by the first 4 layers of the MobileNetv2 network as a new shallow feature map, which is sent into the decoder, so that the model can learn the water surface information better and restore the edge detail information of the target better; the deep feature map obtained by the MobileNetv2 network and the ASPP module of the Deeplabv3+ network is sent into the decoder as a new deep feature map after passing through the CBAM attention module. The CBAM attention module adds weight information to each feature channel, fully utilizes the channel information and spatial information of the feature map, focuses on the water surface edge area on the basis of obtaining the overall feature, reduces the transmission of useless information, and improves the feature extraction capability of the network.

[0038] Finally, the shallow and deep feature maps sent into the decoder are fused, and the feature information is increased, such as Figure 3The improved Deeplabv3+ model shown, by comprehensively utilizing feature information of different levels and depths, the improved image semantic segmentation network can more accurately identify the boundary and details of the water surface area, reduces the missegmentation problem caused by factors such as light change and weather condition, guarantees the accuracy of water surface area segmentation, and has strong robustness.

[0039] Step2: Construct a personnel data set, and obtain the network weight of the target detection network with global perception ability through training. As a preferred mode, the image is processed through data enhancement methods such as flipping and rotating, and the personnel in the image is labeled as a positive sample. Since the camera position of the shooting image is relatively high and remote, the pixel ratio of the personnel is small, which belongs to a small target. In order to reduce the interference of the background on the personnel detection, a part of the original image containing only the background class is introduced as a negative sample after cropping, a training set and a test set are prepared, and the target detection network is trained. The end-to-end target detection framework DETR is preferably used as the target detection network.

[0040] Step3: Obtain the image shot by the monitoring camera, and establish a rectangular coordinate system with the upper left corner of the image as the coordinate origin, the horizontal direction as the x-axis and the vertical direction as the y-axis. Figure 4 The monitoring image obtained by the embodiment of the application.

[0041] Step4: Detect the water surface area in the image using the image semantic segmentation network, obtain the edge contour of the water surface area and the set of edge point coordinates [(x0, x1, ……x k ), (y0, y1, ……y k )] constituting the water surface area, and obtain the maximum value of the edge point coordinates: derive (x water_min , x water_max ) from (x0, x1, ……x k ), derive (y water_min , y water_xax ) from (y0, y1, ……y k ), thereby derive the maximum circumscribed rectangle of the edge contour of the water surface area, and save the upper left corner coordinates (x water_min , y water_min ) and the lower right corner coordinates (x water_xax , y water_xax ) of the circumscribed rectangle, and the edge contour of the water surface area and the maximum circumscribed rectangle thereof are as shown in Figure 5 .

[0042] Step5: Traverse the original image and cut it using a sliding window strategy, and cut the picture into multiple sub-pictures in a sliding window cropping manner, as shown in Figure 6As shown, the subgraph overlap rate is generally set to no less than 50%, and the overlapping area S of the original graph region in the sliding window and the maximum circumscribed rectangle of the water surface region edge contour derived in Step 4 is calculated overlap If S overlap is greater than a set threshold value, it indicates that the region in the sliding window contains water surface and the region adjacent to the water surface, and if there is a person in it, there may be fishing behavior, which needs to be focused on. At this time, the original graph region in the sliding window is cut and saved; if S overlap is less than the set threshold value, it indicates that the region is far from the water surface region in the scene and belongs to the background region where it is impossible to appear fishing personnel, and the original image region in the sliding window does not need to be cut and saved. The sliding window is moved, and the next region is judged.

[0043] The implementation of step 5 is described below.

[0044] Step5.1: Set the number of rows row=4, the number of columns col=6, the length of the original graph Image Height , the width of the original graph Image Width , the overlap rate between the divided subgraphs of the sliding window operation Overlap=0.5, the length of the sliding window Window Height , and the width of the sliding window Window Width .

[0045] Step5.2: According to the length of the original graph Image Height , the width of the original graph Image Width , the overlap rate between the divided subgraphs, the length of the sliding window Window Height , and the width of the sliding window Window Width , the coordinates of all points that will be traversed in the original graph are calculated:

[0046] [(x0,y0),(x1,y0)…(x5,y0),(x0,y1),(x1,y1)…(x5,y1)…(x0,y3),(x1,y3)…(x5,y3)]

[0047] Step5.3: Take (x p ,y q ) from the above coordinate set, and derive (x p ,y q ) through (x p ,y q ):

[0048] x p '=x p +Window Width

[0049] y q '=y q +Window Height

[0050] (x p ,y q ), (x p ',y q ') represent the coordinates of the upper left corner and lower right corner of the sliding window respectively, and the original image area within the sliding window is cropped. In this embodiment, the original image is pre-divided into 4 rows and 6 columns, with a total of 24 sub-images. p,q represent the current traversal of the sliding window to the pth row and the qth column. The coordinates of the sliding window are compared with the coordinates of the maximum circumscribed rectangle (x) derived based on the edge contour of the water surface area in Step 4. water_min ,y water_min ,x water_max ,y water_max ), calculate the overlapping area of ​​the largest circumscribed rectangle of the original image area and the edge contour of the water surface area within the current sliding window. The area is calculated as follows:

[0051] S overlap =(min(x water_max ,x p ')-max(x water_min ,x p ))*(min(y water_max ,y p ')-max(y water_min ,y p ))

[0052] Step 5.4: Determine S overlap And set the threshold value, in this embodiment, the threshold value is set to 0, if S overlap >0, the area within the sliding window is considered to be adjacent to the water surface and needs to be paid special attention to. The original image area within the sliding window is cut and saved. overlap If ≤0, the area is considered to be far away from the water surface area in the scene and belongs to the background area where it is impossible for fishermen to appear. The sliding window moves to judge the next area.

[0053] The present invention believes that only people who appear around the monitored waters are likely to engage in fishing behavior. By adjusting the corresponding S in Step 5.4 overlap The threshold size is set, the selection judgment of the focus area is adjusted, and the sub-image containing the area adjacent to the water surface is cut and saved, thereby pre-excluding people who are not around the water surface area in the image.

[0054] Step6: using a target detection network with global awareness to detect personnel in the subgraph region cut from the original graph in Step5, saving the detection results of each subgraph, merging and removing redundant detection boxes for the same person in each region and saving the coordinates of the detection boxes:

[0055] [(x 0_min ,y 0_min ,x 0_max ,y 0_max ), (x 1_min ,y 1_min ,x 1_max ,y 1_max )…(x k_min ,y k_min ,x k_max ,y k_max )]

[0056] Step6.1: using a target detection network with global awareness to detect personnel in the subgraph region cut from the original graph in Step5, according to the row and column number of the subgraph in the original graph pre-divided into 24 subgraphs, converting the relative coordinates of the personnel detection box in the subgraph into absolute coordinates in the original graph and saving.

[0057] Step6.2: sorting the detection boxes stored in step6.1 in descending order according to the confidence, taking the detection box with the highest confidence as the detection box to be matched each time, calculating the iou between the detection box to be matched and all other detection boxes by the improved method of calculating iou, and the iou calculation formula is as follows:

[0058]

[0059] where S A represents the area of the detection box to be matched, S B represents the area of the detection box currently calculating iou with the detection box to be matched, and the size of the threshold is determined. If it is greater than the threshold, it is considered that the current detection box matches the detection box to be matched, which is a detection box for the same target, and the detection box to be matched and the detection box matched are stored as a matched pair, and the matched detection box is removed from the list of stored detection boxes. Otherwise, it is considered that the two detection boxes are detection boxes for two different targets. Finally, repeat the operation of Step6.2 from the remaining detection boxes until all the detection boxes stored in Step6.1 are stored in the form of matched pairs. If a certain detection box to be matched is left at the end, it is considered that the probability of false detection of the background class is relatively large, and it is directly removed.

[0060] Step6.3: merge and remove redundancy of the detection boxes belonging to the same target, obtain the maximum detection box covering the detection boxes for the same target in the current matching pair as the final detection result, and save the top-left corner coordinates (x result_min ,y result_min ) and the bottom-right corner coordinates (x result_max ,x result_max ) of the final result detection box, corresponding to multiple target detection boxes [(x 0_min ,y 0_min ,x 0_max ,y 0_max ), (x 1_min ,y 1_min ,x 1_max ,y 1_max )…(x k_min ,y k_min ,x k_max ,y k_max )].

[0061] Since the image is cut in the sliding window mode, there may be a target in a sub-image that is not complete, so that the detection box is also not complete, and the traditional calculation of the IOU cannot well remove the incomplete detection box for the same target. The present application improves the traditional IOU calculation method and combines the algorithm logic of merging and removing redundancy of the detection boxes for the same target, thereby well removing the incomplete detection box for the same target and ensuring the accuracy of the detection.

[0062] Step 7: derive the local lowest center point coordinates representing each detected person from the coordinates of the detected person saved in Step 6:

[0063] x k_minimum_center =(x k_min +x k_max ) / 2

[0064] y k_minimum_center =y k_max

[0065] Calculate the shortest distance w minimum from the local lowest center point (x k_minimum_center ,y k_minimum_center ) of each detected person to the water surface area obtained in Step 4, if w minimum is less than or equal to the set threshold value, it is considered that the target is suspected of fishing behavior, a warning is given and the target detection box is displayed in the original image, as shown in Figure 7 , waiting for further verification by human. If w minimum is greater than the set threshold value, it is considered that the target does not have fishing behavior, no warning and detection result display are given. The present application preferably uses a threshold value of 30.

[0066] Step 8: Continue to obtain the image shot by the monitoring camera, repeat the operation of Step 3-7, realize the continuous detection of whether there is a person existing the fishing abnormal behavior under the high monitoring visual angle.

[0067] The present application firstly realizes the elimination of the personnel not appearing around the water surface area in the image by adjusting the threshold of the overlapping area of the original image area and the water surface area in the sliding window, but only this threshold cannot guarantee that the detected personnel in the image exists the fishing behavior, therefore, based on the improved traditional iou calculation method, by adjusting the threshold to remove the redundant frame, on the basis of ensuring the detection accuracy, the present application further introduces the threshold in Step 7, by adjusting the size of the threshold, can ensure that the detected personnel displayed in the original image is very close to the water surface, and the possibility of existing the fishing behavior is very high, so as to perform the early warning.

[0068] The present application can be realized based on a computer program, and based on this, the present application further provides an electronic device, which comprises a processor and a memory, the memory stores at least one instruction or at least one program, the at least one instruction or at least one program is loaded and executed by the processor, and realizes the above-mentioned method for detecting whether the personnel exists the fishing abnormal behavior under the high monitoring visual angle. A computer readable storage medium is also provided, the computer readable storage medium stores at least one instruction or at least one program, and the at least one instruction or one program is executed to realize the above-mentioned method for detecting whether the personnel exists the fishing abnormal behavior under the high monitoring visual angle.

Claims

1. The method for detecting phishing behavior from a high-level monitoring perspective is characterized by: Comprising the following steps: Step1: Construct the binary image dataset of the water surface area in different scenes, and obtain the network weight of the image semantic segmentation network by training; Step2: Construct the personnel dataset, and obtain the network weight of the target detection network with global perception ability by training; Step3: Obtain the image captured by the monitoring camera, taking the upper left corner of the image as the coordinate origin, the horizontal direction as the x-axis, and the vertical direction as the y-axis. Step4: Calculate the distance between the two points in the image. Step5: Convert the distance to the actual distance in the real world. Step4: using image semantic segmentation network to detect water surface area in the image, obtain edge contour of water surface area and set of edge point coordinates constituting water surface area edge point coordinates ),( )], obtain maximum value of edge point coordinates ) and ), so as to deduce maximum circumscribed rectangle of water surface area edge contour, and save upper left corner coordinate ) and right bottom corner coordinate ) of circumscribed rectangle; Step 5: traverse the original image and cut using sliding window strategy, calculate the overlapping area of the original image region in the sliding window and the maximum circumscribed rectangle of the water surface region edge contour derived in Step 4 before cutting , if is greater than the set threshold value, it indicates that the region of the sliding window contains the water surface and the region adjacent to the water surface that needs to be concerned, at this time the original image region in the sliding window is cut and saved, otherwise it indicates that the region in the sliding window is not adjacent to the water surface, which is a non-concerned region, the sliding window is moved and the next region is judged. Step6: Use the target detection network with global perception ability to detect personnel in the subgraph area cut and saved from the original graph in Step5, save the detection results of each subgraph, and obtain the target detection box of the personnel in the subgraph area by merging and eliminating the redundancy of the detection results, and save the coordinates of the detection box: [( )，( ) ( )]； Step7: Calculate the local minimum center point coordinates representing each detected personnel through the detection box saved in Step6: Calculate the local minimum center point of each detected person ) to the shortest distance of the water surface area edge contour obtained in Step 4 , if is less than or equal to the set threshold (no modification), it is considered that the target has suspected phishing behavior, a warning is given, and the target detection frame is displayed in the original image, waiting for further verification by manual; if is greater than the set threshold, it is considered that the target does not have phishing behavior, and no warning and detection result display are performed; Step8: Obtain the image captured by the monitoring camera, and repeat the operations of Steps 3-7 to realize continuous detection of whether there is personnel with fishing abnormal behavior under the high monitoring angle.

2. The method of claim 1, wherein the method further comprises: In Step1, the edges of the binary image of the water surface area in different scenes are marked by edge drawing and point marking, and the training set and test set are prepared for training the image semantic segmentation network.

3. The method of claim 1, wherein the method further comprises: In Step2, after the image is processed by the data enhancement method, the personnel in the image are labeled as positive samples, and the subgraph containing only background classes after cropping the original graph is introduced as negative samples, and the training set and test set are prepared for training the target detection network with global perception ability.

4. The method of claim 1, wherein the method further comprises: The image semantic segmentation network uses an improved Deeplabv3+, ​ The Backbone part of the Deeplabv3+ network is improved as follows: The Backbone of the DeepLabv3+ network uses the MobileNetV2 network, which is stacked by inverted residual blocks. For the inverted residual block using residual edges, a CBAM attention module is introduced after the depth separable convolution in the original branch. For the inverted residual block not using residual edges, three branches are expanded based on the original branch, resulting in four parallel branches. In the three expanded branches, one branch uses 1x1 convolution, and the other two branches set the boundary padding and receptive field expansion factor of the depth separable convolution in the original branch structure to 2 or 3, respectively. Four branches extract features of different scales through different scales of receptive fields, and fuse the features of four branches as the output of the inverted residual block not using residual edges. The decoder part of the Deeplabv3+ network is improved as follows: The two feature maps output by the first 6 layers and the first 14 layers of the MobileNetv2 network are processed through the CBAM attention module, the feature map size is adjusted, and then the feature map is concatenated with the feature map output by the first 4 layers of the MobileNetv2 network as a shallow feature map, which is sent to the decoder. The deep feature map obtained by the MobileNetv2 network and the ASPP module is processed through the CBAM attention module as a new deep feature map, which is upsampled and sent to the decoder. The shallow and deep feature maps sent to the decoder are fused, and the boundaries and details of the water surface area are decoded and output.

5. The method of claim 1, wherein the method further comprises: determining whether the user is a human or a machine based on the user's behavior. Step5 is specifically: ​ Step5.1: Set the number of rows of the split , the number of columns , the length of the original image , the width , the overlap rate between the split sub-images , the length of the sliding window , the width ; Step 5.2: According to the length , width of the original graph, cut the overlapping rate between sub-graphs , the length , width of the sliding window, calculate the coordinate set of all points that the top-left corner of the sliding window will traverse in the original graph: Step5.3: From the above set of coordinates, take out ( ), derive ( ) by ( ): ( ),( ) represent the coordinates of the upper left corner and lower right corner of the sliding window, Indicates that the sliding window is currently traversing to the Row, No. Column, crop the original image area within the sliding window; through the coordinates of the sliding window and the maximum circumscribed rectangle coordinates derived based on the edge contour of the water surface area in Step 4 ( ), calculate the overlapping area of ​​the largest circumscribed rectangle of the original image area and the edge contour of the water surface area within the current sliding window. The area is calculated as follows: Step 5.4: judging and set the size of the threshold, if greater than the threshold, consider the region within the sliding window adjacent to the water surface, as the area of concern, the original region within the sliding window is cut and saved. If the value is less than or equal to the threshold value, it is considered that the region belongs to a background region where it is impossible to appear a phishing person, a sliding window is moved, and a next region is judged.

6. The method of claim 1, wherein the method further comprises: The target detection network with global perception capability is a end-to-end target detection framework DETR.

7. The method for detecting fishing behavior under a high-distance monitoring perspective according to claim 1 is characterized by: Step 6 is specifically: Step 6.1: using the target detection network with global perception capability to detect the personnel in the subgraph region cut from the original graph in Step 5, converting the relative coordinates of the personnel detection frame in the subgraph region into absolute coordinates in the original graph and saving; Step6.2: According to the confidence of the detection box, sort in descending order, take out the detection box with the highest confidence as the detection box to be matched each time, and match it with all the remaining detection boxes through Method to calculate the , The calculation formula is as follows: wherein represents the area of the detection frame to be matched, represents the area of the detection frame to be matched, represents the area of the detection frame to be matched, is compared with the size of a set threshold value, if greater than the threshold value, it is considered that the current detection frame matches the detection frame to be matched, which is for the same target, the detection frame to be matched and the matched detection frame for the same target are stored as a matched pair, and the matched detection frame is removed, otherwise it is considered that the detection frames are for two different targets, and finally the operation of Step 6.2 is repeated from the remaining detection frames until all the stored detection frames are stored in the form of matched pairs, if a certain detection frame to be matched is left at last without a matched detection frame, it is considered as a false detection background class and is directly removed; Step 6.3: merging and eliminating redundancy of the detection frames of the same target, and obtaining the maximum detection frame in the current matching pair as the result of the final target detection.

8. The method for detecting fishing behavior under a high-distance monitoring perspective according to claim 1 is characterized by: In Step 7, if , it is considered that the target exists suspected phishing behavior, a warning is given and the target detection frame is displayed in the original image, and further manual verification is waited for; if , it is considered that the target does not exist phishing behavior, no warning and detection result display are given.

9. An electronic device, characterized by The electronic device includes a processor and a memory, the memory stores at least one instruction or at least one program, the at least one instruction or at least one program is loaded and executed by the processor, and the high-altitude monitoring perspective phishing behavior detection method of any one of claims 1-8 is realized.

10. A computer readable storage medium characterized by The computer readable storage medium stores at least one instruction or at least one program, and when the at least one instruction or the program is executed, the high-altitude monitoring perspective phishing behavior detection method of any one of claims 1-8 is realized.

Images