Method and system for monitoring suspected man-made water station interference behavior and scene

Abstract

The invention relates to a suspected man-made water station interference behavior and scene monitoring method and system, and belongs to the technical field of image processing. Detecting whether a target exists in the image through an EP-YOLO target detection model; if the target exists, carrying out region segmentation on the image by utilizing a semantic segmentation model, and judging whether the position point of the target is located in water or a water bank region or not; if the target is located in water or a water bank area, target type judgment is carried out, and when the target is a ship or an engineering vehicle, warning information is sent out; and when the target is a person, obtaining secondary image data of the target, performing attitude estimation, performing similarity calculation with the attitude library, and when the similarity exceeds a threshold value, sending out alarm information. According to the invention, various water station interference behaviors and scenes can be covered, the generalization is high, and the portability of the system and the application in various scenes are facilitated; the detection accuracy is improved, and the prediction capability is improved without increasing a large amount of calculation.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method and system for monitoring suspected human interference behaviors and scenes at a water station. Background Art

[0002] Automatic water quality monitoring stations are usually used to monitor the water quality of cross-section water bodies and warn of water pollution accidents, which are of great significance for environmental protection and water quality management. However, there are still fraudulent behaviors such as human interference with environmental quality monitoring stations, which seriously affect the normal operation of the automatic monitoring system of water stations and public health. Traditional suspected human interference behavior detection usually adopts on-site inspections or day and night monitoring of monitors. With the increasing number of monitoring points in water stations, the labor cost required is greatly increased. At the same time, the traditional detection efficiency is low and the labor intensity is high, which is difficult to meet the growing supervision needs of law enforcement departments. At the same time, due to the large number of human interference behaviors in water stations, one is that non-operation and maintenance personnel or ships enter the station house and water collection area, tamper with monitoring data, release biological and chemical agents, stir water bodies, touch and drag sampling facilities, lay aquatic plants around the water collection port, and build aeration and oxygenation facilities, etc., which cause abnormal water quality monitoring results; there are also behaviors such as intercepting sewage diversion, building dams, digging ditches and diversion channels, which cause cross-section stations to lose their pollution monitoring function. Using a single deep learning network to target these behaviors cannot achieve comprehensive human interference with water station behaviors, scene monitoring and supervision. In recent years, with the continuous improvement of computer vision detection and recognition technology, the combination of multiple computer vision technologies has become an effective solution for abnormal behavior and scene monitoring around water stations. Summary of the invention

[0003] In order to solve the above-mentioned problems in the prior art, in order to ensure the normal operation of the water station monitoring system, meet the growing supervision needs of regulatory authorities, and realize the intelligent monitoring and management of water stations, the present invention provides a method and system for monitoring suspected human interference in water station behaviors and scenes.

[0004] A method for monitoring suspected human interference behavior and scenes at a water station, comprising the following steps: Step 1: Use the water station monitoring camera to collect video data and convert the video into images; Step 2: Build an EP-YOLO target detection model based on edge-enhanced pyramid network to detect whether there are targets such as people, ships or engineering vehicles in the image; if there are targets, obtain the coordinates of the bottom center point of the anchor box and use the coordinates of the bottom center point as the location point of the target; Step 3: For the image in which the target is detected, the Fusion-Former semantic segmentation model is used to segment the original image into the underwater, waterside and non-water areas, obtain the mask images of the underwater, waterside and non-water areas, and determine whether the location of the target is in the underwater or waterside area; Step 4: If the target is in the water or on the waterfront, the target type is determined. If the target is a ship or an engineering vehicle, an alarm message is issued; if the target is a person, the camera is zoomed in and zoomed in to obtain secondary image data of the target; if there is no target, the surrounding images of the water station are continuously monitored; Step 5: Use the YOLO-Pose model to estimate the posture of the secondary image, obtain the detection frame and key point information of the human body, and calculate the similarity between the key point information and the posture library. If the similarity exceeds the threshold, an alarm message is issued.

[0005] Furthermore, the EP-YOLO target detection model decomposes the image into components of different resolutions through a Laplacian pyramid, and then enhances the detail information of each scale component through a detail enhancement module based on a self-attention mechanism, and finally inputs the detail-enhanced image into the YOLO model for target detection.

[0006] Furthermore, the Laplacian pyramid decomposes the image into components of different resolutions in the following specific process: using the Gaussian pyramid to obtain sub-images of different resolutions; then, defining the Laplacian pyramid with loss information, and when the image needs to be reconstructed, reconstructing the image by performing the inverse operation of the Laplacian pyramid.

[0007] Furthermore, the specific process of the detail enhancement module enhancing the detail information of each scale component is as follows: First, cross convolution is used to extract the detail information of the Laplace component, and then the edge information is further extracted through the convolution filter; Then, the self-attention mechanism is used to perform element-wise addition on the components and detail information of the input image, and the addition result is reshaped to obtain three convolution tensors: , and , and then and Perform matrix product and generate attention matrix A, A and Perform matrix multiplication and reshape it to the same dimension as the input component x. Finally, concatenate it with the input component x and undergo a convolution operation to restore it to the same size as the input image as the output of the detail enhancement module.

[0008] Furthermore, the Fusion-Former semantic segmentation model includes an encoder and a decoder structure; the encoder adopts a hybrid Transformer encoder MiT, which first divides the input image into small blocks of size 4*4, and then uses these small blocks as input to the four-stage MiT to obtain hierarchical and multi-resolution features. ; The decoder stage i sequentially generates refined features by performing information fusion attention ; Upsample the decoder features using bilinear interpolation to match the feature map output by the first stage decoder The height and width of Finally, the concatenated features are processed by a multi-layer perceptron (MLP) to predict a segmentation map of H / 4×W / 4×3, thereby completing the segmentation of the image into the underwater, waterside, and non-water areas.

[0009] Furthermore, the information fusion attention includes two parts: local information enhancement and context information fusion; The input of the local information enhancement part comes from the corresponding encoder feature map, and the input of the context information fusion part comes from the mixed feature map of the encoder and decoder stages; The local information enhancement part uses convolution to assist in enhancing spatial details. The shallow features obtained from the encoder are spliced ​​in the channel dimension, and then passed to a block composed of 3×3 convolution and batch normalization. The local information is aggregated through 3×3 convolution, and then the dimension is compressed using linear projection with activation function and batch normalization to generate detail enhancement weights, which are finally multiplied with the features output by the context information fusion part for fusion; The context information fusion part first divides all pixels of the mixed feature map from the encoder and decoder stages into multiple subgroups, and then uses self-attention to calculate the sparse relationship matrix global relationship estimation, local relationship estimation and context representation.

[0010] Furthermore, the shallow features obtained in the encoder are aligned to the spatial size using spatial average pooling, and the channel size is aligned using 1×1 convolution, and the aligned features are concatenated along the channel dimension to form the input of the local information enhancement part.

[0011] Furthermore, the step 5 specifically includes: Step 501: Collecting behavior images suspected of affecting water station data; Step 502: Use the YOLO-Pose model to obtain the detection frame and 17 key point information of the human body in the image; Step 503: constructing the posture library data, taking the bottom center point of the detection frame as the coordinate origin, normalizing the positions of the 17 key point information with the size of the detection frame, forming a 17-dimensional vector with the coordinate origin, and forming a vector with all the pre-collected data in the same calculation method and storing it in the posture library to form the final posture library data; Step 504: Calculate the posture vector of the human body detection frame and key point information of the human body in the real-time image in the same way, and calculate the similarity between it and the vector in the posture library. If it exceeds the threshold point, an alarm message is issued. If it does not exceed the threshold, no alarm is issued and the monitoring of the surrounding area of ​​the water station continues.

[0012] A suspected human interference behavior and scene monitoring system for a water station comprises a data acquisition module, a memory and a processor; the data acquisition module collects image data of each monitoring source in real time and transmits it to the memory; the memory is used to store computer executable instructions, video image data and a posture library made of human interference behavior image data of a water station; the processor is used to execute the executable instructions and perform data preprocessing on the image data collected by the data acquisition module; target detection is performed on the image to detect whether there are human, ship and engineering vehicle targets in the image; if the above targets exist, the image is segmented into an underwater area, a waterfront area and a non-water area, and it is judged whether the above targets are located in the underwater area or the waterfront area; if the target is located in the underwater area or the waterfront area, it is judged whether the target is a person or a ship or an engineering vehicle, and if the detected target is a ship or an engineering vehicle, an alarm message is issued; if the detected target is a person, the camera is zoomed in and amplified to obtain image data of the target for a second time, the secondly obtained image is input into a posture estimation network, the posture of the human body is estimated, and similarity is calculated with the established posture library to judge whether there is suspected human interference behavior of the water station, and an alarm message is issued if it is an abnormal behavior.

[0013] The beneficial effects of the present invention are as follows: the present invention combines multiple computer vision technologies to realize real-time monitoring of behaviors and scenes of suspected human interference with water stations. The present invention has strong monitoring capabilities, can cover a variety of water station interference behaviors and scenes, has high generalization, and is conducive to the portability of the system and its application in a variety of scenarios. Through target detection and semantic segmentation models, the accuracy of detection is improved, and the prediction ability is improved without increasing the amount of calculation. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a flow chart of the monitoring method in Example 1; Figure 2 This is a schematic diagram of the target detection network structure; Figure 3 This is a schematic diagram of the detail enhancement module structure; Figure 4 Schematic diagram of the semantic segmentation network structure; Figure 5 Schematic diagram of the information fusion attention structure. DETAILED DESCRIPTION

[0015] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0016] Example 1, reference Figure 1 , a suspected human interference water station behavior and scene monitoring system, comprising the following steps: Step 1: Use the water station monitoring camera to collect video data and convert the video into images; Step 2: Build an EP-YOLO target detection model based on edge-enhanced pyramid network to detect whether there are targets such as people, ships or engineering vehicles in the image; if there are targets, obtain the coordinates (x, y) of the center point of the bottom edge of the anchor box and use the coordinates (x, y) of the center point of the bottom edge as the location point of the target; Step 3: For the image in which the target is detected, the Fusion-Former semantic segmentation model is used to segment the original image into the underwater, waterside and non-water areas, obtain the mask images of the underwater, waterside and non-water areas, and determine whether the location of the target is in the underwater or waterside area; Step 4: If the target is in the water or on the waterfront, the target type is determined. If the target is a ship or an engineering vehicle, an alarm message is issued; if the target is a person, the camera is zoomed in and zoomed in to obtain secondary image data of the target; if there is no target, the surrounding images of the water station are continuously monitored; Step 5: Use the YOLO-Pose model to estimate the posture of the secondary image, obtain the detection frame and key point information of the human body, and calculate the similarity between the key point information and the posture library. If the similarity exceeds the threshold, an alarm message is issued.

[0017] In this embodiment, if Figure 2 As shown in the figure, the EP-YOLO target detection model decomposes the image into components of different resolutions through the Laplacian pyramid, and then enhances the detail information of each scale component through the detail enhancement module (DEM) based on the self-attention mechanism, and finally inputs the detail-enhanced image into the YOLO model for target detection.

[0018] Among them, the Laplacian pyramid decomposes the image into components of different resolutions. The specific process is: using the Gaussian pyramid to obtain sub-images of different resolutions, expressed as: , Where I represents the input image, Down represents downsampling, Gaussian represents Gaussian filter, the size of Gaussian kernel is 5*5, and after each Gaussian pyramid operation, the width and height of the image are halved, and the resolution becomes one quarter of the original; In order to restore the original high-resolution image after upsampling, the lost information is needed. The lost information is the component of the Laplacian pyramid, and the Laplacian pyramid is defined as: , in, represents the i-th level of the Laplacian pyramid, Represents the i-th layer of the Gaussian pyramid; when the image needs to be reconstructed, by executing The reverse operation can be performed.

[0019] like Figure 3 As shown in Figure 2, the specific process of the detail enhancement module enhancing the detail information of each scale component is as follows: First, cross convolution is used to extract the detail information of the Laplace component, and then the edge information is further extracted through the convolution filter. This process is expressed as:

[0020] Where E(x) represents the input component X (i.e., the Laplace pyramid output ) edge information, represents the convolution filtering operation, and represents two factorized asymmetric filter banks with receptive fields of 1×m and m×1, represents the convolution operation, and b represents the bias term; Then, the self-attention mechanism is used to perform element-wise addition on the input component x and E(x) to obtain S, which is reshaped to obtain three convolution tensors: , and , then and Perform matrix product and generate attention matrix A, A and Perform matrix multiplication and reshape it to the same dimension as the input component x. Finally, concatenate it with the input component x and undergo a convolution operation to restore it to the same size as the input image. As the output of the detail enhancement module, it can be expressed as: , in, represents the output of the detail enhancement module, Represents a splicing operation, represents the dimension transformation, , represents matrix product, represents element-wise addition, E(x) represents the edge detail information of the component of the input image x, represents the normalization operation, express Conv represents the convolution operation.

[0021] In this embodiment, if Figure 4 As shown in the figure, the Fusion-Former semantic segmentation model includes an encoder and a decoder structure; the encoder adopts a hybrid Transformer encoder MiT, firstly divides the input image into small blocks of size 4*4, and then uses these small blocks as input to the four-stage MiT (MiT is a fixed neural network structure, and the four-stage MiT is used here, indicating that the encoder has four-stage outputs, i.e. 1234), to obtain hierarchical and multi-resolution features , whose size is , that is, to obtain multi-level features with original image resolution of 1 / 4, 1 / 8, 1 / 16, 1 / 32; where H represents height and W represents width, Represents the image feature channel channel of the i-th stage of the encoder.

[0022] The decoder stage i sequentially generates refined features by performing information fusion attention The decoder corresponds to the encoder and also has 4 stages. Decoder stage i represents the decoder of the i-th stage.

[0023] like Figure 5 As shown, the information fusion attention includes local information enhancement and context information fusion parts; The input of the local information enhancement part comes from the corresponding encoder feature map, and the input of the context information fusion part comes from the mixed feature map of the encoder and decoder stages (i.e., the feature map concatenated from the feature maps of the corresponding decoder and encoder); The local information enhancement part uses convolution to assist in enhancing spatial details. The shallow features obtained from the encoder are spliced ​​in the channel dimension and then passed to a block composed of 3×3 convolution and batch normalization. The local information is aggregated by 3×3 convolution, and then the dimension is compressed by linear projection with activation function and batch normalization to generate detail enhancement weights. Finally, it is multiplied with the features output by the context information fusion part for fusion; wherein, the expression for obtaining the shallow features from the encoder part is as follows: , in, represents the j-th layer image feature, {} represents the operation of aligning the spatial size by spatial average pooling, aligning its channel size by 1×1 convolution, and concatenating the aligned features along the channel dimension to form the input of the local information enhancement part .

[0024] The context information fusion part first divides all pixels of the mixed feature map from the encoder and decoder stages into multiple subgroups, and then uses self-attention to calculate the sparse relationship matrix global relationship estimation , local relationship estimation And the context representation, the above process is expressed as: , Among them, X represents the input feature map; In the process of calculating the global relationship, X is rearranged (permute) as a result; then Divided into P groups, each group contains Q adjacent positions ( ): ; In each subset Applying self-attention: , , in, represents the weight matrix calculated on the Pth subset in the process of calculating the global relationship; for The transpose of is the scaling factor, which is used when calculating the self-attention mechanism, and d is used to represent the dimension of each attention head; represents the result of applying self-attention calculation on the Pth subset in the process of calculating the global relationship; and A block consisting of two sets of 1×1 convolutions and batch normalization, and represents a 1×1 convolution, , , Cut the input channels in half, Double the channels.

[0025] All subsets Put it together: ; Then the concatenated output feature map Rearrange to obtain local relations: , in, Represents a local relationship. Represents a rearrangement operation.

[0026] Will Divide into Q groups, each containing P adjacent positions: , In each Applying self-attention independently on , which is similar to the self-attention formula, we can get and , and finally all subsets of Splice them together to get the output of the final context information aggregation part , and finally The channels output by the 1×1 convolution and local information enhancement parts are aligned.

[0027] The decoder features are upsampled using bilinear interpolation to match The height and width of the first stage decoder output. Finally, the spliced ​​features are processed by a multi-layer perceptron (MLP) to obtain a predicted H / 4×W / 4×3 segmentation map, thereby completing the segmentation of the image into the underwater, waterside, and non-water areas.

[0028] Furthermore, the step 5 specifically includes: Step 501: Collecting images of behaviors suspected of affecting water station data on the Internet and project surveillance cameras, including images of illegal behaviors such as placing biological and chemical agents, stirring water bodies, touching and dragging sampling facilities, etc.; Step 502: Use the YOLO-Pose model to obtain the detection frame and 17 key point information of the human body in the image; Step 503: construct the posture library data, take the bottom center point of the detection frame as the coordinate origin (x, y), normalize the positions of the 17 key point information with the size of the detection frame, form a 17-dimensional vector with the coordinate origin (x, y), and store all the pre-collected data in the posture library in the same calculation method to form a vector, thus forming the final posture library data; Step 504: Calculate the posture vector of the human body detection frame and key point information of the human body in the real-time image in the same way, and calculate the similarity between it and the vector in the posture library. If it exceeds the threshold point, an alarm message is issued. If it does not exceed the threshold, no alarm is issued and the monitoring of the surrounding area of ​​the water station continues.

[0029] The present invention combines multiple computer vision technologies to achieve real-time monitoring of suspected human interference behaviors and scenes of water stations. It has strong monitoring capabilities, can cover a variety of water station interference behaviors and scenes, and has high generalization, which is conducive to the portability of the system and its application in a variety of scenarios. The cross convolution in target detection emphasizes edge information by simultaneously utilizing vertical and horizontal gradient information. The parallel design shows that the computational complexity and parameters are less than those of traditional filters with the same receptive field. At the same time, parallel development can also retain more information than sequential design, so that the target detection network can achieve effective real-time monitoring of illegal behaviors when light is limited at night. The information fusion attention in the semantic segmentation model fuses shallow image features and deep contextual semantic information, and uses sparse self-attention mechanism to promote semantic understanding and prediction of contextual information, which improves the prediction ability without increasing the amount of calculation.

[0030] On the other hand, a system for monitoring suspected human interference in water station behavior and scenes includes a data acquisition module, a memory and a processor; the data acquisition module collects image data of each monitoring source in real time and transmits it to the memory; the memory is used to store computer-executable instructions, video image data and a posture library made of human interference in water station behavior image data; the processor is used to execute the executable instructions and perform data preprocessing on the image data collected by the data acquisition module; target detection is performed on the image to detect whether there are people, ships and engineering vehicle targets in the image; if the above targets exist, the image is segmented into underwater areas, waterfront areas and non-water areas, and it is determined whether the above targets are located in the underwater area or the waterfront area; if the target is located in the underwater area or the waterfront area, it is determined whether the target is a person or a ship or an engineering vehicle, and if the detected target is a ship or an engineering vehicle, an alarm message is issued; if the detected target is a person, the camera is zoomed in to obtain the image data of the target for a second time, and the second-acquired image is input into a posture estimation network, the posture of the human body is estimated, and the similarity is calculated with the established posture library to determine whether there is suspected human interference in the water station behavior, and if it is an abnormal behavior, an alarm message is issued.

[0031] In the description of the embodiments of the present invention, it should be understood that the terms "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "center", "top", "bottom", "top", "bottom", "inside", "outside", "inner side", "outer side" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. Among them, "inside" refers to an internal or enclosed area or space. "Periphery" refers to the area surrounding a specific component or a specific area.

[0032] In the description of the embodiments of the present invention, the terms "first", "second", "third", and "fourth" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first", "second", "third", and "fourth" may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "multiple" means two or more.

[0033] In the description of the embodiments of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "install", "connect", "connect", and "assemble" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a direct connection, or an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0034] In the description of the embodiments of the present invention, specific features, structures, materials or characteristics may be combined in a suitable manner in any one or more embodiments or examples.

[0035] In the description of the embodiments of the present invention, it should be understood that "-" and "~" represent a range between two values, and the range includes the endpoints. For example: "AB" represents a range greater than or equal to A and less than or equal to B. "A~B" represents a range greater than or equal to A and less than or equal to B.

[0036] In the description of the embodiments of the present invention, the term "and / or" herein is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " herein generally indicates that the associated objects before and after are in an "or" relationship.

[0037] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for monitoring suspected human interference behaviors and scenes at water stations, characterized in that: The following steps are involved: Step 1: Use the water station monitoring camera to collect video data and convert the video into images; Step 2: Build an EP-YOLO target detection model based on edge-enhanced pyramid network to detect whether there are targets such as people, ships or engineering vehicles in the image; if there are targets, obtain the coordinates of the bottom center point of the anchor box and use the coordinates of the bottom center point as the location point of the target; Step 3: For the image in which the target is detected, the Fusion-Former semantic segmentation model is used to segment the original image into the underwater, waterside and non-water areas, obtain the mask images of the underwater, waterside and non-water areas, and determine whether the location of the target is in the underwater or waterside area; Step 4: If the target is in the water or on the waterfront, the target type is determined. If the target is a ship or an engineering vehicle, an alarm message is issued; if the target is a person, the camera is zoomed in and brought closer to obtain secondary image data of the target; if there is no target, the surrounding area of ​​the water station is continuously monitored; Step 5: Use the YOLO-Pose model to estimate the posture of the secondary image, obtain the detection frame and key point information of the human body, and calculate the similarity between the key point information and the posture library. If the similarity exceeds the threshold, an alarm message is issued.

2. The method for monitoring suspected human interference water station behavior and scene according to claim 1 is characterized in that: The EP-YOLO target detection model decomposes an image into components of different resolutions through a Laplacian pyramid, and then enhances the detail information of each scale component through a detail enhancement module based on a self-attention mechanism. Finally, the detail-enhanced image is input into the YOLO model for target detection.

3. The method for monitoring suspected human interference water station behavior and scene according to claim 2 is characterized in that: The Laplacian pyramid decomposes an image into components of different resolutions in the following specific process: using a Gaussian pyramid to obtain sub-images of different resolutions; then, defining the Laplacian pyramid with loss information, and when the image needs to be reconstructed, performing an inverse operation of the Laplacian pyramid to reconstruct the image.

4. The method for monitoring suspected human interference water station behavior and scene according to claim 2 is characterized in that: The specific process of the detail enhancement module enhancing the detail information of each scale component is as follows: First, cross convolution is used to extract the detail information of the Laplace component, and then the edge information is further extracted through the convolution filter; Then, the input component x and the detail information are element-wise added through the self-attention mechanism, and the dimension of the addition result is reshaped to obtain three convolution tensors , and , and then and Perform matrix product and generate attention matrix A, A and Perform matrix multiplication and reshape it to the same dimension as the input component x. Finally, concatenate it with the input component x and undergo a convolution operation to restore it to the same size as the input image as the output of the detail enhancement module.

5. The method for monitoring suspected human interference water station behavior and scene according to claim 1 is characterized in that: The Fusion-Former semantic segmentation model includes an encoder and a decoder structure; the encoder uses a hybrid Transformer encoder MiT, which first divides the input image into small blocks of size 4*4, and then uses these small blocks as input to the four-stage MiT to obtain hierarchical and multi-resolution features. ; The decoder stage i sequentially generates refined features by performing information fusion attention ; Upsample the decoder features using bilinear interpolation to match the feature map output by the first stage decoder The height and width of Finally, the spliced ​​features are processed by a multi-layer perceptron to predict a segmentation map of H / 4×W / 4×3, thereby completing the segmentation of the image into the underwater, waterside, and non-water areas, where H represents height and W represents width.

6. The method for monitoring suspected human interference water station behavior and scene according to claim 1 is characterized in that: The information fusion attention includes two parts: local information enhancement and context information fusion; The input of the local information enhancement part comes from the corresponding encoder feature map, and the input of the context information fusion part comes from the mixed feature map of the encoder and decoder stages; The local information enhancement part uses convolution to assist in enhancing spatial details. The shallow features obtained from the encoder are spliced ​​in the channel dimension, and then passed to a block composed of 3×3 convolution and batch normalization. The local information is aggregated through 3×3 convolution, and then the dimension is compressed using linear projection with activation function and batch normalization to generate detail enhancement weights, which are finally multiplied with the features output by the context information fusion part for fusion; The context information fusion part first divides all pixels of the mixed feature map from the encoder and decoder stages into multiple subgroups, and then uses self-attention to calculate the sparse relationship matrix global relationship estimation, local relationship estimation and context representation.

7. The method for monitoring suspected human interference water station behavior and scene according to claim 6 is characterized in that: The shallow features obtained in the encoder are aligned to the spatial size using spatial average pooling, and the channel size is aligned using 1×1 convolution, and the aligned features are concatenated along the channel dimension to form the input of the local information enhancement part.

8. The method for monitoring suspected human interference water station behavior and scene according to claim 1 is characterized in that: The step 5 specifically includes: Step 501: Collecting behavior images suspected of affecting water station data; Step 502: Use the YOLO-Pose model to obtain the detection frame and 17 key point information of the human body in the image; Step 503: constructing the posture library data, taking the bottom center point of the detection frame as the coordinate origin, normalizing the positions of the 17 key point information with the size of the detection frame, forming a 17-dimensional vector with the coordinate origin, and forming a vector with all the pre-collected data in the same calculation method and storing it in the posture library to form the final posture library data; Step 504: Calculate the posture vector of the human body detection frame and key point information of the human body in the real-time image in the same way, and calculate the similarity between it and the vector in the posture library. If it exceeds the threshold point, an alarm message is issued. If it does not exceed the threshold, no alarm is issued and the monitoring of the surrounding area of ​​the water station continues.

9. A system for monitoring suspected human interference at a water station and a scene, using the method for monitoring suspected human interference at a water station and a scene as claimed in any one of claims 1 to 8, characterized in that: It includes a data acquisition module, a memory and a processor; the data acquisition module collects image data of each monitoring source in real time and transmits it to the memory; the memory is used to store computer-executable instructions, video image data and a posture library made of image data of human interference with water station behavior; the processor is used to execute the executable instructions and perform data preprocessing on the image data collected by the data acquisition module; target detection is performed on the image to detect whether there are people, ships and engineering vehicle targets in the image; if the above targets exist, the image is segmented into an underwater area, a waterfront area and a non-water area, and it is judged whether the above targets are located in the underwater area or the waterfront area; if the target is located in the underwater area or the waterfront area, it is judged whether the target is a person or a ship or an engineering vehicle, and if the detected target is a ship or an engineering vehicle, an alarm message is issued; if the detected target is a person, the camera is zoomed in to obtain the image data of the target for a second time, and the second-acquired image is input into a posture estimation network, the posture of the human body is estimated, and the similarity is calculated with the established posture library to judge whether there is suspected human interference with the water station behavior, and if it is an abnormal behavior, an alarm message is issued.

Images