Pipeline anomaly intrusion target detection and behavior recognition system of machine vision

Abstract

This invention discloses a machine vision-based system for detecting and recognizing abnormal intrusion targets along pipelines, relating to the field of intelligent security. It includes a pipeline scene modeling module, a target enhancement and detection module, a spatiotemporal tracking module, and a risk warning module. This invention solves problems such as missed detection of small targets at long distances in pipeline scenes, lack of spatial constraints, inaccurate behavior recognition, high false alarm rates, and unreasonable risk classification. It is suitable for real-time safety monitoring of long-distance pipelines such as oil and gas, water conservancy, and natural gas pipelines, and has the advantages of low missed detection, low false alarms, high robustness, and easy deployment.

Description

Technical Field

[0001] This invention relates to the field of intelligent security, specifically to a machine vision-based system for detecting and recognizing abnormal intrusion targets along pipelines. Background Technology

[0002] Long-distance pipelines are crucial infrastructure for energy and water resource transportation, characterized by their long lengths, wide distribution, traversal of complex terrains, and harsh outdoor environments. Illegal excavation, mechanical construction, and unauthorized intrusion along pipeline routes can seriously threaten pipeline safety, potentially leading to major safety accidents such as leaks, explosions, and supply disruptions.

[0003] The existing security monitoring and intelligent analysis technologies have the following key defects in pipeline scenarios: (1) serious missed detection of small targets at long distances: pipeline monitoring cameras are usually installed on poles with a coverage range of hundreds of meters. Personnel, small engineering machinery, flames, smoke and other distant targets are small in size and have weak features in the image, and conventional detection networks are prone to missed detection; (2) lack of spatial constraints and ambiguous regional division: there is a lack of accurate spatial modeling of pipeline protection zones and early warning zones. Vehicle and personnel activities in non-pipeline areas are easily misjudged as intrusion behavior; (3) disconnect between behavior recognition and risk assessment: traditional methods rely on single frames. The appearance feature recognition behavior is difficult to distinguish pipeline sensitive behaviors such as passing by, approaching, staying, crossing, and digging; the risk judgment is based only on the target category or simple distance, and the accuracy of the warning is insufficient; (4) high false alarm rate and poor robustness: interference factors such as dust, light and shadow changes, small animals, vegetation shaking, and camera shaking in the wild environment are very likely to trigger invalid warnings, and there is a lack of effective temporal and spatial consistency verification mechanism; (5) weak scene adaptability: the general target detection and behavior recognition model has not been customized and optimized for the characteristics of pipelines such as narrow strip distribution, long distance imaging, and narrow protection zones, and the actual implementation effect is poor.

[0004] In summary, existing technologies are insufficient to meet the actual needs of high precision, low missed detection, low false alarm, and tiered early warning along pipelines. There is an urgent need for a machine vision intrusion detection and behavior recognition system that is customized for pipeline scenarios, technologically innovative, and engineering-ready. Summary of the Invention

[0005] To address the aforementioned shortcomings in existing technologies, the machine vision-based pipeline abnormal intrusion target detection and behavior recognition system provided by this invention solves core problems such as missed detection of small targets at long distances in pipeline scenarios, insufficient spatial constraints, inaccurate behavior recognition, high false alarm rate, and unreasonable risk classification. It achieves high precision, low missed detection, low false alarm, and graded early warning, thereby improving the reliability and practicality of pipeline safety monitoring.

[0006] To achieve the above-mentioned objectives, the technical solution adopted by this invention is as follows:

[0007] A machine vision-based system for detecting and recognizing abnormal intrusion targets along pipelines includes:

[0008] The pipeline scene modeling module is used to divide the risk areas of different levels in the pipeline scene image according to the principle of spatial coordinate mapping, and generate the region prior weight map.

[0009] The target enhancement detection module is used to process each frame of the pipeline scene image captured continuously during the sampling period through a cross-scale feature alignment fusion method and a region attention mechanism controlled by the region prior weight map, and obtains the detection box, category and confidence of each target during the sampling period through dual detection heads;

[0010] The spatiotemporal tracking module is used to identify different targets based on the detection boxes, categories, and confidence levels of each target during the sampling period, and to obtain the behavior categories of each different target by using trajectory-spatial temporal coding;

[0011] The risk warning module is used to calculate the comprehensive risk value of each different target based on its category and behavior, and output a warning signal.

[0012] Furthermore, the pipeline scene modeling module, based on the principle of spatial coordinate mapping, divides risk regions of different levels in the pipeline scene image and generates a region prior weight map using the following steps:

[0013] A1. Preset different levels of risk areas in the real physical coordinate system;

[0014] A2. Obtain the intrinsic parameter matrix, rotation matrix, translation vector, and homogeneous scale factor of the cameras deployed along the pipeline, and construct the mapping relationship from the real physical coordinate system to the image coordinate system.

[0015] A3. Based on the mapping relationship from the real physical coordinate system to the image coordinate system, the risk areas of different levels in the real physical coordinate system are mapped to the image coordinate system to obtain the risk areas of different levels in the pipeline scene image.

[0016] A4. Generate a prior weight map of the region based on the different levels of risk areas in the scene image along the pipeline.

[0017] Furthermore, the target enhancement detection module includes:

[0018] A backbone feature extraction network is used to obtain multi-layer feature maps of different scales based on the currently input pipeline scene image;

[0019] The cross-scale feature alignment and fusion submodule is used to perform cross-scale feature alignment and fusion on multi-layer feature maps to obtain a cross-scale fused feature map.

[0020] The region attention submodule is used to first downsample the region prior weight map to the cross-scale fusion feature through max pooling. Figure 1 The same size is then multiplied element-wise with the cross-scale fused feature map to obtain the attention-weighted feature map;

[0021] A standard object detection head includes two convolutional layers and one convolutional output layer connected in series, used to obtain the detection box, category, and confidence of each object in the current attention-weighted feature map;

[0022] The small target detection head includes three convolutional layers and one convolutional output layer connected in series, used to obtain the detection box, category and confidence of each target in the current attention-weighted feature map.

[0023] Furthermore, when the size of the target obtained is less than a preset size threshold, the target enhancement detection module adopts the detection box, category, and confidence score of the small target detection head as output; when the size of the target obtained is greater than or equal to the size threshold, it adopts the detection box, category, and confidence score of the conventional target detection head as output.

[0024] Furthermore, the method for the cross-scale feature alignment and fusion submodule to perform cross-scale feature alignment and fusion on multi-layer feature maps to obtain a cross-scale fused feature map includes the following steps:

[0025] B1. By using bilinear upsampling, the dimensions of the feature maps from the second to the last layer are aligned with the feature map of the first layer to obtain the extended feature maps from the second to the last layer.

[0026] B2. Add the first layer feature map and the extended feature maps from the second to the last layer element by element with weights to obtain the cross-scale fused feature map.

[0027] Furthermore, the target enhancement detection module employs the following total detection loss function for labeled supervised learning:

[0028] ,

[0029] ,

[0030] ,

[0031] ,

[0032] in, Let be the total detection loss function. For classification loss function, For regression loss function, For confidence loss function, The total number of samples used for supervised learning. The sample number used for supervised learning. The total number of target categories, For the target category number, For the first Does the sample target belong to the first Class tags, The target enhancement detection module is used for the first Does the sample target belong to the first The class discrimination result, Let be a logarithmic function with the natural constant as its base. For a complete intersection-union function, The detection bounding box output by the target enhancement detection module. The actual label of the detection box. The confidence weighting coefficient is... For the first Confidence labels for the sample The target enhancement detection module is used for the first The confidence level of the sample is output.

[0033] Furthermore, the spatiotemporal tracking module identifies different targets based on the detection bounding boxes, categories, and confidence levels of each target during the sampling period, and uses trajectory-spatial temporal coding to obtain the behavior categories of each different target. The method includes the following steps:

[0034] C1. Remove targets with a confidence level lower than the preset confidence threshold from each target during the sampling period;

[0035] C2. Map the detection boxes of the remaining targets to the corresponding pipeline scene image, and extract the regions within each of the mapped detection boxes as each appearance feature block.

[0036] C3. Calculate the cosine similarity between each appearance feature block, and record the appearance feature blocks with a cosine similarity greater than the preset similarity threshold as the same target, thus identifying the different targets.

[0037] C4. Use the image coordinates of the geometric center of each appearance feature block as the position coordinates of the corresponding target;

[0038] C5. Perform trajectory-spatial temporal encoding on the position coordinates of the same target at different times during the sampling period to generate behavioral feature vectors for each different target;

[0039] C6. Process the behavioral feature vectors of different targets through a fully connected network to obtain the behavioral categories of different targets.

[0040] Furthermore, C5 includes the following sub-steps:

[0041] C51. Combine the position coordinates of the same target at different times during the sampling period to obtain the temporal feature vector sets of each different target:

[0042] ,

[0043] ,

[0044] in, For the target identity number, For time sequence number, The duration of the sampling period. For the first The temporal feature vector set of the target, For the first Target The temporal eigenvector of time, For the first Target The horizontal axis of time position For the first Target The vertical axis representing the location of time. For the first Target Spatial region value of time;

[0045] C52. Feed the temporal feature vector sets of different targets into the LSTM network to generate behavioral feature vectors of different targets:

[0046] ,

[0047] ,

[0048] in, For the first The target's behavioral feature vector For the first Target The time-varying LSTM network encoding vector. For the first Target The time-varying LSTM network encoding vector. For the first The LSTM network encoding vector at the last moment of the target sampling period. It is an LSTM network.

[0049] Furthermore, the method by which the risk warning module calculates the comprehensive risk value of each different target based on its category and behavior category, and outputs a warning signal, includes the following steps:

[0050] D1. Quantify and score the categories and behaviors of the same objective, and obtain the comprehensive risk value for each different objective by weighting them using the following formula:

[0051] ,

[0052] in, For the target identity number, For the first The overall risk value of the target, For the target category risk coefficient, For the first Quantitative scoring of target categories, Risk coefficient for target behavior category For the first Quantitative scoring of target behavior categories;

[0053] D2. Based on the comprehensive risk value of each different target, output graded early warning signals.

[0054] The beneficial effects of this invention are as follows:

[0055] (1) This invention realizes the integrated processing of precise spatial risk classification of targets along pipelines, efficient detection of small targets at long distances, time-sequential identification of target behavior and risk classification and early warning. It does not rely on additional auxiliary equipment and manual intervention. From the system architecture level, it solves the core pain points of existing technologies such as lack of spatial constraints, insufficient detection accuracy, weak behavior recognition and poor early warning effectiveness. It significantly improves the integrity and systematicness of abnormal intrusion monitoring along pipelines and is suitable for the real-time security needs of complex outdoor scenarios of various long-distance pipelines such as oil and gas, water conservancy, and gas.

[0056] (2) The pipeline scene modeling module is based on the principle of spatial coordinate mapping. It accurately maps the multi-level risk areas of the real physical coordinate system to the image coordinate system and generates a prior weight map of the region. This provides clear spatial constraints for subsequent target detection, effectively filters invalid interference targets in non-pipeline protection areas, greatly reduces false alarms caused by factors such as dust, light and shadow changes, and unrelated vehicle and personnel activities, and improves the system's anti-interference ability and scene adaptability.

[0057] (3) The target enhancement detection module adopts a cross-scale feature alignment fusion method to fully integrate shallow detail features and deep semantic features. At the same time, it strengthens the feature weight of high-risk areas through the regional attention mechanism and sets up a dual detection head for regular and small targets to differentiate detection. This addresses the problem of weak features and easy missed detection of small targets at long distances in pipeline scenarios, achieving high-precision and robust detection and significantly reducing the missed detection rate.

[0058] (4) The spatiotemporal tracking module achieves accurate target identification through confidence screening and appearance feature similarity matching. By combining trajectory-spatial temporal coding and LSTM network, the temporal location of the target and spatial risk information are fused to accurately identify pipeline sensitive behaviors. This overcomes the shortcomings of traditional single-frame appearance recognition behavior inaccurate and unable to distinguish behaviors, and realizes the temporal and accurate behavior recognition.

[0059] (5) The risk warning module combines target category and behavior category for quantitative weighting and outputs graded warning signals to achieve accurate risk classification and differentiated handling. This avoids the one-sidedness of single-dimensional risk judgment, ensures rapid response to high-risk intrusion events, reduces invalid alarms for low-risk events, balances the timeliness and accuracy of warnings, reduces operation and maintenance management costs, and improves the efficiency of pipeline safety emergency response. Attached Figure Description

[0060] Figure 1 This is a structural diagram of a machine vision-based pipeline abnormal intrusion target detection and behavior recognition system according to an embodiment of the present invention.

[0061] Figure 2 This is a structural diagram of the target enhancement detection module according to an embodiment of the present invention. Detailed Implementation

[0062] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

[0063] like Figure 1 As shown, in one embodiment of the present invention, a machine vision-based pipeline abnormal intrusion target detection and behavior recognition system includes:

[0064] The pipeline scene modeling module is used to divide the risk areas of different levels in the pipeline scene image according to the principle of spatial coordinate mapping, and generate the region prior weight map.

[0065] The target enhancement detection module is used to process each frame of the pipeline scene image captured continuously during the sampling period through a cross-scale feature alignment fusion method and a region attention mechanism controlled by the region prior weight map, and obtains the detection box, category and confidence of each target during the sampling period through dual detection heads;

[0066] The spatiotemporal tracking module is used to identify different targets based on the detection boxes, categories, and confidence levels of each target during the sampling period, and to obtain the behavior categories of each different target by using trajectory-spatial temporal coding;

[0067] The risk warning module is used to calculate the comprehensive risk value of each different target based on its category and behavior, and output a warning signal.

[0068] In this embodiment, cameras deployed along the pipeline continuously capture images of the scene along the pipeline. Considering that the spatiotemporal tracking module performs "time period" operations, a sampling period is preset. In the real time domain, the sampling period is 10 seconds, but 10 seconds is not the "duration of the sampling period" described in this invention. The value of ", because It is a discrete-domain numerical value. This embodiment uses a sliding time window frame extraction method to extract 10 seconds of consecutive frames of pipeline scene images from continuously captured pipeline scene image frames each time.

[0069] The camera used in this embodiment captures 10 frames per second of images of the scene along the pipeline. Therefore, the duration of the sampling period described later in this embodiment is... 100, time sequence number It is an integer of 100 or less.

[0070] The pipeline scene modeling module, based on the principle of spatial coordinate mapping, divides risk regions of different levels in the pipeline scene image and generates a prior weight map of the regions. The method includes the following steps:

[0071] A1. Preset risk zones at different levels in the real physical coordinate system, including:

[0072] Highest risk zone: If the actual physical distance between the target and the centerline of the pipeline is less than or equal to the preset highest risk zone threshold (set to 5 meters in this embodiment), then the target is determined to be in the highest risk zone;

[0073] In a medium-risk zone, if the actual physical distance between the target and the centerline of the pipeline is greater than the preset highest risk zone threshold but less than or equal to the preset medium-risk zone threshold (set to 20 meters in this embodiment), then the target is determined to be in a medium-risk zone.

[0074] If the actual physical distance of the target is greater than the preset medium-risk zone threshold and less than or equal to the preset outer observation zone threshold (set to 40 meters in this embodiment), then the target is determined to be in the outer observation zone.

[0075] A2. Obtain the intrinsic parameter matrix, rotation matrix, translation vector, and homogeneous scale factor of the cameras deployed along the pipeline. The expression for the intrinsic parameter matrix is:

[0076] ,

[0077] in, This is the intrinsic parameter matrix. The focal length of the camera in the horizontal direction. Let this be the focal length of the camera in the vertical direction. Let x be the x-coordinate of the principal point of the camera. The ordinate of the camera's principal point;

[0078] The rotation matrix has a dimension of 3×3, and the translation vector has a dimension of 3×1;

[0079] In this embodiment, all parameters of the intrinsic parameter matrix, rotation matrix, translation vector, and homogeneous scale factor are obtained by calibrating the cameras deployed along the pipeline using the Zhang Zhengyou calibration method.

[0080] The mapping relationship from the real physical coordinate system to the image coordinate system is constructed using the following formula:

[0081] ,

[0082] in, The horizontal coordinate of the image coordinate system The vertical coordinate of the image coordinate system. It is a homogeneous scaling factor. For rotation matrix, It is a translation vector. The x-axis represents the actual physical coordinate system. The vertical coordinate is the actual physical coordinate system. These are the height coordinates in the actual physical coordinate system.

[0083] A3. Based on the mapping relationship from the real physical coordinate system to the image coordinate system, the risk areas of different levels in the real physical coordinate system are mapped to the image coordinate system to obtain the risk areas of different levels in the pipeline scene image.

[0084] A4. Using the following formula, generate a region prior weight map based on different levels of risk regions in the pipeline scene image:

[0085] ,

[0086] ,

[0087] in, For the region prior weight map in image coordinates The value, This represents the highest-risk area in the pipeline scene images. This is a medium-risk area in the scene image along the pipeline. This refers to the outer observation area in the pipeline scene image. The preset maximum risk weight value, The preset medium risk weight value, This is the preset minimum risk weight value.

[0088] In this embodiment, the following settings are provided: It is 1.5. =1, The value is set to 0.5, which makes the risk zones at different levels clearly distinguishable in terms of weight.

[0089] The pipeline scene modeling module is based on the principle of spatial coordinate mapping. It accurately maps the multi-level risk areas of the real physical coordinate system to the image coordinate system and generates a prior weight map of the region. This provides clear spatial constraints for subsequent target detection, effectively filters invalid interference targets in non-pipeline protection areas, and significantly reduces false alarms caused by factors such as dust, changes in light and shadow, and activities of unrelated vehicles and personnel, thereby improving the system's anti-interference ability and scene adaptability.

[0090] like Figure 2 As shown, the target enhancement detection module includes: a backbone feature extraction network, a cross-scale feature alignment and fusion submodule, a region attention submodule, a conventional target detection head, and a small target detection head.

[0091] In this embodiment, the backbone feature extraction network uses a lightweight deep neural network, MobileNetV2, to obtain multi-layer feature maps of different scales (in this embodiment, it is set to 4 layers of feature maps) based on the currently input pipeline scene image:

[0092] The size of the first layer feature map is 1 / 4 of the scene image along the pipeline, and it is characterized by containing detailed features of small targets; the size of the second layer feature map is 1 / 8 of the scene image along the pipeline; the size of the third layer feature map is 1 / 16 of the scene image along the pipeline; and the size of the fourth layer feature map is 1 / 32 of the scene image along the pipeline, and it is characterized by containing semantic information of the targets.

[0093] The cross-scale feature alignment and fusion submodule is used to perform cross-scale feature alignment and fusion on multi-layer feature maps to obtain a cross-scale fused feature map. In this embodiment, the method includes the following steps:

[0094] B1. By using bilinear upsampling, the dimensions of the feature maps of layers 2 to 4 are aligned with those of the feature map of layer 1 to obtain the extended feature maps of layers 2 to 4.

[0095] B2. Process the first-layer feature map and the extended feature maps from layers 2 to 4 using the following formula to obtain the cross-scale fused feature map:

[0096] ,

[0097] in, For cross-scale fusion of feature maps, This is the first layer feature map. to These are the extended feature maps from layers 2 to 4, respectively. to These are the weighting coefficients for features 1 through 4, respectively. This is an element-wise addition.

[0098] Considering that the most important feature maps in each layer are the fourth and first layers, therefore In this embodiment, Set to 0.4, Set to 0.3, Set to 0.2, Set it to 0.1.

[0099] The region attention submodule is used to first downsample the region prior weight map to the cross-scale fusion feature through max pooling. Figure 1 The size of the feature map is then multiplied element-wise with the cross-scale fusion feature map to obtain the attention-weighted feature map, which is expressed as:

[0100] ,

[0101] in, For attention-weighted feature maps, For max pooling operation, For the region prior weight map, For element-wise multiplication, This is a cross-scale fusion feature map.

[0102] The standard object detection head includes two convolutional layers and one convolutional output layer connected in series, used to obtain the detection box, category, and confidence score of each object in the current attention-weighted feature map.

[0103] In this embodiment, the detailed parameters of the first three layers of conventional target detection are as follows:

[0104] First layer (convolutional layer): 3×3 convolution, 256 output channels, stride 1, padding=1, the output is first normalized by BN, and then activated by SiLU.

[0105] The second layer (convolutional layer) is a 3×3 convolution with 256 output channels, a stride of 1, and padding of 1. The output is first normalized by batch normalization (BN) and then activated by the SiLU activation function.

[0106] The third layer (convolutional output layer) is a 1×1 convolution with 11 output channels (the x-coordinate of the top left corner of the detection box, the y-coordinate of the top left corner of the detection box, the width of the detection box, the height of the detection box, the confidence score, and 0 / 1 labels for 6 target categories).

[0107] The small target detection head includes three convolutional layers and one convolutional output layer connected in series, used to obtain the detection box, category and confidence of each target in the current attention-weighted feature map.

[0108] In this embodiment, the detailed parameters of the four layers of the small target detection head are as follows:

[0109] First layer (convolutional layer): 3×3 convolution, 256 output channels, stride 1, padding=1, the output is first normalized by BN, and then activated by SiLU.

[0110] The second layer (convolutional layer) is a 3×3 convolution with 256 output channels, a stride of 1, and padding of 1. The output is first normalized by batch normalization (BN) and then activated by the SiLU activation function.

[0111] The third layer (convolutional layer) is a 3×3 convolution with 256 output channels, a stride of 1, and padding of 1. The output is first normalized by batch normalization (BN) and then activated by the SiLU activation function.

[0112] The fourth layer (convolutional output layer) is a 1×1 convolution with 11 output channels (the x-coordinate of the top left corner of the detection box, the y-coordinate of the top left corner of the detection box, the width of the detection box, the height of the detection box, the confidence score, and 0 / 1 labels for 6 target categories).

[0113] This embodiment sets six categories: personnel, vehicles, construction machinery, flames, smoke, and animals. If the target belongs to the corresponding category, the value is 1; otherwise, it is 0.

[0114] The small target detection head of this invention has one more convolutional layer than the conventional target detection head, in order to better extract fine features and identify small targets.

[0115] Since the small target detection head and the regular target detection head are completely parallel, after a target is detected, if the size of the target is smaller than a preset size threshold (set to 32×32 pixels in this embodiment), the detection box, category, and confidence score of the small target detection head are adopted as the output; if the size of the target is greater than or equal to the size threshold, the detection box, category, and confidence score of the regular target detection head are adopted as the output.

[0116] The target enhancement detection module uses the following total detection loss function for labeled supervised learning:

[0117] ,

[0118] ,

[0119] ,

[0120] ,

[0121] in, Let be the total detection loss function. For classification loss function, For regression loss function, For confidence loss function, The total number of samples used for supervised learning. The sample number used for supervised learning. This represents the total number of target categories (6 in this example). For the target category number, For the first Does the sample target belong to the first The class label (1 if it is, 0 otherwise). The target enhancement detection module is used for the first Does the sample target belong to the first The class discrimination result, Let be a logarithmic function with the natural constant as its base. For a complete intersection-union function, The detection bounding box output by the target enhancement detection module. The actual label of the detection box. The confidence weighting coefficient is... For the first Confidence labels for the sample The target enhancement detection module is used for the first The confidence level of the sample is output.

[0122] This embodiment focuses on the confidence weighting coefficient. If the target is 32×32 pixels or larger, set it to 1; otherwise, set it to 3. This weights the positive samples of small targets by 3 times, alleviating the problem of imbalanced positive and negative samples.

[0123] The target enhancement detection module adopts a cross-scale feature alignment fusion method, which fully integrates shallow detail features and deep semantic features. At the same time, it strengthens the feature weight of high-risk areas through a region attention mechanism and sets up a dual detection head for regular and small targets to differentiate detection. This addresses the problem of weak features and easy omission of small targets at long distances in pipeline scenarios, and achieves high-precision and robust detection of targets of various sizes such as personnel, construction machinery, flames, and smoke, significantly reducing the false negative rate.

[0124] The spatiotemporal tracking module identifies different targets based on their detection bounding boxes, categories, and confidence levels during the sampling period, and uses trajectory-spatial temporal coding to obtain the behavior categories of each different target. The method includes the following steps:

[0125] C1. Remove targets with a confidence level lower than a preset confidence threshold (set to 0.6 in this embodiment) from each target during the sampling period to eliminate uncertain target detection results.

[0126] C2. Map the detection boxes of the remaining targets to the corresponding pipeline scene image, and extract the regions within each of the mapped detection boxes as each appearance feature block.

[0127] C3. Calculate the cosine similarity between each appearance feature block, and record the appearance feature blocks with a cosine similarity greater than the preset similarity threshold (set to 0.7 in this embodiment) as the same target, thus identifying each different target.

[0128] C4. Use the image coordinates of the geometric center of each appearance feature block as the position coordinates of the corresponding target.

[0129] C5. Perform trajectory-spatial temporal encoding on the position coordinates of the same target at different times during the sampling period to generate behavioral feature vectors for each different target, including the following steps:

[0130] C51. Combine the position coordinates of the same target at different times during the sampling period to obtain the temporal feature vector sets of each different target:

[0131] ,

[0132] ,

[0133] in, For the target identity number, For time sequence number, The duration of the sampling period. For the first The temporal feature vector set of the target, For the first Target The temporal eigenvector of time, For the first Target The horizontal axis of time position For the first Target The vertical axis representing the location of time. For the first Target Spatial region value of time;

[0134] In this embodiment, the first Target Spatial region value of time The setting method is as follows: If the first Target If the time period is in the highest risk zone, set it to 3; if it is in the medium risk zone, set it to 2; if it is in the outer observation zone, set it to 1; otherwise, set it to 0.

[0135] C52. Feed the temporal feature vector sets of different targets into the LSTM network to generate behavioral feature vectors of different targets:

[0136] ,

[0137] ,

[0138] in, For the first The target's behavioral feature vector For the first Target The time-varying LSTM network encoding vector. For the first Target The time-varying LSTM network encoding vector. For the first The LSTM network encoding vector at the last moment of the target sampling period. It is an LSTM network.

[0139] C6. Process the behavioral feature vectors of different targets through a fully connected network to obtain the behavioral categories of different targets.

[0140] The fully connected network in this embodiment uses two fully connected layers with linear transformations and an output layer with the sigmoid function as the activation function. The output layer has 6 channels, which correspond to 6 target behavior categories: "suspected mining", "staying in the highest risk area", "crossing", "lingering in the medium risk area", "approaching the outer observation area", and "staying away", and are still labeled with 0 / 1.

[0141] The spatiotemporal tracking module involves the use of LSTM and fully connected networks, therefore requiring supervised learning training with manual labels. This embodiment uses the common cross-entropy loss function, so it will not be elaborated further.

[0142] The spatiotemporal tracking module achieves accurate target identification through confidence screening and appearance feature similarity matching. By combining trajectory-spatial temporal coding and LSTM network, it integrates the temporal location of the target with spatial risk information, accurately identifying sensitive behaviors such as suspected digging, crossing, and stopping in pipelines. This overcomes the shortcomings of traditional single-frame appearance recognition, which is inaccurate and cannot distinguish between harmless passing and dangerous intrusion, thus achieving temporal and accurate behavior recognition.

[0143] The risk warning module calculates the comprehensive risk value of each different target based on its category and behavior, and outputs a warning signal using the following steps:

[0144] D1. Quantify and score the categories and behaviors of the same objective, and obtain the comprehensive risk value for each different objective by weighting them using the following formula:

[0145] ,

[0146] in, For the target identity number, For the first The overall risk value of the target, For the target category risk coefficient, For the first Quantitative scoring of target categories, Risk coefficient for target behavior category For the first Quantitative scoring of target behavior categories.

[0147] In this embodiment, the quantitative scores set for the six target categories are as follows:

[0148] Construction machinery 1, personnel 0.8, vehicles 0.6, flames 0.5, smoke 0.5, animals 0;

[0149] The quantitative scores for the six target behavior categories are as follows:

[0150] Suspected excavation 1, highest risk zone stay 0.8, cross 0.6, medium risk zone linger 0.4, near the outer observation zone 0.2, far away 0;

[0151] Target category risk coefficient Set to 0.4, the risk coefficient for the target behavior category. Set it to 0.6.

[0152] D2. Based on the comprehensive risk value of each different target, output graded early warning signals.

[0153] If the maximum combined risk value of all targets is greater than or equal to 0.85, the highest level alarm will be triggered immediately, and the emergency response mechanism will be activated.

[0154] If the maximum comprehensive risk value of each target is greater than or equal to 0.6 and less than 0.85, an audible and visual alarm will be triggered, and inspection personnel will be dispatched to conduct on-site verification.

[0155] If the maximum combined risk value of each target is greater than or equal to 0.3 and less than 0.6, an early warning message will be sent to the on-duty personnel.

[0156] If the maximum combined risk value of each target is less than 0.3, only log entries will be recorded and no alarms will be triggered.

[0157] The risk warning module combines target category and behavior category for quantitative weighting and outputs graded warning signals to achieve accurate risk classification and differentiated handling. This avoids the one-sidedness of single-dimensional risk judgment, ensures rapid response to high-risk intrusion events, reduces invalid alarms for low-risk events, balances the timeliness and accuracy of warnings, reduces operation and maintenance management costs, and improves the efficiency of pipeline safety emergency response.

[0158] In summary, this invention achieves integrated processing for precise spatial risk classification of targets along pipelines, efficient detection of small targets at long distances, temporal identification of target behavior, and risk classification and early warning. It does not rely on additional auxiliary equipment or manual intervention. From the system architecture level, it solves the core pain points of existing technologies, such as lack of spatial constraints, insufficient detection accuracy, weak behavior recognition, and poor early warning effectiveness. It significantly improves the completeness and systematicness of abnormal intrusion monitoring along pipelines and is suitable for the real-time security needs of complex field scenarios for various long-distance pipelines such as oil and gas, water conservancy, and gas.

[0159] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A machine vision-based system for detecting and recognizing abnormal intrusion targets along a pipeline, characterized in that, include: The pipeline scene modeling module is used to divide the risk areas of different levels in the pipeline scene image according to the principle of spatial coordinate mapping, and generate the region prior weight map. The target enhancement detection module is used to process each frame of the pipeline scene image captured continuously during the sampling period through a cross-scale feature alignment fusion method and a region attention mechanism controlled by the region prior weight map, and obtains the detection box, category and confidence of each target during the sampling period through dual detection heads; The spatiotemporal tracking module is used to identify different targets based on the detection boxes, categories, and confidence levels of each target during the sampling period, and to obtain the behavior categories of each different target by using trajectory-spatial temporal coding; The risk warning module is used to calculate the comprehensive risk value of each different target based on its category and behavior, and output a warning signal.

2. The machine vision-based pipeline abnormal intrusion target detection and behavior recognition system according to claim 1, characterized in that, The pipeline scene modeling module, based on the principle of spatial coordinate mapping, divides risk regions of different levels in the pipeline scene image and generates a region prior weight map. The method includes the following steps: A1. Preset different levels of risk areas in the real physical coordinate system; A2. Obtain the intrinsic parameter matrix, rotation matrix, translation vector, and homogeneous scale factor of the cameras deployed along the pipeline, and construct the mapping relationship from the real physical coordinate system to the image coordinate system. A3. Based on the mapping relationship from the real physical coordinate system to the image coordinate system, the risk areas of different levels in the real physical coordinate system are mapped to the image coordinate system to obtain the risk areas of different levels in the pipeline scene image. A4. Generate a prior weight map of the region based on the different levels of risk areas in the scene image along the pipeline.

3. The machine vision-based pipeline abnormal intrusion target detection and behavior recognition system according to claim 1, characterized in that, The target enhancement detection module includes: A backbone feature extraction network is used to obtain multi-layer feature maps of different scales based on the currently input pipeline scene image; The cross-scale feature alignment and fusion submodule is used to perform cross-scale feature alignment and fusion on multi-layer feature maps to obtain a cross-scale fused feature map. The region attention submodule is used to first downsample the region prior weight map to the same size as the cross-scale fusion feature map through max pooling, and then multiply it element-wise with the cross-scale fusion feature map to obtain the attention-weighted feature map. A standard object detection head includes two convolutional layers and one convolutional output layer connected in series, used to obtain the detection box, category, and confidence of each object in the current attention-weighted feature map; The small target detection head includes three convolutional layers and one convolutional output layer connected in series, used to obtain the detection box, category and confidence of each target in the current attention-weighted feature map.

4. The machine vision-based pipeline abnormal intrusion target detection and behavior recognition system according to claim 3, characterized in that, When the size of the target is less than a preset size threshold, the target enhancement detection module adopts the detection box, category, and confidence score of the small target detection head as output; when the size of the target is greater than or equal to the size threshold, it adopts the detection box, category, and confidence score of the conventional target detection head as output.

5. The machine vision-based pipeline abnormal intrusion target detection and behavior recognition system according to claim 3, characterized in that, The method for the cross-scale feature alignment and fusion submodule to perform cross-scale feature alignment and fusion on multi-layer feature maps to obtain a cross-scale fused feature map includes the following steps: B1. By using bilinear upsampling, the dimensions of the feature maps from the second to the last layer are aligned with the feature map of the first layer to obtain the extended feature maps from the second to the last layer. B2. Add the first layer feature map and the extended feature maps from the second to the last layer element by element with weights to obtain the cross-scale fused feature map.

6. The machine vision-based pipeline abnormal intrusion target detection and behavior recognition system according to claim 3, characterized in that, The target enhancement detection module uses the following total detection loss function for labeled supervised learning: ， ， ， ， in, Let be the total detection loss function. For classification loss function, For regression loss function, For confidence loss function, The total number of samples used for supervised learning. The sample number used for supervised learning. The total number of target categories, For the target category number, For the first Does the sample target belong to the first Class tags, The target enhancement detection module is used for the first Does the sample target belong to the first The class discrimination result, Let be a logarithmic function with the natural constant as its base. For a complete intersection-union function, The detection bounding box output by the target enhancement detection module. The actual label of the detection box. The confidence weighting coefficient is... For the first Confidence labels for the sample The target enhancement detection module is used for the first The confidence level of the sample is output.

7. The machine vision-based pipeline abnormal intrusion target detection and behavior recognition system according to claim 3, characterized in that, The spatiotemporal tracking module identifies different targets based on the detection bounding boxes, categories, and confidence levels of each target during the sampling period, and uses trajectory-spatial temporal coding to obtain the behavior categories of each different target. The method includes the following steps: C1. Remove targets with a confidence level lower than the preset confidence threshold from each target during the sampling period; C2. Map the detection boxes of the remaining targets to the corresponding pipeline scene image, and extract the regions within each of the mapped detection boxes as each appearance feature block. C3. Calculate the cosine similarity between each appearance feature block, and record the appearance feature blocks with a cosine similarity greater than the preset similarity threshold as the same target, thus identifying the different targets. C4. Use the image coordinates of the geometric center of each appearance feature block as the position coordinates of the corresponding target; C5. Perform trajectory-spatial temporal encoding on the position coordinates of the same target at different times during the sampling period to generate behavioral feature vectors for each different target; C6. Process the behavioral feature vectors of different targets through a fully connected network to obtain the behavioral categories of different targets.

8. The machine vision-based pipeline abnormal intrusion target detection and behavior recognition system according to claim 7, characterized in that, C5 includes the following steps: C51. Combine the position coordinates of the same target at different times during the sampling period to obtain the temporal feature vector sets of each different target: ， ， in, For the target identity number, For time sequence number, The duration of the sampling period. For the first The temporal feature vector set of the target, For the first Target The temporal eigenvector of time, For the first Target The horizontal axis of time position For the first Target The vertical axis representing the location of time. For the first Target Spatial region value of time; C52. Feed the temporal feature vector sets of different targets into the LSTM network to generate behavioral feature vectors of different targets: ， ， in, For the first The target's behavioral feature vector For the first Target The time-varying LSTM network encoding vector. For the first Target The time-varying LSTM network encoding vector. For the first The LSTM network encoding vector at the last moment of the target sampling period. It is an LSTM network.

9. The machine vision-based pipeline abnormal intrusion target detection and behavior recognition system according to claim 1, characterized in that, The risk warning module calculates the comprehensive risk value of each different target based on its category and behavior, and outputs a warning signal using the following steps: D1. Quantify and score the categories and behaviors of the same objective, and obtain the comprehensive risk value for each different objective by weighting them using the following formula: ， in, For the target identity number, For the first The overall risk value of the target, For the target category risk coefficient, For the first Quantitative scoring of target categories, Risk coefficient for target behavior category For the first Quantitative scoring of target behavior categories; D2. Based on the comprehensive risk value of each different target, output graded early warning signals.

Images