Mechanical overrunning identification method, device, equipment, medium and computer program product

Abstract

The application relates to a mechanical occupation identification method, device, equipment, medium and computer program product, which comprises the following steps: inputting the obtained pipeline inspection data into a target detection network embedded with an attention map obtained according to the spatial position information of the pipeline, obtaining the boundary box coordinates and the category confidence of the candidate target; determining the number of pixels covered by the candidate target in the inspection data according to the boundary box coordinates; determining the preliminary risk value of the candidate target according to the category confidence and the number of pixels, and taking the candidate target with the preliminary risk value greater than a preset first risk value threshold as the target; determining the motion speed vector of the target according to the inspection data; determining the distance of the target from the pipeline according to the boundary box coordinates; determining the comprehensive risk value of the target according to the distance, the motion speed vector, the number of pixels and the category confidence; and determining the determination result of the occupation event according to the comprehensive risk value. The application improves the correctness of the pipeline mechanical occupation identification.

Description

Technical Field

[0001] This application relates to the field of mechanical occupancy identification, and in particular to a mechanical occupancy identification method, device, equipment, medium, and computer program product. Background Technology

[0002] Buried pipelines are frequently subject to third-party construction, heavy vehicle traffic, and various mechanical operations. When machinery enters the pipeline's safety protection zone, it may squeeze, disturb, or damage the pipeline, creating safety hazards. To ensure the timely detection and handling of these risks, it is necessary to identify mechanical targets around the pipeline and determine whether there is a risk of obstruction based on their relative position to the pipeline; this is known as pipeline mechanical obstruction identification.

[0003] In related technologies, drones are used to collect aerial images along the pipeline route for inspection, recording corresponding geographical location information during the acquisition process. The images and geographical location information are then transmitted to a ground-based processing unit. After processing, the ground-based unit inputs image patches into a convolutional neural network for comparison with a pre-trained reference model to complete the identification and classification of mechanical targets. The system then outputs the mechanical target category and corresponding geographical location information for display and push notifications.

[0004] In the scenario of pipeline machinery occupancy identification, simply outputting the category of the machinery target and its corresponding geographical location information makes it difficult to reliably distinguish the occupancy risk status of the same type of machinery under different circumstances, including occupancy status and non-occupancy status. Therefore, it is easy to misjudge a non-occupancy status as an occupancy status, or vice versa, resulting in a low accuracy rate of pipeline machinery occupancy identification. Summary of the Invention

[0005] This application provides a method, apparatus, device, medium, and computer program product for identifying mechanical occupancy, in order to solve at least one technical problem existing in the related art.

[0006] According to a first aspect of the embodiments of this application, a method for identifying mechanical occupancy is provided, the method comprising: Based on the spatial location information of the pipeline, an attention map of the pipeline is obtained; The acquired pipeline inspection data is input into a target detection network embedded with the attention map to obtain the bounding box coordinates and class confidence of the candidate targets; the target detection network is used to complete the localization and classification of the candidate targets under the guidance of the attention map, and the bounding box coordinates are used to characterize the location range of the candidate targets; Based on the bounding box coordinates, determine the number of pixels covered by the candidate target in the inspection data; Based on the category confidence level and the number of pixels, a preliminary risk value for the candidate target is determined, and candidate targets whose preliminary risk value is greater than a preset first risk value threshold are selected as targets. Based on the inspection data, determine the target's velocity vector; Based on the bounding box coordinates, determine the distance between the target and the pipeline; The overall risk value of the target is determined based on the distance, the motion velocity vector, the number of pixels, and the category confidence level. Based on the comprehensive risk value, the determination result of the encroachment event is made.

[0007] Preferably, the step of inputting the acquired pipeline inspection data into a target detection network embedded with the attention map to obtain the bounding box coordinates and class confidence of candidate targets includes: The inspection data is input into the feature extraction backbone of the target detection network to obtain a feature map; Based on the attention map, the feature map is weighted to obtain a weighted feature map; The weighted feature map is input into the detection head of the target detection network to obtain the bounding box coordinates and the class confidence.

[0008] Preferably, the target detection network is obtained by training on training samples, which include training inspection data and category labels and bounding box annotations corresponding to the training inspection data; The loss functions used when training the object detection network include a classification loss function, a localization loss function, and a geometric consistency loss function; The classification loss function is used to characterize the degree of inconsistency between the predicted class confidence score output by the object detection network and the class label. The localization loss function is used to characterize the difference between the predicted bounding box coordinates output by the target detection network and the bounding box annotation; The geometric consistency loss function is used to impose a constraint penalty when the shortest Euclidean distance between the center point of the predicted bounding box coordinates and the center line of the pipeline is greater than a preset safety distance threshold.

[0009] Preferably, the spatial location information includes: the location information of the pipeline centerline, and the pipeline burial depth value corresponding to each location of the pipeline centerline; The step of obtaining the attention map of the pipeline based on its spatial location information includes: Based on the safety distance threshold, a two-dimensional planar buffer area is obtained by expanding the area outward with the centerline of the pipeline as the axis. Map the two-dimensional planar buffer region to a raster coordinate system with the same image resolution as the inspection data; Within the grid coordinate system, attention weight values ​​are assigned to each grid point to obtain the attention map; the attention weight values ​​decrease as the distance from each grid point to the center line of the pipeline increases.

[0010] Preferably, the inspection data is determined in the following way: While controlling the drone to fly along the centerline of the pipeline, it collects visible light images, thermal infrared images, and depth maps of the inspection area along the pipeline. While acquiring each frame of visible light image, thermal infrared image, and depth map, the coordinate data and attitude data of the UAV are acquired simultaneously. Based on the coordinate data and the attitude data, coordinate transformation is performed on the thermal infrared image and the depth map to obtain the coordinate-transformed thermal infrared image and the coordinate-transformed depth map. The thermal infrared image and the depth map after coordinate transformation are resampled into the pixel coordinate system of the visible light image to obtain the inspection data.

[0011] Preferably, determining the preliminary risk value of the candidate target based on the category confidence level and the number of pixels includes: The preliminary risk value is obtained by multiplying the ratio of the number of pixels to the number of pixels in the visible light image by the category confidence level.

[0012] Preferably, determining the comprehensive risk value of the target based on the distance, the motion velocity vector, the category confidence level, and the number of pixels includes: The mechanical type of the target is determined based on the category confidence level and the number of pixels; Determine the cosine value between the motion velocity vector and the unit direction vector; the direction of the unit direction vector is the direction from the target to the point closest to the centerline of the pipe; The motion trend of the target is determined based on the cosine value; Based on the number of pixels, determine the ratio of the number of pixels to the number of pixels in the visible light image; The comprehensive risk value is determined based on the distance, the movement trend, the ratio, and the type of machinery.

[0013] Preferably, determining the occupancy event determination result based on the comprehensive risk value includes: The second risk value threshold is determined based on the inspection time, as well as the area category and construction permit status of the inspection area along the route; The determination result is determined based on the comparison between the comprehensive risk value and the second risk value threshold.

[0014] Preferably, determining the judgment result based on the comparison between the comprehensive risk value and the second risk value threshold includes: If the construction permit status is not permitted and the comprehensive risk value is not less than the second risk value threshold, then the judgment result is determined to be a level one encroachment event, and an alarm is triggered. If the construction permit status is "allowed to construct", and the overall risk value is less than the second risk value threshold and not less than the third risk value threshold, then the judgment result is determined to be a monitoring event, and the target is added to the tracking list; if the overall risk value is less than the third risk value threshold and the pipeline is not in a shutdown state, then the judgment result is determined to be "normal operation". The third risk value threshold is determined by reducing the second risk value threshold by a preset ratio.

[0015] According to a second aspect of the embodiments of this application, a mechanical occupancy identification device is provided, the device comprising: The attention map acquisition module is used to obtain the attention map of the pipeline based on the spatial location information of the pipeline; The localization and classification module is used to input the acquired pipeline inspection data into a target detection network embedded with the attention map to obtain the bounding box coordinates and class confidence of the candidate targets; the target detection network is used to complete the localization and classification of the candidate targets under the guidance of the attention map, and the bounding box coordinates are used to characterize the location range of the candidate targets; A pixel count determination module is used to determine the number of pixels covered by the candidate target in the inspection data based on the bounding box coordinates. The target determination module is used to determine the preliminary risk value of the candidate target based on the category confidence level and the number of pixels, and to select the candidate target whose preliminary risk value is greater than a preset first risk value threshold as the target; The motion velocity vector determination module is used to determine the motion velocity vector of the target based on the inspection data. A distance determination module is used to determine the distance between the target and the pipeline based on the bounding box coordinates; The comprehensive risk value determination module is used to determine the comprehensive risk value of the target based on the distance, the motion speed vector, the number of pixels, and the category confidence level. The determination result module is used to determine the determination result of the encroachment event based on the comprehensive risk value.

[0016] According to a third aspect of the embodiments of this application, an electronic device is provided, including one or more processors, for implementing the aforementioned mechanical occupancy identification method.

[0017] According to a fourth aspect of the embodiments of this application, a computer-readable storage medium is provided, on which a program is stored, which, when executed by a processor, implements the aforementioned mechanical occupancy identification method.

[0018] According to a fifth aspect of the embodiments of this application, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement the aforementioned mechanical occupancy identification method.

[0019] The technical solutions provided by the embodiments of this application may include the following beneficial effects: The mechanical occupancy identification method of this application obtains an attention map of the pipeline based on its spatial location information. Inspection data is then input into a target detection network embedded with the attention map. Guided by the attention map, the target detection network locates and classifies candidate targets, obtaining their bounding box coordinates and class confidence scores. In this way, areas near the pipeline receive more attention, while areas far from the pipeline receive less attention. This reduces computation for areas far from the pipeline and minimizes the possibility of missing candidate targets in areas near the pipeline, thus improving the accuracy of candidate target location and classification, and consequently increasing the accuracy of pipeline mechanical occupancy identification.

[0020] Furthermore, the number of pixels covered by candidate targets in the inspection data is determined based on the bounding box coordinates. Combining the category confidence level with the number of pixels, a preliminary risk value for the candidate targets is determined. Then, the candidate targets are filtered based on the preliminary risk value exceeding a preset first risk threshold to obtain targets for subsequent occupancy event determination. This filtering process reduces the probability of candidate targets with low category confidence or small pixel counts entering subsequent determinations, further improving the accuracy of pipeline mechanical occupancy identification.

[0021] Finally, the target's velocity vector is determined based on the inspection data, and the distance between the target and the pipeline is calculated using the bounding box coordinates. Based on this distance, velocity vector, number of pixels, and category confidence, the target's comprehensive risk value is determined, and an occupation event is identified based on this comprehensive risk value. In this process, the proximity of the target to the pipeline, the proximity trend, the target's size, and the target's category are comprehensively considered, further improving the accuracy of pipeline mechanical occupation identification.

[0022] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this application. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a schematic flowchart illustrating a mechanical occupancy identification method according to an embodiment of this application; Figure 2 yes Figure 1 A flowchart illustrating step S102 in the mechanical occupancy identification method shown; Figure 3 yes Figure 1 A flowchart illustrating step S101 in the mechanical occupancy identification method shown; Figure 4 yes Figure 1 A flowchart illustrating the method for determining inspection data in step S102 of the mechanical occupancy identification method shown; Figure 5 yes Figure 1 A flowchart illustrating step S107 in the mechanical occupancy identification method shown; Figure 6 yes Figure 1 A flowchart illustrating step S108 in the mechanical occupancy identification method shown; Figure 7 This is a schematic diagram of a mechanical occupancy identification device according to an embodiment of this application; Figure 8 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0025] The technical solutions in the embodiments of this application will be clearly and completely described herein with reference to the accompanying drawings. In the following description, when referring to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements.

[0026] If the embodiments of this application contain terms relating to directional indications or positional relationships, such as up, down, back, inside, center, etc., these terms are only used to explain the relative positional relationships and movement of the components in a specific posture; if the specific posture changes, the directional indications or positional relationships will also change accordingly. Furthermore, the terms "first," "second," etc., used in the embodiments of this application are for descriptive convenience only and should not be construed as indicating or implying relative importance.

[0027] This application provides a method, apparatus, device, medium, and computer program product for identifying mechanical occupancy. The mechanical occupancy identification method of this application will be described in detail below with reference to the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0028] Figure 1 This application illustrates a method for identifying mechanical occupancy according to an exemplary embodiment. For example... Figure 1 As shown, the method includes steps S101 to S108: S101. Obtain the attention map of the pipeline based on its spatial location information.

[0029] S102. Input the obtained pipeline inspection data into the target detection network with an embedded attention map to obtain the bounding box coordinates and class confidence of the candidate targets.

[0030] The object detection network is used to locate and classify candidate objects under the guidance of the attention map, and the bounding box coordinates are used to represent the location range of the candidate objects.

[0031] S103. Based on the bounding box coordinates, determine the number of pixels covered by the candidate target in the inspection data.

[0032] S104. Based on the category confidence level and the number of pixels, determine the preliminary risk value of the candidate target, and select the candidate target whose preliminary risk value is greater than the preset first risk value threshold as the target.

[0033] S105. Determine the target's velocity vector based on the inspection data.

[0034] S106. Determine the distance between the target and the pipe based on the bounding box coordinates.

[0035] S107. Determine the overall risk value of the target based on distance, motion velocity vector, number of pixels, and category confidence.

[0036] S108. Determine the judgment result of the encroachment event based on the comprehensive risk value.

[0037] The mechanical occupancy identification method in this embodiment obtains an attention map of the pipeline based on its spatial location information. Inspection data is then input into a target detection network embedded with the attention map. Guided by the attention map, the target detection network locates and classifies candidate targets, obtaining their bounding box coordinates and category confidence scores. In this way, areas near the pipeline receive more attention, while areas far from the pipeline receive less attention. This reduces computation for areas far from the pipeline and minimizes the possibility of missing candidate targets in areas near the pipeline, improving the accuracy of candidate target location and classification, and thus increasing the accuracy of pipeline mechanical occupancy identification.

[0038] Furthermore, the number of pixels covered by candidate targets in the inspection data is determined based on the bounding box coordinates. Combining the category confidence level with the number of pixels, a preliminary risk value for the candidate targets is determined. Then, the candidate targets are filtered based on the preliminary risk value exceeding a preset first risk threshold to obtain targets for subsequent occupancy event determination. This filtering process reduces the probability of candidate targets with low category confidence or small pixel counts entering subsequent determinations, further improving the accuracy of pipeline mechanical occupancy identification.

[0039] Finally, the target's velocity vector is determined based on the inspection data, and the distance between the target and the pipeline is calculated using the bounding box coordinates. Based on this distance, velocity vector, number of pixels, and category confidence, the target's comprehensive risk value is determined, and an occupation event is identified based on this comprehensive risk value. In this process, factors such as the proximity of the target to the pipeline, the proximity trend, and the target's size are comprehensively considered, further improving the accuracy of pipeline mechanical occupation identification.

[0040] In one alternative embodiment, such as Figure 2 As shown, S102 includes steps S1021 to S1023: S1021. Input the inspection data into the feature extraction backbone of the target detection network to obtain the feature map.

[0041] Specifically, the object detection network includes a feature extraction backbone and a detection head; the feature extraction backbone can be built based on a convolutional neural network to extract features from the inspection data and output multi-scale feature maps. The feature maps corresponding to each level can be represented as follows: ;in, This indicates the layer number of the feature map, used to distinguish the feature maps output from different convolutional layers or different convolutional stages in the feature extraction backbone.

[0042] S1022. Based on the attention map, the feature map is weighted to obtain the weighted feature map.

[0043] Specifically, in object detection networks, feature representations are modulated using a spatial gating mechanism. Spatial gating refers to using an attention map to weight feature values ​​at different spatial locations in the feature map, selectively enhancing or suppressing these values. Specifically, the attention map is upsampled to the same spatial size as the feature map and multiplied element-wise with it to obtain the weighted feature map. The weighted feature map corresponding to each level can be expressed by the following formula: in, Indicates the first Weighted feature maps corresponding to each level; Indicates the first Feature maps corresponding to each level; Indicates upsampling to the feature map Attention maps with the same spatial dimensions; This indicates element-wise multiplication.

[0044] attention map In spatial location The eigenvalue at that location is represented as Among them, in the attention map With feature map Given the same spatial dimensions, spatial location With feature map The corresponding spatial positions in the text correspond one-to-one.

[0045] Used to characterize feature maps The importance of corresponding spatial locations is determined by the element-wise multiplication of the attention map and the feature map. Therefore, after multiplying the attention map and the feature map element by element, the feature values ​​of the same spatial locations in the feature map corresponding to spatial locations with larger feature values ​​in the attention map will be enhanced, while the feature values ​​of the same spatial locations in the feature map corresponding to spatial locations with smaller feature values ​​in the attention map will be suppressed.

[0046] S1023. Input the weighted feature map into the detection head of the object detection network to obtain the bounding box coordinates and class confidence.

[0047] Specifically, the detection head detects candidate targets based on the weighted feature map, outputting the bounding box coordinates and class confidence score for each candidate target. The bounding box coordinates characterize the location range of the candidate target in the inspection data. The class confidence of each candidate target can be expressed as: ;in, This is the ID of the candidate target. Category confidence. It inherently contains category information. The object detection network will calculate the first... The probability of each candidate target corresponding to each machinery category is calculated, and the highest probability value is selected as the category confidence score. The value, The value range is [0, 1]. For example, when the first... When the probability of a candidate target being predicted as an excavator is 0.95, the class confidence score is... The value is .

[0048] By assigning different weights to different spatial locations in the feature map using the attention map method described above, the feature values ​​corresponding to spatial locations that are more important for identifying pipeline machinery occupancy are enhanced, while the feature values ​​corresponding to spatial locations that are less important for pipeline machinery occupancy are suppressed. This results in the weighted feature map highlighting features that are more conducive to identifying pipeline machinery occupancy. Based on this weighted feature map, the detection head can more accurately output the bounding box coordinates and class confidence of candidate targets, thereby improving the accuracy of pipeline machinery occupancy identification.

[0049] In an optional embodiment, the object detection network in S102 is trained using training samples, which include training inspection data and corresponding category labels and bounding box annotations. The loss function used to train the object detection network includes a classification loss function, a localization loss function, and a geometric consistency loss function.

[0050] The classification loss function is used to characterize the degree of inconsistency between the predicted class confidence and the class label output by the object detection network.

[0051] Specifically, the classification loss function can be expressed by the following formula: in, Represents the classification loss function; This indicates the total number of candidate targets in the current batch; Indicates the total number of categories; Indicates the first A binary indicator for each candidate target; The output of the object detection network represents the first... The candidate targets belong to the category The confidence level of the predicted category.

[0052] The total number of categories refers to the total number of all categories in an object detection task, including... There is one machine category and one background category. The machine category refers to the specific object category that needs to be detected in the target detection task, such as excavator, crane, bulldozer, etc. The background category refers to the area in the training inspection data that does not belong to the machine category, such as open ground, roads, buildings, etc.

[0053] When the The category labels for the candidate targets are hour, When the first The category label of each candidate target is not At that time, In the classification loss function, for the th... For each candidate target, the loss is 0 when the class label does not match the predicted class. hour, It is not included in the loss calculation; classification loss only exists when the class label matches the predicted class. For the th Given 1 candidate target, and assuming the class label matches the predicted class, the predicted class confidence score is... The larger, The smaller the value, the smaller the classification loss, indicating that the predicted class confidence output by the object detection network is more consistent with the class label; when the predicted class confidence corresponding to the class label... The smaller, The larger the value, the greater the classification loss, indicating a higher degree of inconsistency between the predicted class confidence and the class label output by the object detection network.

[0054] The localization loss function is used to characterize the difference between the predicted bounding box coordinates output by the object detection network and the bounding box annotation.

[0055] Specifically, the localization loss function can be expressed by the following formula: in, This represents the localization loss function; Indicates the number of positive samples; Indicates the first The positive sample in the th... Predicted bounding box coordinates on each coordinate component; Indicates the first The positive sample in the th... Boundary box annotations on each coordinate component; The x-coordinate of the center point of the predicted bounding box coordinates and ordinate And the width of the predicted bounding box coordinates. and height .

[0056] in, This can be expressed by the following formula: in, This represents the difference between the predicted bounding box coordinates and the bounding box annotations.

[0057] Positive samples refer to candidate targets that have corresponding category labels and bounding box annotations.

[0058] In the localization loss function, the closer the predicted bounding box coordinates are to the bounding box annotations, the smaller the localization loss. When the difference between the predicted bounding box coordinates and the bounding box annotations is small, the localization loss increases with the increase of the difference, allowing for more precise adjustments to smaller deviations. When the difference between the predicted bounding box coordinates and the bounding box annotations is large, the localization loss still increases with the increase of the difference, but the growth is more gradual, reducing the impact of larger deviations on the training process.

[0059] The geometric consistency loss function is used to penalize situations where the shortest Euclidean distance between the center point of the predicted bounding box coordinates and the center line of the pipe exceeds a preset safe distance threshold.

[0060] Specifically, when training the object detection network, a geometric consistency loss term can be added to the loss function corresponding to the detection head. The geometric consistency loss function is defined by... The shortest Euclidean distance from the center point of the predicted bounding box coordinates of each candidate target to the centerline of the pipeline is used as input. The geometric consistency loss function can be expressed by the following formula: in, Represents the geometric consistency loss function; This indicates the total number of candidate targets in the current batch; Indicates the first The shortest Euclidean distance from the center point of the predicted bounding box coordinates of each candidate target to the center line of the pipeline; This indicates the preset safe distance threshold.

[0061] In the geometric consistency loss function, when At that time, the first The center point of the predicted bounding box coordinates corresponding to each candidate target satisfies the preset safe distance constraint. The corresponding geometric consistency loss is 0; when hour, The larger the value, the greater the loss of geometric consistency. This is because a squared form is used. The greater the distance exceeds the preset safety distance threshold, the more significant the corresponding penalty increase. Therefore, the closer the shortest Euclidean distance from the center point of the predicted bounding box coordinates to the pipe centerline is to or less than the preset safety distance threshold, the smaller the geometric consistency loss; the greater the distance exceeds the preset safety distance threshold, the greater the geometric consistency loss.

[0062] During training, the object detection network can be trained by jointly optimizing the classification loss function, localization loss function, and geometric consistency loss function. The total loss function can be expressed by the following formula: in, Represents the total loss function; Represents the classification loss function; This represents the localization loss function; Represents the geometric consistency loss function; , and These represent the weight coefficients corresponding to the classification loss function, localization loss function, and geometric consistency loss function, respectively.

[0063] In the total loss function, the classification loss function, localization loss function, and geometric consistency loss function are weighted and summed to enable the object detection network to simultaneously optimize the consistency between predicted class confidence and class label, the consistency between predicted bounding box coordinates and bounding box annotations, and the constraint relationship between the shortest Euclidean distance from the center point of the predicted bounding box coordinates to the pipe centerline and the preset safe distance threshold during training. As the value increases, the impact of the classification loss function on the total loss function increases; when... As the value increases, the impact of the localization loss function on the total loss function increases; when As the geometric consistency loss function increases, its impact on the total loss function also increases. Therefore, the gradient descent algorithm can be used to train the object detection network end-to-end based on the total loss function.

[0064] It should be noted that the specific expressions for the classification loss function, localization loss function, geometric consistency loss function, and total loss function mentioned above are just one implementation. Under the following principles, their specific function forms, parameter values, and weight coefficients can all be adjusted: the more consistent the predicted class confidence with the class label, the smaller the value of the classification loss function; the closer the predicted bounding box coordinates are to the bounding box annotations, the smaller the value of the localization loss function; the more the shortest Euclidean distance from the center point of the predicted bounding box coordinates to the center line of the pipe exceeds a preset safe distance threshold, the larger the value of the geometric consistency loss function.

[0065] By employing the above method, when training the object detection network, not only are the predicted class confidence and predicted bounding box coordinates constrained by class labels and bounding box annotations respectively, but the geometric consistency loss function is also used to constrain the shortest Euclidean distance from the center point of the predicted bounding box coordinates to the centerline of the pipeline. Since there is a geometric relationship between the position of the candidate target and the spatial position of the pipeline in the pipeline machinery occupancy scenario, the introduction of the geometric consistency loss function allows the object detection network to learn spatial constraints related to the safety distance threshold while learning class features and bounding box features. This reduces prediction results inconsistent with the actual geometric relationship of the scene, improves the accuracy of predicted class confidence and predicted bounding box coordinates, and ultimately enhances the accuracy and reliability of pipeline machinery occupancy identification.

[0066] In an optional embodiment, the spatial location information in S101 includes: the location information of the pipeline centerline, and the pipeline burial depth value corresponding to each location of the pipeline centerline. The location information of the pipeline centerline can be centerline vector data. , characterizing the geometric shape of the pipe in space; where, This represents the arc length parameter along the pipeline's direction. The pipeline burial depth can be expressed as... This represents the vertical position of the pipeline, i.e., the depth of the pipeline above the ground, and can be obtained based on actual measurement data. For example... Figure 3 As shown, S101 includes steps S1011 to S1013: S1011. Based on the safety distance threshold, expand the area outward with the centerline of the pipeline as the axis to obtain a two-dimensional planar buffer area.

[0067] Specifically, the safety distance threshold is determined by the pipeline safety protection distance specified by the industry, which is usually 50 meters.

[0068] S1012. Map the two-dimensional planar buffer area to a raster coordinate system that matches the image resolution of the inspection data.

[0069] Specifically, this step ensures that the two-dimensional planar buffer area is aligned with the image resolution of the inspection data, facilitating subsequent analysis and processing.

[0070] S1013. In the grid coordinate system, assign attention weight values ​​to each grid point to obtain an attention map. The attention weight values ​​decrease as the distance from each grid point to the pipe centerline increases.

[0071] Specifically, for each grid point in the grid coordinate system, an attention weight value is assigned. The magnitude of this attention weight value decreases as the distance from the grid point to the pipe centerline increases. Specifically, the attention weight value can be assigned according to a Gaussian decay distribution to ensure that the attention weight value is inversely proportional to the distance; the closer the area is to the pipe centerline, the higher the attention weight value, and the farther the area is from the pipe, the lower the attention weight value.

[0072] In this way, a mask with spatial weights can be obtained, which is the attention map. The attention map serves as a guiding signal for channel attention or spatial attention in the subsequent inference stage of the object detection network. This causes the object detection network to focus on the area near the pipe location and suppress the attention given to areas far away from the pipe location, thereby improving the accuracy and efficiency of the object detection network in locating and classifying candidate objects, and thus improving the accuracy of pipe mechanical occupancy identification.

[0073] This method focuses the attention of the object detection network on the region near the pipe location by narrowing the search space and improving the signal-to-noise ratio. Specifically, the object detection network focuses on the region near the pipe location, avoiding excessive computation on regions far from the pipe location, thereby improving processing speed. At the same time, the region near the pipe location receives higher attention, enhancing the object detection network's ability to identify and locate candidate targets in this region, effectively avoiding interference from regions far from the pipe location, and improving the accuracy and reliability of identifying mechanical occupancy of the pipe.

[0074] In one alternative embodiment, such as Figure 4 As shown, the inspection data in S102 is determined in the following way: S001. Control the UAV to fly along the centerline of the pipeline and collect visible light images, thermal infrared images and depth maps of the pipeline inspection area.

[0075] Specifically, the drone's flight path is set to follow the centerline of the pipeline. The drone's flight path remains parallel to the pipeline's direction, ensuring it is always aligned with the pipeline for inspection. During flight, a visible light camera can simultaneously capture visible light images. Thermal infrared cameras acquire thermal infrared images. The depth perception module collects depth maps. Visible light images Thermal infrared images are used to provide shape and texture information about the environment. Used to obtain temperature distribution information, depth map This is used to obtain spatial depth information for each pixel, further enhancing the spatial perception capability of candidate targets.

[0076] S002. While acquiring each frame of visible light image, thermal infrared image and depth map, simultaneously acquire the coordinate data and attitude data of the UAV.

[0077] Specifically, coordinate data can be the UAV's spatial position coordinates obtained through GNSS (Global Navigation Satellite System), and attitude data can be the UAV's attitude information obtained through IMU (Inertial Measurement Unit). Together, they constitute the pose information of each frame of image, used for subsequent image spatial coordinate alignment. The pose information of the i-th frame can be expressed by the following formula: in, This represents the pose information of the i-th frame image; , and This indicates the spatial coordinates of the drone; , and These represent the drone's heading, pitch angle, and roll angle, respectively.

[0078] S003. Based on the coordinate data and attitude data, perform coordinate transformation on the thermal infrared image and depth map to obtain the coordinate-transformed thermal infrared image and the coordinate-transformed depth map.

[0079] Specifically, the thermal infrared image and depth map are first projected and transformed based on the pose information of each frame to adjust the coordinates and ensure that they are in the same coordinate system as the visible light image.

[0080] S004. Resample the coordinate-transformed thermal infrared image and coordinate-transformed depth map into the pixel coordinate system of the visible light image to obtain inspection data.

[0081] Specifically, after coordinate transformation, the thermal infrared image and depth map data are aligned with the visible light image. Next, resampling techniques are used to transform the thermal infrared image and depth map to the pixel coordinate system of the visible light image, ensuring they have the same pixel size and resolution. This spatially aligned image data forms an aligned multimodal data set, which can be viewed as a time-series collection of multimodal data arranged in chronological order. This multimodal data set can be expressed by the following formula: in, Indicates a multimodal data set; Represents a visible light image; Represents thermal infrared images, This represents a depth map.

[0082] The multimodal data sets acquired through the above methods constitute the inspection data, including visible light images, thermal infrared images, and depth maps. Spatial coordinates are unified using pose information, enabling precise alignment of these image data. This alignment process significantly improves the accuracy of data fusion, allowing visible light images, thermal infrared images, and depth maps to effectively complement each other. Specifically, visible light images provide shape and texture details of candidate targets; thermal infrared images provide temperature distribution information of candidate targets, which is particularly advantageous in low-light or nighttime environments; and depth maps provide spatial location and depth information of the target, facilitating target identification from a three-dimensional perspective.

[0083] The multimodal data set acquired through the above method constitutes the inspection data, which includes visible light images, thermal infrared images, and depth maps. This data set utilizes pose information to achieve spatial coordinate unification, enabling precise alignment of the image data. This alignment process significantly improves data fusion accuracy, allowing visible light images, thermal infrared images, and depth maps to effectively complement each other.

[0084] By aligning and fusing multimodal image data, the detection rate and recognition accuracy of object detection models can be improved in complex environments such as fog, nighttime, and vegetation obstruction. This fusion strategy combines early fusion through input-level stitching with mid-term fusion through feature map interaction, and incorporates a modality loss robustness design to ensure that the object detection network can still function normally when image data of a certain modality fails.

[0085] It should be noted that this application does not rely on a specific data acquisition platform to obtain inspection data. Besides the drones used in the above embodiments, it can also be applied to various scenarios such as fixed surveillance cameras, vehicle-mounted mobile platforms, and satellite remote sensing. Only the image resolution, frame rate, viewing angle parameters, and georegistration method need to be adjusted according to the characteristics of different platforms. In fixed camera scenarios, the attention map can be pre-labeled as the pipe projection area within the field of view; in satellite imagery scenarios, the attention map can be generated by combining high-precision GIS (Geographic Information System) data.

[0086] In an optional embodiment, determining the preliminary risk value of a candidate target based on the category confidence and the number of pixels in S104 includes multiplying the ratio of the number of pixels to the number of pixels in the visible light image by the category confidence to obtain the preliminary risk value.

[0087] The preliminary risk value can be expressed by the following formula: in, This indicates the preliminary risk value; Indicates the first Category confidence of each candidate target; Indicates the first The number of pixels for each candidate target; This indicates the number of pixels in a visible light image.

[0088] Using this formula, the preliminary risk value is... The category confidence of the candidate targets was comprehensively considered. Percentage of pixels Among them, category confidence. The probability of a candidate target belonging to a certain category is represented by the percentage of pixels. Characterize the area size of candidate targets in visible light images.

[0089] Preliminary risk value The importance of candidate targets is quantified based on the two indicators mentioned above. When the number of candidate target pixels is large and the category confidence is high, the initial risk value is... A high value indicates a high risk level for the candidate target; if the number of pixels in the candidate target is small or the category confidence is low, the corresponding preliminary risk value is lower. A lower risk level indicates a lower risk level for the candidate target. By using the above method, high-risk candidate targets can be accurately identified for further evaluation, improving the accuracy and reliability of pipeline mechanical occupancy detection.

[0090] In one alternative embodiment, such as Figure 5 As shown, S107 includes steps S1071 to S1075: S1071. Determine the mechanical type of the target based on the category confidence level and the number of pixels.

[0091] Based on the output of the target detection network, the mechanical type of each candidate target is first determined, and a corresponding classification coefficient is assigned according to its mechanical type. The classification coefficient These correspond to small, medium, and heavy construction machinery, respectively. Typically, the classification coefficient for heavy machinery is... The classification coefficient for small machinery is relatively high. Lower.

[0092] S1072. Determine the cosine value between the velocity vector and the unit direction vector; the direction of the unit direction vector is the direction from the target to the point closest to the center line of the pipe.

[0093] Using two or more consecutive aligned visible light images, the optical flow method is used to estimate the value of each target. The velocity vector of the target. Then, calculate the cosine of the angle between this velocity vector and the unit direction vector pointing from the target to the nearest point on the pipe centerline. This cosine value can be expressed by the following formula: in, Let represent the cosine value of the j-th target; This represents the velocity vector of the j-th target. This represents the unit direction vector.

[0094] S1073. Determine the target's motion trend based on the cosine value.

[0095] The motion trend factor can be used to characterize the movement trend of a target. If the target is moving towards the pipe, the motion trend factor is larger, and vice versa. This motion trend factor can be expressed by the following formula: in, This represents the motion trend factor of the j-th target. This represents the gain coefficient, with a value ranging from 0 to 2.0. Let represent the cosine value of the j-th target.

[0096] S1074. Determine the ratio of the number of pixels to the number of pixels in the visible light image based on the number of pixels.

[0097] For each target, its number of pixels is compared to the number of pixels in the visible light image, yielding a ratio. This ratio reflects the target's area in the visible light image; generally, the larger the target, the higher the risk. This ratio can be expressed by the following formula: in, This represents the ratio of the j-th objective; Indicates the first The number of pixels in each target. This indicates the number of pixels in a visible light image.

[0098] S1075. Determine the comprehensive risk value based on distance, movement trend, ratio, and machine type.

[0099] The overall risk value can be calculated using the following formula: in, Indicates the first The overall risk value of each objective; This represents the distance decay steepness parameter, with a value ranging from 3 to 5; This represents the baseline distance threshold, with a value ranging from 5 to 8 meters. Indicates the first The shortest distance from the center point of the bounding box coordinates of each target to the center line of the pipeline.

[0100] The shortest distance can be calculated using the following formula: in, Indicates the first The shortest distance from the center point of the bounding box coordinates of each target to the centerline of the pipeline; and Indicates the arc length parameter corresponding to the centerline of the pipeline. The coordinates; and Indicates the first The coordinates of the center point of the bounding box coordinates of each target.

[0101] In this process, the proximity of the target to the pipeline, the approach trend, the size of the target, and the type of machinery are taken into account, which further improves the accuracy of the judgment.

[0102] This embodiment comprehensively considers multiple factors such as the proximity of the target, the proximity trend, and the type of machinery to generate a quantitative comprehensive risk value, which can more accurately reflect the safety relationship between the target and the pipeline, making the risk assessment more comprehensive, and thus improving the accuracy and reliability of pipeline machinery occupancy identification.

[0103] In one alternative embodiment, such as Figure 6 As shown, S108 includes steps S1081 to S1082: S1081. Determine the second risk value threshold based on the inspection time, the area category and construction permit status of the inspection area along the line.

[0104] Specifically, the second risk threshold can be determined by the following formula: in, This represents the second risk threshold corresponding to the j-th objective; Indicates the basic threshold; t represents time; z represents the region category identifier; This indicates the permission status flag corresponding to the j-th target; , and It is a non-negative adjustment coefficient.

[0105] The time 't' can be obtained from the start timestamp of the current inspection task from the UAV flight control system, and the number of hours can be extracted. The area category identifier 'z' of the current inspection area can be determined by querying the GIS; the more sensitive the area, the higher 'z'. The construction permit database can be accessed to determine if the target's location falls within an authorized construction area; if it does, a permit status flag is set. ,otherwise . , and Examples of possible values ​​are as follows: , , .

[0106] In this embodiment, by dynamically adjusting the second risk value threshold based on time, spatial sensitivity, and construction permit status, the risk of the target can be assessed more accurately. Specifically: The second risk threshold changes dynamically depending on the time of day. During the nighttime period, A larger value increases the second risk threshold, increasing sensitivity to potentially high-risk targets; however, during the daytime, The smaller the threshold, the lower the second risk value threshold, to avoid false alarms for regular targets. Changes over time directly affect the sensitivity of risk assessment, enabling the mechanical occupancy identification method in this embodiment to adapt to different time conditions.

[0107] As the area category identifier z increases, the sensitivity of the corresponding area increases accordingly, and the second risk threshold dynamically increases. This is especially true in sensitive areas such as high-consequence zones and ecological protection zones, which can improve alarm sensitivity and ensure timely identification of potential high-risk targets. The mechanical occupancy identification method in this embodiment can dynamically adjust the risk threshold according to changes in the area, strengthening the monitoring of sensitive areas.

[0108] when When this happens, the second risk threshold will dynamically decrease to avoid false alarms for legitimate construction activities; when At that time, the second risk threshold will dynamically increase, enhancing vigilance against illegal construction. Through this dynamic adjustment method, the mechanical encroachment identification method of this embodiment can achieve accurate risk assessment under different construction permit statuses.

[0109] The combination of multiple dynamic adjustment mechanisms enables the mechanical occupancy identification method in this embodiment to flexibly adapt to different scenarios, ensuring that the second risk value threshold dynamically changes with different conditions, thereby improving the accuracy of risk assessment. By making real-time dynamic adjustments based on three key factors—time, regional sensitivity, and construction permit status—the mechanical occupancy identification method in this embodiment can always maintain high identification accuracy, effectively reducing false alarms and missed alarms.

[0110] It should be explained that the aforementioned formula is merely an example. As long as the parameters of the formula satisfy a specific relationship—that is, the threshold changes with time, regional sensitivity, and construction permit status in a reasonable manner—it is acceptable. For example, when the regional category is identified... As the value increases, the second risk threshold also increases; while the construction permit mark... At that time, the second risk threshold decreases.

[0111] S1082. Determine the judgment result based on the comparison result of the comprehensive risk value and the second risk value threshold.

[0112] In an optional embodiment, step S1082 includes: if the construction permit status is "construction not permitted," and the comprehensive risk value is not less than a second risk value threshold, then the determination result is identified as a Level 1 occupation event, and an alarm is triggered. If the construction permit status is "construction permitted," and the comprehensive risk value is less than the second risk value threshold and not less than a third risk value threshold, then the determination result is identified as a monitoring event, and the target is added to the tracking list; if the comprehensive risk value is less than the third risk value threshold, and the pipeline is not in a shutdown state, then the determination result is identified as normal operation.

[0113] The third risk threshold is determined by reducing the second risk threshold by a preset ratio.

[0114] The third risk threshold is determined by reducing the second risk threshold by a preset ratio. Specifically, the third risk threshold is a part of the second risk threshold and can be determined by the following formula: in, This represents the second risk threshold for the j-th objective; This represents the threshold value for the third risk of the j-th objective. This indicates the preset reduction factor.

[0115] Reduction factor The value range is from 0 to 1. For example, when At that time, the third risk value threshold of the j-th target That is, the second risk threshold. 60%.

[0116] When a pipeline is in a state of inactivity, it means that the pipeline operator has actively set the pipeline section to a state of inactivity or maintenance isolation. At this time, even if there is any encroachment, it will not affect the safety of the pipeline. Therefore, the response level can be reduced and only low-frequency log records can be made for auditing.

[0117] according to and The value of is used to determine the following results: Level 1 suppression event: If and If this occurs, it is determined to be a Level 1 occupancy event, and an alarm is triggered.

[0118] Normal operation: If and If it is running normally, it is considered to be in normal operation and does not require recording or tracking.

[0119] Monitoring events: If and If so, it will be recorded as a monitoring event, without triggering an alarm, but the target will be added to the tracking list. Additionally, if and ≥ Although the overall risk value is high, because the construction is legal, it does not trigger an alarm. It is only recorded as a monitoring event and included in the tracking list for subsequent review to verify whether the construction behavior has always been compliant.

[0120] Special handling when the device is disabled: If and If the pipeline is clearly out of service, this indicator can be ignored or only recorded in low-frequency logs for auditing purposes, without being included in monitoring or alarm logic.

[0121] Event to be observed: When and At that time, although there was no construction permit, the overall risk value did not reach the second risk value threshold. These are considered low-risk abnormal activities, and are recorded in lightweight logs without alarms or tracking, for long-term behavioral analysis.

[0122] The mechanical occupancy identification method in this embodiment selects whether to include the target in the monitoring or judge it as normal operation based on different judgment results, thereby achieving a more flexible and accurate alarm mechanism. Specifically, the mechanical occupancy identification method in this embodiment can flexibly distinguish between normal operation and monitoring targets according to different judgment results, achieving a more accurate alarm mechanism. This method dynamically adjusts the second risk value threshold based on time, regional sensitivity, and construction permit status, adapting to different inspection scenarios. It distinguishes between high-risk alarms and routine monitoring based on construction permit status, and performs special processing for pipeline shutdown status, effectively reducing false alarms and improving the accuracy and reliability of risk assessment.

[0123] In an optional embodiment, after S108, the mechanical occupancy identification method further includes the step of: combining the determination result with the original multimodal data set. Pipeline Attention Guidance Diagram Candidate target attribute information, such as the coordinates of the center point of the bounding box. ,ratio Mechanical type Motion velocity vector and contextual information, such as time. Regional category identifier Construction permit mark The manually verified labels are packaged and stored in a structured occupancy event knowledge base. Newly labeled samples are periodically extracted from this knowledge base to fine-tune the target detection network, achieving incremental updates of model parameters.

[0124] Based on the same inventive concept as the aforementioned mechanical occupancy identification method, this application also provides a mechanical occupancy identification device, such as... Figure 7 As shown, the mechanical occupancy identification device 2 includes: an attention map acquisition module 21, a positioning and classification module 22, a pixel quantity determination module 23, a target determination module 24, a motion speed vector determination module 25, a distance determination module 26, a comprehensive risk value determination module 27, and a judgment result determination module 28.

[0125] Attention map acquisition module 21 is used to obtain the attention map of the pipeline based on the spatial location information of the pipeline. Localization and classification module 22 is used to input the acquired pipeline inspection data into a target detection network embedded with the attention map to obtain the bounding box coordinates and class confidence of candidate targets. The target detection network is used to locate and classify candidate targets under the guidance of the attention map, and the bounding box coordinates represent the location range of the candidate targets. Pixel quantity determination module 23 is used to determine the number of pixels covered by the candidate target in the inspection data based on the bounding box coordinates. Target determination module 24 is used to determine the preliminary risk value of the candidate target based on the class confidence and pixel quantity, and selects candidate targets with a preliminary risk value greater than a preset first risk value threshold as targets. Motion velocity vector determination module 25 is used to determine the motion velocity vector of the target based on the inspection data. Distance determination module 26 is used to determine the distance between the target and the pipeline based on the bounding box coordinates. Comprehensive risk value determination module 27 is used to determine the comprehensive risk value of the target based on the distance, motion velocity vector, pixel quantity, and class confidence. Judgment result determination module 28 is used to determine the judgment result of the occupancy event based on the comprehensive risk value.

[0126] Each module of the aforementioned mechanical occupancy identification device corresponds to the steps of the aforementioned mechanical occupancy identification method. For details on the implementation process of the functions and roles of each module in the aforementioned mechanical occupancy identification device, please refer to the implementation process of the corresponding steps in the aforementioned mechanical occupancy identification method. The same technical effect can be achieved, and will not be repeated here.

[0127] Figure 8 The diagram shown is a structural schematic of the electronic device 30 provided in an embodiment of this application.

[0128] like Figure 8 As shown, the electronic device 30 includes one or more processors 31 for implementing the mechanical occupancy identification method described above.

[0129] In some embodiments, the electronic device 30 may include a storage medium 39. For example, a computer-readable storage medium may store a program that can be invoked by a processor 31, and may include a non-volatile storage medium. In some embodiments, the electronic device 30 may include memory 38 and an interface 37. In some embodiments, the electronic device 30 may also include other hardware depending on the specific application.

[0130] The computer-readable storage medium of this application embodiment stores a program thereon, which, when executed by processor 31, is used to implement the mechanical occupancy identification method described above.

[0131] This application provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the mechanical occupancy identification method as described above.

[0132] This application may take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing program code. Computer-readable storage media include permanent and non-permanent, removable and non-removable media, and information storage can be implemented using any method or technology. Information may be computer-readable instructions, data structures, program modules, or other data. Examples of computer-readable storage media include, but are not limited to: phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other classes of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0133] The aforementioned electronic device can execute the mechanical occupancy identification method provided in the embodiments herein. The aforementioned electronic device may include the aforementioned mechanical occupancy identification device, such as one or more of a processor, controller, and PC (Personal Computer) terminal device. The server terminal device and PC terminal device may include, but are not limited to, a server, desktop computer, tablet computer, or laptop computer.

[0134] It should be noted that the technical solutions or features described in the above embodiments can be combined or supplemented with each other without conflict. The scope of protection of this application is not limited to the precise structures described in the above embodiments and shown in the accompanying drawings; all modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for identifying mechanical occupancy, characterized in that, The method includes: Based on the spatial location information of the pipeline, an attention map of the pipeline is obtained; The acquired pipeline inspection data is input into a target detection network embedded with the attention map to obtain the bounding box coordinates and class confidence of the candidate targets; the target detection network is used to complete the localization and classification of the candidate targets under the guidance of the attention map, and the bounding box coordinates are used to characterize the location range of the candidate targets; Based on the bounding box coordinates, determine the number of pixels covered by the candidate target in the inspection data; Based on the category confidence level and the number of pixels, a preliminary risk value for the candidate target is determined, and candidate targets whose preliminary risk value is greater than a preset first risk value threshold are selected as targets. Based on the inspection data, determine the target's velocity vector; Based on the bounding box coordinates, determine the distance between the target and the pipeline; The overall risk value of the target is determined based on the distance, the motion velocity vector, the number of pixels, and the category confidence level. Based on the comprehensive risk value, the determination result of the encroachment event is made.

2. The mechanical occupancy identification method as described in claim 1, characterized in that, The step of inputting the acquired pipeline inspection data into a target detection network embedded with the attention map to obtain the bounding box coordinates and class confidence of candidate targets includes: The inspection data is input into the feature extraction backbone of the target detection network to obtain a feature map; Based on the attention map, the feature map is weighted to obtain a weighted feature map; The weighted feature map is input into the detection head of the target detection network to obtain the bounding box coordinates and the class confidence.

3. The mechanical occupancy identification method as described in claim 1, characterized in that, The target detection network is trained on training samples, which include training inspection data and category labels and bounding box annotations corresponding to the training inspection data. The loss functions used when training the object detection network include a classification loss function, a localization loss function, and a geometric consistency loss function; The classification loss function is used to characterize the degree of inconsistency between the predicted class confidence score output by the object detection network and the class label. The localization loss function is used to characterize the difference between the predicted bounding box coordinates output by the target detection network and the bounding box annotation; The geometric consistency loss function is used to impose a constraint penalty when the shortest Euclidean distance between the center point of the predicted bounding box coordinates and the center line of the pipeline is greater than a preset safety distance threshold.

4. The mechanical occupancy identification method as described in claim 1, characterized in that, The spatial location information includes: the location information of the pipeline centerline, and the pipeline burial depth value corresponding to each location of the pipeline centerline; The step of obtaining the attention map of the pipeline based on its spatial location information includes: Based on the safety distance threshold, a two-dimensional planar buffer area is obtained by expanding the area outward with the centerline of the pipeline as the axis. Map the two-dimensional planar buffer region to a raster coordinate system with the same image resolution as the inspection data; Within the grid coordinate system, attention weight values ​​are assigned to each grid point to obtain the attention map; the attention weight values ​​decrease as the distance from each grid point to the center line of the pipeline increases.

5. The mechanical occupancy identification method as described in claim 1, characterized in that, The inspection data is determined in the following way: While controlling the drone to fly along the centerline of the pipeline, it collects visible light images, thermal infrared images, and depth maps of the inspection area along the pipeline. While acquiring each frame of visible light image, thermal infrared image, and depth map, the coordinate data and attitude data of the UAV are acquired simultaneously. Based on the coordinate data and the attitude data, coordinate transformation is performed on the thermal infrared image and the depth map to obtain the coordinate-transformed thermal infrared image and the coordinate-transformed depth map. The thermal infrared image and the depth map after coordinate transformation are resampled into the pixel coordinate system of the visible light image to obtain the inspection data.

6. The mechanical occupancy identification method as described in claim 5, characterized in that, Determining the preliminary risk value of the candidate target based on the category confidence level and the number of pixels includes: The preliminary risk value is obtained by multiplying the ratio of the number of pixels to the number of pixels in the visible light image by the category confidence level.

7. The mechanical occupancy identification method as described in claim 5, characterized in that, Determining the comprehensive risk value of the target based on the distance, the motion velocity vector, the category confidence level, and the number of pixels includes: The mechanical type of the target is determined based on the category confidence level and the number of pixels; Determine the cosine value between the motion velocity vector and the unit direction vector; the direction of the unit direction vector is the direction from the target to the point closest to the centerline of the pipe; The motion trend of the target is determined based on the cosine value; Based on the number of pixels, determine the ratio of the number of pixels to the number of pixels in the visible light image; The comprehensive risk value is determined based on the distance, the movement trend, the ratio, and the type of machinery.

8. The mechanical occupancy identification method as described in claim 5, characterized in that, The determination of the occupation event based on the comprehensive risk value includes: The second risk value threshold is determined based on the inspection time, as well as the area category and construction permit status of the inspection area along the route; The determination result is determined based on the comparison between the comprehensive risk value and the second risk value threshold.

9. The mechanical occupancy identification method as described in claim 8, characterized in that, Determining the judgment result based on the comparison between the comprehensive risk value and the second risk value threshold includes: If the construction permit status is not permitted and the comprehensive risk value is not less than the second risk value threshold, then the judgment result is determined to be a level one encroachment event, and an alarm is triggered. If the construction permit status is "allowed to construct", and the overall risk value is less than the second risk value threshold and not less than the third risk value threshold, then the judgment result is determined to be a monitoring event, and the target is added to the tracking list; if the overall risk value is less than the third risk value threshold and the pipeline is not in a shutdown state, then the judgment result is determined to be "normal operation". The third risk value threshold is determined by reducing the second risk value threshold by a preset ratio.

10. A mechanical occupancy identification device, characterized in that, The device includes: The attention map acquisition module is used to obtain the attention map of the pipeline based on the spatial location information of the pipeline; The localization and classification module is used to input the acquired pipeline inspection data into a target detection network embedded with the attention map to obtain the bounding box coordinates and class confidence of the candidate targets; the target detection network is used to complete the localization and classification of the candidate targets under the guidance of the attention map, and the bounding box coordinates are used to characterize the location range of the candidate targets; A pixel count determination module is used to determine the number of pixels covered by the candidate target in the inspection data based on the bounding box coordinates. The target determination module is used to determine the preliminary risk value of the candidate target based on the category confidence level and the number of pixels, and to select the candidate target whose preliminary risk value is greater than a preset first risk value threshold as the target; The motion velocity vector determination module is used to determine the motion velocity vector of the target based on the inspection data. A distance determination module is used to determine the distance between the target and the pipeline based on the bounding box coordinates; The comprehensive risk value determination module is used to determine the comprehensive risk value of the target based on the distance, the motion speed vector, the number of pixels, and the category confidence level. The determination result module is used to determine the determination result of the encroachment event based on the comprehensive risk value.

11. An electronic device, characterized in that, It includes one or more processors for implementing the mechanical occupancy identification method as described in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, It stores a program that, when executed by a processor, implements the mechanical occupancy identification method as described in any one of claims 1 to 9.

13. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instruction is executed by the processor, it implements the mechanical occupancy identification method as described in any one of claims 1 to 9.

Images