Deep Learning-Based Target Detection Method and Equipment for Mechanized Operations on Transmission Lines

Abstract

This invention discloses a deep learning-based target detection method and device for mechanized operations on power transmission lines, relating to the field of image detection technology. The method includes: performing dynamic scene modeling on the input video stream, extracting foreground moving target regions and newly emerging static target regions, and merging them to generate a candidate region of interest (ROI) set; performing feature enhancement on regions within the ROI set using temporal information from adjacent frames, and employing a deep learning model for target recognition and tracking, outputting detection results with an identity ID and target trajectory; performing behavioral semantic analysis based on the detection results, assessing risk levels according to predefined rules, and generating alarm information. This invention addresses the technical problems of inaccurate target detection and high false negative rates in existing mechanized operations by introducing dynamic scene modeling, temporal feature enhancement, and target association recognition mechanisms, thereby improving target detection and tracking accuracy and achieving automatic risk identification and early warning for operations.

Description

Technical Field

[0001] This invention relates to the field of visual inspection technology, specifically to a method and equipment for target detection in mechanized operations of power transmission lines based on deep learning. Background Technology

[0002] Mechanized construction of power transmission lines involves vehicles, hoisting equipment, and various types of engineering machinery. The working environment is often affected by factors such as terrain undulations, lighting changes, occlusion interference, and complex background textures. Targets in images are characterized by small scale, significant appearance differences, and frequent changes in posture. Due to these constraints, image-based object detection often struggles to reliably extract key features, easily leading to problems such as blurred edges and unclear category distinctions. In low-resolution images or dynamic construction scenes, small-sized equipment and moving targets are more easily overlooked, resulting in low recognition accuracy and significant missed detections, affecting the accurate judgment of the state and behavior of the work objects. Summary of the Invention

[0003] This application provides a deep learning-based target detection method and equipment for mechanized operations on power transmission lines, which aims to address the technical problems of inaccurate target detection and high false negative rate in existing mechanized operations.

[0004] In view of the above problems, this application provides a method and equipment for target detection in mechanized operations of power transmission lines based on deep learning.

[0005] The first aspect of this application provides a deep learning-based target detection method for mechanized operations on power transmission lines, the method comprising:

[0006] The system performs dynamic scene modeling on the input video stream, extracts foreground moving target regions and newly emerging static target regions, and merges them to generate a candidate region of interest set. For the regions within the candidate region of interest set, the system performs feature enhancement using temporal information from adjacent frames, and uses a deep learning model for target recognition and tracking, outputting detection results with an identity ID and target trajectory. Based on the detection results, the system performs behavioral semantic analysis, assesses the risk level according to predefined rules, and generates alarm information.

[0007] A second aspect of this application provides a deep learning-based target detection device for mechanized operations on power transmission lines, the device comprising:

[0008] The scene modeling module is used to perform dynamic scene modeling on the input video stream, extract foreground moving target regions and newly emerging static target regions, and merge them to generate a candidate region of interest set; the target recognition module is used to perform feature enhancement on the regions within the candidate region of interest set using temporal information from adjacent frames, and to perform target recognition and tracking using a deep learning model, outputting detection results with an identity ID and target trajectory; the analysis module is used to perform behavioral semantic analysis based on the detection results, assess the risk level according to predefined rules, and generate alarm information.

[0009] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0010] This application performs dynamic scene modeling on the input video stream, extracts foreground moving target regions and newly emerging static target regions, and merges them to generate a candidate region of interest (ROI) set. For regions within the ROI set, feature enhancement is performed using temporal information from adjacent frames, and a deep learning model is used for target recognition and tracking, outputting detection results with an identity ID and target trajectory. Based on the detection results, behavioral semantic analysis is performed, risk levels are assessed according to predefined rules, and alarm information is generated. This invention solves the technical problems of inaccurate target detection and high false negative rates in existing mechanized operations by introducing dynamic scene modeling, temporal feature enhancement, and target association recognition mechanisms, thereby improving target detection and tracking accuracy and achieving automatic identification and early warning of operational risks. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 A schematic diagram of the target detection method for mechanized operations on power transmission lines based on deep learning, provided in an embodiment of this application;

[0013] Figure 2 A schematic diagram of the target detection equipment for mechanized operation of power transmission lines based on deep learning, provided in an embodiment of this application.

[0014] Figure labeling: Scene modeling module 11, target recognition module 12, analysis module 13. Detailed Implementation

[0015] This application provides a deep learning-based target detection method and equipment for mechanized operations on power transmission lines. It addresses the technical problems of inaccurate target detection and high false negative rate in existing technologies by introducing dynamic scene modeling, temporal feature enhancement, and target association recognition mechanisms. This achieves the technical effects of improving target detection and tracking accuracy and realizing automatic identification and early warning of operational risks.

[0016] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0017] It should be noted that any variation of the terms "comprising" and "having" is intended to cover non-exclusive inclusion, for example, a process, method, apparatus, product, or server that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or modules that are not explicitly listed or that are inherent to such process, method, product, or apparatus.

[0018] Example 1, as Figure 1 As shown, this application provides a deep learning-based target detection method for mechanized operations on power transmission lines, the method comprising:

[0019] Step S100: Perform dynamic scene modeling on the input video stream, extract the foreground moving target region and the newly emerging static target region, and merge them to generate a candidate region of interest set.

[0020] In this embodiment, the input video stream consists of a continuously acquired sequence of satellite images, which is used for dynamic scene modeling. During modeling, a dual-background model composed of a dynamic background model and a static reference model is constructed to jointly represent the short-term changes and long-term stable structures of the scene. First, each frame of satellite imagery in the video stream is compared with the dynamic background model to generate a preliminary foreground mask representing motion changes. This foreground moving target region is then extracted through noise suppression, hole optimization, and connected component analysis. Subsequently, each frame of imagery in the video stream is compared with the static reference model using the Structural Similarity Index (SSIM) and edge feature difference analysis. A static difference mask is generated based on these two types of difference information, and connected component filtering is used to obtain newly emerging static target regions representing newly added objects. Finally, the foreground moving target regions and the newly emerging static target regions are spatially fused, and overlapping parts are removed to construct a set of candidate regions of interest.

[0021] Furthermore, the method provided in the application embodiments, which involves performing dynamic scene modeling on the input video stream, extracting foreground moving target regions and newly emerging static target regions, and merging them to generate a candidate region of interest set, also includes:

[0022] A dual-background model is initialized, comprising a dynamic background model and a static reference model. Each frame of the video stream is compared with the dynamic background model to generate a preliminary foreground mask, which is then optimized for noise and holes. The optimized foreground mask is subjected to connected component analysis and filtering to generate a foreground moving target region. Each frame of the video stream is compared with the static reference model using structural similarity index comparison and edge feature difference analysis to generate a static difference mask, which is then filtered using connected component analysis to generate a newly emerging static target region. The foreground moving target region and the newly emerging static target region are merged, and overlapping parts are removed to generate the candidate region of interest set.

[0023] Furthermore, the method provided in the application embodiments also includes:

[0024] The dynamic background model uses an adaptive Gaussian mixture model, while the static reference model uses a clean background snapshot.

[0025] In this embodiment, when performing dynamic scene modeling on the input video stream, a dual-background model is first initialized from a continuously acquired satellite image sequence. This dual-background model consists of a dynamic background model and a static reference model. The dynamic background model employs an adaptive Gaussian mixture model. It establishes multiple Gaussian distributions for each pixel and updates the mean and variance of each Gaussian distribution frame-by-frame based on the difference between the current pixel and the mean of the corresponding Gaussian distribution, ensuring the model continuously represents short-term background changes in the scene. The static reference model uses clean background snapshots. High-quality satellite image frames that do not contain mechanized equipment, moving targets, or obstructions are selected as a long-term stable background to reflect the essentially unchanging static structures in the scene.

[0026] After background initialization, each frame of the video stream is compared with the dynamic background model. A preliminary foreground mask representing areas where pixels do not match the background is generated by calculating the difference between each pixel and its Gaussian distribution mean. Because this preliminary foreground mask contains noise and region breaks, morphological closing operations are performed on it. Dilation fills in holes within the foreground region, and erosion eliminates pseudo-connected regions, ensuring continuous boundaries and a complete internal structure for the foreground region. After mask optimization, connected component analysis is performed on the optimized foreground mask. Regions are grouped based on pixel connectivity, and excessively small or irregular noise blocks are filtered out using region area thresholds. Finally, the foreground moving target region reflecting the moving target in the image is obtained.

[0027] Next, each frame of the video stream is compared with the static reference model using structural similarity index (SSIM) and edge feature difference analysis. In this process, each frame of the video stream and the static reference model are divided into several local blocks of fixed size. The SSIM is calculated for each corresponding local block to reflect the differences in brightness, contrast, and structural components, thus generating an SSIM difference map. Simultaneously, Canny edge detection is performed on each frame and the static reference model to extract edge features, and pixel-level difference operations are performed on the two edge maps to form an edge difference map. Subsequently, the SSIM difference map and the edge difference map are fused, allowing structural and edge change information to be comprehensively represented in the same image. Thresholding is then performed on the fused result, marking significantly different regions as foreground, thereby generating a static difference mask. After connected component analysis, the static difference mask filters out isolated small blocks based on the area and geometric features of the connected regions, ultimately extracting the static target region.

[0028] Finally, after obtaining the foreground moving target region and the newly emerging static target region, spatial overlap detection is performed based on the bounding rectangles of the two types of regions. Regions with overlapping bounding rectangles are merged, and the overlapping parts are uniformly classified into a single target region; regions without overlapping relationships are expressed independently. Through the above merging and overlap removal operations, a set of candidate regions of interest covering both dynamically changing targets and newly added static targets is formed.

[0029] Furthermore, in the method provided in the application embodiments, the method of comparing the structural similarity index and edge feature difference analysis of each frame image in the video stream with the static reference model to generate a static difference mask further includes:

[0030] Each frame of the video stream is divided into multiple local blocks along with the static reference model. The structural similarity index value between each frame and the static reference model is calculated block by block to generate an SSIM difference map. The Canny edge map between each frame and the static reference model is calculated in parallel, and the two edge maps are differentially processed to generate an edge difference map. The SSIM difference map and the edge difference map are weighted and fused, and then binarized using a preset threshold to generate the static difference mask.

[0031] In this embodiment, when comparing the structural similarity index (SSIM) of each frame in the video stream with the static reference model, the current frame image and the static reference model are first registered and resized, and then spatially divided into multiple corresponding local blocks according to a preset block size (e.g., m×n pixels). For each pair of corresponding local blocks, the mean and variance of the grayscale of the local block in the current frame and the mean and variance of the grayscale of the local block in the static reference model are calculated, and the covariance between the two is also calculated. Based on this, the standard formula for calculating the structural similarity index (SSIM) is substituted to obtain the structural similarity index value corresponding to the local block. After completing the SSIM calculation for all local blocks in sequence, the SSIM values ​​of each local block are restored into an SSIM difference map of the same size as the original image according to its spatial position in the image.

[0032] Simultaneously, during the edge feature difference analysis, the Canny edge detection method is used to extract edges from each frame of the image and the static reference model. Specifically, this involves Gaussian smoothing the image to suppress high-frequency noise, calculating the gradient magnitude and direction in the horizontal and vertical directions, performing non-maximum suppression on the gradient magnitude to obtain a refined edge response, and forming a stable Canny edge map through a double-threshold hysteresis connection. After obtaining the Canny edge map of the current frame and the Canny edge map of the static reference model, pixel-by-pixel difference calculation is performed on the two edge maps. When the edge marker of a pixel at the same location differs in the two images, that location is considered a pixel with edge change, thus obtaining an edge difference map representing the addition or disappearance of edges.

[0033] Finally, the SSIM difference map and the edge difference map are weighted and fused. Preferably, the SSIM difference map and the edge difference map are weighted and summed according to a preset ratio using a linear combination method to obtain a comprehensive difference map, so that local structural change information and edge contour change information are jointly expressed in the same image. Subsequently, a preset threshold is applied to the comprehensive difference map for binarization. Regions with pixel values ​​greater than the threshold are marked as foreground difference regions, and regions with pixel values ​​less than or equal to the threshold are marked as background regions, generating a static difference mask describing the static change regions.

[0034] Step S200: For the regions within the candidate region of interest set, feature enhancement is performed using the temporal information of adjacent frames, and a deep learning model is used for target recognition and tracking, outputting detection results with identity ID and target trajectory.

[0035] In this embodiment, when performing feature enhancement on regions within a candidate region of interest set using temporal information from adjacent frames, firstly, a lightweight convolutional neural network is used to extract features from the candidate regions of interest in each frame image, resulting in a corresponding feature map for each frame. Then, the spatially corresponding feature map of the previous frame is obtained, and the previous frame feature map is spatially aligned based on the displacement vector estimated by optical flow, ensuring positional consistency in the feature representations of adjacent frames. The aligned previous frame feature map and the current frame feature map are concatenated along the channel dimension to form a temporal feature input containing cross-frame information. This temporal feature input is then fed into a temporal aggregation module for processing. The temporal aggregation module models the motion relationship between consecutive frames using one-dimensional convolution, achieving the fusion of cross-frame temporal features and outputting an enhanced feature map with temporal consistency.

[0036] Subsequently, a deep learning model is used for target recognition and tracking. In this process, the enhanced feature map is used as input to the deep learning model. The classification branch of the deep learning model predicts the target category, while the regression branch predicts the bounding box. Each detection box in the current frame is assigned a unique identity ID. The identity ID is transferred across frames by calculating the spatial overlap between the detection boxes in the current frame and those in the previous frame. The center points of detection boxes with the same identity ID are connected sequentially to generate the target trajectory. Based on the correspondence between the identity ID and the target trajectory, the detection result, including the target category, bounding box position, and target trajectory, is output.

[0037] Furthermore, in the method provided in the application embodiments, feature enhancement of regions within the candidate region of interest set using temporal information from adjacent frames further includes:

[0038] A lightweight convolutional neural network is used to extract features from candidate regions of interest in each frame of the image, resulting in a feature map for each frame. The feature map of the corresponding position in the previous frame is obtained for each frame of the image. The displacement vector estimated by optical flow is used to spatially align the feature map of the previous frame with the feature map of each frame. The aligned feature map of the previous frame is then concatenated with the feature map of each frame in the channel dimension. The concatenated feature map is then input into a temporal aggregation module, which learns the motion relationship between frames through one-dimensional convolution, fuses cross-frame temporal features, and outputs an enhanced feature map.

[0039] In this embodiment of the application, when using the temporal information of adjacent frames to enhance the features of candidate regions of interest, a lightweight convolutional neural network is first used to extract features from the candidate regions of interest in each frame of the image. The lightweight convolutional neural network maps the original pixels into a feature map of each frame describing the local texture, edge and shape distribution by sequentially performing convolution, nonlinear activation and downsampling operations on the input region.

[0040] Next, the feature map of the corresponding position in the previous frame is obtained for each frame of the image. By selecting feature blocks in the previous frame feature map that have the same spatial coordinates as the candidate region of interest in the current frame, an initial correspondence is established between the two frames based on the same coordinate reference system. After obtaining the previous frame feature map at the corresponding position, the brightness gradient and temporal gradient required by the optical flow method are calculated for the image between the current frame and the previous frame. The pixel-level displacement vector is solved by minimizing the brightness consistency constraint to characterize the local motion changes between adjacent frames. Based on the obtained displacement vector, spatial alignment is performed on the feature map of the corresponding position in the previous frame. Bilinear interpolation is used to resample the non-integer displacement positions in the feature map, ensuring that the previous frame feature map is consistent with the current frame feature map in spatial coordinates.

[0041] After spatial alignment is completed, the aligned feature map of the previous frame is stitched together with the feature map of the current frame in the channel dimension. By directly superimposing the channel numbers of the two, the local texture information, structural information and motion cues across frames are presented together in a unified feature representation.

[0042] The concatenated feature map is then input into the temporal aggregation module. The temporal aggregation module applies one-dimensional convolution in the time dimension and performs local linear combination of features of consecutive frames by sliding the convolution kernel in the time direction to learn the motion relationship between frames and capture temporally dependent features, thereby generating an enhanced feature map that integrates cross-frame temporal information and has temporal consistency.

[0043] Furthermore, in the method provided in the application embodiments, after outputting the enhanced feature map, it further includes:

[0044] The dense optical flow field between adjacent frames is calculated to obtain the motion amplitude information of the optical flow field, which is then normalized and Gaussian smoothed to generate a motion attention heatmap with the same size as the feature map. The motion attention heatmap is then multiplied element-wise with the enhanced feature map output by the temporal aggregation module to complete the feature map optimization.

[0045] In this embodiment, when calculating the dense optical flow field between adjacent frames, the Lucas-Kanade optical flow method is used to analyze the local brightness changes between the current frame and the previous frame. By solving a least-squares problem in the neighborhood of each pixel, the motion vectors of the pixels in the horizontal and vertical directions are obtained, forming a dense optical flow field. Based on the motion vector amplitude of each pixel in the dense optical flow field, the corresponding motion amplitude information is extracted to characterize the local motion intensity between adjacent frames. To ensure that the motion amplitude information is used at a uniform scale, it is normalized to map the motion amplitudes of different pixel positions to a consistent numerical range. Subsequently, the normalized motion amplitude information is smoothed using a Gaussian smoothing method, performing a weighted average within the local neighborhood to suppress noise fluctuations and maintain the spatial continuity of the motion region. After smoothing, a motion attention heatmap with the same spatial size as the enhanced feature map is obtained.

[0046] Next, when optimizing the enhanced feature map output by the temporal aggregation module using the motion attention heatmap, the motion attention heatmap and the enhanced feature map are strictly aligned in spatial location, and element-wise multiplication is performed between them. By multiplying the value of the motion attention heatmap with the corresponding feature value of the enhanced feature map at each pixel position, regions with high motion intensity are enhanced in the result, while regions with low motion intensity are suppressed. Feature map optimization is completed based on element-wise multiplication.

[0047] Furthermore, the method provided in the application embodiments, which employs a deep learning model for target recognition and tracking and outputs detection results with an identity ID and target trajectory, also includes:

[0048] The enhanced feature maps corresponding to each frame of the image are sequentially input into the deep learning model according to their temporal order to predict the target category and bounding box. A unique identity ID is created for the target detected in each frame, and the position of the corresponding detection box is recorded as a trajectory point. The deep learning model includes a classification branch and a regression branch, which are used to predict the target category and bounding box, respectively. The detection box with the largest spatial overlap between all detection boxes in any frame and the detection box in the previous frame is identified. If the maximum overlap is greater than a preset threshold, the identity ID of the target in the previous frame is passed to the identification detection box. The center points of the detection boxes with the same identity ID in each frame are connected in chronological order to obtain the target trajectory. The detection result is generated according to the mapping relationship between the identity ID and the target trajectory.

[0049] In this embodiment, when performing target recognition and tracking on the enhanced feature map corresponding to each frame of image, the enhanced feature maps are first input into the deep learning model in chronological order. This deep learning model is trained using supervised learning methods before deployment. During the training phase, sample data with target category labels and bounding box labels are used, and the network parameters are updated using backpropagation. This enables the model to learn the feature distribution of different target categories in the classification branch and to learn the spatial mapping relationship of bounding box positions in the regression branch. The trained deep learning model consists of a classification branch and a regression branch. The classification branch uses convolutional features for category prediction, and the regression branch uses convolutional features to output bounding box positions, thereby generating candidate target categories and bounding boxes for each frame from the input enhanced feature map.

[0050] After target category prediction and bounding box prediction are completed for each frame of the image, a unique identity ID is assigned to each bounding box in the current frame, and the coordinates of the center point of the bounding box are recorded as trajectory points. To achieve cross-frame target association, all detection boxes in the current frame are compared with all detection boxes in the previous frame using a spatial overlap calculation method. The degree of spatial overlap is measured by calculating the intersection-union ratio (IUU) between the bounding boxes, and the identification detection box with the highest overlap with the detection box in the previous frame is identified in the current frame. If the spatial overlap exceeds a preset threshold, the identity ID of the target in the previous frame is passed to the identification detection box in the current frame to maintain identity continuity in the time series.

[0051] After completing the cross-frame association of identity IDs, the center points of detection boxes with the same identity ID are arranged and connected according to the image time sequence to form the target trajectory of the corresponding target. Finally, based on the mapping relationship between identity ID and target trajectory, the target category, bounding box position, identity ID, and target trajectory are integrated to generate the corresponding detection result.

[0052] Furthermore, the method provided in the application embodiments also includes:

[0053] For any unmatched detection box in any frame, assign a new unique identity ID.

[0054] In this embodiment, during cross-frame association, for any unmatched detection box in any frame, after completing spatial overlap calculation, it is identified that it has not established an association with any detection box in the previous frame, and the detection box is regarded as an independent target appearing for the first time in the current frame. To ensure that it can be tracked uniquely in subsequent frames, a new unique identity ID is assigned to the detection box, and a trajectory starting point is established corresponding to the identity ID and its bounding box center point, so that the unmatched detection box can participate as an independent target in the identity ID transmission and trajectory generation process in subsequent temporal processing.

[0055] Step S300: Perform behavioral semantic analysis based on the detection results, assess the risk level according to predefined rules, and generate alarm information.

[0056] In this embodiment, when performing behavioral semantic analysis based on the detection results, a warning behavioral semantic knowledge base for different preset detection targets is first constructed. Then, the risk level is assessed and alarm information is generated based on the pre-constructed warning behavioral semantic knowledge base, i.e., predefined rules. During this process, the detection results are input into the warning behavioral semantic knowledge base for matching, generating alarm information corresponding to the risk level.

[0057] Furthermore, the method provided in the application embodiment, which performs behavioral semantic analysis based on the detection results, assesses the risk level according to predefined rules, and generates alarm information, further includes:

[0058] Construct a semantic knowledge base of early warning behaviors for different preset detection targets; input the detection results into the semantic knowledge base of early warning behaviors for matching, and generate alarm information corresponding to the risk level.

[0059] In this embodiment, when performing behavioral semantic analysis based on the detection results, a warning behavioral semantic knowledge base for different preset detection targets is first constructed. Using the categories of the preset detection targets as the index, the typical behavioral patterns, spatial constraints, distance thresholds, speed ranges, dwell time thresholds, and trajectory change characteristics of various targets in power transmission line operation scenarios are structurally organized. These behavioral characteristics are then mapped to their corresponding risk levels, forming a set of semantic rules. For engineering vehicles, the warning behavioral semantic knowledge base is divided into semantic entries such as approaching tower foundations, entering restricted areas, and slow-speed stops under conductors. Each entry is configured with numerical conditions such as a safe distance threshold of 8m, a warning distance threshold of 5m, a dangerous distance threshold of 3m, dwell time thresholds of 10 and 20 frames, and a speed threshold. For construction workers, semantic entries such as entering dangerous work areas and approaching hoisting equipment are set, with additional requirements for a distance threshold of 4m, a speed threshold of 0.8m / s, and dwell time requirements. Through this parametric modeling, a warning behavioral semantic knowledge base covering all preset detection target categories is obtained.

[0060] Next, the detection results are input into the semantic knowledge base for early warning behavior matching. The target category, bounding box position, identification, target trajectory, minimum safe distance, average speed, and number of dwell frames in the detection results are used as input features. Each feature is compared against the numerical conditions set in the semantic knowledge base to determine whether the detection result meets a specific semantic rule. For example, if the detection results show that the minimum distance between a piece of machinery and the tower foundation is 2.7m, the number of consecutive dwell frames is 26, and the average speed is 0.4m / s, which are below the danger distance threshold of 3m and exceed the dwell time threshold of 20 frames respectively, and meet the low-speed characteristic, then this record is determined to meet all the constraints of the danger level semantic entry. By matching the semantic knowledge base, the risk level corresponding to this entry is determined.

[0061] Finally, the matched risk level is combined with the time information, spatial location, target category, and trajectory features from the detection results to generate alarm content based on a pre-set alarm information template. This alarm content includes fields such as risk level, alarm location, alarm time, target category, and a summary of triggering conditions. Through the above data integration and template generation, alarm information corresponding to the risk level is obtained.

[0062] In summary, the embodiments of this application have at least the following technical effects:

[0063] This application performs dynamic scene modeling on the input video stream, extracts foreground moving target regions and newly emerging static target regions, and merges them to generate a candidate region of interest (ROI) set. For regions within the ROI set, feature enhancement is performed using temporal information from adjacent frames, and a deep learning model is used for target recognition and tracking, outputting detection results with an identity ID and target trajectory. Based on the detection results, behavioral semantic analysis is performed, risk levels are assessed according to predefined rules, and alarm information is generated. This invention solves the technical problems of inaccurate target detection and high false negative rates in existing mechanized operations by introducing dynamic scene modeling, temporal feature enhancement, and target association recognition mechanisms, thereby improving target detection and tracking accuracy and achieving automatic identification and early warning of operational risks.

[0064] Example 2 is based on the same inventive concept as the deep learning-based target detection method for mechanized operations on power transmission lines described in the previous examples, such as... Figure 2 As shown, this application provides a target detection device for mechanized operations on power transmission lines based on deep learning. The device and method embodiments in this application are based on the same inventive concept. The device includes:

[0065] Scene modeling module 11 is used to perform dynamic scene modeling on the input video stream, extract foreground moving target regions and newly emerging static target regions, and merge them to generate a candidate region of interest set; target recognition module 12 is used to perform feature enhancement on the regions within the candidate region of interest set using temporal information from adjacent frames, and to perform target recognition and tracking using a deep learning model, outputting detection results with identity ID and target trajectory; analysis module 13 is used to perform behavioral semantic analysis based on the detection results, assess the risk level according to predefined rules, and generate alarm information.

[0066] Furthermore, the device is also used to perform the following functions:

[0067] A dual-background model is initialized, comprising a dynamic background model and a static reference model. Each frame of the video stream is compared with the dynamic background model to generate a preliminary foreground mask, which is then optimized for noise and holes. The optimized foreground mask is subjected to connected component analysis and filtering to generate a foreground moving target region. Each frame of the video stream is compared with the static reference model using structural similarity index comparison and edge feature difference analysis to generate a static difference mask, which is then filtered using connected component analysis to generate a newly emerging static target region. The foreground moving target region and the newly emerging static target region are merged, and overlapping parts are removed to generate the candidate region of interest set.

[0068] Furthermore, the device is also used to perform the following functions:

[0069] The dynamic background model uses an adaptive Gaussian mixture model, while the static reference model uses a clean background snapshot.

[0070] Furthermore, the device is also used to perform the following functions:

[0071] Each frame of the video stream is divided into multiple local blocks along with the static reference model. The structural similarity index value between each frame and the static reference model is calculated block by block to generate an SSIM difference map. The Canny edge map between each frame and the static reference model is calculated in parallel, and the two edge maps are differentially processed to generate an edge difference map. The SSIM difference map and the edge difference map are weighted and fused, and then binarized using a preset threshold to generate the static difference mask.

[0072] Furthermore, the device is also used to perform the following functions:

[0073] A lightweight convolutional neural network is used to extract features from candidate regions of interest in each frame of the image, resulting in a feature map for each frame. The feature map of the corresponding position in the previous frame is obtained for each frame of the image. The displacement vector estimated by optical flow is used to spatially align the feature map of the previous frame with the feature map of each frame. The aligned feature map of the previous frame is then concatenated with the feature map of each frame in the channel dimension. The concatenated feature map is then input into a temporal aggregation module, which learns the motion relationship between frames through one-dimensional convolution, fuses cross-frame temporal features, and outputs an enhanced feature map.

[0074] Furthermore, the device is also used to perform the following functions:

[0075] The dense optical flow field between adjacent frames is calculated to obtain the motion amplitude information of the optical flow field, which is then normalized and Gaussian smoothed to generate a motion attention heatmap with the same size as the feature map. The motion attention heatmap is then multiplied element-wise with the enhanced feature map output by the temporal aggregation module to complete the feature map optimization.

[0076] Furthermore, the device is also used to perform the following functions:

[0077] The enhanced feature maps corresponding to each frame of the image are sequentially input into the deep learning model according to their temporal order to predict the target category and bounding box. A unique identity ID is created for the target detected in each frame, and the position of the corresponding detection box is recorded as a trajectory point. The deep learning model includes a classification branch and a regression branch, which are used to predict the target category and bounding box, respectively. The detection box with the largest spatial overlap between all detection boxes in any frame and the detection box in the previous frame is identified. If the maximum overlap is greater than a preset threshold, the identity ID of the target in the previous frame is passed to the identification detection box. The center points of the detection boxes with the same identity ID in each frame are connected in chronological order to obtain the target trajectory. The detection result is generated according to the mapping relationship between the identity ID and the target trajectory.

[0078] Furthermore, the device is also used to perform the following functions:

[0079] For any unmatched detection box in any frame, assign a new unique identity ID.

[0080] Furthermore, the device is also used to perform the following functions:

[0081] Construct a semantic knowledge base of early warning behaviors for different preset detection targets; input the detection results into the semantic knowledge base of early warning behaviors for matching, and generate alarm information corresponding to the risk level.

[0082] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, the above description focuses on specific embodiments of this specification. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired results. In some implementations, multitasking and parallel processing are possible or may be advantageous.

[0083] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A deep learning-based target detection method for mechanized operations on power transmission lines, characterized in that, include: Dynamic scene modeling is performed on the input video stream, foreground moving target regions and newly emerging static target regions are extracted, and then merged to generate a set of candidate regions of interest; For regions within the candidate region of interest set, feature enhancement is performed using temporal information from adjacent frames, and a deep learning model is used for target recognition and tracking, outputting detection results with identity ID and target trajectory; Based on the detection results, behavioral semantic analysis is performed, the risk level is assessed according to predefined rules, and alarm information is generated. Dynamic scene modeling is performed on the input video stream, extracting foreground moving target regions and newly emerging static target regions, and merging them to generate a candidate region of interest set, including: Initialize a dual-background model, which includes a dynamic background model and a static reference model; Each frame of the video stream is compared with the dynamic background model to generate a preliminary foreground mask and perform noise and hole optimization. The optimized foreground mask is then subjected to connected component analysis and filtered to generate the foreground moving target region. Each frame of the video stream is compared with the static reference model using a structural similarity index and edge feature difference analysis. A static difference mask is generated and filtered using connected component analysis to generate a newly emerging static target region. Merge the foreground moving target region with the newly emerging static target region, remove the overlapping parts, and generate the candidate region of interest set; The dynamic background model uses an adaptive Gaussian mixture model, while the static reference model uses a clean background snapshot.

2. The target detection method for mechanized operations on power transmission lines based on deep learning as described in claim 1, characterized in that, Each frame of the video stream is compared with the static reference model using a structural similarity index and edge feature difference analysis to generate a static difference mask, including: Each frame of the video stream is divided into multiple local blocks along with the static reference model. The structural similarity index value between each frame and the static reference model is calculated block by block to generate an SSIM difference map. The Canny edge map of each frame image and the static reference model is calculated in parallel, and the two edge maps are differentially processed to generate an edge difference map. The SSIM difference map and the edge difference map are weighted and fused, and then binarized using a preset threshold to generate the static difference mask.

3. The target detection method for mechanized operations on power transmission lines based on deep learning as described in claim 1, characterized in that, For regions within the candidate region of interest set, feature enhancement is performed using temporal information from adjacent frames, including: A lightweight convolutional neural network is used to extract features from candidate regions of interest in each frame of the image, resulting in a feature map for each frame. The feature map of the corresponding position of the previous frame is obtained for each frame of the image. The displacement vector estimated by the optical flow method is used to spatially align the feature map of the previous frame. The aligned feature map of the previous frame is then concatenated with the feature map of each frame in the channel dimension. The concatenated feature map is input into the temporal aggregation module, which learns the motion relationship between frames through one-dimensional convolution, fuses cross-frame temporal features, and outputs an enhanced feature map.

4. The deep learning-based target detection method for mechanized operations on power transmission lines as described in claim 3, characterized in that, After outputting the enhanced feature map, the following is included: Calculate the dense optical flow field between adjacent frames, obtain the motion amplitude information of the optical flow field, and normalize and Gaussian smooth it to generate a motion attention heatmap with the same size as the feature map; The motion attention heatmap is multiplied element-wise with the enhanced feature map output by the temporal aggregation module to complete the feature map optimization.

5. The deep learning-based target detection method for mechanized operations on power transmission lines as described in claim 3, characterized in that, A deep learning model is used for target recognition and tracking, outputting detection results with target ID and trajectory, including: The enhanced feature maps corresponding to each frame of the image are sequentially input into the deep learning model according to the temporal order to perform target category prediction and bounding box prediction. An independent identity ID is created for the target detected in each frame of the image, and the position of the corresponding detection box is recorded as a trajectory point. The deep learning model includes a classification branch and a regression branch, which are used to predict the target category and bounding box, respectively. Identify the identification detection box with the largest spatial overlap between all detection boxes in any frame and the detection box in the previous frame. If the maximum overlap is greater than a preset threshold, pass the identity ID of the target in the previous frame to the identification detection box. Connect the center points of the detection boxes with the same identity ID in each frame in chronological order to obtain the target trajectory; The detection results are generated based on the mapping relationship between identity ID and target trajectory.

6. The deep learning-based target detection method for mechanized operations on power transmission lines as described in claim 5, characterized in that, For any unmatched detection box in any frame, assign a new unique identity ID.

7. The deep learning-based target detection method for mechanized operations on power transmission lines as described in claim 1, characterized in that, Based on the detection results, behavioral semantic analysis is performed, and the risk level is assessed according to predefined rules to generate alarm information, including: Construct a semantic knowledge base for early warning behaviors of different preset detection targets; The detection results are input into the semantic knowledge base of the early warning behavior for matching, and alarm information corresponding to the risk level is generated.

8. A deep learning-based target detection device for mechanized operations on power transmission lines, characterized in that, The device is used to perform the deep learning-based target detection method for mechanized operations on power transmission lines as described in any one of claims 1-7, and the device includes: The scene modeling module is used to perform dynamic scene modeling on the input video stream, extract foreground moving target regions and newly emerging static target regions, and merge them to generate a candidate region of interest set. The target recognition module is used to enhance the features of regions within the candidate region of interest set using the temporal information of adjacent frames, and to perform target recognition and tracking using a deep learning model, outputting detection results with an identity ID and target trajectory. The analysis module is used to perform behavioral semantic analysis based on the detection results, assess the risk level according to predefined rules, and generate alarm information.

Images