Image processing-based line dance target tracking detection method and system

Abstract

The application discloses a line dance target tracking detection method and system based on image processing. The method comprises the following steps: collecting the use environment data and use time of the line, generating a reference feature map by using a static reference image, and generating real-time frame features of each frame by using real-time line images. The position and amplitude of the line dance are determined by combining the two, and the dance frequency and the position of the antinode point are calculated. Based on the above data, the real-time dance detection risk coefficient is determined, and the trained model is used to predict the fault occurrence time according to the environment, use time, dance amplitude and frequency. Finally, the line tracking detection report is generated according to the real-time dance detection risk coefficient and the predicted fault time. Through the method of the application, intelligent prediction of the line fault occurrence condition is realized, the safety and reliability of the power system are greatly improved, and the power failure accidents caused by line faults are reduced.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method and system for tracking and detecting galloping targets based on image processing. Background Technology

[0002] In current power system maintenance techniques, the galloping phenomenon on transmission lines, especially high-voltage lines, is typically monitored using cameras installed in fixed locations. While this method can detect galloping to some extent, the fixed camera locations limit the viewing angle and coverage, making it difficult to comprehensively and accurately capture the specific details of the galloping. Furthermore, traditional image processing methods have limitations in extracting line image features under specific conditions, resulting in insufficiently accurate assessments of the line's condition.

[0003] The main challenges faced by the aforementioned methods are as follows: First, due to environmental factors such as weather changes and lighting conditions, existing image processing techniques struggle to reliably and accurately extract key features from line images. Second, even if certain features can be extracted, the relationship between these features and the actual operating environment of the line is not fully understood and utilized, thus limiting the ability to effectively predict line fault risks. Therefore, improving the accuracy of image feature extraction and using it to predict line fault occurrences has become an urgent problem to be solved.

[0004] Therefore, it is necessary to design a new method that can not only extract feature information from line images more accurately, but also comprehensively consider various influencing factors of the line operating environment, so as to realize intelligent prediction of line fault occurrence, greatly improve the safety and reliability of the power system, and reduce power outage accidents caused by line faults. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method and system for tracking and detecting line galloping targets based on image processing.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: Its further technical solution is as follows: The advantages of this invention compared to existing technologies are as follows: By acquiring data on the line's operating environment and the time elapsed, and combining this with a reference feature map generated from a static reference image and real-time frame features generated from a real-time line image, the invention accurately determines the location, amplitude, frequency, and antinode position of line galloping, thereby calculating the real-time galloping detection risk coefficient. Simultaneously, it utilizes a trained line fault occurrence time prediction model to comprehensively analyze the line's operating environment and status, predicting possible fault occurrence times, and ultimately generating a line tracking and detection report containing all the aforementioned information. This method not only extracts feature information from line images more accurately but also comprehensively considers various influencing factors of the line's operating environment, achieving intelligent prediction of line fault occurrence conditions. This significantly improves the safety and reliability of the power system and effectively reduces power outages caused by line faults.

[0007] The present invention will be further described below with reference to the accompanying drawings and specific embodiments. Attached Figure Description

[0008] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0009] Figure 1 A schematic flowchart of the line galloping target tracking and detection method based on image processing provided in an embodiment of the present invention; Figure 2 A schematic block diagram of an image processing-based line galloping target tracking and detection system provided in an embodiment of the present invention; Figure 3 A schematic block diagram of a computer device provided for an embodiment of the present invention. Detailed Implementation

[0010] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0011] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0012] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0013] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0014] Please see Figure 1 , Figure 1 This is a flowchart illustrating the image processing-based line galloping target tracking and detection method provided in this embodiment of the invention. The method is applied to a server. By acquiring line usage environment data and time, static reference images, and real-time line images, spatial attention and temporal attention mechanisms are used to determine the position and amplitude of line galloping, and the galloping frequency and antinode positions are generated. Further, the real-time galloping detection risk coefficient is calculated by combining galloping features and operating environment factors. A trained model is used to predict the time of line fault occurrence, and finally, a line tracking and detection report is generated. This method not only accurately extracts feature information from line images but also comprehensively considers various influencing factors such as the environment, achieving intelligent prediction of line fault conditions. This significantly improves the safety and reliability of the power system and effectively reduces power outages caused by line faults.

[0015] Figure 1 This is a flowchart illustrating the line galloping target tracking and detection method based on image processing provided in an embodiment of the present invention. Figure 1 As shown, the method includes the following steps S110 to S180.

[0016] S110. Obtain the line's usage environment data and the line's usage time.

[0017] In this embodiment, environmental data refers to information obtained through sensor devices that reflects the external conditions of the line operation, including ambient temperature and ambient snowfall.

[0018] Line usage time refers to the time span from when the line was put into use to the present moment, and is used to assess the aging and fatigue condition of the line.

[0019] In this embodiment, environmental data refers to external condition information directly related to the operation of the power line, specifically including but not limited to ambient temperature and snowfall. This data is crucial for assessing the power line's operational status because it directly affects the physical properties of the power line materials and the impact of snow and ice loads on the line. For example, extreme low temperatures may cause conductors to harden or become brittle, increasing the risk of breakage; while heavy snowfall may exacerbate the load on the line, leading to sagging or even breakage. Therefore, by deploying sensor devices (such as temperature sensors and snowfall sensors) around the power line to monitor these parameters in real time, potential faults can be effectively predicted and prevented.

[0020] Ambient temperature: This refers to the atmospheric temperature of the area where the power line is located, usually measured by a temperature sensor installed on-site. This data helps analyze the length changes of the power line caused by thermal expansion and contraction and their impact on the line tension.

[0021] Environmental snowfall: This refers to the amount of snow that accumulates on the surface of power transmission lines over a certain period of time, and is generally monitored using specially designed snowfall sensors. This is of great significance for assessing the line's ability to bear additional weight and potential safety hazards.

[0022] Line service life refers to the time span from the completion of line construction and its official commissioning to the present moment. It reflects the actual service life the line has undergone since its commissioning and is one of the important indicators for measuring its aging degree. Understanding this helps assess the aging condition of line materials, structural fatigue, and other potential problems that may arise over time. Line service life can be determined by reviewing the line's construction records or maintenance logs, ensuring the accuracy of the data used. This step plays a crucial role in developing reasonable maintenance plans and timely updating aging facilities to ensure the stable operation of the power system.

[0023] S120. Obtain a static reference image and generate a reference feature map based on the static reference image.

[0024] In this embodiment, the reference feature map refers to the digital representation of the precise location and shape of the line obtained by analyzing and processing a static reference image.

[0025] Specifically, for example, static reference images of the current railway line, captured under windless or light-wind conditions, are fed as input data into a lightweight railway line detection network (e.g., a model improved based on the DeeplabV3+ architecture). This model can effectively extract key features of the railway line from the static reference image, including but not limited to the line's position coordinates, curvature, and orientation. Through this series of processing steps, a high-precision reference feature map is ultimately generated. This reference feature map not only reflects the initial set position and shape of the railway line but also provides a reference standard for subsequent monitoring of any possible displacement or deformation. Therefore, the reference feature map plays a crucial role in assessing changes in the railway line's state.

[0026] S130. Obtain real-time line images and generate real-time frame features for each real-time frame based on the real-time line images.

[0027] In this embodiment, real-time frame features refer to multi-scale feature representations obtained by analyzing and processing real-time acquired line images, used to describe the position, shape and changes of the target line in each frame image.

[0028] During the current tracking and detection cycle, the pan-tilt unit is controlled to roughly align the observation direction of the shooting device with the area where the target line is located, and coarsely track the line's approximate trajectory. During this process, the observation and aiming component is responsible for acquiring high-definition images of the target line, i.e., real-time line images. The pan-tilt system uses automatic target tracking technology to calculate the line's positional offset in the image in real time, and converts this offset into an angle command that the pan-tilt unit needs to rotate to keep the line near the center of the field of view. Simultaneously, for high-frequency, small-amplitude residual deviations caused by environmental factors (such as wind), the precision mechanism inside the observation and aiming component compensates for them using active optical image stabilization technology. For example, it drives a lightweight mirror to make a small and rapid deflection, thereby correcting the optical path and ensuring that the target image point is stably located at the center of the image sensor.

[0029] Next, these preliminarily calibrated and stabilized real-time line images are input into a specially designed image encoder (e.g., a ViT backbone architecture such as Hiera based on SAM2) to extract multi-scale feature information contained in each frame. The "real-time frame features" referred to here specifically include, but are not limited to, the following aspects: Position and direction information: accurately captures the position coordinates and direction of the target line in each frame of the image.

[0030] Morphological characteristics: Reflecting the geometric properties of the line, such as curvature and degree of bending.

[0031] Dynamic change characteristics: By comparing frame characteristics at different time points, dynamic behavior patterns such as line displacement and vibration can be analyzed.

[0032] Therefore, real-time frame features not only provide fundamental data support for accurate monitoring of line status changes, but also lay a solid technical foundation for further analysis of line operating status and safety assessment. This method can effectively improve the accuracy and response speed of line monitoring systems, which is of great significance for ensuring the safe operation of power transmission networks.

[0033] S140. Determine the line galloping position and amplitude based on the reference feature map and the real-time frame features.

[0034] In this embodiment, the line galloping position refers to the precise calculation of the offset of the target line relative to its original position in the current frame by analyzing the differences between the real-time frame and the reference feature map, combined with spatial attention and temporal attention mechanisms. Specifically, it reflects the positional changes of the line at different points in time, especially the specific position coordinates when the line sways due to external environmental influences (such as wind).

[0035] Galloping amplitude measures the degree of positional change of a railway line from one point in time to another. This variation is typically quantified by comparing changes in line position across multiple consecutive frames, and can be measured by maximum deviation distance, average deviation distance, or other metrics that effectively describe the dynamic behavior of the line.

[0036] In one embodiment, step S140 described above may include steps S141 to S143.

[0037] S141. The reference feature map and the real-time frame features are processed according to the spatial attention mechanism to obtain the real-time line segmentation mask and real-time line features of each real-time frame.

[0038] In this embodiment, the real-time line segmentation mask refers to a binarized image generated by applying a spatial attention mechanism to each frame of the image, used to distinguish between line portions and non-line portions in the image. This mask is crucial for subsequent accurate identification of line locations.

[0039] Real-time line features refer to the high-dimensional feature representations extracted from the enhanced feature map, which contain rich information about the line shape, appearance and direction, providing basic data support for subsequent time series analysis.

[0040] S142. Construct a memory bank based on the real-time line segmentation mask and the real-time line features.

[0041] In this embodiment, the memory bank refers to a storage structure used to store the real-time line segmentation mask and real-time line features of all historical frames processed by the spatial attention mechanism. The existence of the memory bank enables the system to utilize information from previous frames for more accurate spatiotemporal analysis, thereby improving detection accuracy.

[0042] S143. Determine the position and amplitude of the line dance based on the memory bank and time attention mechanism.

[0043] In one embodiment, step S143 described above may include steps S1431 to S1432.

[0044] S1431. Determine the historical line segmentation mask and historical line features of the historical frame corresponding to the real-time frame to be processed according to the memory bank.

[0045] In this embodiment, the historical line segmentation mask refers to the binary mask corresponding to a specific historical moment stored in the memory bank, which is used to compare and analyze with the current frame to help identify the changing trend of the line position.

[0046] Historical route features refer to the high-dimensional feature representation corresponding to the historical route segmentation mask, which contains information about the state of the route at that moment.

[0047] S1432. Based on the time attention mechanism, the historical line segmentation mask and historical line features of the historical frame are processed to determine the line dancing position and dancing amplitude.

[0048] By integrating and analyzing relevant information from historical frames through a time-attention mechanism, it is possible not only to track changes in the line's position over time but also to assess its galloping amplitude. This method effectively combines precise spatial positioning capabilities with dynamic change tracking capabilities in the temporal dimension, thus providing a comprehensive and detailed line status monitoring solution.

[0049] In this embodiment, for example, the reference feature map and real-time frame features are processed according to the trained spatial attention mechanism to obtain the real-time line segmentation mask and real-time line features of each real-time frame. For example, the core task of the trained spatial attention mechanism is to determine the difference between the actual position of the line in the current real-time frame and the reference feature map. In the cross-attention layer of the mask decoder, the trained spatial attention mechanism will simultaneously calculate the correlation between the real-time frame features of the current real-time frame and the features of the frames before the current real-time frame (inheriting SAM2 for temporal smoothing), and the correlation between the real-time frame features of the current real-time frame and the reference feature map (used to guide the spatial attention mechanism to focus on the offset region from the static reference. For example, when the background is complex (e.g., forest, mountains), the spatial attention mechanism will prioritize the region near the reference line position and with obvious feature differences, thereby significantly improving the robustness of detecting low-contrast, fine lines). The spatial attention mechanism outputs enhanced feature maps for each real-time frame (a high-dimensional, dense feature representation containing rich information such as the shape, appearance, and orientation of the line, suitable for subsequent temporal attention mechanisms for tracking and matching). The enhanced feature maps are fed into a lightweight mask decoder. The decoder classifies each pixel based on the enhanced features (i.e., determines whether each pixel is a pixel at the location of the line or not), generating a pixel-accurate binary mask, i.e., the real-time line segmentation mask. The enhanced feature map itself is the extracted real-time line feature. Based on the real-time line segmentation mask and real-time line features obtained by processing each real-time frame using the trained spatial attention mechanism, a memory bank is constructed. The output of the trained spatial attention mechanism stored in the memory bank is processed by the temporal attention mechanism to obtain the line's dancing position and dancing amplitude.

[0050] Based on the memory bank, the historical line segmentation mask and historical line features of the historical frames corresponding to the current real-time frame to be processed are determined. For example, when the time attention mechanism processes the t-th real-time frame, the time attention mechanism will retrieve the real-time line segmentation mask and real-time line features from the 1st real-time frame to the (t-1)th real-time frame (i.e., the historical line segmentation mask and historical line features of the historical frames corresponding to the current real-time frame to be processed) from the memory bank as input to determine the line galloping position and galloping amplitude of the line in the t-th real-time frame.

[0051] In one embodiment, the training process of the above-described spatial attention mechanism includes: Acquire multiple historical training route images; Perform pixel-level analysis on each historical training line image and extract the set of all pixels. For each pixel in the set, background line identification is performed to form a background line identification result, and feature classification weight, result uncertainty weight, and line shape weight are calculated. Based on the historical training route images, construct historical training baseline feature maps and historical training frame features; The spatial attention mechanism is used to perform deep processing on the historical training baseline feature map and the historical training frame features to obtain the training line segmentation mask and training line features. Calculate the mean value of line pixel features and the mean value of background pixel features based on the training line features; The horizontal and vertical gradients are calculated using the training line segmentation mask to obtain gradient information; Combining the training line segmentation mask, background line recognition results, feature classification weights, result uncertainty weights, line shape weights, mean line pixel features, mean background pixel features, and gradient information, a training loss function for the spatial attention mechanism is constructed; wherein, the training loss function for the spatial attention mechanism includes: ;in, Let be the training loss function for the spatial attention mechanism. This represents the background line recognition result for the j-th pixel in the k-th historical training line image. The training line segmentation mask for the j-th pixel in the k-th historical training line image. This is the compressed value of the training line segmentation mask for the j-th pixel in the k-th historical training line image. The feature classification weights for the j-th pixel in the k-th historical training line image are: Let the uncertainty weight of the result be the j-th pixel in the k-th historical training line image. The weight of the j-th pixel in the k-th historical training line image is the line shape weight. Let be the mean value of the line pixel features of the k-th historical training line image. The mean value of the background pixel features of the k-th historical training line image. The mean values ​​of the line pixel features and the mean values ​​of the background pixel features of the k-th historical training line image. Norm, The horizontal gradient of the training line segmentation mask for the j-th pixel in the k-th historical training line image. Let K be the vertical gradient of the training line segmentation mask for the j-th pixel in the k-th historical training line image, where K is the number of historical training line images (k≤K), m is the number of pixels (j≤m), and k, K, j, and m are all positive integers.

[0052] The spatial attention mechanism is trained using the aforementioned training loss function.

[0053] The background line recognition result refers to determining whether each pixel belongs to a line (marked as 1) or the background (marked as 0) based on manually labeled data. This step provides basic ground truth data for training.

[0054] Feature classification weights measure how much each pixel's features lean towards the lines or the background. They are calculated by comparing the distances of each pixel to the known foreground and background feature vectors; the smaller the distance, the greater the classification weight.

[0055] The uncertainty weight represents the degree of uncertainty of the spatial attention mechanism in pixel classification. It is usually calculated based on the absolute value of the difference between the entropy of the pixel feature vector and the predicted probability and 0.5. The higher the entropy or the closer the predicted probability is to 0.5, the higher the uncertainty weight.

[0056] Line shape weights emphasize the importance of pixels in the line structure. They consider the distance of a pixel to the center line of the actual mask and its gradient at the mask output; pixels that are closer and have smoother gradients receive higher shape weights.

[0057] Historical training baseline feature map refers to the feature representation of line images under windless or light wind conditions, used to provide a stable reference point.

[0058] Historical training frame features refer to the feature extraction results of each frame of historical training line image, which serve as the basis for real-time frame comparison.

[0059] The mean feature value of line pixels refers to the average feature vector of all pixels predicted by the spatial attention mechanism as the line.

[0060] The mean background pixel feature refers to the average feature vector of all pixels predicted as background by the spatial attention mechanism.

[0061] Gradient information refers to the partial derivatives of the training line segmentation mask in the horizontal and vertical directions, which are used to evaluate the sharpness and smoothness of the segmentation mask edges.

[0062] Using the above metrics, a comprehensive training loss function is constructed to optimize the model's sensitivity to key features, reduce classification uncertainty, make the segmentation results more consistent with the actual line shape, and improve visual quality and structural accuracy.

[0063] By minimizing the loss function using the backpropagation algorithm, model parameters are adjusted to gradually improve model performance until the desired goal is achieved. This method effectively enhances the robustness and accuracy of spatial attention mechanisms in complex environments.

[0064] In this embodiment, for example, multiple manually labeled historical training route images are acquired; the pixel set of each historical training route image is determined according to its size; based on the manually labeled training dataset, the background route recognition result of each pixel in the historical training route image is determined (the background route recognition result is 0 when the pixel is background, and 1 when the pixel is route). The feature classification weight measures whether the features of each pixel are more biased towards route or background. The feature classification weight is calculated as follows: the feature mean of all real foreground pixels (pixels labeled as route) and the feature mean of all real background pixels (pixels labeled as background) in the historical training route image are calculated. The spatial attention mechanism calculates the distance difference between the feature vector extracted from the pixel and the real foreground and background, and maps this distance difference to a feature classification weight using the tanh function. A larger feature classification weight indicates a more precise model classification. The uncertainty weight represents the uncertainty of the spatial attention mechanism's pixel classification. A higher uncertainty weight indicates greater classification uncertainty, and the more attention the model pays to pixels that are difficult to classify. The uncertainty weight is calculated as follows: the entropy of the feature vector extracted from the pixel by the spatial attention mechanism is calculated; a higher entropy indicates a more chaotic feature vector and higher uncertainty. The probability that the spatial attention mechanism predicts the pixel belongs to a line is obtained, and this probability is then compared with 0.The absolute value of the difference of 5 determines the confidence level of the combined probability prediction. When the probability is close to 1 (definitely the pixel is a line) or close to 0 (definitely the pixel is background), the confidence level of the combined probability prediction is high, indicating that the spatial attention mechanism is very confident in its judgment. The uncertainty weight of the result is determined by multiplying the entropy of the feature vector by 1 minus the confidence level of the combined probability prediction. The line shape weight is used to indicate the shape of the line (thin, continuous) of the spatial attention mechanism, avoiding the spatial attention mechanism learning an incorrect model that violates the laws of physics. The line shape weight is calculated as follows: calculate the distance from the pixel to the center line of the labeled real mask, calculate the gradient of the mask output by the spatial attention mechanism at that pixel, and determine the line shape weight by multiplying the distance and gradient. The larger the line shape weight, the more important the pixel is structurally (e.g., located near the center line of the real line, and the mask output by the spatial attention mechanism changes smoothly at this point), and the higher the attention of the spatial attention mechanism to that pixel. Based on the historical training line images, the historical training baseline feature map and historical training frame features are determined. For example, images of the line in the historical training line images under windless or light wind conditions are obtained, and the features are analyzed by light wind. A large-scale line detection network extracts features from the image, obtaining historical training baseline feature maps. An image encoder then acquires features from each frame of the historical training line images, i.e., historical training frame features. A spatial attention mechanism is used to process the historical training baseline feature maps and historical training frame features to obtain training line segmentation masks and training line features. Based on the training line features, the mean features of line pixels and the mean features of background pixels are determined. For example, the mean feature vector of pixels predicted as lines by the spatial attention mechanism is calculated, i.e., the mean feature vector of pixels predicted as background by the spatial attention mechanism is calculated. The average feature vector, i.e., the mean value of background pixel features; based on the training line segmentation mask, determine the partial derivatives of the training line segmentation mask in the horizontal direction, i.e., the horizontal gradient of the training line segmentation mask, and the partial derivatives of the training line segmentation mask in the vertical direction, i.e., the vertical gradient of the training line segmentation mask; based on the training line segmentation mask, background line recognition results, feature classification weights, result uncertainty weights, line shape weights, mean value of line pixel features, mean value of background pixel features, horizontal gradient of the training line segmentation mask, and vertical gradient of the training line segmentation mask, determine the training loss function for the spatial attention mechanism.

[0065] According to one embodiment of the present invention, the training loss function of the spatial attention mechanism is determined based on the training line segmentation mask, the background line recognition result, the feature classification weight, the result uncertainty weight, the line shape weight, the mean value of line pixel features, the mean value of background pixel features, the horizontal gradient of the training line segmentation mask, and the vertical gradient of the training line segmentation mask. .

[0066] In one embodiment, This is the compressed value of the training line segmentation mask for the j-th pixel in the k-th historical training line image (determined by the Sigmoid function), which represents the probability that the model predicts this pixel to be a line. Between 0 and 1, This represents the base loss value, determined by comparing the model's predicted probability of the pixel being a line with the background line identification result. The smaller the difference between the model's predicted probability of the pixel being a line and the background line identification result, the smaller the base loss value; conversely, the larger the difference, the larger the base loss value. The feature classification weights for the j-th pixel in the k-th historical training line image are based on... Training allows for higher feature discrimination of pixels, resulting in a larger loss value and stronger training intensity. This guides the spatial attention mechanism to focus on pixels with high feature discrimination, strengthens its utilization of existing knowledge, avoids wasting effort on pixels with ambiguous features, and enhances its sensitivity to key features. Training allows for higher pixel feature entropy, resulting in a larger loss value and stronger training intensity. This can guide the spatial attention mechanism to actively learn from difficult samples with cluttered feature information, reducing prediction uncertainty. Training can be performed to make pixels closer to the center line, with smaller gradients, larger loss values, and stronger training intensity. This allows the segmentation results of the spatial attention mechanism to better match the shape of the actual line. This represents the first loss function value determined based on the base loss value, feature classification weight, result uncertainty weight, and line morphology weight.

[0067] In one embodiment, This represents the squared Euclidean distance between the mean value of the line pixel features and the mean value of the background pixel features in the k-th historical training line image. The larger this value, the greater the difference between the line and background feature prototypes, and the easier it is for the spatial attention mechanism to distinguish between the line and the background. This loss term serves as a regularization to prevent the spatial attention mechanism from overfitting to the noise in the training data and encourages the learning of more generalized features. This represents the normalized loss term.

[0068] In one embodiment, The loss function is the sum of the absolute values ​​of the horizontal and vertical gradients of the training line segmentation mask for the j-th pixel in the k-th historical training line image. A larger loss value indicates more drastic average changes in the mask image. During training, minimizing this loss value results in smaller gradients in the training line segmentation mask generated by the spatial attention mechanism, leading to a smoother mask image. This improves the visual quality and structural accuracy of the segmentation results. This represents the smoothed loss term after normalization.

[0069] In this way, the training loss function of the spatial attention mechanism can be determined based on the background line recognition result, feature classification weight, result uncertainty weight, line shape weight, mean line pixel feature, mean background pixel feature, horizontal gradient of the training line segmentation mask, and vertical gradient of the training line segmentation mask. During the calculation process, the first loss term, regularization loss term, and smoothing loss term can be determined based on the differences between the background line recognition result and the training line segmentation mask, the differences between the mean line pixel feature and the mean background pixel feature, and the horizontal and vertical gradients of the training line segmentation mask. The first loss term is then weighted according to the feature classification weight, result uncertainty weight, and line shape weight to determine the training loss function of the spatial attention mechanism. Training with this training loss function can specifically improve the sensitivity of the spatial attention mechanism to key features, making the segmentation result of the spatial attention mechanism more consistent with the actual line shape and improving the visual quality and structural accuracy of the segmentation result.

[0070] S150. Generate the dancing frequency and antinode position based on the real-time frame features.

[0071] In this embodiment, the galloping frequency refers to the vibration frequency of the line under the action of wind or other external forces. By analyzing the displacement of specific points on the centerline of the line over time (usually the coordinate change perpendicular to the line direction), the vibration modes of these points can be identified, and the time-domain signal can be converted into a frequency-domain signal by using Fast Fourier Transform (FFT) to determine the frequency with the highest energy as the galloping frequency.

[0072] Antinodes are points where the vibration amplitude is largest during railway line vibration. These points are key locations in the vibration mode, and their vertical displacement (usually in Y-coordinate) is more significant compared to other points. Identifying the locations of these antinodes allows for a better understanding of the vibration characteristics and distribution of the railway line.

[0073] In one embodiment, step S150 described above may include steps S151 to S154.

[0074] S151. Refine the real-time line segmentation mask to determine the line centerline.

[0075] In this embodiment, the line centerline refers to a single-pixel wide line after thinning, representing the geometric center of the actual line. This centerline can be extracted from the original binary image by performing morphological operations (such as skeletonization or thinning algorithms) on the real-time line segmentation mask.

[0076] S152. Based on the centerline of the line, take points at equal intervals on the centerline and analyze the changes of the specified coordinates over time to identify the location of the antinode with the largest amplitude.

[0077] On the established centerline of the route, select a series of reference points at equal intervals.

[0078] For each point, pay special attention to the curve of its coordinate (usually the Y coordinate) perpendicular to the line direction as a function of time.

[0079] Analyze these curves to identify the points with the largest amplitudes, i.e., the antinodes. Amplitude is defined as half the difference between a wave crest and a wave trough; the points with the largest amplitudes indicate that they are the most active parts of the vibration mode.

[0080] S153. Generate a vibration signal based on the time sequence of the antinode position change.

[0081] In this embodiment, the vibration signal refers to a data sequence showing the change in the position of the antinode over time. This sequence records the fluctuations in the vertical displacement of the antinode over time, forming a clear vibration signal. This signal reflects the dynamic behavior of the line under external influences.

[0082] S154. Determine the peak with the highest energy in the spectrum diagram based on the vibration signal to obtain the dancing frequency.

[0083] Apply Fast Fourier Transform (FFT) to the vibration signal obtained in the previous step to transform it from the time domain to the frequency domain.

[0084] The spectrum diagram shows the energy distribution of different frequency components, with the frequency corresponding to the highest energy peak being the galloping frequency. This frequency value is crucial for assessing the safety and stability of the line, as it directly relates to whether the line will be damaged by vibration.

[0085] Through the above steps, a complete calculation process from real-time frame features to galloping frequency and antinode location is realized. This not only helps to improve the understanding of the line's operating status, but also provides important data support for subsequent maintenance and safety assessment.

[0086] S160. Determine the real-time dance detection risk factor based on the dance amplitude, the dance position of the line, the dance frequency, and the antinode position.

[0087] In this embodiment, the galloping detection hazard factor refers to a comprehensive assessment of the risk level faced by transmission lines due to galloping. This factor considers multiple factors, including but not limited to the galloping amplitude, line spacing, the hazard of the galloping location, and the hazard of the antinode location. By quantifying these factors, the safety status of the line can be monitored and risk assessed in real time.

[0088] In one embodiment, step S160 described above may include steps S161 to S164.

[0089] S161. Obtain the distance between the current line and the adjacent lines.

[0090] In this embodiment, line spacing refers to the minimum distance between the current line and its nearest other transmission line, obtained from a static reference image or actual measurement. This parameter is crucial for assessing electrical hazards because a smaller spacing increases the risk of arcing caused by galloping.

[0091] S162. Determine the hazard identification result of the line galloping position based on the line galloping position.

[0092] In this embodiment, the hazard identification result of the line galloping location refers to a binary value (0 or 1) to indicate whether the galloping location is close to high-voltage power lines or other critical areas. If the galloping location is close to high-voltage power lines, there is a serious safety threat, and the identification result is 1; otherwise, if there is no obvious safety threat, the identification result is 0.

[0093] S163. Determine the hazard identification result of the antinode position based on the antinode position.

[0094] In this embodiment, the hazard identification result of the antinode location is a binary value (0 or 1), used to assess whether the antinode location is close to the suspension point of the line or other areas of mechanical stress concentration. If the antinode is located in such a high-risk area, it indicates that a serious accident such as line breakage or tower collapse may occur, and the identification result is 1; otherwise, the identification result is 0.

[0095] S164. Based on the galloping amplitude, the line spacing, the galloping position hazard identification result, and the antinode position hazard identification result, determine the real-time galloping detection hazard coefficient, wherein the real-time galloping detection hazard coefficient integrates electrical hazard and tension hazard to quantify the risk level of line galloping.

[0096] The formula for the real-time dance detection risk factor is expressed as follows: ; To detect the risk level of the dance in real time, The results of the line galloping position hazard identification are as follows. , The results of hazard identification at the antinode location. , For the range of motion, For line spacing, This is a preset threshold for the amplitude of the dance.

[0097] In this embodiment, the line spacing between the current line and adjacent lines under static conditions is obtained through a static reference image. Based on the line galloping position, the hazard identification result of the line galloping position is determined. For example, if the line galloping position is close to a high-voltage power line, there is a serious safety hazard, and the hazard identification result is 1; otherwise, it is 0. If the antinode is close to the suspension point of the line, serious accidents such as line breakage or tower collapse may occur soon, with an extremely high risk of mechanical damage; the hazard identification result is 1, and otherwise, it is 0. Based on the galloping amplitude, line spacing, hazard identification result of the line galloping position, and hazard identification result of the antinode, the hazard status of the line galloping is assessed in real time, and the real-time galloping detection hazard coefficient is determined.

[0098] In this embodiment, This is the ratio of the arcing amplitude to the distance between lines. The larger this ratio, the closer the line is to adjacent lines. In this case, high voltage can directly break down the air, causing an electric arc discharge, resulting in a more severe electrical hazard and a higher risk factor. This is the ratio of the galloping amplitude to a preset galloping amplitude threshold. The larger this ratio, the greater the absolute amplitude of the line galloping. Even if the electrical distance is sufficient, it means that the tension change on the conductor itself is extremely large, posing a risk of overload. The more severe the tension danger, the greater the risk factor. This indicates that the real-time galloping detection hazard coefficient is determined based on four aspects: the hazard identification results of the line galloping location, the hazard identification results of the antinode location, the electrical hazard status, and the tension hazard status.

[0099] In this way, the real-time galloping detection hazard coefficient can be determined based on the galloping amplitude, line spacing, galloping location hazard identification results, and antinode location hazard identification results. During the calculation process, the real-time galloping detection hazard coefficient can be determined based on four aspects: the galloping location hazard identification results, the antinode location hazard identification results, the electrical hazard status of the galloping, and the tension hazard status of the galloping, thus improving the comprehensiveness and accuracy of the real-time galloping detection hazard coefficient.

[0100] S170. The usage environment data, the line usage time, the galloping amplitude, and the galloping frequency are processed according to the trained line fault occurrence time prediction model to generate a predicted line fault occurrence time.

[0101] In this embodiment, predicting the time of line failure refers to calculating the probability that the line may fail at a future point in time by using a pre-trained machine learning or statistical model, combined with current line operating environment data (such as temperature, humidity, wind speed, etc.), the line's usage time, the amplitude of galloping, and the frequency of galloping. This predicted time provides an important reference indicator for assessing the impact of current operating conditions on the line's lifespan and guiding when to perform maintenance work.

[0102] Collect input data: Use environmental data, including but not limited to temperature, humidity, wind speed, and rainfall, which can be obtained in real time through weather stations or sensors installed on power transmission lines.

[0103] Line usage time: refers to the time span from the completion of line installation to the present moment, usually measured in years.

[0104] Sway amplitude: the range of swaying of a power transmission line caused by external forces (such as wind).

[0105] Galloping frequency: refers to the number of times a transmission line gallops per unit of time.

[0106] The collected data were standardized or normalized to ensure they were within the same scale range, which facilitated subsequent model processing.

[0107] The prepared input data is analyzed using a pre-trained model. This model may be built based on various algorithms, such as linear regression, decision trees, random forests, and neural networks, and aims to identify the potential relationship between input variables and line failure times.

[0108] Based on the model output, the probability of a transmission line fault occurring within a specific future time period is determined. The "predicted time of line fault occurrence" mentioned here is actually an estimate, representing the length of time the line is expected to continue operating safely under current galloping conditions and environmental factors.

[0109] The shorter the predicted time of a line fault, the greater the damage caused to the line by the current operating conditions, requiring inspection and maintenance to be arranged as soon as possible; conversely, a longer predicted time indicates that the line is currently in a relatively stable state.

[0110] The resulting predicted line fault occurrence time not only helps power companies to rationally plan maintenance schedules and reduce unnecessary downtime, but also improves the overall reliability and security of the power grid.

[0111] In summary, the core of step S170 lies in using advanced data analysis methods and technologies to accurately predict the future health status of transmission lines, thereby achieving more scientific and effective asset management.

[0112] Therefore, by processing usage environment data, line usage time, galloping amplitude, and galloping frequency based on the trained line fault occurrence time prediction model, the next fault occurrence time of the line under the current galloping and environmental conditions can be predicted. In other words, the line fault occurrence time is predicted. The shorter the predicted line fault occurrence time, the greater the impact of the current galloping and environmental conditions on the line's lifespan, and the more maintenance is required.

[0113] The training process of the line fault occurrence time prediction model includes: Acquire historical usage environment data, historical usage time, historical galloping amplitude, and historical galloping frequency of multiple similar lines during historical testing periods. The historical usage environment data includes historical ambient temperature and historical ambient snowfall. Obtain the historical actual line fault occurrence times for multiple similar lines; The historical usage environment data, historical line usage time, historical galloping amplitude, and historical galloping frequency are processed according to the line fault occurrence time prediction model to obtain the historical sample predicted line fault occurrence time. Based on the actual historical line fault occurrence time, the predicted line fault occurrence time from the historical samples, the historical usage environment data, the historical line usage time, the historical galloping amplitude, and the historical galloping frequency, the training loss function of the line fault occurrence time prediction model is determined. The line fault occurrence time prediction model is trained using the training loss function of the model to obtain the trained model.

[0114] In this embodiment, for example, historical usage environment data, historical line usage time, historical galloping amplitude, and historical galloping frequency of multiple similar lines of the same type as the currently tracked and detected line (usually lines that have been used for a long time under normal use) are obtained during historical test cycles. The historical ambient temperature, historical galloping amplitude, and historical galloping frequency are all average values ​​within the historical test cycles. Accelerated aging experiments are conducted on the similar lines during the historical test cycles to determine the actual historical line fault occurrence time during the test cycles. The historical usage environment data, historical line usage time, historical galloping amplitude, and historical galloping frequency are processed according to the line fault occurrence time prediction model to obtain the predicted line fault occurrence time for historical samples. Based on the actual historical line fault occurrence time, the predicted line fault occurrence time for historical samples, the historical usage environment data, the historical line usage time, the historical galloping amplitude, and the historical galloping frequency, the training loss function of the line fault occurrence time prediction model is determined. The method for determining the training loss function of the line fault occurrence time prediction model is as follows: it can be based on... Determine the training loss function for the line fault occurrence time prediction model. ,in, Let i be the actual historical line fault occurrence time of the i-th similar line in the r-th historical test period. For the i-th similar line, predict the line fault occurrence time based on historical samples in the r-th historical test period. The historical ambient temperature of the i-th similar line in the r-th historical test cycle. To preset the ambient temperature threshold, Let represent the historical environmental snowfall for the i-th similar route during the r-th historical test period. To preset the snowfall threshold, The historical usage time of the i-th similar line in the r-th historical test period. To preset the used time threshold, Let represent the historical galloping amplitude of the i-th similar line in the r-th historical test period. To preset the threshold for the amplitude of the dance, Let be the historical galloping frequency of the i-th similar line in the r-th historical test cycle. The preset dancing frequency threshold is defined as follows: n is the number of similar lines, R is the number of historical test cycles, i≤n, r≤R, and i, r, R and n are all positive integers. This is the ratio of the historical galloping amplitude of the i-th similar line in the r-th historical test period to the preset galloping amplitude threshold. The larger this ratio, the larger the historical galloping amplitude. The preset galloping amplitude threshold can be set to 10cm. This is the ratio of the historical galloping frequency of the i-th similar line in the r-th historical test cycle to the preset galloping frequency threshold. The larger this ratio, the higher the historical galloping frequency. The preset galloping frequency threshold can be set to 1Hz. This indicates a negative correlation between the predicted time of line faults from historical samples and the historical galloping amplitude and frequency. For example, the larger the historical galloping amplitude, the greater the tension on the line itself, the greater the risk of overload, and the more likely the line is to fail. The smaller the predicted time of line faults from historical samples, the higher the historical galloping frequency, the greater the metal fatigue effect, and the more likely the line is to experience fatigue fracture. Therefore, placing the terms related to historical galloping amplitude and frequency in the denominator indicates that the smaller the historical galloping amplitude and frequency, the larger the predicted time of line faults from historical samples, and the greater the impact on the error of the predicted time of line faults from historical samples. This is the ratio of the absolute value of the difference between the historical ambient temperature and the preset ambient temperature threshold of the i-th similar line in the r-th historical test cycle to the preset ambient temperature threshold. The larger this ratio, the greater the difference between the historical ambient temperature and the preset ambient temperature threshold. The preset ambient temperature threshold can be set to 15 degrees Celsius. This is the ratio of the historical environmental snowfall amount to the preset snowfall threshold for the i-th similar route in the r-th historical test period. The larger this ratio, the greater the historical environmental snowfall amount. The preset snowfall threshold can be set to 1mm. This is the ratio of the historical line usage time of the i-th similar line in the r-th historical test period to a preset usage time threshold. The larger this ratio, the longer the historical line has been used. The preset usage time threshold can be set to 1 year. This indicates a negative correlation between the predicted fault occurrence time of historical samples and the absolute value of the difference between historical ambient temperature and the preset ambient temperature threshold, historical snowfall, and historical line usage time. For example, higher historical ambient temperatures result in higher line temperatures, leading to annealing effects on the aluminum strands and steel core. The greater the decrease in tensile strength of aluminum and steel, the more prone the line is to faults, and the shorter the predicted fault occurrence time. Conversely, lower historical ambient temperatures result in lower line temperatures, causing the metal materials within the line to transition from a ductile to a brittle state at low temperatures. The greater the decrease in impact resistance, the more prone the line is to faults, and the shorter the predicted fault occurrence time. The larger the historical sample data, the smaller the predicted line fault occurrence time. When the historical ambient snowfall is high, the line load is greater due to snow adhesion, and snow covering the insulator strings shortens their effective creepage distance, making flashover more likely. The high-temperature arc generated by flashover can burn the insulator enamel surface, damaging its insulation performance and permanently reducing its service life. A smaller predicted line fault occurrence time indicates a shorter remaining lifespan for lines with longer service lives. Therefore, placing the absolute value of the difference between historical ambient temperature and the preset ambient temperature threshold, the historical ambient snowfall, and the historical line service life in the denominator indicates that the smaller these values, the larger the predicted line fault occurrence time, and the greater the impact on the error in the historical sample prediction of line fault occurrence time. To determine the relative error between the actual historical fault occurrence time and the predicted fault occurrence time from historical samples for the i-th similar line in the r-th historical test cycle, we use... and A weighted average is calculated between the relative errors of historical actual line fault occurrence times and historical sample predictions of line fault occurrence times to obtain the training loss function for the line fault occurrence time prediction model. During training, this training loss function is reduced, thereby decreasing the error between the historical actual line fault occurrence times and the historical sample predictions of line fault occurrence times. This improves the prediction accuracy of the line fault occurrence time prediction model, thus enhancing its overall accuracy. The line fault occurrence time prediction model is then trained using this training loss function to further improve its prediction accuracy, resulting in a trained line fault occurrence time prediction model. or If the denominator is 0, then the term is directly set to 1. For example, if Then ignore directly. Only calculate The value.

[0115] S180. Generate a line tracking and detection report based on the real-time galloping detection risk coefficient and the predicted line fault occurrence time.

[0116] In this embodiment, the line tracking and detection report refers to a detailed line status assessment document automatically generated based on the analysis results of real-time galloping detection risk factors and predicted line fault occurrence times. This report aims to provide a comprehensive perspective for assessing the current safety status of transmission lines and to provide a basis for subsequent maintenance or emergency response measures.

[0117] Specifically, a line tracing and testing report can include the following key parts: Report Structure and Content: Basic Information: Line Name or Number; Testing Date and Time; Testing Equipment Information (e.g., Sensor Type, Installation Location, etc.); Real-time goofing detection risk factor: describes the goofing situation within the current measurement period, including the goofing amplitude, frequency and its changing trend.

[0118] It provides a specific numerical value for the real-time vibration detection risk factor. For example, if the factor is greater than 0.5, it indicates that the current line is in a high-risk state; conversely, it indicates that it is relatively safe.

[0119] If the risk level exceeds the threshold, it is recommended to take immediate action and notify relevant personnel to conduct an on-site inspection.

[0120] Predicting line fault occurrence time: Analyzing the length of time the line is expected to operate safely under existing operating conditions (environmental factors and galloping behavior).

[0121] If the predicted failure time is less than 30 days, it indicates a potential major safety hazard, and maintenance work needs to be arranged as soon as possible; otherwise, the tracking and monitoring can continue as planned.

[0122] Based on the above two indicators, an overall safety assessment of the line should be made. If either indicator shows an anomaly, the possible causes and recommended countermeasures should be explained in detail.

[0123] Suggest specific maintenance measures or directions for further investigation, such as whether it is necessary to adjust the inspection frequency, add temporary support devices, or replace damaged parts.

[0124] Charts are used to illustrate the changing trends of galloping parameters over a period of time, helping to more intuitively understand the evolution of line conditions. The curve showing the change of the real-time galloping detection risk factor over time is displayed, highlighting high-risk periods. The calculation results for predicting the time of failure and their range of uncertainty are presented graphically.

[0125] Summarize the key findings of the current testing cycle and propose a clear action plan for the next steps.

[0126] If emergency intervention is required, a list of the highest priority tasks should be compiled, and the relevant personnel responsible for carrying them out should be designated.

[0127] In this way, the S180 procedure not only ensures continuous monitoring of the health status of transmission lines but also provides scientific decision support, contributing to improved grid operation efficiency and safety. Furthermore, this systematic reporting mechanism helps record historical data, facilitating long-term trend analysis and the improvement of maintenance strategies.

[0128] The method in this embodiment can accurately analyze the galloping location, amplitude, frequency, and antinode location in real-time line images. By precisely measuring these parameters, it further enables the assessment of real-time dangers associated with line galloping and the prediction of line fault occurrence times, thereby achieving real-time monitoring of line galloping conditions and effectively reducing the likelihood of line faults.

[0129] In determining the training loss function for the spatial attention mechanism, this embodiment considers factors such as background line recognition results, feature classification weights, result uncertainty weights, line shape weights, mean line pixel features, mean background pixel features, and the horizontal and vertical gradients of the training line segmentation mask. By calculating the differences between the background line recognition results and the training line segmentation mask, the differences between the mean line pixel features and the mean background pixel features, and the horizontal and vertical gradients of the training line segmentation mask, the first loss term, regularization loss term, and smoothing loss term are determined respectively. Subsequently, the first loss term is weighted using feature classification weights, result uncertainty weights, and line shape weights to finally determine the training loss function for the spatial attention mechanism. This method not only improves the sensitivity of the spatial attention mechanism to key features but also makes its segmentation results more closely resemble the actual line shape, improving the visual quality and structural accuracy of the segmentation results.

[0130] Furthermore, this embodiment comprehensively considers multiple factors when determining the real-time galloping detection hazard factor, including galloping amplitude, line spacing, galloping location hazard identification results, and antinode location hazard identification results. Specifically, the real-time galloping detection hazard factor is determined through four aspects: the galloping location hazard identification results, the antinode location hazard identification results, the electrical hazard status of the galloping, and the tension hazard status. This method greatly improves the comprehensiveness and accuracy of the real-time galloping detection hazard factor, helping to more accurately assess the safety status of the lines.

[0131] In summary, the method presented in this embodiment not only enables precise tracking and analysis of line galloping states at the technical level, but also significantly improves the system's ability to capture key features by introducing advanced algorithmic models, such as spatial attention mechanisms and their training loss functions, ensuring the quality and reliability of the segmentation results. Furthermore, the real-time galloping detection risk coefficient, determined through multi-dimensional considerations, further enhances the accuracy of judging the line safety status, providing a solid technical guarantee for the safe operation of the power grid.

[0132] Specifically, the method in this embodiment accurately analyzes the galloping position, amplitude, frequency, and antinode position of the line in the real-time line image. Furthermore, by assessing the real-time danger of line galloping based on the galloping position, amplitude, frequency, and antinode position, and predicting the time of line fault occurrence, the method can detect line galloping in real time and reduce the possibility of line faults. When determining the training loss function of the spatial attention mechanism, it can be based on the background line recognition result, feature classification weight, result uncertainty weight, line shape weight, mean line pixel feature, mean background pixel feature, horizontal gradient of the training line segmentation mask, and vertical gradient of the training line segmentation mask. During the calculation, the first loss term, regularization loss term, and smoothing loss term can be determined based on the differences between the background line recognition result and the training line segmentation mask, the differences between the mean line pixel feature and the mean background pixel feature, and the horizontal and vertical gradients of the training line segmentation mask. The first loss term is then weighted according to the feature classification weight, result uncertainty weight, and line shape weight to determine the training loss function of the spatial attention mechanism. Training with this loss function can specifically improve the sensitivity of the spatial attention mechanism to key features, making the segmentation results more closely resemble the actual line shape and improving the visual quality and structural accuracy of the segmentation results. When determining the risk factor for real-time galloping detection, the risk factor can be determined based on the galloping amplitude, line spacing, the results of hazard identification at the galloping position, and the results of hazard identification at the antinode position. During the calculation process, the risk factor for real-time galloping detection can be determined based on four aspects: the results of hazard identification at the galloping position, the results of hazard identification at the antinode position, the electrical hazard status of the galloping, and the tension hazard status of the galloping. This improves the comprehensiveness and accuracy of the real-time galloping detection risk factor.

[0133] The aforementioned image processing-based line galloping target tracking and detection method accurately determines the position, amplitude, frequency, and antinode location of line galloping by acquiring line usage environment data and usage time, and combining the baseline feature map generated from static baseline images with the real-time frame features generated from real-time line images. This allows for the calculation of the real-time galloping detection risk coefficient. Simultaneously, a trained line fault occurrence time prediction model is used to comprehensively analyze the line operating environment and status, predicting possible fault occurrence times. Finally, a line tracking and detection report containing all the aforementioned information is generated. This method not only extracts feature information from line images more accurately but also comprehensively considers various influencing factors of the line operating environment, enabling intelligent prediction of line fault occurrence conditions. This significantly improves the safety and reliability of the power system and effectively reduces power outages caused by line faults.

[0134] Figure 2This is a schematic block diagram of a line galloping target tracking and detection system 300 based on image processing provided in an embodiment of the present invention. Figure 2 As shown, corresponding to the above-described image processing-based method for tracking and detecting galloping targets on railway lines, the present invention also provides an image processing-based system 300 for tracking and detecting galloping targets on railway lines. This image processing-based system 300 includes a unit for executing the above-described image processing-based method for tracking and detecting galloping targets on railway lines, and the system can be configured in a server. Specifically, please refer to... Figure 2 The image processing-based line galloping target tracking and detection system 300 includes an environmental data acquisition unit 301, a baseline feature generation unit 302, a real-time feature generation unit 303, a line data generation unit 304, an advanced data generation unit 305, a real-time detection unit 306, a fault prediction unit 307, and a report generation unit 308.

[0135] The system comprises the following components: an environmental data acquisition unit 301, used to acquire the line's usage environment data and the line's usage time; a baseline feature generation unit 302, used to acquire a static baseline image and generate a baseline feature map based on the static baseline image; a real-time feature generation unit 303, used to acquire a real-time line image and generate real-time frame features for each real-time frame based on the real-time line image; a line data generation unit 304, used to determine the line galloping position and amplitude based on the baseline feature map and the real-time frame features; an advanced data generation unit 305, used to generate the galloping frequency and antinode position based on the real-time frame features; a real-time detection unit 306, used to determine the real-time galloping detection risk coefficient based on the galloping amplitude, the line galloping position, the galloping frequency, and the antinode position; a fault prediction unit 307, used to process the usage environment data, the line's usage time, the galloping amplitude, and the galloping frequency based on a trained line fault occurrence time prediction model to generate a predicted line fault occurrence time; and a report generation unit 308, used to generate a line tracking and detection report based on the real-time galloping detection risk coefficient and the predicted line fault occurrence time.

[0136] In one embodiment, the line data generation unit 304 includes

[0137] The first processing subunit is used to process the reference feature map and the real-time frame features according to the spatial attention mechanism to obtain the real-time line segmentation mask and real-time line features of each real-time frame; the memory bank construction subunit is used to construct a memory bank according to the real-time line segmentation mask and the real-time line features; the line data determination subunit is used to determine the line dancing position and dancing amplitude according to the memory bank and the temporal attention mechanism.

[0138] In one embodiment, the line data determination subunit includes: The historical data determination module is used to determine the historical line segmentation mask and historical line features of the historical frame corresponding to the current real-time frame to be processed based on the memory bank; the historical data processing module is used to process the historical line segmentation mask and historical line features of the historical frame according to the time attention mechanism to determine the line dancing position and dancing amplitude.

[0139] In one embodiment, the advanced data generation unit 305 includes: The centerline determination subunit is used to refine the real-time line segmentation mask in order to determine the line centerline; Based on the centerline of the line, points are taken at equal intervals on the centerline, and the changes of specified coordinates over time are analyzed to identify the location of the antinode with the largest amplitude. Vibration signals are generated based on the time-varying sequence of the antinode positions. The highest energy peak in the spectrum is determined based on the vibration signal to obtain the dancing frequency.

[0140] In one embodiment, the real-time detection unit 306 includes: The system includes a spacing acquisition subunit for acquiring the spacing between the current line and adjacent lines; a galloping hazard identification subunit for determining the galloping location hazard identification result based on the galloping location; an antinode hazard identification subunit for determining the antinode location hazard identification result based on the antinode location; and a coefficient determination subunit for determining a real-time galloping detection hazard coefficient based on the galloping amplitude, the line spacing, the galloping location hazard identification result, and the antinode location hazard identification result. The real-time galloping detection hazard coefficient integrates electrical and tension hazards to quantify the risk level of line galloping.

[0141] It should be noted that those skilled in the art can clearly understand that the specific implementation process of the above-mentioned image processing-based line galloping target tracking and detection system 300 and its various units can be referred to the corresponding descriptions in the foregoing method embodiments. For the sake of convenience and brevity, these details will not be repeated here.

[0142] The aforementioned image processing-based line galloping target tracking and detection system 300 can be implemented as a computer program, which can be used in, for example... Figure 3 It runs on the computer device shown.

[0143] Please see Figure 3 , Figure 3This is a schematic block diagram of a computer device provided in an embodiment of this application. The computer device 500 can be a server, wherein the server can be a standalone server or a server cluster composed of multiple servers.

[0144] See Figure 3 The computer device 500 includes a processor 502, a memory, and a network interface 505 connected via a system bus 501. The memory may include a non-volatile storage medium 503 and internal memory 504.

[0145] The non-volatile storage medium 503 may store an operating system 5031 and a computer program 5032. The computer program 5032 includes program instructions that, when executed, cause the processor 502 to perform a line galloping target tracking and detection method based on image processing.

[0146] The processor 502 provides computing and control capabilities to support the operation of the entire computer device 500.

[0147] The internal memory 504 provides an environment for the operation of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute a line dancing target tracking and detection method based on image processing.

[0148] This network interface 505 is used for network communication with other devices. Those skilled in the art will understand that... Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device 500 to which the present application is applied. The specific computer device 500 may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0149] The processor 502 is used to run a computer program 5032 stored in the memory to implement all the steps of the image processing-based line dancing target tracking and detection method.

[0150] It should be understood that in the embodiments of this application, the processor 502 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0151] It will be understood by those skilled in the art that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program includes program instructions and can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the process steps of the embodiments of the above methods.

[0152] Therefore, the present invention also provides a storage medium. This storage medium can be a computer-readable storage medium. The storage medium stores a computer program, wherein when executed by a processor, the computer program causes the processor to perform all the steps of the image processing-based line galloping target tracking and detection method.

[0153] The storage medium can be any computer-readable storage medium capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory (ROM), magnetic disk, or optical disk.

[0154] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0155] In the embodiments provided by this invention, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the division of each unit is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0156] The steps in the method of this invention can be adjusted, merged, or reduced in order according to actual needs. The units in the system of this invention can be merged, divided, or reduced according to actual needs. Furthermore, the functional units in the various embodiments of this invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0157] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a terminal, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0158] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for tracking and detecting galloping targets on railway lines based on image processing, characterized in that, include: Obtain data on the usage environment of the line and the time the line has been used; Acquire a static reference image and generate a reference feature map based on the static reference image; Acquire real-time line images and generate real-time frame features for each real-time frame based on the real-time line images; The line galloping position and amplitude are determined based on the reference feature map and the real-time frame features; The dancing frequency and antinode positions are generated based on the real-time frame features. The real-time vibration detection risk factor is determined based on the vibration amplitude, the vibration position of the line, the vibration frequency, and the antinode position. The usage environment data, the line usage time, the galloping amplitude, and the galloping frequency are processed according to the trained line fault occurrence time prediction model to generate a predicted line fault occurrence time. A line tracking and detection report is generated based on the real-time vibration detection risk factor and the predicted line fault occurrence time.

2. The image processing-based method for tracking and detecting galloping targets according to claim 1, characterized in that, The step of determining the line galloping position and amplitude based on the reference feature map and the real-time frame features includes: The baseline feature map and the real-time frame features are processed according to the spatial attention mechanism to obtain the real-time line segmentation mask and real-time line features of each real-time frame. A memory bank is constructed based on the real-time line segmentation mask and the real-time line features; The position and amplitude of the line dance are determined based on the memory bank and time attention mechanism.

3. The image processing-based method for tracking and detecting galloping targets according to claim 2, characterized in that, The training process of the spatial attention mechanism includes: Acquire multiple historical training route images; Perform pixel-level analysis on each historical training line image and extract the set of all pixels. For each pixel in the set, background line identification is performed to form a background line identification result, and feature classification weight, result uncertainty weight, and line shape weight are calculated. Based on the historical training route images, construct historical training baseline feature maps and historical training frame features; The spatial attention mechanism is used to perform deep processing on the historical training baseline feature map and the historical training frame features to obtain the training line segmentation mask and training line features. Calculate the mean value of line pixel features and the mean value of background pixel features based on the training line features; The horizontal and vertical gradients are calculated using the training line segmentation mask to obtain gradient information; By combining the training line segmentation mask, background line recognition results, feature classification weights, result uncertainty weights, line shape weights, mean line pixel features, mean background pixel features, and gradient information, a training loss function for the spatial attention mechanism is constructed. The spatial attention mechanism is trained using the aforementioned training loss function.

4. The image processing-based method for tracking and detecting galloping targets according to claim 3, characterized in that, The training loss function of the spatial attention mechanism includes: ;in, Let be the training loss function for the spatial attention mechanism. This represents the background line recognition result for the j-th pixel in the k-th historical training line image. The training line segmentation mask for the j-th pixel in the k-th historical training line image. This is the compressed value of the training line segmentation mask for the j-th pixel in the k-th historical training line image. The feature classification weights for the j-th pixel in the k-th historical training line image are: Let the uncertainty weight of the result be the j-th pixel in the k-th historical training line image. The weight of the j-th pixel in the k-th historical training line image is the line shape weight. Let be the mean value of the line pixel features of the k-th historical training line image. The mean value of the background pixel features of the k-th historical training line image. The mean values ​​of the line pixel features and the mean values ​​of the background pixel features of the k-th historical training line image. Norm, The horizontal gradient of the training line segmentation mask for the j-th pixel in the k-th historical training line image. Let K be the vertical gradient of the training line segmentation mask for the j-th pixel in the k-th historical training line image, where K is the number of historical training line images (k≤K), m is the number of pixels (j≤m), and k, K, j, and m are all positive integers.

5. The image processing-based method for tracking and detecting galloping targets according to claim 3, characterized in that, The determination of the line dancing position and amplitude based on the memory bank and time attention mechanism includes: Based on the memory bank, determine the historical line segmentation mask and historical line features of the historical frame corresponding to the current real-time frame to be processed. The historical line segmentation mask and historical line features of historical frames are processed using a time attention mechanism to determine the line galloping position and amplitude.

6. The image processing-based method for tracking and detecting galloping targets according to claim 1, characterized in that, The step of generating the dancing frequency and antinode positions based on the real-time frame features includes: The real-time line segmentation mask is refined to determine the line centerline; Based on the centerline of the line, points are taken at equal intervals on the centerline, and the changes of specified coordinates over time are analyzed to identify the location of the antinode with the largest amplitude. Vibration signals are generated based on the time-varying sequence of the antinode positions. The highest energy peak in the spectrum is determined based on the vibration signal to obtain the dancing frequency.

7. The image processing-based method for tracking and detecting galloping targets according to claim 1, characterized in that, The determination of the real-time galloping detection risk factor based on the galloping amplitude, the galloping position, the galloping frequency, and the antinode position includes: Get the distance between the current line and adjacent lines; The hazard identification result of the line galloping position is determined based on the described line galloping position; The hazard identification result of the antinode location is determined based on the location of the antinode; Based on the galloping amplitude, the line spacing, the hazard identification results of the line galloping position, and the hazard identification results of the antinode position, a real-time galloping detection hazard coefficient is determined. The real-time galloping detection hazard coefficient integrates electrical hazards and tension hazards to quantify the risk level of line galloping.

8. The image processing-based method for tracking and detecting galloping targets according to claim 7, characterized in that, The formula for the real-time dance detection risk factor is expressed as follows: ; To detect the risk level of the dance in real time, The results of the line galloping position hazard identification are as follows. , The results of hazard identification at the antinode location. , For the range of motion, For line spacing, This is a preset threshold for the amplitude of the dance.

9. The image processing-based method for tracking and detecting galloping targets according to claim 1, characterized in that, The training process of the line fault occurrence time prediction model includes: Acquire historical usage environment data, historical usage time, historical galloping amplitude, and historical galloping frequency of multiple similar lines during historical testing periods. The historical usage environment data includes historical ambient temperature and historical ambient snowfall. Obtain the historical actual line fault occurrence times for multiple similar lines; The historical usage environment data, historical line usage time, historical galloping amplitude, and historical galloping frequency are processed according to the line fault occurrence time prediction model to obtain the historical sample predicted line fault occurrence time. Based on the actual historical line fault occurrence time, the predicted line fault occurrence time from the historical samples, the historical usage environment data, the historical line usage time, the historical galloping amplitude, and the historical galloping frequency, the training loss function of the line fault occurrence time prediction model is determined. The line fault occurrence time prediction model is trained using the training loss function of the model to obtain the trained model.

10. A line galloping target tracking and detection system based on image processing, characterized in that, include: The environmental data acquisition unit is used to acquire data on the usage environment of the line and the time the line has been used. A reference feature generation unit is used to acquire a static reference image and generate a reference feature map based on the static reference image; A real-time feature generation unit is used to acquire real-time line images and generate real-time frame features for each real-time frame based on the real-time line images. The line data generation unit is used to determine the line galloping position and amplitude based on the reference feature map and the real-time frame features; An advanced data generation unit is used to generate the dancing frequency and antinode positions based on the real-time frame features. A real-time detection unit is used to determine the real-time galloping detection risk factor based on the galloping amplitude, the galloping position of the line, the galloping frequency, and the antinode position. The fault prediction unit is used to process the usage environment data, the line usage time, the galloping amplitude and the galloping frequency according to the trained line fault occurrence time prediction model to generate a predicted line fault occurrence time. The report generation unit is used to generate a line tracking and detection report based on the real-time galloping detection hazard coefficient and the predicted line fault occurrence time.

Images