Intelligent inspection method for preventing power transmission line from being damaged by external force based on visual model and computing equipment

Abstract

The invention provides an intelligent inspection method for preventing external damage of a power transmission line based on a visual model and computing equipment, and is applied to the technical field of power transmission line monitoring. The intelligent inspection method comprises the following steps: carrying out data acquisition through a camera and an environment monitoring sensor; preprocessing the collected camera image data and environment monitoring sensor data; performing sequence-to-sequence operation on the preprocessed data by adopting a first visual architecture model; and monitoring and analyzing the output of the first visual architecture model in real time, and triggering early warning when an abnormal condition of the power transmission line is detected. The method can improve the recognition precision of the power transmission line, improves the detection flexibility, and achieves the more efficient inspection and maintenance of the power transmission line.

Description

Technical Field

[0001] The present invention relates to the technical field of power transmission line monitoring, and in particular to an intelligent inspection method and computing equipment for preventing power transmission lines from being damaged by external forces based on a visual model. Background Art

[0002] With the development of industry and urbanization, transmission lines are an important part of the power system, and the safe and stable operation of transmission lines is crucial to ensuring social and economic activities. However, the operating environment of transmission lines is complex and changeable, and they face a variety of external threats such as natural disasters, mechanical damage, and illegal intrusion. In order to ensure the safety of transmission lines, traditional manual inspection methods are widely used, but this method has problems such as high labor intensity, low efficiency, and difficulty in real-time monitoring.

[0003] Therefore, considering the uniqueness of different transmission line environments and the changes in monitoring requirements, a method is needed to identify and handle various unknown external force damage situations in various complex environments and improve the recognition rate and detection accuracy of abnormal situations in transmission lines. Summary of the invention

[0004] The present invention aims to provide an intelligent inspection method and computing equipment for preventing power transmission lines from being damaged by external forces based on a visual model, so as to perform real-time monitoring and intelligent analysis on the power transmission lines, so as to prevent and reduce power transmission line damage events caused by external forces.

[0005] According to one aspect of the present invention, a method for intelligent inspection of power transmission lines against external force damage based on a visual model is proposed, comprising:

[0006] Data collection through cameras and environmental monitoring sensors;

[0007] Preprocessing the collected camera image data and environmental monitoring sensor data;

[0008] Using a first visual architecture model to perform a sequence-to-sequence operation on the preprocessed data;

[0009] The output of the first visual architecture model is monitored and analyzed in real time, triggering an early warning when an abnormality in the transmission line is detected.

[0010] According to some embodiments, the first visual architecture model performs sequence-to-sequence operations on the preprocessed data, including: the first visual architecture model simultaneously performs multiple data tasks including target detection and instance segmentation on the preprocessed data, wherein:

[0011] The target detection identifies components and external force equipment in the power transmission line and extracts visual features of the components and external force equipment;

[0012] The instance segmentation segments different visual features in an image.

[0013] According to some embodiments, the first visual architecture model performs a sequence-to-sequence operation on the preprocessed data, further comprising:

[0014] The different visual features are semantically analyzed, the different visual features are converted into natural language descriptions, and subtitle generation processing is performed based on the image data.

[0015] According to some embodiments, the first visual architecture model performs a sequence-to-sequence operation on the preprocessed data, further comprising:

[0016] The image and text prompts are converted into a unified representation to achieve multi-task parallel processing, wherein the text prompts are used to guide the first visual architecture model to understand the specific content of the task, and the text prompts include instructions to the first visual architecture model.

[0017] According to some embodiments, extracting visual features of the component and the external force device includes extracting a location, type, shape, and state of a target object.

[0018] According to some embodiments, the step further includes preparing data for model pre-training:

[0019] Performing automated data annotation and generation, using a second data model to automatically identify target objects in the transmission line scene, using image segmentation technology to generate a segmentation data set for the target objects, and using a large language model to automatically generate detailed text labels describing the image content; performing data enhancement processing on the image, including scaling, cropping, rotating and flipping the image, and simulating different environmental conditions to perform data enhancement on the image;

[0020] Perform multimodal data alignment between the visual features of the image and the generated text labels to ensure data consistency and accuracy;

[0021] All data are standardized and preprocessed according to the format required by the first visual architecture.

[0022] According to some embodiments, it also includes: performing quality audit on the automatically generated data and the data processed by the data enhancement through an error feedback mechanism, and feeding back the labeling errors and misdetections into the automated data labeling and generation and data enhancement processing flow.

[0023] According to some embodiments, the method further includes pre-training the first visual architecture model:

[0024] Use the cross entropy loss function to optimize the language modeling objective:

[0025]

[0026] Among them, L represents the loss value, n represents the total number of categories, i represents the i-th category currently being calculated, yi represents the true label value of category i, and pi represents the probability of the model predicting category i;

[0027] The algorithm is optimized for training using a variant of stochastic gradient descent:

[0028]

[0029] Among them, θ t+1 represents the updated parameter value, weight, θ t Represents the current parameter value, α represents the learning rate, the step size of the control parameter update, m^t represents the deviation correction value of the first-order momentum of the gradient, that is, the weighted average of the gradient, v^t represents the deviation correction value of the second-order momentum of the gradient, that is, the weighted average of the square of the gradient, and ∈ represents a constant.

[0030] According to another aspect of the present invention, there is provided an intelligent inspection system for preventing external force damage to power transmission lines based on a visual model, comprising:

[0031] A data acquisition device, through which data acquisition is performed;

[0032] A data collection layer, preprocessing the collected data;

[0033] An intelligent analysis layer, using a first visual architecture model to perform sequence-to-sequence operations on the preprocessed data through the intelligent analysis layer;

[0034] The response execution layer monitors and analyzes the output of the intelligent analysis layer in real time through the response execution layer, and triggers an early warning when an abnormal situation of the transmission line is detected.

[0035] According to another aspect of the present invention, there is provided a computing device, comprising:

[0036] Processor; and

[0037] A memory stores a computer program, and when the computer program is executed by the processor, the processor is caused to perform any of the aforementioned methods.

[0038] According to an example embodiment of the present invention, data is collected through cameras and environmental monitoring sensors, and combined with various types of data information, the transmission line and its surrounding environment can be monitored in all directions to ensure that no key information is missed. Preprocessing the collected camera image data and environmental monitoring sensor data can significantly improve the quality of the data, facilitate subsequent model input, and make the analysis more accurate and reliable; the use of a visual architecture model can handle multiple tasks at the same time, and sequence-to-sequence operations allow the model to understand dynamic changes in time series data and adapt to complex real-life scenarios; real-time monitoring and analysis of the output of the first visual architecture model, the automated early warning system reduces the pressure of manual monitoring, improves efficiency, and continuously runs real-time monitoring. When an abnormal situation in the transmission line is detected, an early warning is triggered, which can help relevant departments respond quickly to ensure the safe and stable operation of the transmission line and reduce economic losses and social impacts.

[0039] Through the method of the present invention, a visual architecture model can be used to simultaneously perform multiple visual tasks such as target detection, instance segmentation, and image caption generation, thereby realizing multi-task parallel processing, improving the resource utilization of the model, reducing the cost of building and maintaining multiple independent models, and simplifying the overall design of the system.

[0040] It is to be understood that the foregoing general description and the following detailed description are exemplary only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for describing the embodiments are briefly introduced below.

[0042] Figure 1 A flow chart of a method for real-time orderly charging regulation of a charging station area according to an example embodiment is shown.

[0043] Figure 2 A graph showing a learning rate variation for configuring a cosine annealing strategy according to an example embodiment is shown.

[0044] Figure 3 A block diagram of an intelligent inspection system for power transmission lines against external force damage based on a visual model according to an example embodiment is shown.

[0045] Figure 4 A block diagram of a computing device according to an example embodiment of the invention is shown. DETAILED DESCRIPTION

[0046] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that the present invention will be comprehensive and complete and fully convey the concepts of the example embodiments to those skilled in the art. The same reference numerals in the figures represent the same or similar parts, and thus their repeated description will be omitted.

[0047] In addition, the described features, structures or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present invention. However, those skilled in the art will appreciate that the technical solution of the present invention can be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid blurring various aspects of the present invention.

[0048] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0049] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0050] It should be understood that although the terms first, second, third, etc. may be used herein to describe various components, these components should not be limited by these terms. These terms are used to distinguish one component from another component. Therefore, the first component discussed below can be referred to as the second component without departing from the teachings of the present inventive concept. As used herein, the term "and / or" includes any one of the associated listed items and all combinations of one or more.

[0051] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the present invention are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0052] Those skilled in the art will appreciate that the drawings are merely schematic diagrams of example embodiments, and the modules or processes in the drawings are not necessarily necessary for implementing the present invention, and therefore cannot be used to limit the protection scope of the present invention.

[0053] With the development of drone technology, remote sensing technology, computer vision and artificial intelligence technology, power transmission line inspection has begun to shift towards intelligence and automation. Using drones equipped with high-definition cameras to inspect lines can quickly obtain a large amount of image data. However, how to accurately and efficiently identify hidden dangers of external force damage from a large amount of images remains a technical challenge.

[0054] Hidden dangers of damage by external forces, such as construction machinery approaching transmission lines, earthwork construction near transmission lines, or the use of lifting equipment on construction sites, may cause the machinery to collide with the lines due to the close distance, causing damage to the wires, line short circuits, and even power outages and other serious consequences.

[0055] Due to the interference of complex background, identifying hidden dangers of external damage is a great technical challenge. Mechanical equipment is easily confused with the environment, which increases the difficulty of detection. The distance between the machinery and the line is constantly changing, requiring real-time monitoring to accurately identify hidden dangers. There are many types of construction machinery and their appearances vary greatly, which increases the complexity of the recognition model. Different lighting and weather conditions affect the image quality, increasing the difficulty of detection. In addition, hidden dangers are dynamically changing, requiring rapid image processing and timely warning. The model needs to take into account both accuracy and speed.

[0056] There has been some research and application in the field of intelligent inspection of power transmission lines at home and abroad. For example, image processing technology is used to analyze images taken by drones to identify obvious faults such as broken wires and damaged insulators; machine learning algorithms are used to classify the different states of power transmission lines. However, most existing technologies focus on a single type of fault detection, and their recognition capabilities are limited for more complex and subtle external force destructive behaviors, such as construction machinery approaching and illegal buildings. In addition, existing inspection systems often rely on a large amount of manually annotated data for training, which is not only costly, but also difficult to adapt to changing environments and complex scenarios.

[0057] Therefore, the present invention proposes an intelligent inspection method for power transmission lines to prevent external force damage based on a visual model. The intelligent inspection method can process multi-source and multi-modal data and has self-learning and self-adaptation capabilities, which is of great significance for improving the safety management level of power transmission lines.

[0058] Before describing the embodiments of the present application, some terms or concepts involved in the embodiments of the present application are explained.

[0059] As a model in the field of visual language models, Florence-2 demonstrates the ability to handle various visual tasks in a multi-task learning environment and has advantages in learning the joint representation of images and texts.

[0060] Albumentations is an efficient and easy-to-use image data augmentation library that is widely used in computer vision tasks. It provides rich image transformation functions to help simulate image changes in various real-world scenarios, thereby improving the generalization and robustness of the model.

[0061] AdamW is an optimization algorithm widely used in deep learning optimization tasks. It combines the advantages of the Adam optimizer and the regularization effect of weight decay. AdamW shows better generalization ability and stability when processing large-scale models.

[0062] Exemplary embodiments of the present invention are described below with reference to the accompanying drawings.

[0063] The present invention performs intelligent inspection on the protection of power transmission lines against external force damage based on a visual model. The visual model utilizes the Florence-2 model and combines the specific needs of power transmission line inspection to develop an intelligent inspection method based on a visual architecture to improve recognition accuracy and inspection efficiency.

[0064] Figure 1 A flow chart of a method for intelligent inspection of power transmission lines against external force damage based on a visual model according to an example embodiment is shown.

[0065] In S101, data is collected through cameras and environmental monitoring sensors.

[0066] According to the example embodiment, cameras installed at key locations continuously capture the transmission line and its surrounding environment, use video streams for data collection, and capture abnormal conditions such as physical damage and foreign object intrusion through image information. Environmental monitoring sensors can collect environmental data related to the safety of transmission lines, such as temperature, humidity, wind speed, vibration, etc., and various types of environmental data to monitor environmental factors that may affect the safety of transmission lines. For example, high wind speeds may cause the line to swing too much, and vibration sensors can help detect whether mechanical equipment is approaching or hitting the power pole tower.

[0067] In S103, the collected camera image data and environment monitoring sensor data are preprocessed.

[0068] According to the example embodiment, the collected camera image data and environmental monitoring sensor data are preprocessed, and the image preprocessing steps include denoising, contrast enhancement and size standardization to ensure that the image data is clear and consistent for easy analysis. Different types of environmental monitoring sensor data are standardized to ensure that they are within the same dimensional range and to remove noise and outliers. For time series sensor data, smoothing filters or other time series analysis methods can be used for preliminary processing.

[0069] In S105, a first visual architecture model is used to perform a sequence-to-sequence operation on the preprocessed data.

[0070] According to an example embodiment, multi-task unified processing is performed through the first visual architecture model, and the first visual architecture model is used to perform sequence-to-sequence operations on the preprocessed data. The first visual architecture model simultaneously performs multiple data tasks of target detection and instance segmentation on the preprocessed data. This embodiment takes the Florence-2 model as an example. The Florence-2 model has multimodal processing and understanding capabilities, and can realize the processing of multiple tasks in a single model. The multi-task learning paradigm of the Florence-2 model enables different tasks to share visual and language representations, thereby realizing multi-task processing.

[0071] Components and external force equipment in the transmission line are identified through target detection, and visual features of the components and external force equipment are extracted. The Florence-2 model is used to extract features from the image data. For example, components in the transmission line and external force threats such as construction machinery and illegal buildings are identified, and important visual features are extracted. The visual features can be, but are not limited to, extracting the position, type, shape and state of the target object.

[0072] Different visual features in the image are segmented through instance segmentation. Florence-2 is used for instance segmentation to segment different objects in the image. For example, components such as wires and insulators in transmission lines can be distinguished from external threats such as cranes and excavators. The data after instance segmentation is the mask of each object, including the boundary information and category label of the object, which is used to accurately identify and analyze the independent state of each object.

[0073] Semantic analysis is performed on different visual features, the different visual features are converted into natural language descriptions, and subtitle generation is performed based on the image data. The features in the image are semantically analyzed through the Florence-2 model, and subtitles are generated based on the image data to describe the scenes and objects in the image, such as "there is a crane operating near the transmission line, and there is a risk of insufficient safety distance", to help inspection personnel better understand the image content.

[0074] The Florence-2 model converts images and text prompts into a unified representation through a sequence-to-sequence learning paradigm to achieve multi-task parallel processing. The text prompts are used to guide the Florence-2 model to understand the specific content of the task, and the text prompts include instructions to the Florence-2 model. For example, text prompts may include instructions to the model, such as "detect power line components in the image" or "describe the scene in the image." These prompts are processed as inputs together with the image data to achieve the same representation of image features and text descriptions, enabling the model to generate outputs for specific tasks such as target detection results or image captions.

[0075] The sequence-to-sequence operation in this example embodiment is widely used in tasks such as machine translation, text generation and speech recognition. In the present invention, the image is cut into multiple smaller areas (patches) and converted into a sequence form together with the text as input, so that the model can understand the local features more carefully and achieve a unified representation form, thereby improving the ability to handle complex scenes and having better adaptability and generalization performance when facing new environments or unseen data.

[0076] In S107, the output of the first visual architecture model is monitored and analyzed in real time, and an early warning is triggered when an abnormality in the power transmission line is detected.

[0077] According to the example embodiment, the output of the first visual architecture model is monitored and analyzed in real time, and the pre-processed data is input into the trained visual model. The potential threats to the transmission line, such as foreign object intrusion, vegetation being too close, etc., are identified through the visual model. The system quickly identifies abnormal behavior or changes through real-time analysis. When an abnormal situation of the transmission line is detected, whether it is physical damage or potential threat, an early warning is immediately triggered, and an early warning notification is sent to the maintenance personnel through multiple channels, such as SMS, email, application push message, etc.

[0078] The intelligent inspection method of this example embodiment utilizes advanced artificial intelligence algorithms to achieve in-depth understanding and accurate judgment of image and environmental data. It is applicable to power transmission line inspection tasks in various complex environments, and can detect and handle problems in the first place, thus reducing the possibility of accidents.

[0079] Compared with the artificially set detection rules in the prior art, which often cannot fully cover the open scenes and unpredictable external force damage that may be encountered in the inspection of power transmission lines, this example embodiment has a model with self-learning and self-adaptive capabilities, which can identify and process various unknown external force damage situations and meet the detection needs of open scenes of power transmission lines.

[0080] Compared with traditional visual tasks such as object detection and instance segmentation, which usually rely on a dedicated single expert model for processing, multiple models are required to handle different tasks respectively, increasing the complexity of the system. The present invention uses the unified large visual model of the Florence-2 architecture to implement multiple visual tasks through a single model, which not only improves efficiency but also reduces computing costs.

[0081] With respect to the organization and enhancement of the visual model training data, the present invention combines automated tools and artificial intelligence large models to improve the quality and diversity of transmission line image data, thereby enhancing the generalization ability of the model.

[0082] First, data for model pre-training is prepared. According to the example embodiment, automatic data annotation and generation are performed. The second data model is used to automatically identify the target object in the transmission line scene. The image segmentation technology is used to generate a segmentation data set for the target object. The large language model is used to automatically generate detailed text labels describing the image content. In this embodiment, the Grounding DINO model is used to automatically generate target detection data in the transmission line scene. The Grounding DINO model can identify key objects in the image and generate accurate bounding boxes. The SAM (Segment Anything Model) model is used to automatically generate an example segmentation data set of the scene. The SAM model is an image segmentation model. The SAM model can identify and segment any object based on text instructions or image content, realize flexible and general segmentation capabilities, and is applicable to a variety of scenes and targets. It has the characteristics of "detecting everything and segmenting everything". The SAM model is used to generate image segmentation examples as a data set to help the subsequent training of the Florence-2 model. The large language models Qwen2 and Llama3 are used to automatically generate detailed text labels describing the image content for the ImageCaption (image description generation technology) task of training the transmission line scene.

[0083] The image is enhanced by data processing, scaling, cropping, rotating and flipping the image, and simulating different environmental conditions to enhance the image data. This embodiment uses the Albumentations library to enhance the image data and simulate different environmental conditions, including rain, snow, fog, dusk, light changes and lens distortion. Multi-scale and directional enhancements help the model maintain high performance in a variety of real-world environments.

[0084] Align the visual features of the image with the generated text labels for multi-modal data alignment to ensure data consistency and accuracy. The visual features of the image refer to the image features of the original image in vector form, such as image texture, spatial relationship, and semantic relationship, etc. The alignment is to calculate the semantic vector similarity between the image feature vector and the text feature vector in the vector space for alignment. Multi-modal data alignment is a key step in realizing multi-modal learning of the visual unified model.

[0085] Preprocess all data in the format required by the first visual architecture. All data is preprocessed in the format required by the Florence-2 architecture, including text labels and image data. The standardized preprocessing ensures the efficiency and consistency of model training. Example format:

[0086]

[0087]

[0088] Finally, perform quality control and verification. Conduct quality audits on the automatically generated data and the data after data augmentation processing through an error feedback mechanism, and feedback the cases of annotation errors and false detections to the automated data annotation and generation and data augmentation processing processes. Conduct strict quality audits on the automatically generated and augmented data to ensure data accuracy and reliability.

[0089] After preparing the pre-training data for the model, train the visual model, and optimize the model performance through advanced training strategies, so that the first visual architecture model of the present invention can perform excellently in various visual tasks.

[0090] According to some embodiments, the pre-training of the first visual architecture model is based on a large-scale multi-task image dataset, namely the FLD-5B dataset. The FLD-5B dataset contains 126 million images and more than 5 billion annotations, covering various different visual tasks. The pre-training of this embodiment adopts a unified multi-task learning paradigm, and through large-scale automated image annotation and model refinement strategies, comprehensive visual annotations are generated.

[0091] In the pre-training phase, the visual model is designed to perform a variety of visual tasks, including but not limited to image classification, object detection, image caption generation, and visual question answering. Multiple data formats are trained in an epoch-crossing manner. All data is uniformly processed through a sequence-to-sequence learning framework. The model receives task instructions through text prompts and generates corresponding text outputs. Model training uses a standard cross-entropy loss function to optimize the language modeling objective. For each task, the loss function calculates the difference between the model output and the target and guides the model parameter update. The formula for using the cross-entropy loss function to optimize the language modeling objective is:

[0092]

[0093] Among them, L represents the loss value, n represents the total number of categories, i represents the i-th category currently being calculated, yi represents the true label value of category i, and pi represents the probability of the model predicting category i;

[0094] The model training uses the AdamW optimizer, which combines momentum and adaptive learning rate adjustment. It is an effective variant of stochastic gradient descent. The learning rate uses a cosine annealing strategy and implements linear warm-up at the beginning of training. For the learning rate change diagram with the cosine annealing strategy, see Figure 2 According to the example embodiment, the ordinate is the learning rate, the abscissa is the number of iterations, and the learning rate reaches a peak value after the warmup phase at the beginning of the iteration, and then decreases smoothly according to the cosine function. The cosine annealing strategy is a learning rate scheduling method used to dynamically adjust the learning rate during the deep learning training process, so that the learning rate gradually decreases during the training process, thereby improving the convergence effect and avoiding the model from falling into oscillation or over-updating in the later stage of training.

[0095] The formula for algorithm optimization training using a variant of stochastic gradient descent is:

[0096]

[0097] in,

[0098] θ t+1 represents the updated parameter value and weight,

[0099] θ t Indicates the current parameter value.

[0100] α represents the learning rate and the step size of the control parameter update.

[0101] m^t represents the deviation correction value of the first-order momentum of the gradient, that is, the weighted average of the gradient,

[0102] v^t represents the deviation correction value of the second-order momentum of the gradient, that is, the weighted average of the square of the gradient,

[0103] ∈ represents a constant.

[0104] The trained visual model is used to perform intelligent analysis on the images collected from the transmission line terminals, and real-time monitoring is carried out on hoisting, lifting, excavation, loading and unloading of goods, sand mining and other behaviors in and near the protection zone of the overhead transmission line. The monitored objects can be construction machinery and vehicles such as cranes, excavators, dump trucks, forklifts, cement pump trucks, tower cranes, and rammers.

[0105] The types of line faults caused by external force damage include: insufficient safety distance caused by tower cranes, hoists, etc. lifting and hoisting heavy objects in and near the line protection zone; insufficient safety distance caused by excavators, forklifts and other machinery working in and near the line protection zone; insufficient safety distance caused by cement pump trucks, dump trucks, sand mining ships, etc. crossing overhead transmission lines; failures caused by illegal vehicles hitting transmission towers, guy wires, etc.; insufficient safety distance caused by illegal construction of buildings in and near the protection zone; and failures caused by overhead transmission lines due to wire ropes, transfer ropes, cables, etc. touching the wires.

[0106] Foreign matter on-line short circuit mainly refers to the interphase short circuit or grounding fault caused by foreign matter hanging on the line and shorting the air gap. Foreign matter sources mainly include plastic sheets, color steel tiles, advertising cloth, billboards, tin foil, dust-proof nets, sunshade nets, plastic films, balloons, kites, etc. In severe weather such as strong winds, foreign matter sources hang on the line, thus causing line faults. Construction hoisting mainly refers to faults caused by insufficient safety distance during hoisting in and near the line protection area; when the channel safety distance is insufficient, safety measures need to be implemented in place, and power should be cut off if necessary, otherwise it is very easy to cause grounding faults in the transmission line. Illegal fishing touching the line refers to when the overhead transmission line passes through ponds, rivers and other areas, and the fishing rod or fishing line is not at a safe distance from the conductor due to fishing near the channel. Channel fireworks are mainly caused by the burning of combustible and flammable materials such as tree barriers, weeds, and buildings in the overhead transmission line channel. If necessary, emergency shutdown is required to avoid risks.

[0107] In response to the above-mentioned external force damage causing line failures and short circuits caused by foreign objects, the present invention can automatically fuse the results of multiple visual tasks based on the collected visual images and output detection box coordinates, instance segmentation mask boxes, and image subtitles. To cope with unpredictable open scenes in power transmission line inspections, the training mechanism of the present invention uses a large-scale language model to conduct deep learning on a large amount of text data, extract rich semantic information and contextual knowledge, and then align and fuse these semantic information with the visual features extracted from the image by the Florence-2 model, so as to train the model to recognize and understand complex scene changes. By combining the semantic understanding ability of the large-scale language model with the visual feature recognition ability, the detection of various potential external force destructive behaviors in open scenes is achieved.

[0108] A quantitative analysis is performed on the intelligent inspection method for power transmission lines against external force damage based on the visual model of this embodiment. Through quantitative evaluation, the average intersection over union (mIOU) of the classic visual algorithm yolov5_v7.0 is 73.85% under the same validation set, while the method of the present invention reaches 84.89%, and the detection rate of the anti-external force damage data set is also better than that of the classic visual algorithm.

[0109] Figure 3 A block diagram of an intelligent inspection system for power transmission lines against external force damage based on a visual model according to an example embodiment is shown.

[0110] According to an example embodiment, a vision model-based intelligent inspection system for power transmission lines to prevent external force damage includes a data acquisition device, a data acquisition layer, a data acquisition layer, an intelligent analysis layer, and a response execution layer.

[0111] Data collection is performed through the data collection equipment, which integrates a variety of equipment such as high-resolution cameras carried by drones, low-power cameras in fixed positions, environmental monitoring sensors, and ground inspection robots, ensuring the comprehensiveness and diversity of data collection. Among them, the high-definition camera carried by the drone can flexibly collect high-definition images of the transmission line at close range, capture subtle changes and potential threats in the line, and the fixed cameras deployed in key areas are responsible for all-weather long-term monitoring of the transmission line to ensure comprehensive coverage of the system. Environmental sensors are responsible for collecting various environmental data related to the safety of the transmission line, such as temperature, humidity, wind speed, etc. These data are crucial for analyzing the operation of the transmission line.

[0112] The collected data is preprocessed at the data acquisition layer, and the efficient data preprocessing module can perform operations such as denoising, contrast enhancement and size standardization on the collected images and sensor data to ensure that the image data is clear and consistent, so as to improve the accuracy of subsequent processing.

[0113] The intelligent analysis layer uses the first visual architecture model to perform visual tasks such as target detection, instance segmentation, and image caption generation, and performs multi-task unified processing. The first visual architecture model performs sequence-to-sequence operations on the pre-processed data; images and text prompts are converted into a unified representation to achieve in-depth analysis and understanding of transmission line images. An adaptive learning mechanism is introduced in the intelligent analysis layer. Through online learning and incremental updates, the system enables the visual model to continuously adapt to new data and scenarios and improve generalization capabilities.

[0114] The response execution layer includes a real-time monitoring and early warning system, which monitors and analyzes the output of the intelligent analysis layer in real time and triggers an early warning when an abnormality in the transmission line is detected. Potential threats to the transmission line are identified, and maintenance personnel are notified in a timely manner once an abnormality such as foreign object intrusion or excessive proximity of vegetation is detected.

[0115] According to some embodiments, the visual model-based intelligent inspection system for power transmission lines against external force damage also includes a human-computer interaction layer and a maintenance update layer. The human-computer interaction layer provides an interactive visualization platform for the system, enabling monitoring personnel to interact with the system through a user interface. The user interface provides an intuitive operation interface for monitoring personnel, and allows them to easily view system status, historical data, and warning records, supporting users to conduct in-depth analysis of inspection data to discover potential patterns and trends. The interactive visualization platform supports customized data display methods, facilitates viewing data analysis results and warning notifications, and achieves more efficient transmission line inspection and maintenance, which helps to ensure the safe and stable operation of transmission lines.

[0116] The maintenance and update layer is responsible for the iteration and update of the model to ensure that the system can adapt to new challenges. The system regularly updates the model based on the latest inspection data to adapt to new scenarios and changes. The model is regularly retrained and optimized in combination with user feedback. User feedback and system performance evaluation results are used to further optimize the model and algorithm, so that the transmission line inspection system can not only meet current task requirements, but also flexibly respond to various new situations that may arise in the future, maintaining the long-term stability and reliability of the system.

[0117] The design of the intelligent inspection system for preventing external force damage to power lines based on visual models in this embodiment takes into account generalization capability and scalability, can adapt to different power transmission line environments and monitoring requirements, and has better practicality and promotion value. The integrated real-time monitoring and early warning system can promptly identify abnormal situations and issue early warnings, thereby improving the safety management level and response speed of power transmission lines. By providing accurate monitoring data and in-depth data analysis tools, it provides strong support for the maintenance and decision-making of power transmission lines. Combined with regular model iterations and updates, the present invention can adapt to new challenges and environmental changes, ensuring long-term system effectiveness.

[0118] The intelligent inspection method for preventing external force damage to power transmission lines based on visual models of the present invention uses advanced image processing technology and deep learning models. The adopted visual model has self-learning and self-adaptive capabilities, can cope with unpredictable external force damage in open scenes, does not require manual setting of detection rules, can accurately identify power transmission lines and their components, and can maintain high recognition accuracy even under harsh visual conditions, thereby improving the flexibility and accuracy of detection. By adopting automated data sorting methods and efficient data enhancement technology, the speed and quality of data processing are greatly improved, making model training more efficient.

[0119] In summary, the present invention not only improves the intelligence level of power transmission line inspection, but also achieves efficient, accurate and reliable monitoring effects through technological innovation. It has important practical value and broad market prospects.

[0120] Figure 4 A block diagram of a computing device according to an example embodiment of the invention is shown.

[0121] like Figure 4 As shown, computing device 30 includes processor 12 and memory 14. Computing device 30 may also include bus 22, network interface 16, and I / O interface 18. Processor 12, memory 14, network interface 16, and I / O interface 18 may communicate with each other via bus 22.

[0122] The processor 12 may include one or more general-purpose CPUs (Central Processing Units, processors), microprocessors, or application-specific integrated circuits, etc., for executing relevant program instructions.

[0123] The memory 14 may include a machine system readable medium in the form of a volatile memory, such as a random access memory (RAM), a read-only memory (ROM) and / or a cache memory. The memory 14 is used to store one or more programs including instructions and data. The processor 12 can read the instructions stored in the memory 14 to execute the above-mentioned method according to the embodiment of the present invention.

[0124] The computing device 30 may also communicate with one or more networks via the network interface 16. The network interface 16 may be a wireless network interface.

[0125] The bus 22 may include an address bus, a data bus, a control bus, etc. The bus 22 provides a path for exchanging information between components.

[0126] It should be noted that, in the specific implementation process, the computing device 30 may also include other components necessary for normal operation. In addition, those skilled in the art may understand that the above device may only include components necessary for implementing the embodiments of this specification, and need not include all components shown in the figure.

[0127] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the above method when executed by a processor. The computer-readable storage medium may include, but is not limited to, any type of disk, including a floppy disk, an optical disk, a DVD, a CD-ROM, a microdrive, and a magneto-optical disk, a ROM, a RAM, an EPROM, an EEPROM, a DRAM, a VRAM, a flash memory device, a magnetic card or an optical card, a nanosystem (including a molecular memory IC), a network storage device, a cloud storage device, or any type of medium or device suitable for storing instructions and / or data.

[0128] An embodiment of the present invention also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute part or all of the steps of any method recorded in the above method embodiments.

[0129] Those skilled in the art can clearly understand that the technical solution of the present invention can be implemented with the help of software and / or hardware. "Unit" and "module" in this specification refer to software and / or hardware that can independently complete or cooperate with other components to complete specific functions, where the hardware can be, for example, a field programmable gate array, an integrated circuit, etc.

[0130] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0131] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0132] In the several embodiments provided by the present invention, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0133] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0134] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0135] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the whole or part of the technical solution can be embodied in the form of a software product, which is stored in a memory and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the methods of various embodiments of the present invention.

[0136] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0137] The exemplary embodiments of the present invention are specifically shown and described above. It should be understood that the present invention is not limited to the detailed structures, configurations or implementations described herein; on the contrary, the present invention is intended to cover various modifications and equivalent configurations included in the spirit and scope of the appended clauses.

[0138] Those skilled in the art can clearly understand that the technical solution of the present invention can be implemented with the help of software and / or hardware. "Unit" and "module" in this specification refer to software and / or hardware that can independently complete or cooperate with other components to complete specific functions, where the hardware can be, for example, a field programmable gate array, an integrated circuit, etc.

[0139] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0140] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0141] In the several embodiments provided by the present invention, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0142] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0143] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0144] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the whole or part of the technical solution can be embodied in the form of a software product, which is stored in a memory and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the methods of various embodiments of the present invention.

[0145] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0146] The exemplary embodiments of the present invention are specifically shown and described above. It should be understood that the present invention is not limited to the detailed structure, configuration or implementation method described herein; on the contrary, the present invention is intended to cover various modifications and equivalent configurations included in the spirit and scope of the attached clauses.

Claims

1. An intelligent inspection method for power transmission lines to prevent external force damage based on a visual model, characterized in that: include: Data collection through cameras and environmental monitoring sensors; Preprocessing the collected camera image data and environmental monitoring sensor data; Using a first visual architecture model to perform a sequence-to-sequence operation on the preprocessed data; The output of the first visual architecture model is monitored and analyzed in real time, triggering an early warning when an abnormality in the transmission line is detected.

2. The intelligent inspection method according to claim 1, characterized in that: The first visual architecture model performs a sequence-to-sequence operation on the preprocessed data, including: the first visual architecture model simultaneously performs a plurality of data tasks including target detection and instance segmentation on the preprocessed data, wherein: The target detection identifies components and external force equipment in the power transmission line and extracts visual features of the components and external force equipment; The instance segmentation segments different visual features in an image.

3. The intelligent inspection method according to claim 2, characterized in that: The first visual architecture model performs a sequence-to-sequence operation on the preprocessed data, further comprising: The different visual features are semantically analyzed, the different visual features are converted into natural language descriptions, and subtitle generation processing is performed based on the image data.

4. The intelligent inspection method according to claim 3, characterized in that: The first visual architecture model performs a sequence-to-sequence operation on the preprocessed data, further comprising: The image and text prompts are converted into a unified representation to achieve multi-task parallel processing, wherein the text prompts are used to guide the first visual architecture model to understand the specific content of the task, and the text prompts include instructions to the first visual architecture model.

5. The intelligent inspection method according to claim 2, characterized in that: Extracting visual features of the components and external force devices includes extracting the location, type, shape and state of the target object.

6. The intelligent inspection method according to claim 1, characterized in that: Also included is data prepared for model pre-training: Performing automated data annotation and generation, using a second data model to automatically identify target objects in the transmission line scene, using image segmentation technology to generate a segmentation data set for the target objects, and using a large language model to automatically generate detailed text labels describing the image content; performing data enhancement processing on the image, including scaling, cropping, rotating and flipping the image, and simulating different environmental conditions to perform data enhancement on the image; Perform multimodal data alignment between the visual features of the image and the generated text labels to ensure data consistency and accuracy; All data are standardized and preprocessed according to the format required by the first visual architecture.

7. The intelligent inspection method according to claim 6, characterized in that: Also includes: The automatically generated data and the data processed by the data enhancement are quality audited through an error feedback mechanism, and the labeling errors and false detections are fed back to the automated data labeling and generation and data enhancement processing flow.

8. The intelligent inspection method according to claim 1, characterized in that: Also included is the pre-trained model of the first visual architecture: Use the cross entropy loss function to optimize the language modeling objective: Among them, L represents the loss value, n represents the total number of categories, i represents the i-th category currently being calculated, yi represents the true label value of category i, and pi represents the probability of the model predicting category i; The algorithm is optimized for training using a variant of stochastic gradient descent: Among them, θ t+1 represents the updated parameter value, weight, θ t Represents the current parameter value, α represents the learning rate, the step size of the control parameter update, m^t represents the deviation correction value of the first-order momentum of the gradient, that is, the weighted average of the gradient, v^t represents the deviation correction value of the second-order momentum of the gradient, that is, the weighted average of the square of the gradient, and ∈ represents a constant.

9. An intelligent inspection system for power transmission lines to prevent external force damage based on a visual model, characterized in that: include: A data acquisition device, through which data acquisition is performed; A data collection layer, preprocessing the collected data; An intelligent analysis layer, using a first visual architecture model to perform sequence-to-sequence operations on the preprocessed data through the intelligent analysis layer; The response execution layer monitors and analyzes the output of the intelligent analysis layer in real time through the response execution layer, and triggers an early warning when an abnormal situation of the transmission line is detected.

10. A computing device, characterized in that: include: processor; as well as A memory storing a computer program, which, when executed by the processor, enables the processor to perform the method according to any one of claims 1 to 8.

Images