Power distribution network overhead line picture hierarchical knowledge mining method based on intelligent image recognition

Abstract

The invention relates to the technical field of picture processing, in particular to a power distribution network overhead line picture hierarchical knowledge mining method based on intelligent image recognition, and the method comprises the following steps: data standard examination, model design, training and iterative optimization. The method comprises the following steps: acquiring unmanned aerial vehicle inspection data of an overhead line of a power distribution network, carrying out manual annotation on the inspection data to form a fine annotation data set, carrying out model training and evaluation based on the fine annotation data set, and carrying out iterative optimization according to an evaluation result until a model effect reaches a practical level; the knowledge based on hierarchical mining supports power distribution digital intelligence high-level application service, auditing of equipment and elements, and sample collection of algorithm cultivation. Defect identification, collection, re-checking and work order pushing processes taking the tower as a unit are highly automatic, so that the efficiency and accuracy of defect processing are greatly improved, human errors are reduced, and the timeliness and effectiveness of power grid maintenance work are ensured.

Description

Technical Field

[0001] The invention relates to the technical field of image processing, and in particular to a method for hierarchical mining of knowledge of overhead line images of a distribution network based on intelligent image recognition. Background Art

[0002] In the current field of overhead line operation and maintenance of distribution networks, although some solutions based on image recognition technology have been applied to drone inspections, they still face technical limitations and challenges. Most existing technologies focus on single-level image analysis and lack the ability to conduct in-depth hierarchical mining and knowledge integration of inspection data, which limits their application efficiency and adaptability in complex scenarios. The shortcomings of existing technologies are as follows: (a) Insufficient hierarchical knowledge mining: Existing technologies usually only perform surface image recognition, and their comprehensive analysis capabilities for the deep-level spatiotemporal correlations, tower structures, equipment status, and component defects behind the images are weak. This makes it difficult to build a comprehensive hierarchical knowledge system, which affects the accurate assessment of the health status of the distribution network.

[0003] (b) Lack of systematic defect management: Defect identification and management of towers and their ancillary equipment are usually carried out in isolation, lacking an effective data collection and review mechanism. This makes the defect handling process complex and inefficient, and it is difficult to form a systematic optimization strategy and timely maintenance response.

[0004] (c) Insufficient support for advanced application services: In the context of the digital transformation of power distribution, existing technologies do not provide sufficient support for high-level application services and cannot efficiently provide high-quality samples and data support for equipment audits and algorithm iterations, limiting the potential for intelligent upgrades of power distribution systems.

[0005] In view of this, the present invention proposes a method for hierarchical mining of knowledge of overhead line images of distribution network based on intelligent image recognition, which solves the above technical problems. Summary of the invention

[0006] In order to address the deficiencies of the prior art, the present invention provides a method for hierarchical mining of knowledge from images of overhead lines in a distribution network based on intelligent image recognition. This method addresses the above-mentioned problems by using intelligent image recognition technology to perform hierarchical mining of knowledge from images of overhead lines in a distribution network, and constructs a hierarchical management and visualization management system for image knowledge data with poles and towers as units. At the same time, the method can automatically process defect identification and collection, support optimization of operation and maintenance decisions in a data-driven manner, and provide a solid foundation for the digital transformation of power distribution. Therefore, the present invention aims to solve the technical difficulties of the prior art in handling complex scenarios, improving the level of intelligence, achieving efficient operation and maintenance management, and promoting the intelligent upgrade of distribution systems.

[0007] The technical solution adopted by the present invention to solve the technical problem is a method for hierarchical mining of knowledge of overhead line images of distribution network based on intelligent image recognition, comprising the following steps: S1. Data review, model design, training and iterative optimization. Obtain the inspection data of overhead lines of the distribution network by drones, manually annotate the inspection data to form a precisely annotated data set, conduct model training and evaluation based on the precisely annotated data set, and iterate and optimize according to the evaluation results until the model effect reaches a practical level. S2. Based on the trained model, the image data is hierarchically mined for knowledge, including at least the basic layer, spatiotemporal layer, tower layer, equipment layer, component layer, and defect layer, to achieve hierarchical management and visual management of image knowledge data; S3, identify and collect defects based on the tower as a unit, and collect the data of multiple tower images by merging, deduplication, and filtering to achieve defect distribution display, review, and push work orders based on the tower as a unit; S4. Collect samples for knowledge-based hierarchical mining to support high-level application services of digital power distribution, audit of equipment and components, and cultivation of algorithms.

[0008] Preferably, the data review, model training and iterative optimization in S1 are specifically as follows: Formulate sample annotation standards based on the acquired data, compile systematic training materials, and develop training and examination systems, including at least course management, question bank management, and examination management; During the model design phase, we develop algorithm evaluation standards that are close to practical applications, adopt closed-loop testing methods that can fully demonstrate the algorithm's capabilities and open-loop testing methods that are closer to actual combat, and develop visual model evaluation tools.

[0009] Preferably, the design, training and optimization of the model are as follows: construct a large electric power vision model, use "text" or "text + sample image" as the prompt word, perform target detection on the prompt word object in the image, use the BERTBASE text encoder obtained by CN-CLIP pre-training as the text instruction encoder, and its parameter amount is 100M, use EVA-02-L as the selected visual encoder to fusion the encoder through the instruction image feature, the input is the encoding result E of the text instruction encoder and the encoding result X of the image encoder, and the output is the text encoding result E′ and the image encoding result X′ after feature fusion, and after the candidate target generator, select the positions corresponding to the first K image features for constructing candidate targets, and the decoder fuses the information between "candidate target-image-text", and the calculation result is recorded as Z=[z1,z2,…zK], where K is the number of candidate targets output by the candidate target generator; The similarity between Z and the text encoding result E′ is calculated to obtain the classification result; After a series of fully connected calculations, the position update value of the target box is obtained, and finally the predicted box is obtained; Given the true value of the target box {y1, y2, ... yn} and a series of prediction boxes {p1, p2, ... pK}, the model uses Hungarian matching to match the true value with the prediction box. The matching loss function includes the classification loss and the prediction box loss. The matching result is M = {m1, m2, ... mn}, which means that the prediction box successfully matched with yi is pmi; After the matching is completed, the model will calculate the loss function for all {(yi,pmi)} and unmatched prediction boxes {pi;i∉M}; the loss function for {(yi,pmi)} includes the classification loss and the loss of the prediction box, while for the unmatched prediction box, only the classification loss is calculated; the mathematical form of the classification loss is the Focal loss function; the loss of the prediction box includes the L1 loss and the GIOU loss; after obtaining the loss function, the model gradient is calculated and the parameters are updated according to the chain rule.

[0010] Preferably, during the model training and iterative optimization stage, algorithm cultivation is performed based on the large electric power vision model, and the cultivated algorithm at least includes the identification of defects, components, equipment, poles and towers, and the surrounding environment.

[0011] Preferably, the hierarchical mining of the image data based on the trained model in S2 includes at least a basic layer, a spatiotemporal layer, a tower layer, an equipment layer, a component layer and a defect layer, so as to realize hierarchical management and visual management of the image knowledge data.

[0012] Preferably, the hierarchical mining of image data based on the trained model in S2 is specifically as follows: (1) The knowledge mined at the basic layer includes at least the shooting location, shooting method, exposure type, shooting time, storage time, update time, resolution and image size; (2) The knowledge mined in the spatiotemporal layer includes at least the unit to which it belongs, the location where it was generated, longitude, latitude, season and month, terrain classification, environmental safety area, crossing situation and update time; (3) The knowledge of tower layer excavation should at least include the line to which it belongs, the tower to which it belongs, the tower material, tower function, tower circuit, signboard and update time; (4) The knowledge mined at the device level includes at least the device name, quantity, and classification; (5) The knowledge mined at the component level includes at least the component name, quantity, and attributes or classification; (6) The knowledge of defect layer mining includes at least defect content, quantity, and attributes or classification.

[0013] Preferably, the statistics and visualization display based on the basic layer at least include the display of the trend of the number of stored pictures, the statistics of the number of pictures taken at different parts of the picture, the distribution of the number of pictures taken by different shooting methods, and the distribution of the number of pictures by different exposure methods.

[0014] Preferably, the statistics and visualization display based on the spatiotemporal layer at least include the distribution of the number of pictures in each city, the distribution of the seasons and months in which the pictures were produced, the ranking of the number of pictures of crossing conditions, the ranking of the number of pictures of different terrains, and the ranking of the number of pictures of different environmental safety areas. The terrain classification, environmental safety area and crossing condition knowledge are first queried from the source database to see whether there are manual annotations. If there are manual annotations, the manually annotated information is displayed. If there are no manual annotations, the user can perform online reasoning through the surrounding environment detection model, and can switch the model version to view the reasoning results of different versions; Preferably, the statistics and visualization display based on the tower layer at least include the distribution of the number of images of different tower materials, the distribution of the number of images of different labels, the distribution of the number of images of different tower functions, and the distribution of the number of images of different tower circuits. The tower material, tower function, tower circuit, and signboard knowledge are first queried from the source database to see if they have been manually annotated. If they have been manually annotated, the manually annotated information is displayed. If there is no manual annotation, the user can perform online reasoning through the tower detection model, and can switch the model version to view the reasoning results of different versions.

[0015] Preferably, the statistics and visualization display based on the device layer at least include the distribution of the number of device images and the distribution of the number of device labels. The device name and quantity knowledge is first queried from the source database to see if there is any manual annotation. If there is manual annotation, the manual annotation information is displayed. If there is no manual annotation, the user can perform online reasoning through the device detection model, and can switch the model version to view the reasoning results of different versions.

[0016] Preferably, the statistics and visualization displays based on the component layer include at least the distribution of the number of component images and the statistics of component labels, wherein the component name, quantity and attribute or classification knowledge is first queried from the source database to see whether there is manual annotation. If there is manual annotation, the manually annotated information is displayed. If there is no manual annotation, the user can perform online reasoning through the component detection model and switch the model version to view the reasoning results of different versions.

[0017] Preferably, the statistics and visualization displays based on the defect layer include at least the distribution of the number of defect images and the statistics of defect labels, wherein the defect content, quantity and attributes or classification knowledge are first queried from the source database to see whether there are manual annotations. If there are manual annotations, the manually annotated information is displayed. If there are no manual annotations, the user can perform online reasoning through the defect detection model and switch the model version to view the reasoning results of different versions.

[0018] Preferably, the defect identification and collection in S3 is performed based on the pole tower as a unit, and the defect distribution display, review, and push of work orders based on the pole tower as a unit are realized by merging, deduplicating, and filtering multiple pictures of the pole tower for collection, which is specifically as follows: a. Identify and classify defects using the tower as a unit; traverse all images of the tower, obtain all categories of the tower by merging and removing duplicates, and obtain the number of labels for each category. Take the largest number of labels in this category among all images to obtain the tower defect distribution showing the type and number of defects of the tower; b. Display, review and push work orders for defect distribution based on pole towers.

[0019] Preferably, the defect distribution display, review and push of work orders based on towers are implemented as follows: 1) Click on the tower under the line to display the defect distribution with the tower as the unit; 2) Click to enter the manual review operation interface; or directly pass and review the next tower; 3) After the display tags are manually added, deleted, modified and reviewed, the algorithm identifies the detection rate and false detection ratio of the current tower dimension; 4) Display the types of defects of the tower, as well as the number of defects identified by the algorithm and the number after manual review for each type of defect. The number of defects after manual review is automatically updated; 5) Display the number of defect labels added, deleted, and modified after manual review; 6) Click on the tower under the line to automatically locate the picture with the most defects under the current tower; 7) Click Push to generate a defect elimination work order and push it to the supply service system.

[0020] Preferably, the sample collection of knowledge-based hierarchical mining-based high-level application services, equipment and component auditing, and algorithm cultivation for digital power distribution in S4 is specifically as follows: The knowledge-based hierarchical mining-based high-level application services for digital power distribution include at least: data-reality consistency management; automatic comparison with the power grid resource business middle-end data based on the identified tower information, equipment information, and geographic coordinates, and visualization of the comparison results on a power grid map to assist in the management of data-reality consistency in equipment; Statistics of components and equipment for line project acceptance; at least including: chassis chuck recognition, photo copy recognition, line-laying violation recognition, irregular collection reminder, defect recognition, material autonomous recognition, spatial distance measurement, and OCR image text recognition; Analyze the intactness rate of lightning arresters; identify the different states of lightning arresters, including intact, damaged, aged, and abnormal position, and perform hierarchical mining on lightning arrester-related images, including the basic layer, spatiotemporal layer, equipment layer, and defect layer; through hierarchical knowledge management and visualization, display the distribution and state changes of lightning arresters, especially the display of intactness rate, provide decision support based on the knowledge of hierarchical mining, and analyze the level of distribution automation.

[0021] Beneficial effects of the present invention: In the present invention, a standardized sample annotation system is formulated and combined with a systematic training material and examination system to ensure the professional skills of the annotation personnel and greatly improve the accuracy and efficiency of the annotation. This full-process annotation training system is relatively lacking in the existing technology and is particularly important for large-scale image data processing.

[0022] Accurate evaluation and optimization of algorithm performance: An innovative test method combining closed sets and open sets is adopted, combined with a self-developed visual evaluation tool, which not only fully demonstrates the algorithm capabilities, but also meets actual combat needs, accelerates the iterative optimization process of the algorithm, and ensures that the large electric power vision model can serve the target detection task more accurately.

[0023] Depth and breadth of knowledge mining and application: The rich information in image data is deeply mined in a hierarchical manner, from the basic layer to the defect layer, to achieve comprehensive knowledge extraction and statistical analysis. In particular, in the absence of manual annotation, it allows users to use the model for online reasoning, greatly broadening the scenarios of knowledge application and enhancing the adaptability and practicality of the system.

[0024] Intelligent and automated defect management: The defect identification, collection, review and work order push process based on towers is highly automated, which greatly improves the efficiency and accuracy of defect handling, reduces human errors, and ensures the timeliness and effectiveness of power grid maintenance.

[0025] Decision support and resource optimization: Through hierarchical knowledge mining, high-level application services for digital power distribution are supported, such as data-reality consistency management, line project acceptance, and lightning arrester integrity analysis. This provides detailed data support and decision-making basis, helps to rationally allocate resources and accurately maintain equipment, and improves the overall operating efficiency and safety of the power distribution system.

[0026] Efficiency of sample collection and model cultivation: The sample collection strategy targeting specific targets significantly improves the efficiency of sample labeling and model cultivation, provides a high-quality industry knowledge base for large model training, and accelerates the maturity and application of algorithm models. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0028] Figure 1 It is a schematic diagram of the overall process of the present invention. DETAILED DESCRIPTION

[0029] In order to make the technical means, creative features, objectives and effects of the present invention easy to understand, the following is a summary of the present invention. The present invention is further described with reference to specific implementation modes.

[0030] like Figure 1 As shown, the embodiment of the present invention provides a method for hierarchical mining of knowledge of overhead line images of a distribution network based on intelligent image recognition, comprising the following steps: S1. Data review, model design, training and iterative optimization. Obtain the inspection data of overhead lines of the distribution network by drones, manually annotate the inspection data to form a precisely annotated data set, conduct model training and evaluation based on the precisely annotated data set, and iterate and optimize according to the evaluation results until the model effect reaches a practical level. S2. Based on the trained model, the image data is hierarchically mined for knowledge, including at least the basic layer, spatiotemporal layer, tower layer, equipment layer, component layer, and defect layer, to achieve hierarchical management and visual management of image knowledge data; S3, identify and collect defects based on the tower as a unit, and collect the data of multiple tower images by merging, deduplication, and filtering to achieve defect distribution display, review, and push work orders based on the tower as a unit; S4. Collect samples for knowledge-based hierarchical mining to support high-level application services of digital power distribution, audit of equipment and components, and cultivation of algorithms.

[0031] Preferably, the data review, model training and iterative optimization in S1 are specifically as follows: Formulate sample annotation standards based on the acquired data, compile systematic training materials, and develop training and examination systems, including at least course management, question bank management, and examination management; During the model design phase, we develop algorithm evaluation standards that are close to practical applications, adopt closed-loop testing methods that can fully demonstrate the algorithm's capabilities and open-loop testing methods that are closer to actual combat, and develop visual model evaluation tools.

[0032] Preferably, the design, training and optimization of the model are as follows: construct a large electric power vision model, use "text" or "text + sample image" as the prompt word, perform target detection on the prompt word object in the image, use the BERTBASE text encoder obtained by CN-CLIP pre-training as the text instruction encoder, and its parameter amount is 100M, use EVA-02-L as the selected visual encoder to fusion the encoder through the instruction image feature, the input is the encoding result E of the text instruction encoder and the encoding result X of the image encoder, and the output is the text encoding result E′ and the image encoding result X′ after feature fusion, and after the candidate target generator, select the positions corresponding to the first K image features for constructing candidate targets, and the decoder fuses the information between "candidate target-image-text", and the calculation result is recorded as Z=[z1,z2,…zK], where K is the number of candidate targets output by the candidate target generator; The similarity between Z and the text encoding result E′ is calculated to obtain the classification result; After a series of fully connected calculations, the position update value of the target box is obtained, and finally the predicted box is obtained; Given the true value of the target box {y1, y2, ... yn} and a series of prediction boxes {p1, p2, ... pK}, the model uses Hungarian matching to match the true value with the prediction box. The matching loss function includes the classification loss and the prediction box loss. The matching result is M = {m1, m2, ... mn}, which means that the prediction box successfully matched with yi is pmi; After the matching is completed, the model will calculate the loss function for all {(yi,pmi)} and unmatched prediction boxes {pi;i∉M}; the loss function for {(yi,pmi)} includes the classification loss and the loss of the prediction box, while for the unmatched prediction box, only the classification loss is calculated; the mathematical form of the classification loss is the Focal loss function; the loss of the prediction box includes the L1 loss and the GIOU loss; after obtaining the loss function, the model gradient is calculated and the parameters are updated according to the chain rule.

[0033] Preferably, during the model training and iterative optimization stage, algorithm cultivation is performed based on the large electric power vision model, and the cultivated algorithm at least includes the identification of defects, components, equipment, poles and towers, and the surrounding environment.

[0034] Preferably, the hierarchical mining of the image data based on the trained model in S2 includes at least a basic layer, a spatiotemporal layer, a tower layer, an equipment layer, a component layer and a defect layer, so as to realize hierarchical management and visual management of the image knowledge data.

[0035] Preferably, the hierarchical mining of image data based on the trained model in S2 is specifically as follows: (1) The knowledge mined at the basic layer includes at least the shooting location, shooting method, exposure type, shooting time, storage time, update time, resolution and image size; (2) The knowledge mined in the spatiotemporal layer includes at least the unit to which it belongs, the location where it was generated, longitude, latitude, season and month, terrain classification, environmental safety area, crossing situation and update time; (3) The knowledge of tower layer excavation should at least include the line to which it belongs, the tower to which it belongs, the tower material, tower function, tower circuit, signboard and update time; (4) The knowledge mined at the device level includes at least the device name, quantity, and classification; (5) The knowledge mined at the component level includes at least the component name, quantity, and attributes or classification; (6) The knowledge of defect layer mining includes at least defect content, quantity, and attributes or classification.

[0036] Preferably, the statistics and visualization display based on the basic layer at least include the display of the trend of the number of stored pictures, the statistics of the number of pictures taken at different parts of the picture, the distribution of the number of pictures taken by different shooting methods, and the distribution of the number of pictures by different exposure methods.

[0037] Preferably, the statistics and visualization display based on the spatiotemporal layer at least include the distribution of the number of pictures in each city, the distribution of the seasons and months in which the pictures were produced, the ranking of the number of pictures of crossing conditions, the ranking of the number of pictures of different terrains, and the ranking of the number of pictures of different environmental safety areas. The terrain classification, environmental safety area and crossing condition knowledge are first queried from the source database to see whether there are manual annotations. If there are manual annotations, the manually annotated information is displayed. If there are no manual annotations, the user can perform online reasoning through the surrounding environment detection model, and can switch the model version to view the reasoning results of different versions; Preferably, the statistics and visualization display based on the tower layer at least include the distribution of the number of images of different tower materials, the distribution of the number of images of different labels, the distribution of the number of images of different tower functions, and the distribution of the number of images of different tower circuits. The tower material, tower function, tower circuit, and signboard knowledge are first queried from the source database to see if they have been manually annotated. If they have been manually annotated, the manually annotated information is displayed. If there is no manual annotation, the user can perform online reasoning through the tower detection model, and can switch the model version to view the reasoning results of different versions.

[0038] Preferably, the statistics and visualization display based on the device layer at least include the distribution of the number of device images and the distribution of the number of device labels. The device name and quantity knowledge is first queried from the source database to see if there is any manual annotation. If there is manual annotation, the manual annotation information is displayed. If there is no manual annotation, the user can perform online reasoning through the device detection model, and can switch the model version to view the reasoning results of different versions.

[0039] Preferably, the statistics and visualization displays based on the component layer include at least the distribution of the number of component images and the statistics of component labels, wherein the component name, quantity and attribute or classification knowledge is first queried from the source database to see whether there is manual annotation. If there is manual annotation, the manually annotated information is displayed. If there is no manual annotation, the user can perform online reasoning through the component detection model and switch the model version to view the reasoning results of different versions.

[0040] Preferably, the statistics and visualization displays based on the defect layer include at least the distribution of the number of defect images and the statistics of defect labels, wherein the defect content, quantity and attributes or classification knowledge are first queried from the source database to see whether there are manual annotations. If there are manual annotations, the manually annotated information is displayed. If there are no manual annotations, the user can perform online reasoning through the defect detection model and switch the model version to view the reasoning results of different versions.

[0041] Preferably, the defect identification and collection in S3 is performed based on the pole tower as a unit, and the defect distribution display, review, and push of work orders based on the pole tower as a unit are realized by merging, deduplicating, and filtering multiple pictures of the pole tower for collection, which is specifically as follows: a. Perform defect identification and classification on the tower as a unit; traverse all images of the tower, obtain all categories of the tower by merging and removing duplicates, and obtain the number of labels for each category, take the largest number of labels in the category among all images, and obtain the tower defect distribution showing the type and number of defects of the tower; b. Display, review and push work orders for defect distribution based on pole towers.

[0042] Preferably, the defect distribution display, review and push of work orders based on towers are implemented as follows: 1) Click on the tower under the line to display the defect distribution with the tower as the unit; 2) Click to enter the manual review operation interface; or directly pass and review the next tower; 3) After the display tags are manually added, deleted, modified and reviewed, the algorithm identifies the detection rate and false detection ratio of the current tower dimension; 4) Display the types of defects of the tower, as well as the number of defects identified by the algorithm and the number after manual review for each type of defect. The number of defects after manual review is automatically updated; 5) Display the number of defect labels added, deleted, and modified after manual review; 6) Click on the tower under the line to automatically locate the picture with the most defects under the current tower; 7) Click Push to generate a defect elimination work order and push it to the supply service system.

[0043] Preferably, the sample collection of knowledge-based hierarchical mining-based high-level application services, equipment and component auditing, and algorithm cultivation for digital power distribution in S4 is specifically: The knowledge-based hierarchical mining-based high-level application services for digital power distribution include at least: data-reality consistency management; automatic comparison with the power grid resource business middle-end data based on the identified tower information, equipment information, and geographic coordinates, and visualization of the comparison results on a power grid map to assist in the management of data-reality consistency in equipment; Statistics of components and equipment for line project acceptance; at least including: chassis chuck recognition, photo copy recognition, line-laying violation recognition, irregular collection reminder, defect recognition, material autonomous recognition, spatial distance measurement, and OCR image text recognition; Analyze the intactness rate of lightning arresters; identify the different states of lightning arresters, including intact, damaged, aged, and abnormal position, and perform hierarchical mining on lightning arrester-related images, including the basic layer, spatiotemporal layer, equipment layer, and defect layer; through hierarchical knowledge management and visualization, display the distribution and state changes of lightning arresters, especially the display of intactness rate, provide decision support based on the knowledge of hierarchical mining, and analyze the level of distribution automation.

[0044] 1: Constructing a sample annotation and training system Technical problem solving process: In view of the complexity of power inspection image recognition, we first developed detailed sample annotation standards by analyzing a large number of power line images taken on site, covering identification marks in multiple dimensions such as defects, components, equipment, poles, and surrounding environments. Subsequently, we compiled systematic training materials based on these standards, and developed a training and examination system that includes tutorial management, question bank management, and examination management. This system not only provides theoretical learning materials, but also ensures that annotators can complete image annotation tasks accurately and efficiently through physical training, classroom guidance, and on-the-job training, thereby fundamentally improving the quality and efficiency of annotation.

[0045] 2: Construction and application of the electric power vision big model Technical problem solving process: In the model design and optimization stage, the electric power vision big model was used to build an electric power vision big model that can accept "text" or "text + sample image" as prompts. First, by formulating algorithm evaluation standards close to actual applications and using closed-loop and open-loop testing methods, it is ensured that the model can comprehensively and accurately evaluate the status of power facilities. The further developed visual model evaluation tool enables users to intuitively understand the performance of the model and make optimization adjustments. Through continuous iterative training, the model can accurately identify various elements in the power line, significantly improving the accuracy and efficiency of defect detection.

[0046] 3: Hierarchical mining and knowledge application examples Technical problem solving process: Taking the pole tower as an example, it shows how to manage and apply knowledge through hierarchical mining technology. First, the system automatically counts and visualizes multi-dimensional information about the pole tower, such as the material, function, circuit, etc. of the pole tower. For unlabeled information, the system enables the online reasoning model for real-time recognition, and users can select the model version as needed to obtain the best reasoning result. In addition, the system can identify and collect defects for each pole tower, automatically summarize the defect type and quantity, and form an intuitive defect distribution map. Through manual review and algorithm collaboration, the system dynamically updates the defect label and automatically pushes the work order to the service system, greatly improving the response speed and accuracy of defect handling.

[0047] IV: Technical problem solving process of digital high-level application examples: In terms of digital-real consistency management, the identified towers, equipment information and geographic coordinates are used to automatically compare with the business middle-end data to discover and visualize any data inconsistencies, facilitate rapid corrections, and ensure the accuracy of equipment information. For the analysis of lightning arrester integrity, the system makes a detailed division of the lightning arrester status, and through hierarchical knowledge management and visualization, it provides a scientific basis for formulating lightning arrester maintenance strategies, and improves the predictability and pertinence of equipment maintenance. Through the above embodiments, the present invention provides a comprehensive solution from data labeling to model training, knowledge mining to high-level applications, which effectively solves multiple technical challenges in the field of power inspection and promotes the intelligent transformation of distribution systems.

[0048] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of the present invention is defined by the attached claims and their equivalents.

Claims

1. A method for hierarchical mining of knowledge from overhead line images of distribution network based on intelligent image recognition, characterized in that: The following steps are involved: S1. Data review, model design, training and iterative optimization. Obtain the inspection data of overhead lines of the distribution network by drones, manually annotate the inspection data to form a precisely annotated data set, conduct model training and evaluation based on the precisely annotated data set, and iterate and optimize according to the evaluation results until the model effect reaches a practical level. S2. Based on the trained model, the image data is hierarchically mined for knowledge, including at least the basic layer, spatiotemporal layer, tower layer, equipment layer, component layer, and defect layer, to achieve hierarchical management and visual management of image knowledge data; S3, identify and collect defects based on the tower as a unit, and collect the data of multiple tower images by merging, deduplication, and filtering to achieve defect distribution display, review, and push work orders based on the tower as a unit; S4. Collect samples for knowledge-based hierarchical mining to support high-level application services of digital power distribution, audit of equipment and components, and cultivation of algorithms.

2. The method for hierarchical mining of knowledge of overhead line images of distribution network based on intelligent image recognition according to claim 1 is characterized by: The data review, model training and iterative optimization in S1 are specifically as follows: Formulate sample annotation standards based on the acquired data, compile systematic training materials, and develop training and examination systems, including at least course management, question bank management, and examination management; During the model design phase, we develop algorithm evaluation standards that are close to practical applications, adopt closed-loop testing methods that can fully demonstrate the algorithm's capabilities and open-loop testing methods that are closer to actual combat, and develop visual model evaluation tools.

3. The method for hierarchical mining of knowledge of overhead line images of distribution network based on intelligent image recognition according to claim 2 is characterized by: The design, training and optimization of the model are as follows: construct a large electric power vision model, use "text" or "text + sample image" as the prompt word, perform target detection on the prompt word object in the image, use the BERTBASE text encoder obtained by CN-CLIP pre-training as the text instruction encoder, and its parameter volume is 100M. Use EVA-02-L as the selected visual encoder to fusion the encoder through the instruction image feature. The input is the encoding result E of the text instruction encoder and the encoding result X of the image encoder. The output is the text encoding result E′ and the image encoding result X′ after feature fusion. After the candidate target generator, select the positions corresponding to the first K image features to construct the candidate target. The decoder fuses the information between "candidate target-image-text", and the calculation result is recorded as Z=[z1,z2,…zK], where K is the number of candidate targets output by the candidate target generator; The similarity between Z and the text encoding result E′ is calculated to obtain the classification result; After a series of fully connected calculations, the position update value of the target box is obtained, and finally the predicted box is obtained; Given the true value of the target box {y1, y2, ... yn} and a series of prediction boxes {p1, p2, ... pK}, the model uses Hungarian matching to match the true value with the prediction box. The matching loss function includes the classification loss and the prediction box loss. The matching result is M = {m1, m2, ... mn}, which means that the prediction box successfully matched with yi is pmi; After the matching is completed, the model will calculate the loss function for all {(yi,pmi)} and unmatched prediction boxes {pi;i∉M}; the loss function for {(yi,pmi)} includes the classification loss and the loss of the prediction box, while for the unmatched prediction box, only the classification loss is calculated; the mathematical form of the classification loss is the Focal loss function; the loss of the prediction box includes the L1 loss and the GIOU loss; after obtaining the loss function, the model gradient is calculated and the parameters are updated according to the chain rule.

4. The method for hierarchical mining of knowledge of overhead line images of distribution network based on intelligent image recognition according to claim 2 is characterized by: During the model training and iterative optimization stage, algorithm cultivation is carried out based on the large electric power vision model. The cultivated algorithm at least includes the identification of defects, components, equipment, poles and towers, and the surrounding environment.

5. The method for hierarchical mining of knowledge of overhead line images of distribution network based on intelligent image recognition according to claim 2 is characterized by: The hierarchical mining of the image data based on the trained model in S2 includes at least the basic layer, the spatiotemporal layer, the tower layer, the equipment layer, the component layer and the defect layer, so as to realize the hierarchical management and visual management of the image knowledge data.

6. The method for hierarchical mining of knowledge from overhead line images of distribution network based on intelligent image recognition according to claim 5 is characterized by: The hierarchical mining of image data based on the trained model in S2 is specifically as follows: (1) The knowledge mined at the basic layer includes at least the shooting location, shooting method, exposure type, shooting time, storage time, update time, resolution and image size; (2) The knowledge mined in the spatiotemporal layer includes at least the unit to which it belongs, the location where it was generated, longitude, latitude, season and month, terrain classification, environmental safety area, crossing situation and update time; (3) The knowledge of tower layer excavation should at least include the line to which it belongs, the tower to which it belongs, the tower material, tower function, tower circuit, signboard and update time; (4) The knowledge mined at the device level includes at least the device name, quantity, and classification; (5) The knowledge mined at the component level includes at least the component name, quantity, and attributes or classification; (6) The knowledge of defect layer mining includes at least defect content, quantity, and attributes or classification.

7. The method for hierarchical mining of knowledge from distribution network overhead line images based on intelligent image recognition according to claim 6 is characterized by: The statistics and visualization display based on the basic layer at least include the display of the trend of the number of stored pictures, the statistics of the number of pictures taken from different parts of the picture, the distribution of the number of pictures taken by different shooting methods, and the distribution of the number of pictures by different exposure methods.

8. The method for hierarchical mining of knowledge from distribution network overhead line images based on intelligent image recognition according to claim 7, characterized in that: The statistics and visualization displays based on the spatiotemporal layer include at least the distribution of the number of pictures in each city, the distribution of the seasons and months when the pictures were produced, the ranking of the number of pictures of crossing situations, the ranking of the number of pictures of different terrains, and the ranking of the number of pictures of different environmental safety areas. Among them, the terrain classification, environmental safety area and crossing situation knowledge are first queried from the source database to see if there are any manual annotations. If there are manual annotations, the manually annotated information will be displayed. If there are no manual annotations, users can perform online reasoning through the surrounding environment detection model, and can switch the model version to view the reasoning results of different versions.

9. The method for hierarchical mining of knowledge of overhead line images of distribution network based on intelligent image recognition according to claim 7, characterized in that: The statistics and visualization display based on the tower layer at least include the distribution of the number of pictures of different tower materials, the distribution of the number of pictures with different labels, the distribution of the number of pictures of different tower functions and the distribution of the number of pictures of different tower loops. Among them, the tower material, tower function, tower loop and signboard knowledge are first queried from the source database to see whether there are manual annotations. If there are manual annotations, the manually annotated information will be displayed. If there are no manual annotations, users can perform online reasoning through the tower detection model, and can switch the model version to view the reasoning results of different versions.

10. The method for hierarchical mining of knowledge from distribution network overhead line images based on intelligent image recognition according to claim 7, characterized in that: The statistics and visualization displays based on the device layer include at least the distribution of the number of device images and the distribution of the number of device labels. The device name and quantity knowledge are first queried from the source database to see if there is any manual annotation. If there is manual annotation, the manually annotated information is displayed. If there is no manual annotation, the user can perform online reasoning through the device detection model, and can switch the model version to view the reasoning results of different versions.

11. The method for hierarchical mining of knowledge from overhead line images of distribution network based on intelligent image recognition according to claim 7, characterized in that: At the same time, the statistics and visualization based on the component layer include at least the distribution of the number of component images and the statistics of component labels. The component name, quantity, attribute or classification knowledge is first queried from the source database to see if there is any manual annotation. If there is manual annotation, the manually annotated information is displayed. If there is no manual annotation, the user can perform online reasoning through the component detection model, and can switch the model version to view the reasoning results of different versions.

12. The method for hierarchical mining of knowledge from overhead line images of distribution network based on intelligent image recognition according to claim 7, characterized in that: At the same time, the statistics and visualization based on the defect layer include at least the distribution of the number of defect images and the statistics of defect labels. The defect content, quantity, attributes or classification knowledge are first queried from the source database to see whether there are manual annotations. If there are manual annotations, the manually annotated information is displayed. If there are no manual annotations, users can perform online reasoning through the defect detection model, and can switch the model version to view the reasoning results of different versions.

13. The method for hierarchical mining of knowledge from overhead line images of distribution network based on intelligent image recognition according to claim 7, characterized in that: The defect identification and collection in S3 is performed based on the pole tower as a unit, and the defect distribution display, review, and push of work orders based on the pole tower as a unit are realized by merging, deduplicating, and filtering multiple pictures of the pole tower for collection, which is specifically as follows: a. Identify and classify defects using the tower as a unit; traverse all images of the tower, obtain all categories of the tower by merging and removing duplicates, and obtain the number of labels for each category. Take the largest number of labels in this category among all images to obtain the tower defect distribution showing the type and number of defects of the tower; b. Display, review and push work orders for defect distribution based on pole towers.

14. The method for hierarchical mining of knowledge from distribution network overhead line images based on intelligent image recognition according to claim 13, characterized in that: The details of displaying, reviewing and pushing work orders for defect distribution based on towers are as follows: 1) Click on the tower under the line to display the defect distribution with the tower as the unit; 2) Click to enter the manual review operation interface; or directly pass and review the next tower; 3) After the display tags are manually added, deleted, modified and reviewed, the algorithm identifies the detection rate and false detection ratio of the current tower dimension; 4) Display the types of defects of the tower, as well as the number of defects identified by the algorithm and the number after manual review for each type of defect. The number of defects after manual review is automatically updated; 5) Display the number of defect labels added, deleted, and modified after manual review; 6) Click on the tower under the line to automatically locate the picture with the most defects under the current tower; 7) Click Push to generate a defect elimination work order and push it to the supply service system.

15. The method for hierarchical mining of knowledge from distribution network overhead line images based on intelligent image recognition according to claim 13, characterized in that: The knowledge based on hierarchical mining in S4 supports the sample collection of high-level application services, equipment and component audits, and algorithm cultivation for digital power distribution, specifically: The knowledge-based hierarchical mining-based high-level application services for digital power distribution include at least: data-reality consistency management; automatic comparison with the power grid resource business middle-end data based on the identified tower information, equipment information, and geographic coordinates, and visualization of the comparison results on a power grid map to assist in the management of data-reality consistency in equipment; Statistics of components and equipment for line project acceptance; at least including: chassis chuck recognition, photo copy recognition, line-laying violation recognition, irregular collection reminder, defect recognition, material autonomous recognition, spatial distance measurement, and OCR image text recognition; Analyze the intactness rate of lightning arresters; identify the different states of lightning arresters, including intact, damaged, aged, and abnormal position, and perform hierarchical mining on lightning arrester-related images, including the basic layer, spatiotemporal layer, equipment layer, and defect layer; through hierarchical knowledge management and visualization, display the distribution and state changes of lightning arresters, especially the display of intactness rate, provide decision support based on the knowledge of hierarchical mining, and analyze the level of distribution automation.

Images