Substation drawing label multi-modal extraction method, device, equipment, medium and program product

Abstract

The present disclosure provides a substation drawing label multi-modal extraction method, device, equipment, medium and program product, the method comprising: dynamically and adaptively slicing the substation drawing based on information entropy feature density, determining the position mapping relationship between each image slice and the substation drawing; performing symbol detection and text label extraction on the image slice in parallel to obtain a symbol set and a text set; constructing a heterogeneous graph, calculating a multi-criteria edge weight, and using a graph neural network to infer the heterogeneous graph based on the multi-criteria edge weight to obtain a correlation probability matrix, and matching the correlation probability matrix to obtain a mapping relationship set; constructing a substation electrical topology graph, and using power system topology constraints to perform consistency checking and correction on the substation electrical topology graph and the correlation pairs in the mapping relationship set to obtain substation structured data. The dynamic slicing mechanism based on information entropy density solves the problem of being cut off or blurred in large-resolution drawings, and improves the recognition completeness.

Description

Technical Field

[0001] This disclosure relates to the interdisciplinary fields of power system automation and computer vision, and in particular to a method, apparatus, equipment, medium, and program product for multimodal extraction of substation drawing annotations. Background Technology

[0002] With the deepening of the digital transformation of the power grid, the power system has accumulated a massive amount of historical paper and vector substation drawings, such as primary system diagrams and secondary wiring diagrams. Automated recognition and digital modeling of these drawings are fundamental to building a digital twin power grid and realizing automated voltage detection and fault-assisted diagnosis. In recent years, object detection and optical character recognition (OCR) technologies based on deep learning have made significant progress in general fields, but in the field of power system drawing recognition, challenges remain, including extremely large image sizes, complex relationships between symbols and text, and strong professional semantic constraints.

[0003] Substation drawing recognition solutions in related technologies typically employ the following process: Preprocessing: Large-size engineering drawings are uniformly scaled to a fixed size, such as 1024x1024, or the large drawing is divided into several sub-drawings using a sliding window slicing method with a fixed step size. Symbol detection: Object detection algorithms such as Faster R-CNN or YOLO are used to locate and classify power symbols in the sub-drawings. Text recognition: Traditional OCR engines such as Tesseract or PaddleOCR are used to extract text from the drawings. Post-processing association: Based on simple physical distance judgments such as the nearest neighbor principle, the recognized text labels are assigned to the nearest equipment symbol.

[0004] However, the above scheme has the following drawbacks: directly scaling the drawing globally will cause small symbols and text to become blurry, resulting in a sharp drop in recognition rate; fixed step size slicing is very likely to cut off symbols or long strings located on the cutting line, resulting in fragmented recognition results and failing to guarantee the integrity of cross-slice targets, that is, there is a problem of information loss and truncation caused by high resolution processing. Summary of the Invention

[0005] In view of this, the purpose of this disclosure is to provide a method, device, equipment, medium and program product for multimodal extraction of substation drawing annotations.

[0006] To achieve the above objectives, the first aspect of this disclosure provides a method for extracting multimodal data from substation drawings, including:

[0007] The original substation drawing is obtained, and the original substation drawing is dynamically and adaptively sliced ​​based on the information entropy feature density to obtain multiple image slices. The positional mapping relationship between each image slice and the original substation drawing is determined. Symbol detection and text annotation extraction are performed in parallel on the image slices to obtain a symbol set and a text set. The symbol set includes at least one symbol and its location information, and the text set includes at least one text and its location information. Construct a heterogeneous graph with each symbol in the symbol set as a first type of node and each text in the text set as a second type of node. Calculate and fuse the multi-criteria edge weights between the first type of node and the second type of node. Based on the multi-criteria edge weights, use a graph neural network to infer the heterogeneous graph to obtain an association probability matrix. Match the association probability matrix to obtain a mapping relationship set. Based on the location mapping relationship, the symbol set, and the text set, a substation electrical topology diagram is constructed. Then, the consistency of the substation electrical topology diagram and the associated pairs in the mapping relationship set is checked and corrected using power system topology constraints to obtain substation structured data.

[0008] The second aspect of this disclosure provides a device for extracting multimodal data from substation drawings, comprising: The slicing module is configured to acquire the original substation drawing, perform dynamic adaptive slicing of the original substation drawing based on the information entropy feature density to obtain multiple image slices, and determine the positional mapping relationship between each image slice and the original substation drawing. The extraction module is configured to perform symbol detection and text annotation extraction on the image slices in parallel to obtain a symbol set and a text set. The symbol set includes at least one symbol and its location information, and the text set includes at least one text and its location information. The mapping module is configured to construct a heterogeneous graph with each symbol in the symbol set as a first type of node and each text in the text set as a second type of node, calculate and fuse the multi-criteria edge weights between the first type of node and the second type of node, use a graph neural network to infer the heterogeneous graph based on the multi-criteria edge weights to obtain an association probability matrix, and match the association probability matrix to obtain a mapping relationship set. The construction and correction module is configured to construct a substation electrical topology diagram based on the location mapping relationship, the symbol set, and the text set, and to perform consistency verification and correction on the association pairs in the substation electrical topology diagram and the mapping relationship set using power system topology constraints, thereby obtaining substation structured data.

[0009] A third aspect of this disclosure provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the method described in the first aspect.

[0010] A fourth aspect of this disclosure provides a non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform the method described in the first aspect.

[0011] The fifth aspect of this disclosure provides a computer program product including computer program instructions that, when executed on a computer, cause the computer to perform the method as described in the first aspect.

[0012] As can be seen from the above, the substation drawing annotation multimodal extraction method, apparatus, equipment, medium, and program product provided in this disclosure include: acquiring the original substation drawing; dynamically and adaptively slicing the original substation drawing based on information entropy feature density to obtain multiple image slices, and determining the positional mapping relationship between each image slice and the original substation drawing; performing symbol detection and text annotation extraction on the image slices in parallel to obtain a symbol set and a text set, wherein the symbol set includes at least one symbol and its position information, and the text set includes at least one text and its position information; and constructing a system based on the symbols... A heterogeneous graph is constructed where each symbol in the set represents a first-class node and each text in the text set represents a second-class node. Multi-criteria edge weights between the first-class and second-class nodes are calculated and fused. Based on these weights, a graph neural network is used to infer the heterogeneous graph, yielding an association probability matrix. This matrix is ​​then matched to obtain a mapping relationship set. Based on the location mapping relationships, the symbol set, and the text set, a substation electrical topology map is constructed. Power system topology constraints are used to verify and correct the consistency of the association pairs in the substation electrical topology map and the mapping relationship set, resulting in structured substation data. By employing a dynamic slicing mechanism based on information entropy density, the problem of long texts and subtle symbols being cut off or blurred in ultra-high resolution drawings is solved, improving the recognition completeness rate. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in this disclosure or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 A flowchart illustrating an exemplary method for extracting multimodal annotations from substation drawings provided in this embodiment of the present disclosure is shown.

[0015] Figure 2 A schematic diagram of an exemplary substation drawing annotation multimodal extraction device provided in an embodiment of this disclosure is shown.

[0016] Figure 3 A schematic diagram of the hardware structure of an exemplary computer device provided in an embodiment of this disclosure is shown. Detailed Implementation

[0017] It is understood that before using the technical solutions of the various embodiments in this disclosure, users will be informed of the type, scope of use, and usage scenarios of the personal information involved in an appropriate manner, and user authorization will be obtained.

[0018] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose, based on the prompt message, whether to provide personal information to the software or hardware such as electronic devices, applications, servers, or storage media performing the operations of this disclosed technical solution.

[0019] As an optional but not limited implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0020] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0021] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0022] To make the objectives, technical solutions, and advantages of this disclosure clearer, the principles and spirit of this disclosure will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided merely to enable those skilled in the art to better understand and implement this disclosure, and are not intended to limit the scope of this disclosure in any way. Rather, these embodiments are provided to make this disclosure more thorough and complete, and to fully convey the scope of this disclosure to those skilled in the art.

[0023] In this article, it is important to understand that any number of elements in the accompanying figures is for illustrative purposes and not for limitation, and any naming is for distinction only and has no limiting meaning.

[0024] It should be noted that, unless otherwise defined, the technical or scientific terms used in the embodiments of this disclosure should have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms "first," "second," and similar words used in the embodiments of this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly. The article "a" or "an" preceding an element does not exclude the existence of multiple such elements.

[0025] The principles and spirit of this disclosure will be explained in detail below with reference to several representative embodiments.

[0026] The substation drawing recognition scheme described in the background art also has the following drawbacks: Lack of business logic validation leads to a high false recognition rate. General-purpose AI models rely solely on visual features for judgment and lack electrical expertise. For example, visually, "1" and "I" or "0" and "O" are easily confused, and current technologies cannot automatically correct this based on substation equipment numbering rules. Existing solutions may identify connections that are physically and logically impossible, such as two circuit breakers directly connected without conductors, lacking strong constraints based on power topology relationships.

[0027] Multimodal mapping relationships are fragile: The system relies solely on physical proximity to match text and symbols. In substation drawings with densely packed equipment, such as closely arranged relays or pressure plates, this simple geometric relationship is prone to errors and cannot handle complex guide line pointing relationships or one-to-many and many-to-one annotation scenarios.

[0028] Waste of computing resources: The fixed-slice method repeatedly calculates the background color of a large number of blank areas in the drawing, which has no effective information. When processing very large drawings with tens of thousands of pixels, the reasoning efficiency is extremely low.

[0029] The purpose of this invention is to overcome the shortcomings of existing technologies in processing power engineering drawings, such as low recognition accuracy, poor logical consistency, and waste of computing resources. It provides a multimodal extraction system and method for substation drawing annotations that combines topological constraints and dynamic slicing. Specifically, the technical problems to be solved include: Solving the problems of information loss and target truncation in the preprocessing stage of ultra-high resolution drawings: To address the issues of pixel blurring caused by global scaling in existing technologies, and the tendency of fixed-step slicing methods to cut off power symbols and long text annotations in the middle, rendering them unrecognizable, this invention aims to propose a dynamic adaptive slicing mechanism. This mechanism can dynamically adjust the slicing window and overlapping area according to the feature distribution density of the drawing content, ensuring that subtle symbols and complete text annotations in ultra-high resolution drawings maintain the integrity of image features before entering the recognition model.

[0030] Addressing the identification error issue caused by the lack of power business logic in general AI models: To address the shortcomings of existing technologies that rely solely on visual feature recognition, failing to distinguish between ambiguous characters and producing recognition results that violate the physical laws of power systems, this invention aims to introduce power topology constraint logic. By establishing a priori knowledge base and numbering semantic rules for substation equipment connections, the original output results of the model are logically verified and forcibly corrected, thereby eliminating recognition errors that do not conform to the physical connection logic of power systems and improving the system's recognition accuracy in specialized scenarios.

[0031] Solving the problem of inaccurate association between text annotations and graphic symbols in dense layouts: To address the problem that existing technologies, which rely on the simple nearest geometric distance principle, are prone to mismatching text and symbols in densely packed symbol areas such as cabinets and terminal blocks, this invention aims to achieve a deep cross-modal mapping mechanism. By fusing spatial location, guide line geometric features, and semantic association attributes, a multimodal feature fusion model is constructed to achieve a high-accuracy mapping association between text annotations and graphic symbols, effectively handling one-to-many, many-to-one, and complex annotation scenarios with guide line orientations.

[0032] Solving the computational efficiency problem of large-size redundant information: To address the problem of low inference efficiency caused by indiscriminate calculation of large blank areas in existing technologies, this invention aims to quickly filter invalid information areas in drawings through a dynamic slicing sampling strategy, focusing on key feature areas containing symbols and annotations, thereby significantly improving the system's calculation speed for ultra-large drawings with tens of thousands of pixels while ensuring full recognition coverage.

[0033] Therefore, this invention provides a multimodal extraction method for substation drawing annotations that combines topological constraints and dynamic slicing. Its core process is divided into the following four key stages: dynamic adaptive slicing stage based on feature density, parallel extraction stage of multimodal features, cross-modal spatial association stage based on heterogeneous graph neural network, and deep error correction stage based on power business logic and topology path.

[0034] The following is an explanation of the key technologies involved: Multimodal extraction: This refers to simultaneously extracting graphic symbols such as circuit breakers, disconnectors, and transformers, as well as text information such as equipment numbers and specifications from drawings, and establishing the relationship between the two.

[0035] Dynamic tiling: For ultra-high resolution images, large images are transformed into small-sized tile images that can be processed by the model through a sliding window with a non-fixed step size or a sampling strategy based on feature density.

[0036] Topological constraints: Using the physical connection logic of the power system, such as the connection order of buses, branches, and bays, as prior knowledge, constraints and corrections are applied to logical contradictions identified by AI.

[0037] Cross-modal mapping: Establishing spatial and semantic correspondences between text labels and graphic symbols in feature space.

[0038] Figure 1 A flowchart illustrating an exemplary method for extracting multimodal annotations from substation drawings provided in this embodiment of the present disclosure is shown.

[0039] Methods for extracting multimodal data from substation drawings include: S101. Obtain the original substation drawing, perform dynamic adaptive slicing on the original substation drawing based on information entropy feature density to obtain multiple image slices, and determine the positional mapping relationship between each image slice and the original substation drawing.

[0040] In some exemplary embodiments, the dynamic adaptive slicing of the original substation drawing based on information entropy feature density to obtain multiple image slices includes: The original substation drawings are downsampled to generate thumbnails; Calculate the local information entropy of the thumbnail and generate an information entropy density map; Based on the information entropy density map, the original substation drawings are divided into information-dense areas and information-sparse areas; In the information-dense area, sliding slices are performed using a first window size and a first overlap ratio; In the information sparse region, sliding slicing is performed using a second window size and a second overlap rate, where the second window size is larger than the first window size and the second overlap rate is smaller than the first overlap rate. For connected regions located at the edge of the slice, the image slice is generated centered on the break point of the connected region in the original substation drawing.

[0041] Among them, the information entropy density map refers to the probability distribution map generated by calculating the pixel change gradient or feature richness in a local area of ​​the original image, which is used to guide the dynamic slicing algorithm to identify key information areas in the drawing.

[0042] In practical implementation, for substation drawings with tens of thousands of pixels, this invention does not use mechanical slicing with a fixed step size, but instead performs the following dynamic adaptive slicing method based on information entropy feature density: As an example, tens of thousands of pixels includes 10,000×10,000 and above.

[0043] Global low-resolution density map generation: First, the original image is downsampled proportionally to a low-resolution thumbnail. Then, a convolutional neural network is used to extract the feature response values ​​of the entire image and calculate the local pixel information entropy H.

[0044] As an example, low resolution includes 1024×1024.

[0045] Non-uniform grid partitioning: Based on the distribution of information entropy H, the entire image is divided into dense feature regions and sparse feature regions.

[0046] As an example, dense feature regions contain symbols and text; sparse feature regions contain long lines or blank spaces.

[0047] In dense areas, dynamically set the slice window size. (e.g., 640×640), with an overlap rate set to 35%. 50%.

[0048] In the sparse region, increase the sliding step size and set the window size to [value missing]. (e.g., 1280×1280), the overlap rate is reduced to 10%.

[0049] Edge-truncated target resampling mechanism: During the slicing process, if the target detection operator identifies that there is a truncated connected component at the edge of the slice, such as a long device number being cut, then a compensation slice (Anchor Patch) is generated by backtracking with the coordinates of the break point as the center, ensuring that the target appears in complete form in at least one slice.

[0050] Coordinate mapping restoration: Record the offset of each slice relative to the top left corner of the original image ( A layered coordinate transformation matrix is ​​established to subsequently restore the local detection results to the physical coordinate system of the entire image without loss.

[0051] Through the above exemplary embodiments, fixed-step cutting is abandoned, and on-demand slicing is achieved through global density scanning. Compensation sampling blocks are introduced to handle edge-crossing targets. This solves the problems of long annotation text truncation and missed detection of overlapping symbols caused by traditional slicing modes for ultra-high resolution drawings, improving the overall recognition completeness rate of tens of thousands of pixel drawings by about 25%, while reducing the amount of invalid computation by more than 40% by skipping blank areas.

[0052] For ultra-high resolution processing, in addition to the dynamic slicing method based on information entropy density mentioned above, the following alternative technical paths can also be adopted: A two-stage scaling scheme based on feature map region proposals (ROIs) is adopted: First, the entire image is scaled down to a very low resolution, and a lightweight detection network is used to predict all potential regions of interest (ROIs) containing symbols and text. Then, local matting and recognition are performed only on these ROIs at the original resolution, rather than slicing the entire image.

[0053] This approach avoids the complex logic of slicing and stitching, and can achieve the same goal of saving computing resources and maintaining resolution by combining a global coarse view with a local fine view.

[0054] A global feature mapping scheme based on Feature Pyramid Network (FPN) is adopted: By employing a backbone network with extremely deep downsampling ratio, such as ResNet with dilated convolutions, a very large receptive field is directly generated at the feature layer. Through multi-scale feature fusion, semantic understanding of large images is achieved in the feature space without physical slicing.

[0055] S102. Perform symbol detection and text annotation extraction on the image slices in parallel to obtain a symbol set and a text set. The symbol set includes at least one symbol and its location information, and the text set includes at least one text and its location information.

[0056] In practice, after obtaining image slices, the following multimodal feature parallel extraction and structuring steps are performed: Symbol detection branch: The improved Faster R-CNN framework is adopted, and a feature pyramid enhancement module (FPN+) is added after the ResNet-101 feature extraction layer. It is specifically designed for multi-scale detection of small symbols in substations (such as surge arresters and pressure plates), and outputs the symbol category, center point coordinates and confidence score.

[0057] Text annotation extraction branch: OCR recognition is performed using a Transformer structure with a self-attention mechanism.

[0058] Text line positioning: Extract the location boxes of all text annotations in the drawing.

[0059] Semantic recognition: Identify text content and perform preliminary normalization of characters according to power specifications (such as correcting tilt angles).

[0060] Result structure: Output symbol set S={ } and text set T={ }, where each element contains location, category / content and its feature vector.

[0061] S103. Construct a heterogeneous graph with each symbol in the symbol set as a first type of node and each text in the text set as a second type of node. Calculate and fuse the multi-criteria edge weights between the first type of node and the second type of node. Based on the multi-criteria edge weights, use a graph neural network to infer the heterogeneous graph to obtain an association probability matrix. Match the association probability matrix to obtain a mapping relationship set.

[0062] In some exemplary embodiments, the calculation and fusion of multi-criteria edge weights between the first type of nodes and the second type of nodes includes: Calculate the geometric distance and relative angle between the first type of node and the second type of node, and generate geometric constraint weights; Extract the guide lines from the original substation drawings. If a guide line is detected connecting a second type node and a first type node, then generate a visual guide line weight. Calculate the semantic matching score between the text content of the second type of node and the category label of the first type of node, and generate semantic affinity weights; The geometric constraint weights, visual guide line weights, and semantic affinity weights are weighted and fused to obtain the multi-criteria edge weights.

[0063] In substation drawings, a leader line is a thin straight or broken line used to clarify the relationship between the labeled content and the labeled object, connecting the text label box and specific power equipment symbols.

[0064] In some exemplary embodiments, the step of matching the association probability matrix to obtain a set of mapping relationships includes: The bipartite graph optimal matching algorithm is used to process the association probability matrix to determine the symbol associated with each text, thus obtaining the mapping relationship set.

[0065] In practice, this stage executes the algorithm flow for cross-modal mapping based on heterogeneous graph neural networks (HGT): Heterogeneous graph construction: Symbol node S and text node T are treated as two different types of vertices in the graph.

[0066] Among them, a heterogeneous graph refers to a graph data structure that contains multiple node types such as symbol nodes and text nodes, and multiple edge relationships such as geometric associations, semantic associations, and leader line associations.

[0067] Multi-criteria edge weight calculation includes geometric constraints, visual guide line features, and semantic affinity: Geometric constraints: calculation and Distance between the centroids and relative angles .

[0068] Visual guide line features: Guide lines in the drawing are extracted using the LSD (Line Segment Detector) algorithm. If one end of a line segment falls within... Within the expanded bounding box, the other end points to Then the visual weight of that side will be... Set to highest priority.

[0069] Semantic affinity: Using a pre-trained power corpus model, the semantic matching score between text content, such as "110kV #1 main transformer", and symbol categories, such as transformer, is calculated. .

[0070] Graph attention reasoning: Through the heterogeneous graph attention mechanism, neighbor features are aggregated in the local subgraph to calculate the association probability matrix of each symbol-text pair.

[0071] Optimal association matching: The KM algorithm (Kuhn-Munkres) is used to perform bipartite graph weighted matching on the probability matrix to ensure that each text annotation is accurately associated with its corresponding device symbol.

[0072] The above exemplary embodiments change the previous approach of relying solely on geometric distance for label matching. Instead, the physical orientation of the guide line, relative orientation preference, and electrical semantic matching degree are all encoded into the heterogeneous graph neural network. In areas of substation wiring diagrams with extremely dense data, such as terminal blocks and secondary circuits, the problems of label misalignment and confusion among multiple targets are solved, and the label association accuracy rate is improved from 82% to over 99%.

[0073] In addition to heterogeneous graph neural networks (HGT) and guided line tracing, the following solutions can also be used for cross-modal mapping of text and symbols: Association scheme based on offset vector regression: While performing text recognition, the model additionally predicts an offset vector pointing to the center point of its corresponding symbol. In post-processing, based on the coordinates pointed to by this vector, the text is automatically classified to the nearest target symbol. This is an end-to-end regression approach that does not rely on explicit graph matching algorithms, but establishes spatial mapping relationships through the autonomous learning of neural networks.

[0074] Joint embedding scheme based on multimodal Transformer: The identified symbol image features (Visual Token) and text character semantic features (Text Token) are input into a multimodal Transformer encoder. The association weight between the two is automatically learned through the self-attention mechanism, and the pair with the highest weight is the matching pair.

[0075] For guide line recognition, in addition to the probabilistic Hough transform (LSD), the following methods can also be used: Connection extraction based on semantic segmentation: The guide lines are treated as a special semantic category, and pixel-level masks of the guide lines are extracted using segmentation networks such as U-Net or DeepLab. The association between text box pixels and symbol box pixels is determined by whether they are within the same connected component of the mask. This segmentation scheme is more robust to curved and discontinuous guide lines than the Line Detection Scheme (LSD).

[0076] S104. Based on the location mapping relationship, the symbol set, and the text set, construct the substation electrical topology diagram, and use power system topology constraints to perform consistency verification and correction on the association pairs in the substation electrical topology diagram and the mapping relationship set to obtain substation structured data.

[0077] In some exemplary embodiments, the symbols in the symbol set include device graphic symbols and wire connection line symbols; The construction of the substation electrical topology diagram based on the location mapping relationship, the symbol set, and the text set includes: Based on the coordinate mapping relationship, the coordinates of each symbol in the symbol set are restored; Extract the conductor connection symbols and their connection relationships from the restored symbol set; Based on the extracted conductor connection symbols and their connection relationships, the electrical topology diagram of the substation is constructed.

[0078] In some exemplary embodiments, the step of using power system topology constraints to perform consistency verification on the electrical topology diagram of the substation and the associated pairs in the mapping relationship set includes: Based on the electrical topology diagram of the substation, check whether the number of port connections of the symbol nodes conforms to the preset physical rules. If not, trigger a neighborhood search to complete the connection or mark it as missing. Starting from a node with a known voltage level, the voltage level attribute is propagated along the connected path of the substation electrical topology diagram. If the voltage information in the text associated with the symbol node on the path does not match the propagation result, it is determined to be a logical conflict. For texts with an identification confidence level below a preset threshold, character correction is performed based on the naming rules of the topological path containing the text.

[0079] In practice, this stage involves in-depth error correction that combines topological constraints with business logic: Topology Reconstruction: Constructing the electrical topology of the substation based on the extracted conductor connections. .

[0080] Physical conformance verification includes port connectivity verification and voltage level propagation: Port connectivity verification: For example, if a visual system identifies an isolating switch, but only one end of it is connected to a wire in the topology diagram, the logic engine will automatically trigger a neighborhood search to correct any possible missed detections.

[0081] Voltage level propagation: Starting from the main transformer or the initial outgoing node, the voltage level attribute is propagated along the topology path. If a path is marked as 220kV, but the associated text of the symbol on the path is recognized by OCR as 110kV, the error correction mechanism is activated.

[0082] Among them, voltage level propagation refers to the reasoning process that uses the topological connectivity identified in the drawings to automatically assign the voltage attributes of known nodes to other unmarked or vaguely identified device symbols along the path.

[0083] Business semantic feedback error correction includes fuzzy character re-validation and logical backtracking: Fuzzy character re-verification: If the text recognition confidence is lower than 0.8, such as recognizing 1O1, then based on the device naming rules of the interval where the topology is located, which is usually three digits, the edit distance algorithm is used to automatically correct it to 101.

[0084] Logical backtracking: When a logical conflict is found, such as two circuit breakers being illegally connected in series, backtracking to the above stage, and retrieving a high-resolution compensation slice for the conflict area for secondary fine extraction.

[0085] Through the above exemplary embodiments, the physical constraints of the power system, such as voltage level consistency, port connectivity, and equipment naming conventions, are transformed into posterior probabilistic constraints for the algorithm model. This enables the algorithm to understand professional drawings. Through feedback from topological logic, errors caused by damaged drawings or visual ambiguity can be automatically corrected, achieving a qualitative leap from visual extraction to semantic modeling, and ensuring that the final exported structured data conforms to power engineering specifications.

[0086] In addition to topology path search and Bayesian error correction, the following methods can also be used for error correction in power business logic: A power semantic rewriting scheme based on Large Language Model (LLM): The original string recognized by OCR and its contextual location information are input into a large language model that has been fine-tuned with a power industry corpus. By leveraging the contextual understanding capabilities of the large model and combining it with Prompt constraints such as "Please correct the following garbled characters according to the substation naming conventions", semantic-level text repair is achieved.

[0087] By using the generalization ability of pre-trained models to replace hard-coded business rules, complex text errors can be handled more flexibly.

[0088] Knowledge Graph-based reasoning solutions: The identification results are converted into triples and aligned with existing power equipment standard knowledge graphs. Utilizing the graph's path reasoning capabilities—for example, if A connects to B, then the voltage levels of A and B must be consistent—identification nodes that do not conform to the graph's logic are identified and corrected.

[0089] This invention achieves significant technological advancements in the field of substation drawing digitization by deeply integrating dynamic sampling technology, cross-modal graph association algorithms, and power business topology logic. The specific beneficial effects are reflected in the following aspects: Breakthrough in the accuracy and efficiency bottlenecks of ultra-high resolution drawing processing: Eliminating information loss: Traditional global scaling methods can cause small electrical symbols such as surge arresters and pressure plates to become blurred due to pixel merging. However, the dynamic adaptive slicing technology of this invention can extract local features at the original scale, ensuring the bottom line of recognition accuracy.

[0090] Solving the problem of target truncation: Through the edge target compensation mechanism and the design of compensation slice (AnchorPatch), the problem of recognition failure caused by long device numbers and complex connection lines being cut off at the edge of the slice is completely solved.

[0091] Significantly improves computational efficiency: Compared with blind calculation with a fixed step size across the entire image, this method skips a large number of blank areas without actual information by scanning with information entropy density, which improves the inference speed of tens of thousands of pixel images by more than 40% and greatly reduces hardware resource consumption.

[0092] Accurate annotation and association were achieved in complex and dense scenarios: Overcoming the problem of misattribution: In substation secondary wiring diagrams or cabinet diagrams, equipment symbols are arranged extremely densely. This invention uses heterogeneous graph neural networks (HGT) to replace the traditional simple distance matching. By comprehensively considering geometric distance, relative orientation, visual guide line direction, and semantic matching degree, it achieves a highly reliable one-to-one correspondence between text annotations and power symbols.

[0093] It has robust guide line resolution capabilities: It can automatically identify and track the guide lines of annotations in drawings, solving visual problems that traditional algorithms cannot handle, such as annotation offset and multiple annotations pointing at each other. The correlation accuracy has been improved from less than 85% to more than 98%.

[0094] The AI ​​model is endowed with the ability to correct errors in professional power business logic: Dual verification using both visual and logical methods: This invention does not rely on computer vision in isolation, but introduces power topology constraints. Even when poor drawing quality, such as creases, stains, or blurred handwriting, leads to low AI visual confidence, the system can still use physical connection rules, such as voltage level consistency and port connectivity, to infer and correct erroneous recognition results.

[0095] Completely eliminate professional semantic errors: By establishing a substation equipment naming standard library, the system can automatically correct visually easily confused characters such as the number 0 and the letter O, 1 and I, 8 and B in accordance with business logic, ensuring that the extracted data is logically consistent and absolutely correct at the electrical engineering level.

[0096] This supports a closed loop for fully automated digital modeling of power systems: Directly connect to production systems: The output of this method is no longer fragmented text and coordinates, but structured data such as CIM models containing physical topology, semantic attributes and spatial relationships, which can be directly imported into digital power grid platforms, automatic voltage detection systems or digital twin systems.

[0097] Reduce manual review costs: Traditional identification schemes often require a large amount of manual secondary verification, while this invention, with its extremely high business consistency performance, reduces the need for manual intervention by more than 90%, greatly accelerating the process of digital transformation of substation drawings.

[0098] It has extremely strong scene adaptability and generalization ability: This method is not only applicable to primary system diagrams of substations, but can also be seamlessly transferred to secondary circuit diagrams, terminal block diagrams, and other engineering drawing fields with strong logical constraints, such as petrochemical process diagrams and water conservancy engineering diagrams, and has broad industrial application value.

[0099] It should be noted that the method of this disclosure embodiment can be executed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario, where multiple devices cooperate to complete the task. In such a distributed scenario, one of these devices may execute only one or more steps of the method of this disclosure embodiment, and the multiple devices will interact with each other to complete the method described.

[0100] It should be noted that the above description describes some embodiments of this disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in a different order than that shown in the above embodiments and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0101] Based on the same inventive concept, corresponding to any of the above embodiments, this disclosure also provides a substation drawing annotation multimodal extraction device.

[0102] refer to Figure 2 The device includes: The slicing module 11 is configured to acquire the original substation drawing, perform dynamic adaptive slicing of the original substation drawing based on the information entropy feature density to obtain multiple image slices, and determine the positional mapping relationship between each image slice and the original substation drawing. Extraction module 12 is configured to perform symbol detection and text annotation extraction on the image slice in parallel to obtain a symbol set and a text set. The symbol set includes at least one symbol and its position information, and the text set includes at least one text and its position information. Mapping module 13 is configured to construct a heterogeneous graph with each symbol in the symbol set as a first type of node and each text in the text set as a second type of node, calculate and fuse the multi-criteria edge weights between the first type of node and the second type of node, use a graph neural network to infer the heterogeneous graph based on the multi-criteria edge weights to obtain an association probability matrix, and match the association probability matrix to obtain a mapping relationship set. The construction and correction module 14 is configured to construct a substation electrical topology diagram based on the location mapping relationship, the symbol set, and the text set, and to perform consistency verification and correction on the association pairs in the substation electrical topology diagram and the mapping relationship set using power system topology constraints, thereby obtaining substation structured data.

[0103] In some exemplary embodiments, the slicing module 11 is configured as follows: The original substation drawings are downsampled to generate thumbnails; Calculate the local information entropy of the thumbnail and generate an information entropy density map; Based on the information entropy density map, the original substation drawings are divided into information-dense areas and information-sparse areas; In the information-dense area, sliding slices are performed using a first window size and a first overlap ratio; In the information sparse region, sliding slicing is performed using a second window size and a second overlap rate, where the second window size is larger than the first window size and the second overlap rate is smaller than the first overlap rate. For connected regions located at the edge of the slice, the image slice is generated centered on the break point of the connected region in the original substation drawing.

[0104] In some exemplary embodiments, the mapping module 13 is configured as follows: Calculate the geometric distance and relative angle between the first type of node and the second type of node, and generate geometric constraint weights; Extract the guide lines from the original substation drawings. If a guide line is detected connecting a second type node and a first type node, then generate a visual guide line weight. Calculate the semantic matching score between the text content of the second type of node and the category label of the first type of node, and generate semantic affinity weights; The geometric constraint weights, visual guide line weights, and semantic affinity weights are weighted and fused to obtain the multi-criteria edge weights.

[0105] In some exemplary embodiments, the mapping module 13 is configured as follows: The bipartite graph optimal matching algorithm is used to process the association probability matrix to determine the symbol associated with each text, thus obtaining the mapping relationship set.

[0106] In some exemplary embodiments, the symbols in the symbol set include device graphic symbols and wire connection line symbols, and the construction and correction module 14 is configured to: Based on the coordinate mapping relationship, the coordinates of each symbol in the symbol set are restored; Extract the conductor connection symbols and their connection relationships from the restored symbol set; Based on the extracted conductor connection symbols and their connection relationships, the electrical topology diagram of the substation is constructed.

[0107] In some exemplary embodiments, the construction and correction module 14 is configured to: Based on the electrical topology diagram of the substation, check whether the number of port connections of the symbol nodes conforms to the preset physical rules. If not, trigger a neighborhood search to complete the connection or mark it as missing. Starting from a node with a known voltage level, the voltage level attribute is propagated along the connected path of the substation electrical topology diagram. If the voltage information in the text associated with the symbol node on the path does not match the propagation result, it is determined to be a logical conflict. For texts with an identification confidence level below a preset threshold, character correction is performed based on the naming rules of the topological path containing the text.

[0108] For ease of description, the above apparatus is described in terms of its functions, divided into various modules. Of course, in implementing this disclosure, the functions of each module can be implemented in one or more software and / or hardware.

[0109] The apparatus of the above embodiments is used to implement the corresponding method in any of the foregoing embodiments and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0110] Based on the same inventive concept, corresponding to any of the above embodiments, this disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in any of the above embodiments.

[0111] Figure 3 This embodiment illustrates a more specific hardware structure of an electronic device. The device may include a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, memory 1020, input / output interface 1030, and communication interface 1040 are interconnected internally via the bus 1050.

[0112] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0113] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.

[0114] The input / output interface 1030 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components within the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touchscreens, microphones, various sensors, etc., while output devices may include displays, speakers, vibrators, indicator lights, etc.

[0115] The communication interface 1040 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0116] Bus 1050 includes a pathway for transmitting information between various components of the device, such as processor 1010, memory 1020, input / output interface 1030, and communication interface 1040.

[0117] It should be noted that although the above-described device only shows the processor 1010, memory 1020, input / output interface 1030, communication interface 1040, and bus 1050, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.

[0118] The electronic devices described above are used to implement the corresponding methods in any of the foregoing embodiments and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0119] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this disclosure also provides a non-transitory computer-readable storage medium that stores computer instructions for causing the computer to perform the methods described in any of the above embodiments.

[0120] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0121] The computer instructions stored in the storage medium of the above embodiments are used to cause the computer to perform the methods described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0122] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this disclosure also provides a computer program product, which includes a computer program. In some embodiments, the computer program is executable by one or more processors to cause the processors to perform the method. Corresponding to the execution entity for each step in each embodiment of the method, the processor performing the corresponding step may belong to the corresponding execution entity.

[0123] The computer program products of the above embodiments are used to cause a processor to execute the methods described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0124] Those skilled in the art will recognize that embodiments of this disclosure can be implemented as a system, method, or computer program product. Therefore, this disclosure can be implemented as entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software, generally referred to herein as a "circuit," "module," or "system." Furthermore, in some embodiments, this disclosure can also be implemented as a computer program product contained in one or more computer-readable media, which includes computer-readable program code.

[0125] Any combination of one or more computer-readable media may be used. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example,, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (not exhaustive) of a computer-readable storage medium may include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device.

[0126] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0127] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0128] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0129] It should be understood that each block of a flowchart and / or block diagram, as well as combinations of blocks in a flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine that, when executed by a computer or other programmable data processing device, creates means for implementing the functions / operations specified in the blocks of the flowchart and / or block diagram.

[0130] These computer program instructions may also be stored in a computer-readable medium that enables a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable medium produce a product comprising an instruction apparatus that implements the functions / operations specified in the boxes of a flowchart and / or block diagram.

[0131] Computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, such that the instructions that execute on the computer or other programmable apparatus can provide a process for implementing the functions / operations specified in the boxes of a flowchart and / or block diagram.

[0132] Furthermore, although the operations of the methods of this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all of the operations shown must be performed to achieve the desired result. Rather, the steps depicted in the flowcharts may be executed in a different order. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0133] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0134] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of this application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0135] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of this disclosure (including the claims) is limited to these examples; within the framework of this disclosure, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of the embodiments of this disclosure as described above, which are not provided in detail for the sake of brevity.

[0136] Additionally, to simplify the description and discussion, and to avoid obscuring the embodiments of this disclosure, the provided drawings may or may not show well-known power / ground connections to integrated circuit (IC) chips and other components. Furthermore, the apparatus may be shown in block diagram form to avoid obscuring the embodiments of this disclosure, and this also takes into account the fact that the details of implementation of these block diagram apparatuses are highly dependent on the platform on which the embodiments of this disclosure will be implemented (i.e., these details should be fully understood by those skilled in the art). While specific details (e.g., circuits) have been set forth to describe exemplary embodiments of this disclosure, it will be apparent to those skilled in the art that the embodiments of this disclosure can be implemented without these specific details or with variations thereof. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0137] Although this disclosure has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed.

[0138] This disclosure is intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for extracting multimodal data from substation drawings, comprising: The original substation drawing is obtained, and the original substation drawing is dynamically and adaptively sliced ​​based on the information entropy feature density to obtain multiple image slices. The positional mapping relationship between each image slice and the original substation drawing is determined. Symbol detection and text annotation extraction are performed in parallel on the image slices to obtain a symbol set and a text set. The symbol set includes at least one symbol and its location information, and the text set includes at least one text and its location information. Construct a heterogeneous graph with each symbol in the symbol set as a first type of node and each text in the text set as a second type of node. Calculate and fuse the multi-criteria edge weights between the first type of node and the second type of node. Based on the multi-criteria edge weights, use a graph neural network to infer the heterogeneous graph to obtain an association probability matrix. Match the association probability matrix to obtain a mapping relationship set. Based on the location mapping relationship, the symbol set, and the text set, a substation electrical topology diagram is constructed. Then, the consistency of the substation electrical topology diagram and the associated pairs in the mapping relationship set is checked and corrected using power system topology constraints to obtain substation structured data.

2. The method according to claim 1, wherein the dynamic adaptive slicing of the original substation drawing based on information entropy feature density to obtain multiple image slices includes: The original substation drawings are downsampled to generate thumbnails; Calculate the local information entropy of the thumbnail and generate an information entropy density map; Based on the information entropy density map, the original substation drawings are divided into information-dense areas and information-sparse areas; In the information-dense area, sliding slices are performed using a first window size and a first overlap ratio; In the information sparse region, sliding slicing is performed using a second window size and a second overlap rate, where the second window size is larger than the first window size and the second overlap rate is smaller than the first overlap rate. For connected regions located at the edge of the slice, the image slice is generated centered on the break point of the connected region in the original substation drawing.

3. The method according to claim 1, wherein calculating and fusing the multi-criteria edge weights between the first type of nodes and the second type of nodes includes: Calculate the geometric distance and relative angle between the first type of node and the second type of node, and generate geometric constraint weights; Extract the guide lines from the original substation drawings. If a guide line is detected connecting a second type node and a first type node, then generate a visual guide line weight. Calculate the semantic matching score between the text content of the second type of node and the category label of the first type of node, and generate semantic affinity weights; The geometric constraint weights, visual guide line weights, and semantic affinity weights are weighted and fused to obtain the multi-criteria edge weights.

4. The method according to claim 1, wherein matching the association probability matrix to obtain the mapping relationship set includes: The bipartite graph optimal matching algorithm is used to process the association probability matrix to determine the symbol associated with each text, thus obtaining the mapping relationship set.

5. The method according to claim 1, wherein the symbols in the symbol set include device graphic symbols and wire connection line symbols; The construction of the substation electrical topology diagram based on the location mapping relationship, the symbol set, and the text set includes: Based on the coordinate mapping relationship, the coordinates of each symbol in the symbol set are restored; Extract the conductor connector symbols and their connection relationships from the restored symbol set; Based on the extracted conductor connection symbols and their connection relationships, the electrical topology diagram of the substation is constructed.

6. The method according to claim 1, wherein the consistency verification of the association pairs in the substation electrical topology diagram and the mapping relationship set using power system topology constraints includes: Based on the electrical topology diagram of the substation, check whether the number of port connections of the symbol nodes conforms to the preset physical rules. If not, trigger a neighborhood search to complete the connection or mark it as missing. Starting from a node with a known voltage level, the voltage level attribute is propagated along the connected path of the substation electrical topology diagram. If the voltage information in the text associated with the symbol node on the path does not match the propagation result, it is determined to be a logical conflict. For texts with an identification confidence level below a preset threshold, character correction is performed based on the naming rules of the topological path containing the text.

7. A device for extracting multi-modal annotations from substation drawings, comprising: The slicing module is configured to acquire the original substation drawing, perform dynamic adaptive slicing of the original substation drawing based on the information entropy feature density to obtain multiple image slices, and determine the positional mapping relationship between each image slice and the original substation drawing. The extraction module is configured to perform symbol detection and text annotation extraction on the image slices in parallel to obtain a symbol set and a text set. The symbol set includes at least one symbol and its location information, and the text set includes at least one text and its location information. The mapping module is configured to construct a heterogeneous graph with each symbol in the symbol set as a first type of node and each text in the text set as a second type of node, calculate and fuse the multi-criteria edge weights between the first type of node and the second type of node, use a graph neural network to infer the heterogeneous graph based on the multi-criteria edge weights to obtain an association probability matrix, and match the association probability matrix to obtain a mapping relationship set. The construction and correction module is configured to construct a substation electrical topology diagram based on the location mapping relationship, the symbol set, and the text set, and to perform consistency verification and correction on the association pairs in the substation electrical topology diagram and the mapping relationship set using power system topology constraints, thereby obtaining substation structured data.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the method as claimed in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform the method according to any one of claims 1 to 6.

10. A computer program product, characterized in that, It includes computer program instructions that, when run on a computer, cause the computer to perform the method as described in any one of claims 1 to 6.

Images