Work order intelligent identification analysis method, system, device and medium for power system operation

Abstract

This invention discloses a method, system, equipment, and medium for intelligent identification and analysis of work orders in power system operations, relating to the fields of power information technology and artificial intelligence. The method includes acquiring and preprocessing paper work order images, performing layout analysis and region segmentation, OCR recognition of text regions and structural recognition of table regions, constructing a unified document structure representation, performing multimodal feature fusion and sequence modeling, extracting key business entities, determining work order type and conducting compliance review based on entities and features, and outputting an analysis report. This invention effectively solves the shortcomings of existing technologies, such as fragmented processing, superficial information understanding, and lack of intelligent analysis capabilities, by constructing a technical closed loop from image perception, heterogeneous graph document modeling, multimodal adaptive fusion to intelligent analysis of business rules.

Description

Technical Field

[0001] This invention relates to the fields of power information technology and artificial intelligence, specifically to a method, system, equipment, and medium for intelligent identification and analysis of work orders for power system operations. Background Technology

[0002] With the advancement of smart grid construction, the number of power system operation and maintenance work orders has surged. Traditional manual processing methods are inefficient and cannot meet real-time requirements. In recent years, advancements in optical character recognition and natural language processing technologies have provided support for automated work order processing. Existing technologies have achieved basic document digitization and information extraction through layout analysis and OCR.

[0003] However, existing technologies still have significant limitations when processing complex power work orders: First, text recognition and layout analysis are often separated, making it difficult to accurately capture cell logic in table areas, leading to misalignment or omission of key business information; second, existing methods mostly rely on single text or visual features, lacking an effective multimodal feature fusion mechanism, ignoring the auxiliary role of layout coordinates and design in semantic understanding, and reducing recognition robustness; finally, existing compliance reviews are mostly based on hard rules, lacking the ability to intelligently identify risks based on deep learning, and are unable to perform deep semantic reasoning and logical verification. Summary of the Invention

[0004] In view of the above-mentioned existing problems, the present invention provides a method, system, equipment and medium for intelligent identification and analysis of work orders for power system operations, in order to solve the problems of misalignment or omission of key business information, reduced identification robustness and inability to perform deep semantic reasoning and logical verification in the prior art.

[0005] To address the aforementioned technical issues, a method for intelligent identification and analysis of work orders for power system operations is proposed, including: The system acquires original images of paper work orders using image acquisition equipment, preprocesses these images to obtain standardized digital images, performs layout analysis on these standardized digital images, and automatically detects and segments regions. Optical character recognition (OCR) is performed on text regions to obtain text sequences and corresponding layout coordinates. Structure recognition is performed on segmented table regions, generating structured table data by detecting cells and associating text content, and constructing a unified document structure representation. Based on this unified document structure representation, multimodal deep feature extraction and fusion are performed to generate composite feature representations, which are then automatically identified and extracted using sequence modeling methods to extract predefined categories of key business entities. Based on the extracted key business entity information and composite feature representations, the work order type is automatically determined, and content compliance is automatically reviewed and risk points are automatically identified according to preset business logic, generating and outputting a structured analysis report.

[0006] As a preferred embodiment of the intelligent identification and analysis method for work orders in power system operations described in this invention, the layout analysis includes: taking the preprocessed work order image as input, predicting the category label for each pixel, the training data consisting of labeled historical work order images, and after inference, the model aggregates adjacent pixels of the same category into independent layout regions through post-processing operations, and generates a minimum bounding rectangle bounding box for each region; for table regions, the model distinguishes between the table title region and the table body region.

[0007] As a preferred embodiment of the intelligent identification and analysis method for work orders in power system operations described in this invention, the construction of a unified document structure representation includes abstracting each independent semantic unit in the text sequence and each logical cell in the structured table data as a structure node. Based on the inherent relationships of structured table data determined by page coordinate information, various types of relationship edges connecting structural nodes are established to construct a document model based on heterogeneous graphs. The model uses nodes to represent content units and edges to represent various relationships between units.

[0008] As a preferred embodiment of the intelligent identification and analysis method for work orders in power system operations described in this invention, the step of performing multimodal deep feature extraction and fusion includes: performing graph structure learning on a document model based on heterogeneous graphs, capturing the topological relationships between nodes and updating the features of each node to obtain a node feature representation that includes global structure and local semantics. Visual features are extracted from non-text regions, and a composite feature representation is generated by dynamically balancing and integrating node feature representations and visual features through an adaptive feature fusion strategy.

[0009] As a preferred embodiment of the intelligent identification and analysis method for work orders in power system operations described in this invention, the execution structure identification includes: using a cell detection model based on deep learning to predict the bounding box position of each cell from the table area image; calculating the intersection-union ratio (IU) between the bounding box and all predicted cell bounding boxes for each text unit of optical character recognition; assigning the current text unit to the cell with the largest IU that exceeds a preset threshold; and reorganizing the scattered text content into the cell to obtain a machine-readable structured table. The construction of the heterogeneous graph-based document model includes using words and lexical units obtained from optical character recognition as structural nodes. The initial feature vector of each structural node is obtained by concatenating the semantic embedding vector of the current word, and the center coordinates, width, and height of the bounding box after image size normalization, as expressed by the formula: in, Let be the initial feature vector of the k-th structural node in the graph. This is a vector concatenation operation. For words The semantic vector obtained after word embedding. To extract the a-th word from the work order image using optical character recognition, For words The x-coordinate of the center point of the corresponding bounding box in the image. For words The ordinate of the center point of the corresponding bounding box in the image. For words The width corresponding to the bounding box, For words Corresponding to the height of the bounding box, The total width of the preprocessed, normalized digital image. The total height of the preprocessed, normalized digital image. These are auxiliary features.

[0010] As a preferred embodiment of the intelligent identification and analysis method for work orders in power system operations described in this invention, the graph structure learning includes: using a graph attention network to calculate the attention weights of the edges relating to all neighboring nodes for each structural node in the model through a multi-head attention mechanism; and weighting and summing the features of neighboring nodes based on the attention weights; and updating the features of the current structural node by combining nonlinear transformations. After iterative updates, the final feature representation of each structural node is obtained, that is, the node feature representation containing global structure and local semantics. The formula for calculating attention weights is expressed as follows: in, In the l-th layer of the graph neural network, Let be the attention weight of the target node r to its neighbor node u. To perform Softmax normalization along the dimension u of neighboring nodes, It is a non-linear activation function. For learnable attention vectors, Let be the learnable linear transformation weight matrix of the l-th layer. For the current feature representation of the target node r at layer l, For the current feature representation of neighbor node u at layer l, Index the target node. Indexing neighboring nodes, This is a vector concatenation operation. Let be the feature vector of the edge connecting node r and node u; The iterative update formula is expressed as: in, This is the new feature representation of node r at the (l+1)th layer obtained after the calculation at the l-th layer. Let r be the set of neighboring nodes in the graph; The adaptive feature fusion strategy includes a gated fusion mechanism that receives concatenated node feature representations and visual features through a learnable neural network layer and outputs normalized fusion weights; the composite feature representation includes the sum of the node feature representation and the visual feature after multiplying them by their respective fusion weights.

[0011] As a preferred embodiment of the intelligent identification and analysis method for work orders in power system operations described in this invention, the sequence modeling method includes: using a model combining a bidirectional long short-term memory network and a conditional random field for sequence modeling; inputting composite feature representations as sequences into the bidirectional long short-term memory network; obtaining the hidden state of context information at each location; inputting the hidden state sequence into the conditional random field model; decoding the optimal entity label sequence by modeling the transition probabilities between labels; and extracting entities such as work ticket number, operation ticket number, equipment number, work location, work content, safety measures, hazard points, control measures, personnel, work group, start time, and end time. The automatic type determination of work orders includes: using a text convolutional neural network to input the pooling result of the composite feature representation and the global text feature vector of the work order into a classifier; using a fully connected layer and a Softmax function to calculate the probability distribution of the current work order belonging to work ticket, operation ticket, work instruction card and the remaining categories; and taking the category corresponding to the highest probability as the work order type determination result. The formula for determining the output work order type is as follows: in, The probability distribution of work order types predicted by the model. For the Softmax function, This is the weight matrix of the classification layer. This represents the composite feature representation obtained after multimodal gating fusion. This is the bias vector for the classification layer. This is a work order type label.

[0012] The beneficial effects of this preferred technical solution are as follows: through image preprocessing and layout analysis, structured modeling based on heterogeneous graphs, and multimodal deep feature fusion, accurate structured parsing and robust feature extraction of complex work orders are achieved. Combined with sequence modeling and business logic verification, automatic extraction of key business information, accurate determination of work order types, and intelligent identification of compliance risks are achieved, thereby improving the automation level and management security of power system operations.

[0013] As a preferred embodiment of the intelligent identification and analysis system for power system operations described in this invention, it is characterized by including a multimodal perception and structuring module, a graph semantic understanding and entity extraction module, an adaptive decision-making and classification module, and a rule-driven analysis and output module.

[0014] The multimodal perception and structuring module is used to preprocess the collected work order images, automatically segment the text, table and signature functional areas through layout analysis technology, extract the text sequence and position by optical character recognition in the text area, and perform cell detection and content association in the table area to generate structured table data.

[0015] The graph semantic understanding and entity extraction module is used to construct a heterogeneous graph based on the text units and their positions and table structure relationships output by OCR, use a graph neural network to encode and learn the current graph, capture the associations and semantics between document elements, generate feature representations, and extract key entities of predefined categories from the features through a sequence labeling model.

[0016] The adaptive decision and classification module receives semantic features and visual features, dynamically calculates the weight of each modality feature through a gating fusion mechanism, and fuses them to obtain a composite feature representation that reflects the characteristics of the current work order content. It then uses a classifier to determine the specific type of the work order and outputs the classification confidence score.

[0017] The rule-driven analysis and output module is used to perform logical verification and reasoning on the extracted key entities, classification results and structured tables based on the business rule knowledge base, automatically perform compliance checks, risk factor identification and work order correlation tracing analysis, and integrate all intermediate results and analysis conclusions to generate a standardized analysis report.

[0018] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of a method for intelligent identification and analysis of work orders for power system operations.

[0019] A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of a method for intelligent identification and analysis of work orders for power system operations.

[0020] The beneficial effects of this invention are as follows: This invention achieves standardization and fine-grained region segmentation of work order images through image preprocessing and layout analysis, laying the foundation for subsequent processing; it combines optical character recognition with deep learning-based cell detection and association to construct a unified document structure representation based on heterogeneous graphs, preserving the physical layout and internal logic of the document and solving the problems of misalignment and omission of complex table information; it performs multimodal deep feature extraction through graph attention networks and adaptive gating fusion strategies, dynamically integrating semantic information and visual context, endowing the model with the ability to capture long-distance dependencies and improving the robustness of feature representation; and it utilizes a model combining bidirectional long short-term memory networks and conditional random fields for sequence modeling, extracting key business entities to ensure the accuracy and completeness of data extraction; based on composite feature representation and entity information, it performs work order type determination and automatic content compliance review, realizing the automation of work processes and intelligent risk identification, effectively ensuring the safety and management efficiency of power system operations. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 The above is a flowchart of an intelligent identification and analysis method for work orders in power system operations, provided as an embodiment of the present invention.

[0023] Figure 2 This is a schematic diagram illustrating the feature extraction and aggregation of a work order intelligent identification and analysis method for power system operations provided in an embodiment of the present invention.

[0024] Figure 3 This is a schematic diagram of work order text classification for an intelligent identification and analysis method for work orders in power system operations, provided in an embodiment of the present invention.

[0025] Figure 4 The flowchart of a system scheme for an intelligent identification and analysis system for power system operations is provided in one embodiment of the present invention. Detailed Implementation

[0026] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0027] Example 1, referring to Figure 1 As an embodiment of the present invention, a method for intelligent identification and analysis of work orders for power system operations is provided, including: S100: Acquires the original image of the paper work order through the image acquisition device, preprocesses the original image to obtain a standardized digital image, performs layout analysis on the standardized digital image, and automatically detects and segments regions.

[0028] S200: Performs optical character recognition on the text area to obtain the text sequence and the corresponding page coordinate information. Performs structure recognition on the segmented table area. By detecting cells and associating text content, it generates structured table data and constructs a unified document structure representation.

[0029] S300: Based on a unified document structure representation, it performs multimodal deep feature extraction and fusion to generate composite feature representations, and automatically identifies and extracts predefined categories of key business entity information through sequence modeling methods.

[0030] S400: Based on the extracted key business entity information and composite feature representation, the system automatically determines the type of work orders, automatically reviews the content compliance and identifies risk points according to preset business logic, and generates and outputs a structured analysis report.

[0031] It should be noted that this invention achieves highly robust and precise segmentation of complex layouts through pixel-level semantic segmentation, laying a precise perceptual foundation for subsequent processing. It also constructs a heterogeneous graph document model that integrates text, tables, and layout relationships, unifying the multi-dimensional relational representation of documents. It utilizes graph attention networks to achieve deep semantic understanding of relationships and adopts an adaptive gating mechanism to dynamically fuse multimodal features and extract composite feature representations. Through sequence modeling, it can extract scattered key business entities with high precision and automatically determine and review work order types, content compliance, and risk points in conjunction with a preset business rule base.

[0032] Example 2, refer to Figure 1 - Figure 3 This is a second embodiment of the present invention, which provides a method for intelligent identification and analysis of work orders for power system operations, including: In step S100, the preprocessing includes steps S101 to S103: S101: For paper work orders printed from the Power Production Management Information System (PMIS) platform, images are acquired using an A3 / A4 high-speed scanner or a mobile terminal camera to obtain the original digital image I, and the original digital image I is preprocessed to improve the recognition quality.

[0033] S102: Denoising includes using a Gaussian filtering algorithm to reduce image scanning noise and salt-and-pepper noise.

[0034] S103: The image tilt angle is detected by the Hough transform algorithm and rotation correction is performed to keep the text lines horizontal.

[0035] S104: Scale or interpolate the image to a standard resolution (300 DPI) to obtain a preprocessed, normalized image.

[0036] Furthermore, in this embodiment of the application, step S100, the page layout analysis includes steps S111 to S114: S111: Page layout analysis is performed using a semantic segmentation model based on deep learning.

[0037] S112: The model takes the preprocessed work order image as input and predicts the category label for each pixel: body text, table, image / signature, background.

[0038] S113: The data consists of a large number of labeled historical work order images. After model inference, post-processing operations (connected component analysis) are used to aggregate adjacent pixels of the same type into independent layout regions, and a minimum bounding box is generated for each region.

[0039] S114: For table areas, the model can distinguish between the table title area and the table body area; for signatures, it can be separated as an independent non-text area, which is convenient for subsequent separate verification of authenticity or standardization.

[0040] In an optional implementation, step S100, the layout analysis further includes marking the binarized image with connected components to obtain all connected components, and calculating the features of each connected component, such as aspect ratio, pixel density, and relative position with adjacent components. The region is then classified according to a preset rule set: a region with an aspect ratio close to 1 and high pixel density is a signature; multiple rectangular blocks of similar width and vertical alignment constitute a table; and a large area composed of multiple small text line connected components is the main text.

[0041] In another optional implementation, in step S100, the layout analysis may further include projecting pixels of the image in the vertical and horizontal directions, dividing rows and columns by analyzing the peaks and troughs of the projection histogram, determining continuous text rows as text areas, and identifying areas with periodic troughs in the projection image as tables. Image and signature areas are usually represented as independent blocks isolated from text areas on the projection image.

[0042] In this embodiment of the application, step S200, which involves performing optical character recognition on the text region, includes steps S201 to S203: S201: For each text region detected in the image, use an optical character recognition engine (based on a Transformer OCR model) to recognize it, output the character or word sequence in the current region, and simultaneously record the specific bounding box position information of each word in the image.

[0043] S202: Enter the table structure recognition stage, and use the YOLO object detection model based on deep learning to predict the set of bounding boxes of all cells within the table area.

[0044] S203: Perform content association, assign the OCR-recognized words to the corresponding cells according to the degree of positional overlap, and select the cells with the highest matching degree for each word by calculating the intersection-union ratio of the bounding box of each word and the bounding boxes of each cell. When the intersection-union ratio of the two reaches the preset threshold of 0.25, the current word is assigned to the corresponding cell, that is, the table in the image is converted into structured data with row and column logical relationship and each cell is filled with the correct text content; The formula for calculating the intersection-union ratio is expressed as: in, The index of the cell finally assigned to the nth word. The optimal cell index is calculated for the nth word. For all possible cell indexes, Let be the intersection-union ratio function. The bounding box of the nth word. This is the bounding box of the m-th cell.

[0045] In an optional implementation, in step S200, performing optical character recognition on the text region further includes binarizing the table region image, removing noise using morphological operations, determining the row and column positions of the table through horizontal and vertical projection contour analysis, dividing the cell grid, and performing independent optical character recognition on the region within each grid cell. When there are multiple lines of text within a cell, row segmentation is performed in conjunction with connectivity component analysis.

[0046] In another optional implementation, in step S200, performing optical character recognition on the text area may further include matching a preset table template based on the work order classification results. The template predefines the coordinate areas and field names of key cells. During recognition, the system directly sends the image slices within the corresponding coordinate areas into the optical character recognition engine and fills the results into the predefined fields. For dynamically changing rows, rule-based judgment is used to split multiple records by recognizing the numbers or bullet points under the safety measures column.

[0047] Furthermore, in step S200, the construction of the unified document structure representation includes steps S211-S212: S211: Nodes in the graph are built using lexical units as the basic unit. Each lexical unit corresponds to a node in the graph. To fully represent the node attributes, the feature vector of the node is composed of semantic information and spatial location features. Specifically, it includes pre-trained word embeddings, the center coordinates and width and height information of the bounding box after normalization, and auxiliary features such as font or confidence. Node features are represented as follows: in, Let be the initial feature vector of the k-th structural node in the graph. This is a vector concatenation operation. For words The semantic vector obtained after word embedding. To extract the a-th word from the work order image using optical character recognition, For words The x-coordinate of the center point of the corresponding bounding box in the image. For words The ordinate of the center point of the corresponding bounding box in the image. For words The width corresponding to the bounding box, For words Corresponding to the height of the bounding box, The total width of the preprocessed, normalized digital image. The total height of the preprocessed, normalized digital image. These are optional auxiliary features.

[0048] S212: Based on the established node features, three types of edges are constructed to connect nodes to form complex topological relationships, including: adjacent edges connecting adjacent words in the same row according to the reading order; table edges connecting words in the same row or column; and syntactic dependency edges constructed based on dependency parsing, to construct a complete graph structure that integrates text semantics, spatial layout and grammatical logic.

[0049] Furthermore, in this embodiment of the application, in step S212, the construction of the three types of edges includes steps A1 to A3: A1: Adjacent Edges: Based on the spatial coordinates of nodes, connect the nodes that are closest in horizontal distance within the same text line, conform to the reading order (from left to right), to form a chain structure and capture the semantics of continuous sentences.

[0050] A2: Table Edges: Connect all nodes within the same table cell, and also connect nodes in different cells of the same row (or column) that have the same semantic role, strengthening the row and column relationships and attribute alignment within the table.

[0051] A3: Syntactic Dependency Edges: Perform dependency parsing in natural language processing on paragraphs composed of consecutive lines of text. Based on the analyzed grammatical relations, establish directed edges between corresponding word nodes in the graph to capture deep semantic logic.

[0052] In an optional implementation, in step S212, the construction of the three types of edges further includes: spatial proximity edges: for each node, an undirected edge is established between it and the K nodes closest to it in Euclidean distance, while capturing spatial associations within and across rows; attribute consistency edges: based on simple rules, a connection is established when two nodes share the same format features or are located in a predefined key field area.

[0053] In another optional implementation, in step S212, the construction of the three types of edges may also include using a depth-first traversal strategy of "from top to bottom and from left to right" to connect nodes. Starting from the first node in the upper left corner of the page, the nodes with the closest vertical distance in the same column are connected first. When the column ends, the connection moves to the top node of the next column to the right and continues to connect. In obvious table areas, a grid connection method is used.

[0054] In step S300, the multimodal deep feature extraction and fusion includes steps S301 to S304: S301: Utilizes a graph attention network (GAT) as the core architecture, and performs deep feature transfer by stacking 2 to 4 network layers.

[0055] S302: In each layer update, the node updates its own state by aggregating information from neighboring nodes. A multi-head attention mechanism is used, with 4 to 8 attention heads set up to enhance the feature capture capability. The formula for calculating attention weights is expressed as follows: in, In the l-th layer of the graph neural network, Let be the attention weight of the target node r to its neighbor node u. To perform Softmax normalization along the dimension u of neighboring nodes, It is a non-linear activation function. For learnable attention vectors, Let be the learnable linear transformation weight matrix of the l-th layer. For the current feature representation of the target node r at layer l, For the current feature representation of neighbor node u at layer l, Index the target node. Indexing neighboring nodes, This is a vector concatenation operation. Let be the feature vector of the edge connecting node r and node u; The iterative update formula is expressed as: in, This is the new feature representation of node r at the (l+1)th layer obtained after the calculation at the l-th layer. Let be the set of neighboring nodes of node r in the graph.

[0056] S303: When calculating attention weights, the feature mappings of the target node and its neighboring nodes are combined, and vector concatenation operations and edge feature information are introduced, and nonlinear transformation is performed in conjunction with the LeakyReLU activation function.

[0057] S304: After multiple iterations, each node outputs a feature vector containing rich contextual structure and spatial location information.

[0058] Furthermore, in step S300, the extraction of predefined categories of key business entity information includes steps S311~S313: S311: Input the feature vector output by the graph neural network into the Bidirectional Long Short-Term Memory Network-Conditional Random Field (BiLSTM-CRF) model, extract the key entity set, encode the input features through the BiLSTM layer, and output the hidden state containing contextual information, as expressed by the formula: in, To represent the hidden state output at the nth position (corresponding to the nth word) after encoding by a Bidirectional Long Short-Term Memory (BiLSTM) network, It is a bidirectional long short-term memory network. The feature vector output by the nth term in a graph neural network (GNN).

[0059] S312: The CRF layer uses the hidden state to calculate the emission score and combines it with the tag transition matrix to perform globally optimal decoding of the tag sequence, ensuring the logical rationality of the prediction results. The formula is expressed as: in, To predict the probability of entity label sequence Z given the BiLSTM hidden state sequence O at all locations, It is an exponential function. The label transition score in a Conditional Random Field (CRF) model. The label transition matrix is ​​a learnable matrix. For the entity label predicted at position n−1, For the entity label predicted at position n, For the launch fraction, This is the position index in the sequence.

[0060] S313: Identify and extract key entity information such as work ticket number, operation ticket number, equipment number, work location, work content, safety measures, hazard points and control measures, personnel / teams, and start and end times through the model.

[0061] In step S400, the automatic type determination of the work order includes steps S401~S402: S401: Perform multimodal gating fusion, concatenate text features, table features and image features, calculate the attention weights of each modality through learnable weight parameters, and perform weighted summation of the three features according to the weights to generate fusion features that can comprehensively reflect multi-source information.

[0062] S402: Utilize the current fusion features or the full text of the work order, input the convolutional neural network or Transformer classifier for deep analysis, and output the type label of the work order and the corresponding confidence score. The work order type is specifically divided into work ticket, operation ticket, work instruction card or remaining category. The formula for determining the output work order type is as follows: in, The probability distribution of work order types predicted by the model. For the Softmax function, This is the weight matrix of the classification layer. This represents the composite feature representation obtained after multimodal gating fusion. This is the bias vector for the classification layer. This is a work order type label.

[0063] Furthermore, in this embodiment of the application, in step S401, the execution of multimodal gating fusion includes steps B1 to B3: B1: Concatenate the node feature representations of the text / table (from a graph neural network) with the visual features extracted from the signature / illustration area (from a convolutional neural network) into vectors.

[0064] B2: The concatenated vector is input into a fully connected neural network layer. The current layer learns to generate a set of normalized scalar weights (i.e., gating weights) related to the input content. The formula for calculating the gating weights is expressed as: in, The calculated gate weight vector, For the learnable weight matrix of the gated weight generation layer, Text feature representation, This is a representation of the table's structural features. For image / visual feature representation, The learnable bias vector for the gating weight generation layer. This is a vector concatenation operation.

[0065] B3: The three weights correspond to the importance of text features, table structure features, and visual features, respectively. The fused feature is obtained by multiplying each of the three features by its corresponding gating weight and then summing the results. The fused feature is represented as: in, The final generated fusion feature representation, , and For the calculated gating weights, For element-wise addition of vectors, This is a scalar-vector multiplication.

[0066] In an optional implementation, in step S401, the multimodal gating fusion further includes a static fusion strategy based on weighted average, wherein fixed empirical weight coefficients are preset for text features, table features and visual features respectively. The coefficients are determined based on domain knowledge before model training (text feature weight 0.6, table feature weight 0.3, visual feature weight 0.1). During fusion, each feature vector is directly multiplied by its corresponding fixed weight and then added together.

[0067] In another optional implementation, in step S401, performing multimodal gating fusion may further include using visual features as a query, text / table features as keys and values, calculating the attention distribution of visual features on text / table features, and performing a weighted summation of text / table features based on the current attention distribution to obtain a text / table context vector modulated by visual information, and concatenating or adding the context vector with the original visual features.

[0068] Furthermore, in step S400, the generation and output of the structured analysis report includes steps S411-S414: S411: Utilize the obtained entity set, work order classification results, and preset business rule base and knowledge base to carry out compliance verification, risk identification, correlation tracing and statistical analysis tasks, and output structured work orders and detailed analysis reports.

[0069] S412: Compliance verification is achieved by checking the completeness of required fields, the standardization of format, and the validity of signatures and seals; risk identification is performed by identifying hazards, missing safety measures, and high-risk operations; the correlation and tracing of work sequences are analyzed by linking work order numbers with historical records; and statistical analysis is performed on the number of work orders generated, their type distribution, and compliance rate indicators.

[0070] S413: During the training phase, the model uses historical samples of power work orders to construct a training set containing images, type labels, and entity sequence labels, and divides it into training set, validation set, and test set in an 8:1:1 ratio. Multiple rounds of training are performed using the AdamW optimizer and preset hyperparameters. The AdamW optimizer learning rate is set to 1×10. -4 The batch size was set to 16, the training rounds were 30, and the cross-entropy loss function was used for model training.

[0071] S414: During the inference phase, the output includes structured fields extracted from the work order, reconstructed tabular data, work order types with confidence levels, and comprehensive analysis results including compliance conclusions, risk point lists, related information, and statistical reports.

[0072] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

[0073] Example 3, referring to Figure 4 This is the third embodiment of the present invention, which provides an intelligent identification and analysis system for work orders in power system operations, including a multimodal perception and structuring module, a graph semantic understanding and entity extraction module, an adaptive decision-making and classification module, and a rule-driven analysis and output module.

[0074] The multimodal perception and structuring module is used to preprocess the collected work order images, automatically segment the text, table and signature functional areas through layout analysis technology, extract the text sequence and position by optical character recognition in the text area, and perform cell detection and content association in the table area to generate structured table data.

[0075] The graph semantic understanding and entity extraction module is used to construct a heterogeneous graph based on the text units and their positions and table structure relationships output by OCR, use a graph neural network to encode and learn the current graph, capture the associations and semantics between document elements, generate feature representations, and extract key entities of predefined categories from the features through a sequence labeling model.

[0076] The adaptive decision and classification module receives semantic features and visual features, dynamically calculates the weight of each modality feature through a gating fusion mechanism, and fuses them to obtain a composite feature representation that reflects the characteristics of the current work order content. It then uses a classifier to determine the specific type of the work order and outputs the classification confidence score.

[0077] The rule-driven analysis and output module is used to perform logical verification and reasoning on the extracted key entities, classification results and structured tables based on the business rule knowledge base, automatically perform compliance checks, risk factor identification and work order correlation tracing analysis, and integrate all intermediate results and analysis conclusions to generate a standardized analysis report.

[0078] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

[0079] Example 4, the fourth embodiment of the present invention, differs from the previous three embodiments in that: If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0080] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-including system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0081] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0082] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

Claims

1. A method for intelligent identification and analysis of work orders for power system operations, characterized by: include, The original image of the paper work order is acquired by the image acquisition device, and the original image is preprocessed to obtain a standardized digital image. The standardized digital image is then analyzed for layout, and the region is automatically detected and segmented. Optical character recognition is performed on the text area to obtain the text sequence and the corresponding page coordinate information. Structure recognition is performed on the segmented table area. By detecting cells and associating text content, structured table data is generated and a unified document structure representation is constructed. Based on a unified document structure representation, multimodal deep feature extraction and fusion are performed to generate composite feature representations. Then, the key business entity information of predefined categories is automatically identified and extracted through sequence modeling methods. Based on the extracted key business entity information and composite feature representation, the system automatically determines the type of work orders, and automatically reviews the content compliance and identifies risk points according to preset business logic, generating and outputting structured analysis reports.

2. The intelligent identification and analysis method for work orders in power system operations as described in claim 1, characterized in that: The layout analysis includes using the preprocessed work order image as input, predicting the category label for each pixel, and training data consisting of labeled historical work order images. After inference, the model aggregates adjacent pixels of the same category into independent layout regions through post-processing operations, and generates a minimum bounding box for each region. For table regions, the model distinguishes between the table title region and the table body region.

3. The intelligent identification and analysis method for work orders in power system operations as described in claim 2, characterized in that: The construction of a unified document structure representation includes, Each independent semantic unit in the text sequence and each logical cell in the structured table data are abstracted into a structure node; Based on the inherent relationships of structured table data determined by page coordinate information, various types of relationship edges connecting structural nodes are established to construct a document model based on heterogeneous graphs. The model uses nodes to represent content units and edges to represent various relationships between units.

4. The intelligent identification and analysis method for work orders in power system operations as described in claim 3, characterized in that: The multimodal deep feature extraction and fusion process includes performing graph structure learning on a heterogeneous graph-based document model, capturing the topological relationships between nodes and updating the features of each node to obtain node feature representations that include global structure and local semantics. Visual features are extracted from non-text regions, and a composite feature representation is generated by dynamically balancing and integrating node feature representations and visual features through an adaptive feature fusion strategy.

5. The intelligent identification and analysis method for work orders in power system operations as described in claim 4, characterized in that: The execution structure recognition includes using a deep learning-based cell detection model to predict the bounding box position of each cell from the table area image; for each text unit of optical character recognition, calculating the intersection-union ratio (IU) between the bounding box and all predicted cell bounding boxes; assigning the current text unit to the cell with the largest IU that exceeds a preset threshold; and reorganizing the scattered text content into the cell to obtain a machine-readable structured table. The construction of the heterogeneous graph-based document model includes using words and lexical units obtained from optical character recognition as structural nodes. The initial feature vector of each structural node is obtained by concatenating the semantic embedding vector of the current word, and the center coordinates, width, and height of the bounding box after image size normalization, as expressed by the formula: in, Let be the initial feature vector of the k-th structural node in the graph. This is a vector concatenation operation. For words The semantic vector obtained after word embedding. To extract the a-th word from the work order image using optical character recognition, For words The x-coordinate of the center point of the corresponding bounding box in the image. For words The ordinate of the center point of the corresponding bounding box in the image. For words The width corresponding to the bounding box, For words Corresponding to the height of the bounding box, The total width of the preprocessed, normalized digital image. The total height of the preprocessed, normalized digital image. These are auxiliary features.

6. The intelligent identification and analysis method for work orders in power system operations as described in claim 5, characterized in that: The graph structure learning process includes using a graph attention network to calculate the attention weights of the edges relating to all neighboring nodes for each structural node in the model through a multi-head attention mechanism. Based on the attention weights, the features of the neighboring nodes are weighted and summed, and the features of the current structural node are updated by combining nonlinear transformations. After iterative updates, the final feature representation of each structural node is obtained, which is a node feature representation that contains global structure and local semantics. The formula for calculating attention weights is expressed as follows: in, In the l-th layer of the graph neural network, Let be the attention weight of the target node r to its neighbor node u. To perform Softmax normalization along the dimension u of neighboring nodes, It is a non-linear activation function. For learnable attention vectors, Let be the learnable linear transformation weight matrix of the l-th layer. For the current feature representation of the target node r at layer l, For the current feature representation of neighbor node u at layer l, Index the target node. Indexing neighboring nodes, This is a vector concatenation operation. Let be the feature vector of the edge connecting node r and node u; The iterative update formula is expressed as: in, This is the new feature representation of node r at the (l+1)th layer obtained after the calculation at the l-th layer. Let r be the set of neighboring nodes in the graph; The adaptive feature fusion strategy includes a gated fusion mechanism that receives concatenated node feature representations and visual features through a learnable neural network layer and outputs normalized fusion weights; the composite feature representation includes the sum of the node feature representation and the visual feature after multiplying them by their respective fusion weights.

7. The intelligent identification and analysis method for work orders in power system operations as described in claim 6, characterized in that: The sequence modeling method includes using a model combining a bidirectional long short-term memory network and a conditional random field for sequence modeling. The composite feature representation is used as the sequence input to the bidirectional long short-term memory network to obtain the hidden state of the context information at each position. The hidden state sequence is then input into the conditional random field model. By modeling the transition probabilities between labels, the optimal entity label sequence is decoded to extract entities such as work ticket number, operation ticket number, equipment number, work location, work content, safety measures, hazard points, control measures, personnel, work group, start time, and end time. The automatic type determination of work orders includes: using a text convolutional neural network to input the pooling result of the composite feature representation and the global text feature vector of the work order into a classifier; using a fully connected layer and a Softmax function to calculate the probability distribution of the current work order belonging to work ticket, operation ticket, work instruction card and the remaining categories; and taking the category corresponding to the highest probability as the work order type determination result. The formula for determining the output work order type is as follows: in, The probability distribution of work order types predicted by the model. For the Softmax function, This is the weight matrix of the classification layer. This represents the composite feature representation obtained after multimodal gating fusion. This is the bias vector for the classification layer. This is a work order type label.

8. A work order intelligent identification and analysis system for power system operations, employing the work order intelligent identification and analysis method for power system operations as described in any one of claims 1 to 7, characterized in that, It includes a multimodal perception and structuring module, a graph semantic understanding and entity extraction module, an adaptive decision and classification module, and a rule-driven analysis and output module; The multimodal perception and structuring module is used to preprocess the collected work order images, automatically segment the text, table and signature functional areas through layout analysis technology, extract the text sequence and position by optical character recognition in the text area, and perform cell detection and content association in the table area to generate structured table data. The graph semantic understanding and entity extraction module is used to construct a heterogeneous graph based on the text units and positions and table structure relationships output by OCR, use graph neural networks to encode and learn the current graph, capture the associations and semantics between document elements, generate feature representations, and extract key entities of predefined categories from the features through a sequence labeling model. The adaptive decision and classification module is used to receive semantic features and visual features, dynamically calculate the weight of each modality feature through a gating fusion mechanism and fuse them to obtain a composite feature representation that reflects the characteristics of the current work order content, and use a classifier to determine the specific type of the work order and output the classification confidence score. The rule-driven analysis and output module is used to perform logical verification and reasoning on the extracted key entities, classification results and structured tables based on the business rule knowledge base, automatically perform compliance checks, risk factor identification and work order correlation tracing analysis, and integrate all intermediate results and analysis conclusions to generate a standardized analysis report.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the work order intelligent identification and analysis method for power system operations as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the work order intelligent identification and analysis method for power system operations as described in any one of claims 1 to 7.

Images