Method, device and medium for generating a multi-modal maintenance plan for a power plant

Abstract

The application relates to a multimodal maintenance scheme generation method, device, equipment and medium of a power device. The method comprises the following steps: acquiring multimodal sensing data and device attribute information of the power device; constructing a structured state object of the power device according to the multimodal sensing data and the device attribute information; determining a target state node corresponding to the power device from a pre-constructed knowledge graph according to the multimodal sensing data and the structured state object; extracting a target processing path of the power device from the knowledge graph according to the target state node and the structured state object; wherein the target processing path at least comprises a fault node connected with the target state node and an operation node sequence corresponding to the fault node; respectively mapping the operation node sequence into a plurality of structured task units; and performing sorting and format conversion on each structured task unit to generate a maintenance scheme of the power device. The method provided by the application can improve the rationality and execution safety of the generated maintenance scheme.

Description

Technical Field

[0001] This application relates to the field of power equipment operation and maintenance technology, and in particular to a method, apparatus, computer equipment, computer-readable storage medium and computer program product for generating multimodal maintenance schemes for power equipment. Background Technology

[0002] With the continuous expansion of my country's power system and the continuous improvement of its intelligence level, the operating environment of power equipment is becoming increasingly complex, and the demand for equipment condition monitoring and maintenance decision-making is growing rapidly.

[0003] Currently, maintenance plan generation still relies on manual experience or semi-automatic template matching, lacking a complete logical chain from state awareness and knowledge matching to task output. This makes it difficult to ensure the safety, logical consistency, and reproducibility of the plan. Especially in complex equipment scenarios, maintenance tasks often involve multi-step, multi-job, and multi-tool collaboration. Traditional generation systems cannot automatically guarantee the operation sequence and safety constraints, resulting in potential operational conflicts and process errors. Summary of the Invention

[0004] Therefore, it is necessary to provide a method, apparatus, computer equipment, and computer-readable storage medium for generating multimodal maintenance schemes for power equipment that can improve operational rationality and safety, in order to address the above-mentioned technical problems.

[0005] Firstly, this application provides a method for generating multimodal maintenance schemes for power equipment, including:

[0006] Acquire multimodal sensing data and equipment attribute information of power equipment;

[0007] Based on the multimodal sensing data and the device attribute information, a structured state object of the power equipment is constructed;

[0008] Based on the multimodal sensing data and the structured state object, the target state node corresponding to the power equipment is determined from the pre-constructed knowledge graph;

[0009] Based on the target state node and the structured state object, the target processing path of the power equipment is extracted from the knowledge graph; wherein, the target processing path includes at least a fault node connected to the target state node and a sequence of operation nodes corresponding to the fault node;

[0010] The operation node sequence is mapped into multiple structured task units respectively;

[0011] The structured task units are sorted and their formats are converted to generate a maintenance plan for the power equipment.

[0012] In one embodiment, the structured state object includes state labels and component locations;

[0013] The step of extracting the target processing path of the power equipment from the knowledge graph based on the target state node and the structured state object includes:

[0014] Starting from the target state node, search the knowledge graph for all reachable processing paths that satisfy the depth constraints.

[0015] The path score of the reachable processing path is determined based on the status label, the component location, the sequence of operation nodes in the reachable processing path, and the corresponding operation sparsity.

[0016] Based on the path scores, the reachable processing paths are sorted, and the reachable processing path with the highest path score is determined as the target processing path.

[0017] In one embodiment, determining the path score of the reachable processing path based on the status label, the component location, the sequence of operation nodes in the reachable processing path, and the corresponding operation sparsity includes:

[0018] For each operation node in the operation node sequence, obtain the structural adaptation function value between the operation node, the status label, and the component position, as well as the operation sparsity regularization value of the operation node. Perform a weighted summation on the structural mismatch function value and the operation sparsity regularization value to determine the node contribution value of the operation node.

[0019] The path score of the reachable processing path is determined based on the average node contribution value of each operation node.

[0020] In one embodiment, the structured state object includes a device number;

[0021] The step of determining the target state node corresponding to the power equipment from a pre-built knowledge graph based on the multimodal sensing data and the structured state object includes:

[0022] Based on the multimodal sensing data, a unified state vector is constructed;

[0023] Based on the device number, a target subgraph containing the power device is determined from the knowledge graph;

[0024] The unified state vector is matched with the embedded representation vectors of each state node in the target subgraph to determine the target state node.

[0025] In one embodiment, the multimodal sensing data includes thermal imaging image data, structured operation and maintenance text data, and sensor sampling data;

[0026] The step of constructing a unified state vector based on the multimodal sensing data includes:

[0027] The thermal imaging image data is processed by an image encoder to extract features and generate an image feature vector.

[0028] The structured operation and maintenance text data is processed by a semantic encoder to extract features and generate a semantic feature vector.

[0029] The sensor sampled data is encoded using a structured numerical encoder to generate a structured numerical feature vector.

[0030] The image feature vector, the semantic feature vector, and the structured numerical feature vector are weighted and summed to generate the unified state vector.

[0031] In one embodiment, the step of sorting and format-converting each of the structured task units to generate a maintenance plan for the power equipment includes:

[0032] A task dependency graph is constructed based on each of the structured task units, and a topological sorting process is performed on each of the structured task units based on the task dependency graph to determine the basic task sequence.

[0033] The task score of the structured task unit is determined based on its response urgency, security weight, resource overhead, and historical risk exposure.

[0034] Based on the task scores of each structured task unit, the structured task units in the basic task sequence are sorted and optimized to generate a task chain;

[0035] The task chain is formatted and a maintenance plan is generated.

[0036] In one embodiment, the step of converting the format of the task chain to generate a maintenance plan includes:

[0037] Determine the target output template from the pre-stored system template library;

[0038] Based on the field mapping rules of the target output template, field mapping and compression processing are performed on each structured task unit in the task chain to obtain the field mapping results;

[0039] The task risk level is determined based on the operating voltage level score, historical associated accident score, and executor qualification requirement label of the structured task unit;

[0040] The maintenance plan is generated based on the field mapping results, the task risk level, and the serialization method of the target output template.

[0041] Secondly, this application provides a multi-modal maintenance scheme generation device for power equipment, comprising:

[0042] The data acquisition module is used to acquire multimodal sensing data and equipment attribute information of power equipment;

[0043] A structured object generation module is used to construct a structured state object of the power equipment based on the multimodal sensing data and the equipment attribute information.

[0044] The processing path extraction module is used to determine the target state node corresponding to the power equipment from a pre-built knowledge graph based on the multimodal perception data and the structured state object; and to extract the target processing path of the power equipment from the knowledge graph based on the target state node and the structured state object; wherein the target processing path includes at least a fault node connected to the target state node and a sequence of operation nodes corresponding to the fault node;

[0045] The maintenance plan generation module is used to map the operation node sequence into multiple structured task units; sort and convert the format of each structured task unit to generate a maintenance plan for the power equipment.

[0046] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the multimodal maintenance scheme generation method for power equipment provided in any of the above embodiments.

[0047] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the multimodal maintenance scheme generation method for power equipment provided in any of the above embodiments.

[0048] The aforementioned method, apparatus, computer equipment, computer-readable storage medium, and computer program product for generating multimodal maintenance schemes for power equipment acquire multimodal sensing data and equipment attribute information of the power equipment. Based on this data, they construct structured state objects for the power equipment. These structured state objects serve as input for knowledge graph reasoning, providing a standardized basis for accurate matching of target state nodes and accurate extraction of target processing paths. This effectively avoids problems such as fault location errors and inappropriate maintenance path selection caused by human subjective bias. Furthermore, based on the multimodal sensing data and structured state objects, the system determines the appropriate maintenance path from the pre-constructed knowledge graph. The system identifies target state nodes corresponding to power equipment and extracts target processing paths for the power equipment from a knowledge graph based on these target state nodes and structured state objects, thus achieving intelligent and automated generation of maintenance paths. Operation node sequences are mapped to multiple structured task units, and these units are sorted to ensure correct execution logic and reduce the risk of operational conflicts and process errors. Finally, the structured task units are format-converted to generate maintenance plans for the power equipment. These structured task units are mapped into maintenance plans that can be recognized by the dispatch system and directly executed by maintenance personnel, thus enabling the implementation of maintenance plans. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 This is a flowchart illustrating a method for generating multimodal maintenance solutions for power equipment in one embodiment.

[0051] Figure 2 This is a flowchart illustrating S108 in one embodiment;

[0052] Figure 3 This is a flowchart illustrating S106 in one embodiment;

[0053] Figure 4 This is a flowchart illustrating step S302 in one embodiment;

[0054] Figure 5 This is a flowchart illustrating S112 in one embodiment;

[0055] Figure 6 This is a flowchart illustrating S508 in one embodiment;

[0056] Figure 7 This is a structural block diagram of a multimodal maintenance scheme generation device for power equipment in one embodiment.

[0057] Figure 8 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0059] In one embodiment, this application provides a method for generating multimodal maintenance schemes for power equipment, such as... Figure 1 As shown, it includes the following S102-S112.

[0060] S102, acquire multimodal sensing data and equipment attribute information of power equipment.

[0061] Multimodal sensing data refers to data collected from the operating scenarios of power equipment through different modal sensing methods, such as image data, sensor-collected data, and text data. Multimodal sensing data can be acquired through infrared cameras, sensors, power monitoring systems, and the main control systems of power equipment. Equipment attribute information may include equipment number, component location, etc. The equipment number can be identified through an asset system, and the component location can be obtained through heat map analysis or manual annotation.

[0062] S104. Based on multimodal sensing data and equipment attribute information, construct a structured state object for the power equipment.

[0063] The structured state object includes a data acquisition timestamp, the equipment number of the power equipment, the component location, and a status label. The data acquisition timestamp refers to the time when the multimodal sensing data was collected, corresponding to the time when a fault in the power equipment was determined to exist. The status label indicates the fault state of the power equipment. For example, the resultant state object can be represented as... For example: "Medium-voltage cabinet #3, status is terminal temperature rise too high, location is upper right connecting piece, time is November 8, 2025, 10:24:00". Here, d represents the equipment number, s represents the status label, l represents the component location, and t represents the data acquisition timestamp.

[0064] The status labels of power equipment can be determined by matching multimodal sensing data with a predefined set of status labels. Elements in the predefined status label set can include terms such as "poor contact," "abnormal temperature rise," and "insulation aging," and this set can be manually defined. For example, multimodal sensing data can be matched with the predefined status label set using vector space matching. First, a unified status vector of the multimodal sensing data can be obtained. and a predefined set of status labels Elements Embedded vector The status label of the power equipment is matched by the minimum Euclidean distance principle (as in equation (1)).

[0065]

[0066] S106, Based on multimodal perception data and structured state objects, determine the target state node corresponding to the power equipment from the pre-built knowledge graph.

[0067] knowledge graph A heterogeneous graph composed of multiple types of nodes, node set It can include device nodes, status nodes, fault nodes, and operation nodes, edge sets. It can contain directed relationships such as "impact," "cause," and "handle." Each node in the knowledge graph has attributes, such as "applicable equipment," "related parts," "risk level," and "execution order." Node attributes can be constructed by manual annotation and historical data. The nodes in this application's knowledge graph have obvious causal relationship characteristics, that is, state nodes point to fault nodes, fault nodes then point to feasible operation nodes, and there may also be sequential edges between operation nodes. For example, a branch in the knowledge graph can be: abnormal temperature rise of wiring terminals (state node) → poor contact (fault node) → tightening terminal screws (operation node) → thermal imaging re-inspection (operation node).

[0068] By using semantic matching, the target state node corresponding to the power equipment can be determined from a pre-built knowledge graph based on multimodal perception data and structured state objects.

[0069] S108: Extract the target processing path of the power equipment from the knowledge graph based on the target state node and structured state object.

[0070] The target processing path includes at least the fault node connected to the target status node and the corresponding operation node sequence. The processing path starts from the target status node and ends at the operation node. For example, the example above, "abnormal temperature rise of terminal block (status node) → poor contact (fault node) → tighten terminal screws (operation node) → thermal imaging re-inspection (operation node)", is a processing path.

[0071] Each node in a knowledge graph has attributes. Starting from the target state node, the state labels, component positions, etc. in the structured state object are matched with the node attributes in the knowledge graph to determine the target processing path.

[0072] S110 maps the sequence of operation nodes into multiple structured task units.

[0073] A structured task unit is the smallest independent engineering execution unit formed by engineering and standardizing the mapping of abstract operation nodes in the target processing path. It clarifies all execution elements of a single-step operation through fixed and unified key fields, possessing engineering executability that can be recognized by the scheduling system and directly implemented by maintenance personnel. A structured task unit may include an operation name, execution object, estimated working hours, required tools, safety requirements, and preconditions. The operation name comes from the operation node; the execution object comes from the equipment number; the estimated working hours and required tools can be obtained directly from the node attributes of the operation node or from a mapping table extracted from historical work orders; safety requirements are obtained from the hazard level label in the node attributes of the operation node, which can be defined by experts when building a knowledge graph; preconditions indicate the logical execution relationship between this structured task unit and other structured task units, for example, "replacing a fuse" must be performed after "power is cut off".

[0074] S112 sorts and converts the format of each structured task unit to generate a maintenance plan for the power equipment.

[0075] Based on the logical execution relationships and risks among the structured task units, the system can sort them to ensure the correct execution order and that high-risk tasks are not included in the preceding steps. The sorted structured task units can then be formatted to generate standardized, executable maintenance plans.

[0076] In this embodiment, by acquiring multimodal perception data and equipment attribute information of power equipment, a structured state object of the power equipment is constructed based on the multimodal perception data and equipment attribute information. This structured state object serves as input for knowledge graph reasoning, providing a standardized basis for accurate matching of target state nodes and accurate extraction of target processing paths. This effectively avoids problems such as fault location errors and inappropriate maintenance path selection caused by human subjective bias. Based on the multimodal perception data and structured state objects, the target state nodes corresponding to the power equipment are determined from the pre-constructed knowledge graph. Based on the target state nodes and structured state objects, the target processing paths of the power equipment are extracted from the knowledge graph, realizing the intelligent and automated generation of maintenance paths. Operation node sequences are mapped to multiple structured task units, and each structured task unit is sorted to ensure the correct execution logic between them, reducing the risk of operational conflicts and process errors. Finally, each structured task unit is format-converted to generate a maintenance plan for the power equipment. The structured task units are mapped into maintenance plans that can be recognized by the scheduling system and directly executed by maintenance personnel, realizing the implementation of the maintenance plan.

[0077] In one embodiment, such as Figure 2 As shown, based on the target state node and structured state object, the target processing path of the power equipment is extracted from the knowledge graph, including the following S202-S206.

[0078] S202, starting from the target state node, searches the knowledge graph for all reachable processing paths that satisfy the depth constraints.

[0079] Depth constraints are used to limit the search depth. For example, a depth constraint can allow a search depth of less than or equal to 4, meaning a processing path can contain at most two operation nodes. A reachable processing path is the processing path obtained by searching from the target state node.

[0080] S204. Determine the path score of the reachable processing path based on the status label, component location, sequence of operation nodes in the reachable processing path, and corresponding operation sparsity.

[0081] Operation sparsity indicates the frequency of occurrence of the operation steps corresponding to a given operation node in historical work orders. Higher operation sparsity means the operation steps appear less frequently in historical work orders, resulting in higher execution risk and should be avoided in preceding steps. Path scoring is a rating of the rationality and feasibility of the processing path based on a comprehensive assessment of the matching degree between the node attributes of the operation node and the status labels and component locations of the power equipment, as well as the operation sparsity of the operation node.

[0082] For each operation node in the operation node sequence, obtain the structural adaptation function value between the operation node, status label, and component position, as well as the operation sparsity regularization term value of the operation node. Perform weighted summation on the structural mismatch function value and the operation sparsity regularization term value to determine the node contribution value of the operation node. Then, based on the average node contribution value of each operation node, determine the path score of the reachable processing path.

[0083]

[0084] As shown in equation (2), This represents the i-th reachable processing path; Indicates path score; This represents the sequence of operation nodes in the reachable processing path; j is the index; Indicates the operation node Status labels and component location The structural adaptation function values ​​between them; The weights representing the structure adaptation function values ​​reflect the degree of recommendation for the execution of steps; Indicates the value of the sparsity regularization term for the operation; This represents the weight of the sparsity regularization term value.

[0085] The structural adaptation function has a value range of [0,1]. It matches the "applicable state" in the node attributes of the operation node with the state label, and matches the "applicable component" in the node attributes with the component position. When both match, the structural adaptation function value is 1; when only one matches, the value is 0.5; and when neither matches, the value is 0. (Weight) This can be obtained based on expert experience rules. The value of the operation sparsity regularization term can be calculated by the frequency of the operation steps corresponding to the operation node in historical work orders. For example, if an operation only appears 3 times in the past 1000 work orders, then... .

[0086] S206. Based on the path scores, sort the reachable processing paths and determine the reachable processing path with the highest path score as the target processing path.

[0087] Based on the path scores, each reachable processing path can be sorted in descending or ascending order, and the reachable processing path with the highest score can be determined as the target processing path.

[0088] In this embodiment, starting from the target state node, all reachable processing paths satisfying the depth constraint are searched from the knowledge graph. Based on the state label, component location, the sequence of operation nodes in the reachable processing path, and the corresponding operation sparsity, the path score of each reachable processing path is determined. When calculating the path score for each reachable processing path, the compatibility between the operation node and the component location and state label of the power equipment is considered to ensure the rationality of the operation steps. Furthermore, the operation sparsity of the operation nodes fully considers the frequency of risky operations in real-world scenarios, ensuring the safety of the operation steps. Based on each path score, the reachable processing paths are sorted, and the reachable processing path with the highest path score is determined as the target processing path, improving the engineering rationality and operational executability of the processing paths.

[0089] In one embodiment, the target processing path can be structured to provide standardized input for subsequent steps. For example, the structured processing path can be represented as follows: ,in, Indicates device number =; Indicates status label; Representing state nodes in a knowledge graph The set of faulty nodes connected downwards; This represents the sequence of operation nodes after sorting and filtering.

[0090] In one embodiment, such as Figure 3 As shown, based on multimodal perception data and structured state objects, the target state nodes corresponding to power equipment are determined from the pre-built knowledge graph, including the following S302-S306.

[0091] S302, construct a unified state vector based on multimodal sensing data.

[0092] The unified state vector is a vector representation of multimodal sensing data. The unified state vector can be obtained by encoding the multimodal sensing data through an encoder.

[0093] S304, Based on the equipment number, determine the target subgraph containing the power equipment from the knowledge graph.

[0094] The knowledge graph of a power system contains multiple subgraphs. The target subgraph containing the power device can be determined from the knowledge graph by the device number, thereby narrowing down the search range for the status label.

[0095] S306, Match the unified state vector with the embedded representation vectors of each state node in the target subgraph to determine the target state node.

[0096] The embedding representation vectors of each state node in the target subgraph can be generated by a structure-preserving graph convolutional network. The target state node can be determined by matching the unified state vector with the embedding representation vectors of each state node in the target subgraph using the nearest neighbor matching method, as shown in Equation (3).

[0097]

[0098] in, This represents the set of state nodes in the subgraph corresponding to the power equipment. State Node Embedded vector, This represents the target state node.

[0099] In one embodiment, multimodal sensing data includes thermal imaging image data, structured operation and maintenance text data, and sensor sampling data.

[0100] Thermal imaging data can reflect the surface temperature field distribution of power equipment, such as localized overheating at switch contact points. It can be acquired using infrared thermal imaging cameras deployed on power equipment such as substations, switchgear, and transformer substations. For example, the sampling frequency is once every 60 seconds, the image resolution is 128×128, and it is a single-channel thermal image.

[0101] The structured operation and maintenance text data comes from the inspection records and defect descriptions filled in by operation and maintenance personnel through handheld terminals. It can be natural language sentences, such as: "The main contact on the left side of the medium-voltage cabinet in area B is obviously hot, and the infrared image shows a reddish color."

[0102] Sensor sampling data is provided by edge data acquisition devices or Supervisory Control and Data Acquisition (SCADA) systems that can be connected to power devices, including time-series data of indicators such as current, voltage, and temperature. For example, SCADA can record data in 10-second cycles, with each record containing the average and range values ​​of each type of sensor over the most recent 60 seconds, forming a 6-dimensional vector.

[0103] Thermal imaging data, structured maintenance text data, and sensor sampling data are highly practical data in power equipment maintenance scenarios, and can accurately reflect the operating conditions of power equipment.

[0104] In one embodiment, such as Figure 4 As shown, a unified state vector is constructed based on multimodal sensing data, including the following S402-S408.

[0105] S402 extracts features from thermal imaging image data using an image encoder to generate image feature vectors.

[0106] An image encoder can consist of two convolutional layers and one fully connected layer. The network structure of an image encoder can be: Conv(1,16,3x3)→ReLU→Conv(16,32,3x3)→ReLU→Flatten→FC(64). The image encoder can extract the distribution characteristics of hot spot regions in thermal imaging data, for example, identifying temperature concentration areas corresponding to main contacts or terminal connection points.

[0107] S404 uses a semantic encoder to extract features from structured operation and maintenance text data and generate semantic feature vectors.

[0108] A semantic encoder can be based on the BERT model. After pre-training the BERT model, the model parameters are frozen, serving as a static encoder. After encoding the structured operational text data using the semantic encoder, the [CLS] vector from the BERT model is used as the semantic feature vector.

[0109] S406 uses a structured numerical encoder to encode sensor sampled data and generate structured numerical feature vectors.

[0110] A structured numerical encoder can consist of two linear mapping layers, and its network structure can be: Linear(6→32)→ReLU→Linear(32→64).

[0111] Furthermore, the image feature vector, semantic feature vector, and structured numerical feature vector all have the same dimension; for example, they are all 1 / 2. .

[0112] S408 performs a weighted summation of the image feature vector, semantic feature vector, and structured numerical feature vector to generate a unified state vector.

[0113] The weights of image feature vectors, semantic feature vectors, and structured numerical feature vectors are calculated using the softmax function. The weights can be dynamically adjusted according to the fault type. For example, in thermal breakdown problems, image feature vectors have higher weights, while in parameter drift problems, structured numerical feature vectors have higher weights.

[0114] As shown in equation (4), the image feature vector, semantic feature vector, and structured numerical feature vector are weighted and summed to generate a unified state vector. For a unified state vector; These represent image, text, and structured numerical feature vectors, respectively. For modal fusion weight coefficients, satisfying .

[0115]

[0116] In one embodiment, such as Figure 5 As shown, the structured task units are sorted and their formats are converted to generate maintenance plans for power equipment, including the following S502-S508.

[0117] S502, construct a task dependency graph based on each structured task unit, perform topological sorting on each structured task unit based on the task dependency graph, and determine the basic task sequence.

[0118] Task Dependency Graph The node set T represents the set of task units, and the edge set... This indicates a constraint that "must be executed after a certain task." The task dependency graph is built based on the sequential edges of operation nodes in the knowledge graph and the logical dependency inference rules of each structured task unit. For example, if a structured task unit contains the keyword "detection" and another structured task unit contains "repair," the system will automatically establish an edge relationship where detection precedes repair.

[0119] After establishing a task dependency graph based on each structured task unit, the structured task units can be topologically sorted based on the task dependency graph. If there are no loops, the basic task sequence is determined based on the topological structure of the task dependency graph. If there are loops, it means that the dependency relationship of at least two structured task units is conflicting, or the graph structure is abnormal. In this case, the rule rollback mechanism will be triggered to request manual intervention or use the default order.

[0120] S504 determines the task score of a structured task unit based on its response urgency, security weight, resource overhead, and historical risk exposure.

[0121] The urgency level reflects the speed of the startup delay of the structured task unit. It can be obtained by extracting the average startup delay of the task under this state from the historical task sequence and then normalizing it. The safety weight can be mapped according to the hazard level label of the structured task unit. Resource overhead can be calculated based on the number of tools required for the structured task unit and the estimated working hours. Historical risk exposure indicates the frequency with which the operation steps corresponding to the structured task unit cause accidents in historical work orders; a higher historical risk exposure indicates a higher task risk.

[0122] As shown in equation (5), Represents structured task units Compared with the current state The urgency of the response; Represents the safety weight, with a range of ; Indicates resource overhead; Indicates historical risk exposure; As a weighting factor, it needs to be set by experts or determined through scheduling simulation optimization during system deployment.

[0123]

[0124] S506: Based on the task scores of each structured task unit, the structured task units in the basic task sequence are sorted and optimized to generate a task chain.

[0125] A task chain is a sequence of tasks composed of optimized, structured task units. The structured task units in the basic task sequence are optimized and ordered based on task scores to prevent high-risk tasks from being placed in earlier steps.

[0126] In some embodiments, the structured task units in the generated task chain, in addition to the aforementioned attribute information such as operation name, execution object, estimated working time, required tools, safety requirements, and preconditions, also possess parallel scheduling tag information. Furthermore, based on whether there are limitations such as resource mutual exclusion, positional conflicts, or operational interference between the structured task units, it can be determined whether there are structured task units that can be scheduled in parallel, and parallel task groups can be formed. For example, for two task units, "replacing terminal blocks" and "replacing temperature control modules," if their execution locations are in different electrical cabinets, the tools do not overlap, and the operation logic has no sequential dependency, they can be grouped into the same parallel group, improving scheduling efficiency and maintenance speed.

[0127] S508 performs format conversion on the task chain and generates a maintenance plan.

[0128] The final maintenance plan must have the structured fields required by the platform scheduling system, and also meet the readability and standardization requirements for manual execution and review. Therefore, it is necessary to convert the task chain into a maintenance plan in the required format.

[0129] In one embodiment, such as Figure 6 As shown, the task chain is formatted and a maintenance plan is generated, including the following S602-S608.

[0130] S602, determine the target output template from the pre-stored system template library.

[0131] The system template library stores various output templates, each including field definitions, format specifications, field mapping rules, and serialization methods. The serialization method is the conversion logic defined in the output template for converting fields to the target format. For example, the system template library... If the goal is to generate a JSON work order for the scheduling platform, then select... The template defines fields such as operation_id, device_code, step_order, estimated_duration, safety_level, and tool_list, and requires the nested structure of these fields to conform to the scheduling interface call rules. If the generated work order is an HTML work order for maintenance personnel's handheld terminal, then it may use... Its structure places greater emphasis on visualization hierarchy and clarity of natural language description.

[0132] S604, based on the field mapping rules of the target output template, performs field mapping and compression processing on each structured task unit in the task chain to obtain the field mapping results.

[0133] After the field format is determined, each structured task unit needs to be... The field space is mapped to the target output template. As shown in Equation (6), the compression function can be based on the field mapping. Control how each field is retained and displayed under different output templates:

[0134]

[0135] in, Indicates task The The importance weight of each field comes from the field priority settings in the system template library for different task scenarios. For example, in high-pressure operations, the "safety level" field has a higher weight. Indicates output template The field's retention coefficient is set to 1 for retention and 0 for ignoring. This mechanism allows the generation process to intelligently tailor information for different application scenarios, improving the adaptability of the solution. For example, if a structured task unit... The value of the field "Required Tools" is ["Insulating Clamps", "Temperature Gun"]. In mobile terminal scenarios, this will be retained as a natural language description: "Prepare insulating clamps and a temperature gun for operation," while in scheduling platform scenarios, it will be encoded as an array of field names in `tool_list`. M represents the number of fields. Field mapping compression function. The output is the field mapping result.

[0136] S606 determines the task risk level based on the operating voltage level score of the structured task unit, the historical associated accident score, and the executor qualification requirement label.

[0137] The operating voltage level risk score indicates the degree of danger of the voltage level of the equipment involved in the task unit. It can be determined based on the voltage level of the equipment. If the equipment involves a voltage level of 10kV or higher, the operating voltage level risk score is 1; otherwise, it is 0. The historical related accident score indicates whether the task unit has experienced any accidents in the past. This can be extracted from historical work order tags. If the task unit has experienced any accidents in the past, the historical related accident score is 1. The executor qualification requirement tag indicates the level of qualification required for the executor in this task unit. If marked "Requires highly qualified personnel," it is set to 1.

[0138] As shown in equation (7), the operating voltage level of the structured task unit is scored. Historical related accident scoring And the label of the executor's qualification requirements Perform summation to determine the task risk level. .

[0139]

[0140] S608 generates a maintenance plan based on the field mapping results, task risk level, and serialization method of the target output template.

[0141] When the task risk level When this happens, a risk warning message needs to be attached to the structured task unit. The template for the risk warning message comes from the operation and maintenance regulations corpus and can be attached to each task unit as supplementary explanation. Based on the execution order and parallel scheduling tag information of each structured task unit in the task chain, the field mapping results can be merged with the risk warning message set, and the merged task content can be serialized into the target format according to the serialization method of the target output template.

[0142] If no risk warning is required, the field mapping result can be serialized into the target format based on the execution order and parallel scheduling label information of each structured task unit in the task chain, as well as the serialization method of the target output template.

[0143] The final maintenance plan output includes the following core fields: equipment number, trigger status, operation steps, field mapping results, task sorting and parallel information, and a set of risk warnings. This maintenance plan can be uploaded to the scheduling platform for work assignment, or it can be synchronized to the operation and maintenance terminal for technicians to view and execute.

[0144] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0145] Based on the same inventive concept, this application also provides a multimodal maintenance scheme generation device for power equipment, used to implement the multimodal maintenance scheme generation method for power equipment described above. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations of one or more embodiments of the multimodal maintenance scheme generation device for power equipment provided below can be found in the limitations of the multimodal maintenance scheme generation method for power equipment described above, and will not be repeated here.

[0146] In one exemplary embodiment, such as Figure 7 As shown, a multi-modal maintenance scheme generation device for power equipment is provided, comprising:

[0147] The data acquisition module 702 is used to acquire multimodal sensing data and equipment attribute information of power equipment;

[0148] The structured object generation module 704 is used to construct structured state objects of power equipment based on multimodal sensing data and equipment attribute information;

[0149] The processing path extraction module 706 is used to determine the target state node corresponding to the power equipment from a pre-built knowledge graph based on multimodal perception data and structured state objects; and to extract the target processing path of the power equipment from the knowledge graph based on the target state node and structured state objects; wherein, the target processing path includes at least the fault node connected to the target state node and the sequence of operation nodes corresponding to the fault node;

[0150] The maintenance plan generation module 708 is used to map the operation node sequence into multiple structured task units; sort and convert the format of each structured task unit to generate a maintenance plan for the power equipment.

[0151] In one embodiment, the processing path extraction module is further configured to search for all reachable processing paths that satisfy the depth constraint conditions from the knowledge graph, starting from the target state node; determine the path score of the reachable processing path based on the state label, component position, operation node sequence in the reachable processing path and the corresponding operation sparsity; sort the reachable processing paths based on each path score, and determine the reachable processing path with the highest path score as the target processing path.

[0152] In one embodiment, the processing path extraction module is further configured to, for each operation node in the operation node sequence, obtain the structural adaptation function value between the operation node, the status label, and the component position, as well as the operation sparsity regularization term value of the operation node; perform weighted summation processing on the structural mismatch function value and the operation sparsity regularization term value to determine the node contribution value of the operation node; and determine the path score of the reachable processing path based on the average node contribution value of each operation node.

[0153] In one embodiment, the processing path extraction module is further configured to construct a unified state vector based on multimodal perception data; determine a target subgraph containing power equipment from the knowledge graph based on the device number; and match the unified state vector with the embedding representation vectors of each state node in the target subgraph to determine the target state node.

[0154] In one embodiment, the processing path extraction module is further configured to extract features from thermal imaging image data using an image encoder to generate an image feature vector; extract features from structured operation and maintenance text data using a semantic encoder to generate a semantic feature vector; encode sensor sampling data using a structured numerical encoder to generate a structured numerical feature vector; and perform weighted summation processing on the image feature vector, semantic feature vector, and structured numerical feature vector to generate a unified state vector.

[0155] In one embodiment, the maintenance plan generation module is further configured to construct a task dependency graph based on each structured task unit, perform topological sorting on each structured task unit based on the task dependency graph to determine a basic task sequence; determine the task score of each structured task unit based on its response urgency, security weight, resource overhead, and historical risk exposure; perform sorting optimization on each structured task unit in the basic task sequence based on its task score to generate a task chain; and convert the task chain to a new format to generate a maintenance plan.

[0156] In one embodiment, the maintenance plan generation module is further configured to determine a target output template from a pre-stored system template library; perform field mapping and compression processing on each structured task unit in the task chain based on the field mapping rules of the target output template to obtain the field mapping results; determine the task risk level based on the operating voltage level score, historical associated accident score, and executor qualification requirement label of the structured task unit; and generate a maintenance plan based on the field mapping results, the task risk level, and the serialization method of the target output template.

[0157] Each module in the aforementioned multimodal maintenance scheme generation device for power equipment can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0158] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 8 As shown, the computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements a method for generating multimodal maintenance schemes for power equipment. The display unit is used to form a visually visible image and can be a display screen, projection device, or virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0159] Those skilled in the art will understand that Figure 8The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0160] In one exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the multimodal maintenance scheme generation method for power equipment provided in any of the above embodiments.

[0161] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the method for generating multimodal maintenance schemes for power equipment provided in any of the above embodiments.

[0162] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the method for generating multimodal maintenance schemes for power equipment provided in any of the above embodiments.

[0163] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0164] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0165] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for generating multimodal maintenance schemes for power equipment, characterized in that, The method includes: Acquire multimodal sensing data and equipment attribute information of power equipment; Based on the multimodal sensing data and the device attribute information, a structured state object of the power equipment is constructed; Based on the multimodal sensing data and the structured state object, the target state node corresponding to the power equipment is determined from the pre-constructed knowledge graph; the structured state object includes state labels and component locations; Starting from the target state node, search the knowledge graph for all reachable processing paths that satisfy the depth constraints. The path score of the reachable processing path is determined based on the status label, the component location, the sequence of operation nodes in the reachable processing path, and the corresponding operation sparsity. Based on the path scores, the reachable processing paths are sorted, and the reachable processing path with the highest path score is determined as the target processing path; wherein, the target processing path includes at least a fault node connected to the target state node and a sequence of operation nodes corresponding to the fault node; The operation node sequence is mapped into multiple structured task units respectively; The structured task units are sorted and their formats are converted to generate a maintenance plan for the power equipment.

2. The method according to claim 1, characterized in that, The step of determining the path score of the reachable processing path based on the status label, the component location, the sequence of operation nodes in the reachable processing path, and the corresponding operation sparsity includes: For each operation node in the operation node sequence, obtain the structural adaptation function value between the operation node, the status label, and the component position, as well as the operation sparsity regularization value of the operation node. Perform a weighted summation on the structural adaptation function value and the operation sparsity regularization value to determine the node contribution value of the operation node. The path score of the reachable processing path is determined based on the average node contribution value of each operation node.

3. The method according to claim 1, characterized in that, The structured state object includes the device number; The step of determining the target state node corresponding to the power equipment from a pre-built knowledge graph based on the multimodal sensing data and the structured state object includes: Based on the multimodal sensing data, a unified state vector is constructed; Based on the device number, a target subgraph containing the power device is determined from the knowledge graph; The unified state vector is matched with the embedded representation vectors of each state node in the target subgraph to determine the target state node.

4. The method according to claim 3, characterized in that, The multimodal sensing data includes thermal imaging image data, structured operation and maintenance text data, and sensor sampling data; The step of constructing a unified state vector based on the multimodal sensing data includes: The thermal imaging image data is processed by an image encoder to extract features and generate an image feature vector. The structured operation and maintenance text data is processed by a semantic encoder to extract features and generate a semantic feature vector. The sensor sampled data is encoded using a structured numerical encoder to generate a structured numerical feature vector. The image feature vector, the semantic feature vector, and the structured numerical feature vector are weighted and summed to generate the unified state vector.

5. The method according to claim 1, characterized in that, The step of sorting and format-converting each of the structured task units to generate a maintenance plan for the power equipment includes: A task dependency graph is constructed based on each of the structured task units, and a topological sorting process is performed on each of the structured task units based on the task dependency graph to determine the basic task sequence. The task score of the structured task unit is determined based on its response urgency, security weight, resource overhead, and historical risk exposure. Based on the task scores of each structured task unit, the structured task units in the basic task sequence are sorted and optimized to generate a task chain; The task chain is formatted and a maintenance plan is generated.

6. The method according to claim 5, characterized in that, The process of converting the format of the task chain to generate a maintenance plan includes: Determine the target output template from the pre-stored system template library; Based on the field mapping rules of the target output template, field mapping and compression processing are performed on each structured task unit in the task chain to obtain the field mapping results; The task risk level is determined based on the operating voltage level score, historical associated accident score, and executor qualification requirement label of the structured task unit; The maintenance plan is generated based on the field mapping results, the task risk level, and the serialization method of the target output template.

7. A multi-modal maintenance scheme generation device for power equipment, characterized in that, include: The data acquisition module is used to acquire multimodal sensing data and equipment attribute information of power equipment; A structured object generation module is used to construct a structured state object of the power equipment based on the multimodal sensing data and the equipment attribute information; the structured state object includes state labels and component locations; The processing path extraction module is used to determine the target state node corresponding to the power equipment from a pre-built knowledge graph based on the multimodal perception data and the structured state object; Starting from the target state node, search the knowledge graph for all reachable processing paths that satisfy the depth constraints. The path score of the reachable processing path is determined based on the status label, the component location, the sequence of operation nodes in the reachable processing path, and the corresponding operation sparsity. Based on the path scores, the reachable processing paths are sorted, and the reachable processing path with the highest path score is determined as the target processing path; wherein, the target processing path includes at least a fault node connected to the target state node and a sequence of operation nodes corresponding to the fault node; The maintenance plan generation module is used to map the operation node sequence into multiple structured task units respectively; The structured task units are sorted and their formats are converted to generate a maintenance plan for the power equipment.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images