Power equipment detection report generation method, device and system and medium

Abstract

The invention relates to the technical field of power equipment detection, and provides a power equipment detection report generation method, device and system and a medium. According to the implementation scheme, time alignment is carried out on multi-modal event data of the power equipment and a sensor test data stream, and a sensor data segment corresponding to the power equipment is obtained; respectively carrying out feature extraction on the multi-modal event data, the sensor data segment and a preset report template to construct a heterogeneous graph; in response to a detection request of to-be-detected power equipment, based on to-be-detected equipment data in the detection request, performing retrieval in the heterogeneous graph to obtain a retrieval result; constructing structured input data based on the retrieval result and a preset prompt template; and taking the structured input data as the input of the visual language big language model, and generating a detection report of the to-be-detected power equipment. According to the embodiment of the invention, the accuracy of the power equipment detection report generation content can be improved.

Description

Technical Field

[0001] This invention relates to the field of power equipment testing technology, and in particular to a method, apparatus, system and medium for generating power equipment testing reports. Background Technology

[0002] Live-line testing of power equipment is a crucial step in ensuring the safe and stable operation of the power grid, and the resulting testing reports are the core basis for assessing equipment health and making operational and maintenance decisions. Traditional machine learning models often rely on single-modality or low-dimensional fusion strategies, which often struggle to establish reliable correspondences when processing data from different time scales, resulting in insufficient data correlation and making it difficult for the model to accurately reflect the true operating status of the equipment. Furthermore, existing machine learning models frequently exhibit limited feature representation capabilities when faced with massive and complex power system monitoring data.

[0003] As a result, the generated reports are prone to inaccurate descriptions, logical inconsistencies, or a lack of professionalism, failing to meet the accuracy requirements in power equipment testing scenarios. Summary of the Invention

[0004] This invention provides a method, apparatus, system, and medium for generating power equipment inspection reports, which can solve at least one of the above-mentioned technical problems.

[0005] In a first aspect, embodiments of the present invention provide a method for generating a power equipment inspection report, comprising: Time alignment is performed between the multimodal event data of the power equipment and the sensor test data stream to obtain the sensor data segment corresponding to the power equipment; Feature extraction is performed on the multimodal event data, the sensor data segments, and the preset report template to construct a heterogeneous graph; In response to a testing request from a power device to be tested, a search is performed on the heterogeneous graph based on the data of the device to be tested in the testing request to obtain the search results. Based on the search results and the preset prompt template, structured input data is constructed; The structured input data is used as input to a visual language large language model to generate a detection report for the power equipment to be detected.

[0006] Secondly, embodiments of the present invention provide a power equipment inspection report generation device, comprising: The time alignment module is used to perform time alignment between the multimodal event data of the power equipment and the sensor test data stream to obtain the sensor data segment corresponding to the power equipment. The graph construction module is used to extract features from the multimodal event data, the sensor data segments, and the preset report template to construct a heterogeneous graph. The retrieval module is used to respond to a detection request from the power equipment to be detected, and to perform a retrieval in the heterogeneous graph based on the data of the equipment to be detected in the detection request, so as to obtain retrieval results; A structured input data construction module is used to construct structured input data based on the search results and preset prompt templates; The inspection report generation module is used to take the structured input data as input to the visual language large language model and generate an inspection report for the power equipment to be inspected.

[0007] Thirdly, embodiments of the present invention also provide a power equipment testing report generation system, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method described in any one of the embodiments of the present invention.

[0008] Fourthly, embodiments of the present invention also provide a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to perform the method described in any one of the embodiments of the present invention.

[0009] By employing the technical solution of this invention, time alignment is performed on the multimodal event data and sensor test data streams of power equipment to obtain the corresponding sensor data segments for the power equipment. This ensures that monitoring information from different data sources establishes a correspondence under the same time reference. Furthermore, feature extraction is performed on the aligned sensor data segments, multimodal event data, and preset report templates, and a heterogeneous graph is constructed based on the extraction results. This allows different types of data to express their semantic associations and complementary features within a unified graph structure. Upon receiving a detection request from the device to be tested, a search is performed on the heterogeneous graph based on the device data. This enables the accurate acquisition of sensor test data, event data, or template data most relevant to the device's state, thereby improving the quality of data for subsequent report generation. Based on this, structured input data is constructed using the search results and preset prompt templates. This provides clear, standardized, and context-complete input content for the visual language big data model, allowing the model to more accurately understand the device's state and professional semantics. Finally, the structured input data is input into the visual language big data model to generate a detection report. This effectively improves the accuracy of the detection report content.

[0010] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0011] The accompanying drawings are provided for a better understanding of this solution and do not constitute a limitation of the invention. Wherein: Figure 1 This is a flowchart of a method for generating a power equipment inspection report according to an embodiment of the present invention; Figure 2 This is a structural block diagram of a power equipment testing report generation device according to an embodiment of the present invention; Figure 3 This is a schematic block diagram of an electronic device used to implement the methods of embodiments of the present invention. Detailed Implementation

[0012] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of the present invention, including various details to aid understanding. These details should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of the invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0013] Figure 1 This is a flowchart of a method for generating a power equipment inspection report according to an embodiment of the present invention.

[0014] like Figure 1 As shown, the method for generating the power equipment inspection report may include: S110, Time-align the multimodal event data of the power equipment with the sensor test data stream to obtain the corresponding sensor data segment of the power equipment; S120: Features are extracted from multimodal event data, sensor data segments, and preset report templates to construct a heterogeneous graph; S130, in response to the detection request of the electrical equipment to be tested, the heterogeneous graph is searched based on the data of the equipment to be tested in the detection request to obtain the search results; S140, Based on the search results and the preset prompt template, construct structured input data; S150 uses structured input data as input to a visual language large language model to generate an inspection report for the electrical equipment to be inspected.

[0015] For example, multimodal event data refers to a non-continuous collection of information, containing multiple data types (modals), collected around the device under test during a specific detection event. It is typically recorded manually on-site around a detection task and includes information such as descriptive text of the detection item, descriptive text of the environment (e.g., temperature, humidity, wind speed, altitude, surrounding electromagnetic environment), and descriptive information of instantaneous states (e.g., images).

[0016] For example, suppose a transformer is being inspected, and the inspector uses a mobile app to collect the following "multimodal event data": Inspection item description text: {"Equipment ID":"BYQ-01-A","Model":"S11-M","Rated Capacity":"500kVA"}; where kVA is kilovolt-ampere; Environmental description text: "December 5, 2025, sunny weather, ambient temperature 15℃. No obvious oil leakage from the equipment body, but slight signs of heating at the A-phase bushing joint." Images (visual information): A photograph of the overall appearance of the equipment, and a thermal image of the A-phase bushing joint taken by an infrared thermal imager.

[0017] In this example, the data sources for multimodal event data include, but are not limited to: Manual input: Inspectors enter unstructured text, such as descriptions of the on-site environment, preliminary judgments of equipment appearance, and textual records of abnormal phenomena, through the text input boxes of the application (APP). Form filling: The APP provides structured forms, and inspectors fill in parameters such as equipment identity (ID), model, and voltage level according to the equipment nameplate information. Photo / video recording: Inspectors use the terminal's built-in camera to take photos of the equipment's overall appearance, close-up details, and the equipment's operating environment. External device acquisition: The terminal can connect to portable infrared thermal imagers and other devices via Bluetooth or Universal Serial Bus (USB) to directly acquire and import thermal images.

[0018] For example, a sensor test data stream refers to time-series monitoring data containing continuous timestamps, continuously collected by various sensors installed on electrical equipment during testing.

[0019] For example, when performing partial discharge detection on a transformer, a 3-minute data stream was collected using an ultra-high frequency (UHF) sensor. This "sensor test data stream" is a series of time-amplitude pairs, such as [(10:01:00.001, 15.2mV), (10:01:00.002, 18.1mV), (10:01:00.003, 16.5mV), ... , (10:04:00.000, 14.9mV)].

[0020] For example, before time alignment of multimodal event data of power equipment with sensor test data streams, the testing task constitutes an "event" when an inspector is preparing to test a power device on-site. Sensor test data streams can be automatically acquired as follows: First, multimodal event data is automatically acquired and preprocessed using a handheld mobile terminal to form a raw event sequence with alignable timestamps. Specifically, after the inspector activates the mobile acquisition module on-site, they take a picture of the nameplate of the device under test using a camera and simultaneously record a voice description of the environment. The system then uses the time of the image capture as the initial time point of the event and performs lightweight preprocessing on the inputs from the image and voice channels respectively. For the image channel, the system detects the blurriness, illumination uniformity, and contrast of the nameplate image in real time, and triggers an intelligent re-capture mechanism when these indicators fail to reach a set threshold. When the image quality meets the requirements, the system sequentially performs illumination equalization, anti-distortion correction, and sharpening / denoising processing. A lightweight Optical Character Recognition (OCR) model deployed on the mobile device is then used to recognize the processed image, resulting in an image recognition event containing text content, text location, and recognition confidence. For the voice channel, the system utilizes a multi-channel audio preprocessing strategy to analyze the spectral characteristics of environmental noise in real time, performing dynamic noise reduction in the frequency domain to preserve the main frequency bands of the voice. Based on an anti-industrial noise voice recognition model deployed on the mobile device, the system converts the verbal information of the inspectors into text, forming a voice recognition event containing sentence content, word-level timestamps, and semantic confidence.

[0021] After image recognition and speech recognition events are generated, the system immediately invokes the built-in power business rule engine to perform standardized parsing and professional terminology correction on the events. The rule engine first performs error correction, lexical normalization, and field extraction on the OCR and speech recognition texts based on a power professional terminology knowledge base. Further, through spatial topology analysis, regular expression matching, and semantic role labeling technologies, it identifies and extracts key information such as equipment model, rated voltage, manufacturer, ambient temperature, humidity, and appearance condition, assigning a confidence score to each piece of information. The structured information generated by the above parsing, together with its corresponding event timestamp, constitutes the source data sequence of the multimodal event data.

[0022] After generating image events, voice events, and structured field events, the mobile device records the timestamps corresponding to all events, including image capture time, voice start and end time, and the parsing time when the rule engine completes field localization, and stores them uniformly as a sortable time-stamped event stream. Subsequently, when the testing personnel begin to use the power test sensors to collect equipment operating data, the sensor test data stream, carrying continuous timestamps, is uploaded to the mobile device in real time. In step S110, based on the timestamps in the aforementioned event stream, the multimodal event data and the sensor data stream are time-aligned to determine the specific sensor data segments corresponding to the equipment nameplate information, environmental conditions, and human observation behavior, providing complete and synchronous input data for subsequent diagnostic algorithms.

[0023] For example, the data source for sensor test data streams can also be as follows: Sensor deployment: According to the requirements of the testing project, the testing personnel install or place specific sensors (such as UHF sensors for partial discharge detection, transient ground voltage sensors, ultrasonic sensors, etc.) at specific locations on the power equipment under test. Instrument connection and setup: The sensor is connected to a dedicated data acquisition instrument or testing host via a cable. The testing personnel set parameters such as sampling rate, test duration, and trigger threshold on the instrument. Data acquisition: After the instrument is started, it samples and digitizes the sensor's analog signal at frequencies in the megahertz (MHz) or gigahertz (GHz) range within a set time period, forming a massive amount of time-series data points. Data export: After the test is completed, this raw data stream containing the complete waveform is exported as a file from the instrument and then uploaded to the data management system or associated with the current testing event.

[0024] It should be noted that, in the aforementioned examples, image recognition and speech recognition methods are preferred over manual data collection.

[0025] For example, a preset report template refers to a standardized text structure used to generate power equipment inspection reports, which contains multiple fields. These fields include, for example,

Equipment Overview

Key Test Data

Diagnostic Conclusion

Risk Level Determination

Handling Recommendations

[0026] For example, the equipment data to be tested refers to the structured data input by the business system or user in the testing request, used to identify the equipment and the testing requirements. For example, {"query_equipment_id":"TR-110kV-023", "equipment type":"110kV on-load tap-changing transformer"}. Here, kV is also known as kilovolt (kV).

[0027] For example, the search results refer to the structured data set obtained by reasoning and searching based on "data of the device to be tested" in the heterogeneous graph, which is used to populate the report template.

[0028] For example, {"Template Mapping List": [{"Field Name":"Appearance Inspection Status","Source Node":"Field Record 001","Content":"Slight signs of heating at the A-phase bushing connector"},{"Field Name":"Maximum Discharge Amount Detected for Partial Discharge","Source Node":"Sensor Data Segment 001","Value":"150pC"}]}. Here, pC stands for PicoCoulomb (pC).

[0029] For example, visual language large language models include Generative Pre-trained Transformer (GPT), Large Language and VisionAssistant (LLaVA) series models, Qwen-VL (Ali Tongyi Qianwen Visual Language Model), etc.

[0030] In this example, structured input data is fed into a visual language big language model to infer information from the data and output a detection report for the electrical equipment to be detected.

[0031] According to the above implementation method, firstly, the multimodal event data of power equipment and the sensor test data stream are time-aligned to establish an accurate correspondence between data from different sources on a unified time scale, thereby obtaining sensor data segments reflecting the actual operating status of the equipment. Subsequently, features are extracted from the multimodal event data, sensor data segments, and preset report templates, and the extraction results are constructed into a heterogeneous graph, allowing the semantic relationships between various types of data to be presented in a structured manner. Upon receiving a detection request, a search is performed on the heterogeneous graph based on the data of the equipment to be detected, which can accurately locate key features and related information related to the equipment to obtain search results. Based on the search results and prompt templates, structured input data is constructed, providing a clear, standardized, and complete input expression for the visual language big data model. Finally, the visual language big data model generates a power equipment inspection report. In this way, the quality of information organization can be improved in the context of complex multi-source data, enhancing the semantic fit and data support during report generation, thereby significantly improving the accuracy and reliability of the generated power equipment inspection report content.

[0032] In one implementation, time alignment is performed on the multimodal event data of power equipment and the sensor test data stream to obtain the corresponding sensor data segment of the power equipment. This includes: using the timestamp of the equipment information record event in the multimodal event data as the reference anchor point and expanding the time window of the reference anchor point to obtain a reference time series; standardizing the sensor test data stream to obtain a standard test sequence; constructing a local cost matrix based on the Euclidean distance between each first time point in the reference time series and each second time point in the standard test sequence; calculating the local cost matrix according to the dynamic time warping algorithm and setting global path constraints and local monotonic constraints to obtain a cumulative cost matrix; starting from the end point of the cumulative cost matrix, backtracking along the direction of minimum cumulative cost to obtain a target curved path connecting the reference time series and the standard test sequence; extracting the start and end mapping points in the target curved path to obtain the start and end times, and intercepting the standard test sequence based on the start and end times to obtain the sensor data segment.

[0033] For example, firstly, the high-precision timestamp T of the device information record event in the multimodal event data is read. record ; Select the window half-width δ based on the preset time tolerance (e.g., in seconds); in T record Extend δ forward and backward from the center to determine the time range [T] of the reference time series. record δ,T record +δ];Within this time range, a reference time point sequence R={r1,…,r] is obtained by sampling on the time axis at an appropriate time resolution (e.g., consistent with the sensor's highest sampling rate or reference sampling rate). m If necessary, the signals corresponding to the device information recording events can be smoothed or resampled to eliminate isolated noise and ensure the temporal uniformity of the reference sequence.

[0034] For example, if the device information record event timestamp is 2025-06-01 10:00:00, and δ=30 seconds with a reference resolution of 1 second is selected, then the reference time series is 61 time points r1,…,r from 09:59:30 to 10:00:30. 61 .

[0035] For example, the original time series segments covering the reference time range are extracted from the sensor data stream, and preprocessing such as missing value interpolation, detrending, and low-pass / high-pass filtering is performed on the sensor time series as needed; the preprocessed time series data is resampled or interpolated at a selected sampling rate to obtain time grids that are compatible with or computationally comparable to the reference sequence; the resampled test sequence T={t1,…,t nPerform Z-score standardization (i.e., subtract the mean from the sequence and divide by the standard deviation) or other normalization methods to eliminate the influence of different physical dimensions and amplitude scales on subsequent distance calculations; if there are multi-channel sensors, independent standardization can be performed at the channel level or principal component / feature extraction can be performed first and then standardization can be performed.

[0036] For example, if a temperature sensor samples the original sequence at 100Hz from 09:58:00 to 10:05:00, and then resamples it to 1Hz by extending the range by 10s before and after the reference window (09:59:30 to 10:00:30), and then performs Z-score processing on the 1Hz sequence to obtain the standard test sequence T.

[0037] For example, let the reference sequence sample value be R[i] (or the feature vector of the reference sequence at this time point), and the standard test sequence sample value be T[j]. For all i∈[1,m] and j∈[1,n], calculate the local cost D[i,j]=(R[i]-T[j]). 2 (For multidimensional features, the squared norm can be used) The results are organized into a cost matrix D of size m×n. To improve computational efficiency, vectorization or block computation can be used. Threshold pruning can be performed on D before computation (a high penalty value can be assigned if the distance is too large) so that it can be used in conjunction with subsequent path constraints.

[0038] For example, if the reference sequence R has 61 points and the test sequence T has 301 points, the local cost matrix D∈R is calculated. 61×301 Where D[10,120]=(R

[10] -T

[120] ) 2 .

[0039] For example, the cumulative cost matrix C is initialized and filled point by point according to the recursive relation C[i,j]=D[i,j]+min{C[i-1,j],C[i,j-1],C[i-1,j-1]}. The first row and first column of the matrix are initialized according to boundary conditions (e.g., accumulating from (1,1) or setting a large value to force matching of the starting point); simultaneously, global path constraints are applied during the calculation process (e.g., Sakoe-Chiba band: only index pairs |j-αi|≤w are allowed to participate in the calculation, the weight α can be preset according to actual needs, and the bandwidth w is determined by the preset time tolerance) to limit the out-of-bounds deformation of the path, and local monotonicity constraints are applied (i.e., the path can only move to the right, down, or down-right to maintain time monotonicity). In implementation, only in-band elements can be calculated based on the bandwidth w, and out-of-band elements can be set to infinity to save computation.

[0040] For example, in the aforementioned 61×301 scenario, if the Sakoe-Chiba bandwidth w=10 (corresponding to ±10 seconds tolerance), then only the D[i,j] terms that satisfy |j-(n / m)i|≤10 are calculated, and C is filled according to the recursive formula.

[0041] For example, after filling the cumulative cost matrix C, starting from the predetermined endpoint (usually i=m and a suitable j endpoint, such as j=n or the position with the minimum cost in the last row), backtracking sequentially in the direction that minimizes the previous cumulative cost in {(i-1,j),(i,j-1),(i-1,j-1)} until reaching the starting point (1,1), the index pairs along the way are recorded in order to form the optimal curved path (i.e., the target curved path). In the formula, k represents the index of the point; K represents the total number of points. To avoid backtracking getting stuck in local equivalence cycles, a preference can be introduced for equivalence selection during backtracking (e.g., prioritizing diagonal directions), and the backtracking path can be post-processed to remove isolated mappings or brief jitters.

[0042] For example, the target curved path obtained through backtracking is extracted, which consists of a series of (i,j) coordinate pairs. The first coordinate pair (i_start, j_start) and the last coordinate pair (i_end, j_end) in the path are found. Here, j_start is the starting index of the matching segment in the standard test sequence, and j_end is the ending index. Based on these two indices, all data points from j_start to j_end are precisely extracted from the standard test sequence. This extracted subsequence is the sensor data segment that is finally precisely aligned with the multimodal event data.

[0043] According to the above implementation method, the device information record event timestamp in the multimodal event data is used as the reference anchor point. A reference time series is formed by expanding the time window, and the sensor test data stream is standardized to generate a standard test sequence. Then, a local cost matrix is ​​constructed based on the Euclidean distance between the time points in the reference time series and the standard test sequence, and the cumulative cost matrix is ​​calculated using a dynamic time warping algorithm under global path constraints and local monotonic constraints. Then, the target curved path that can optimally align the two types of time series is obtained by backtracking from the endpoint of the cumulative cost matrix along the minimum cost path. Finally, the start and end mapping points are extracted from the target curved path to determine the corresponding start and end times, and the standard test sequence is then truncated to obtain accurate sensor data segments. In this way, more stable and accurate time alignment can be achieved when there are differences in the time scale of multi-source heterogeneous data, so that the extracted sensor data segments are highly semantically matched with the device events, thereby improving the reliability and accuracy of subsequent feature analysis and report generation.

[0044] In one implementation, feature extraction is performed on multimodal event data, sensor data segments, and a preset report template to construct a heterogeneous graph. This includes: visualizing the sensor data segments to generate sensor data curve images, and encoding these images using a pre-trained visual encoder to obtain feature vectors for the sensor data nodes; encoding the field description text in the report template, the environmental description text in the multimodal event data, and the detection item description text using a pre-trained text encoder to obtain feature vectors for the report template field nodes, environmental description nodes, and detection item nodes, respectively; and extracting device features from the multimodal event data. Nameplate information is structured into numerical parameters to obtain feature vectors for device nodes. Based on each feature vector, cross-modal semantic similarity is calculated between sensor data nodes, environmental description nodes, detection item nodes, and device nodes and report template field nodes, respectively. Semantic association edges are established between node pairs with similarity greater than a preset threshold. Connection edges are established between sensor data nodes, environmental description nodes, detection item nodes, report template field nodes, and device nodes. Based on sensor data nodes, environmental description nodes, detection item nodes, report template field nodes, and device nodes, as well as semantic association edges and connection edges, a heterogeneous graph is constructed.

[0045] For example, the input one-dimensional time-series sensor data segment (e.g., an array containing 1024 data points) is first rendered into a two-dimensional, standard-sized grayscale or color image, i.e., a sensor data curve image, using a graphics library (such as Matplotlib). Next, the system calls a visual encoder pre-trained on large-scale data (e.g., the Vision Transformer in a Contrastive Language-Image Pre-training (CLIP) model or a ResNet), taking this generated curve image as input. After passing through multiple layers of convolution or attention computation within the encoder, the image is finally mapped and output as a high-dimensional floating-point array representing the core semantics of the image; this array is the feature vector of the sensor data nodes.

[0046] For example, the system sequentially retrieves the field description text from the report template, the environment description text from the multimodal event data, and the detection item description text. For each text segment, the system invokes a pre-trained text encoder (e.g., the Text Transformer in the CLIP model or a BERT (Bidirectional Encoder Representation) model). The text encoder first decomposes the text string into individual tokens, then processes these token sequences through a multi-layer attention network to capture their deep semantics, ultimately outputting a high-dimensional floating-point array with the same dimension as the visual feature vector, serving as the feature vector for the corresponding node.

[0047] For example, key information about the device nameplate is parsed from multimodal event data, typically in key-value pair form. The system then processes this information in a structured and numerical manner: for category information such as type and model, one-hot encoding is used to convert it into a binary vector; for numerical information such as rated voltage and rated capacity, normalization is performed. Finally, these processed values ​​are concatenated into a long vector, which serves as the feature vector for the device node.

[0048] In this example, assume the device nameplate information is {"Model":"SFZ11","Voltage Rating":"110kV","Capacity":"50MVA"}. The system may convert "SFZ11" into a vector [0, 0, 1], normalize "110kV" to 0.5, and normalize "50MVA" to 0.7. The final concatenated device node feature vector is [0, 0, 1, 0.5, 0.7]. This vector directly reflects the physical properties of the device. Here, MVA stands for Mega Volt Ampere (MVA).

[0049] For example, the system iterates through all node pairs that require association calculation, such as each sensor data node and each report template field node. For each pair of nodes, the system extracts their respective feature vectors (both from previous steps) and then calculates the cosine similarity between the two vectors. The cosine similarity is a value between -1 and 1; the closer the value is to 1, the more consistent the two vectors are in direction, i.e., the more semantically similar they are. The system compares the calculated similarity with a pre-set threshold (e.g., 0.8). If the similarity is greater than the threshold, an edge is established between the two nodes, and the similarity can be used as the weight of the edge; this edge is the semantic association edge.

[0050] For example, all the independent nodes (containing their respective feature vectors) and all the edges (containing edge types and weights) generated in the previous steps are integrated. The system creates a graph data structure, adding all nodes to the node set and all edges (semantic related edges and all other connecting edges) to the edge set. Since the graph contains multiple types of nodes and multiple types of edges, the final constructed structure is a heterogeneous information network that can comprehensively and multidimensionally describe the entire detection event, i.e., a heterogeneous graph.

[0051] According to the above implementation method, the sensor data segments are visualized and encoded to transform the raw time-series signals into image features that can be understood by the model. Then, a pre-trained text encoder is used to extract semantic vectors from report template fields, environmental description information, and detection item descriptions, and the device nameplate information is structured into parameter vectors, thereby achieving the expression of multimodal data in a unified vector space. Based on this, cross-modal semantic similarity is calculated between various nodes and report template field nodes to select semantically highly related node pairs and establish semantic association edges. Simultaneously, connection edges are established by combining the inherent structural relationships between sensors, environment, devices, and detection tasks, ultimately forming a heterogeneous graph containing multiple types of nodes and multiple relationship edges. This achieves comprehensive association modeling of multimodal data, enabling subsequent retrieval and inference based on the heterogeneous graph to simultaneously utilize time-series features, textual semantics, device attributes, and task structure information, thereby significantly improving the accuracy of association analysis and laying a solid data foundation for more accurate report generation.

[0052] In one implementation, establishing connection edges between sensor data nodes, environment description nodes, detection item nodes, report template field nodes, and device nodes includes: establishing edges between sensor data nodes and environment description nodes within the same time window to obtain time-series correlation edges; establishing membership edges connecting device nodes with their corresponding detection item nodes, sensor data nodes, and environment description nodes based on the inherent structure of multimodal event data; establishing logical edges connecting node pairs that conform to the power business rule base based on a preset power business rule base; wherein, a node pair consists of any two nodes from the sensor data node, environment description node, detection item node, report template field node, and device node; and determining connection edges based on time-series correlation edges, membership edges, and logical edges.

[0053] For example, first, all sensor data nodes and environment description nodes in the graph are traversed. For each node, its associated time information is extracted: the sensor data node corresponds to a time period (start time to end time), while the environment description node corresponds to a specific timestamp. Next, it is determined whether the timestamp of the environment description node falls within the time period (i.e., time window) of the sensor data node. If this time proximity condition is met, an undirected edge is established between the two nodes. This edge is defined as a temporally associated edge, indicating that the data from these two different modalities were collected at almost the same time.

[0054] For example, suppose a sensor data node A represents the partial discharge signal collected from 10:01:30 to 10:03:30. Simultaneously, there is an environmental description node B, which is the "clear discharge sound heard inside the equipment" recorded by the inspector at 10:02:15. Because the timestamp 10:02:15 falls perfectly within the time window [10:01:30, 10:03:30], a temporal correlation edge can be established between node A and node B. This edge objectively reflects the strong temporal correlation between the events "heard the discharge sound" and "collected the partial discharge signal".

[0055] For example, by analyzing the inherent structure of multimodal event data, the core entity (device node) of this detection task is identified. Then, all other relevant information nodes collected or involved in this task are identified (including all sensor data nodes, all environmental description nodes, and the detection item nodes being performed). Finally, starting from the device node representing the core entity, a directed edge is established to each of these related subordinate nodes. These directed edges clearly define the hierarchical relationship that "all this data belongs to this device," and they are collectively referred to as membership edges.

[0056] For example, suppose the object of a detection task is "#1 main transformer" (equipment node). In this task, a segment of "UHF partial discharge signal" (sensor data node) is collected, a "phase A bushing has oil contamination" is recorded (environment description node), and the detection item performed is "partial discharge live detection" (detection item node). In this way, three membership edges can be established: from "#1 main transformer" to "UHF partial discharge signal", from "#1 main transformer" to "phase A bushing has oil contamination", and from "#1 main transformer" to "partial discharge live detection".

[0057] For example, a pre-defined power business rule base containing knowledge and operation and maintenance procedures of the power industry is loaded. This rule base defines the logical relationships that may exist between different types of nodes, based on expert experience. All possible node pairs in the graph (consisting of any two nodes between sensor data nodes, environmental description nodes, detection item nodes, report template field nodes, and equipment nodes) are traversed and matched against the rules in the rule base one by one. If a node pair fully satisfies a rule, an edge is established between the two nodes; this edge is the logical edge, representing a priori association at the level of expert knowledge.

[0058] For example, the temporal association edges, membership edges, and logical edges obtained in the previous example are used together as connection edges.

[0059] According to the above implementation method, by establishing temporal association edges between sensor data nodes and environmental description nodes within the same time window, and utilizing the inherent structure of multimodal event data to construct membership edges between device nodes and their corresponding detection item nodes, sensor data nodes, and environmental description nodes, and establishing logical edges between any node pairs that conform to the rules based on the power business rule base, the temporal relationship, semantic attribution relationship, and business rule relationship are uniformly incorporated into the same heterogeneous graph structure. Finally, the connection edges are determined by integrating the above edges, enabling the graph structure to comprehensively express the multi-source information associations in the equipment detection process. This not only improves the accuracy of expressing the associations between nodes but also provides a more accurate, structured, and interpretable association foundation for subsequent graph-based retrieval, reasoning, and report generation, thereby enhancing the intelligent processing capability and output quality of the overall detection process.

[0060] In one implementation, in response to a detection request from a power device to be detected, a search is performed in a heterogeneous graph based on the device data in the detection request to obtain search results. This includes: using the power device to be detected as a target node, searching the heterogeneous graph for nodes and edges associated with the target node to construct a task subgraph, wherein the task subgraph includes each report template field node corresponding to a first node type and nodes corresponding to each second node type; aggregating and updating the task subgraph using a graph neural network to obtain updated feature representations for each node; based on the updated feature representations for each node, calculating the confidence scores between each node corresponding to each second node type and the report template field node for each report template field node, obtaining confidence scores for each second node type; for each second node type, selecting the node corresponding to the maximum value of each confidence score as a target matching node, obtaining target matching nodes; and using the structured data of each target matching node as the search results.

[0061] For example, suppose the detection request is for "Cable Terminal #2". The system locates the device node "Cable Terminal #2" in the global graph. Then, the system finds that this node is connected to a "Sensor Data - Partial Discharge Pulse" node, an "Environmental Description - White Powder on Surface" node, and a "Detection Item - Ultrasonic Detection" node via membership edges. The graph also contains several preset report template field nodes such as "Partial Discharge Amplitude" and "Appearance Status". The system copies all these nodes and all edges between them (semantic edges, logical edges, etc.) to form a task subgraph containing only information related to this task.

[0062] It should be noted that the first node type is the report template field node; the second node type is all other nodes besides the report template field node, such as sensor data node, environment description node, detection item node, and device node.

[0063] For example, the task subgraph and the initial feature representation of each node are input into a pre-trained graph neural network (GNN) model. The GNN model performs multiple rounds (layers) of message passing. In each round, for each node in the graph, the model collects the feature representations of all its neighboring nodes. Due to the heterogeneous graph, the model assigns different learning weights to the information from different neighbors based on the type of the connection edges (e.g., semantic edges, logical edges). The model then fuses the weighted aggregated neighbor information with the node's own current feature representation (e.g., through a neural network layer) to compute a new feature representation for the node in the next round, which contains richer contextual information. This process is repeated multiple times, so that the final feature representation of each node indirectly incorporates the structural and semantic information of the entire subgraph.

[0064] For example, after the GNN processing is complete, the system iterates through each report template field node in the task subgraph. For a selected report template field node, the system again iterates through all nodes belonging to the "second node type" (i.e., all potential data source nodes, such as sensor data nodes, environmental description nodes, etc.). For each such data source node, the system extracts its feature representation updated by the GNN, as well as the updated feature representation of the current report template field node. Then, the system calculates the cosine similarity between these two high-dimensional vectors or calculates a score using a learnable scoring function. This calculated similarity score is defined as the link confidence between the data source node and the report template field node.

[0065] For example, all nodes are first grouped according to their type (e.g., "sensor data node," "environmental description node"). Then, for each report template field node, the system independently compares and selects within each group. For instance, for the report field "partial discharge amplitude," the system finds all nodes with calculated confidence scores within the "sensor data node" group and selects the node with the highest score as the target matching node for that category. Similarly, the system finds the node with the highest score for "partial discharge amplitude" within the "environmental description node" group. This process is repeated for all report fields.

[0066] For example, based on all the target matching nodes obtained in the previous example, for each target matching node, the system accesses the data stored internally or the raw data associated with it. The system extracts this data and performs structured processing. All of this structured data extracted from the target matching nodes and prepared to populate the report constitutes the final search results.

[0067] For example, [{"Data Source Type":"Sensor","Value":"150.5 pC","Unit":"pC"}, {"Data Source Type":"Text","Content":"There is a crisp, intermittent discharge sound inside the device"}]. These two structured data objects, along with the data matched for other report fields, together constitute the final result of this search, ready for use in subsequent report generation.

[0068] According to the above implementation method, taking the power equipment to be detected as the target node, its associated nodes and edges are retrieved in a heterogeneous graph, and a task subgraph is constructed based on this. This allows the multi-source data related to the equipment and the report template fields to be jointly modeled in the same context. A graph neural network is used to aggregate and update the features of the task subgraph, ensuring that the representation of each node fully integrates neighborhood information, thereby more accurately reflecting its business semantics and relationships. Subsequently, based on the updated node features, a confidence score is calculated, and the node with the highest confidence score in each node category is selected as the target matching node. The structured data of these target matching nodes is used as the final retrieval result. This ensures that each template field obtains the most representative and relevant data source, significantly improving the reliability of the retrieval results and the accuracy of the generated report content.

[0069] In one implementation, structured input data is constructed based on the search results and a preset prompt template, including: classifying the structured data in the search results into sensor data and non-sensor data according to data type; generating at least one sensor data curve image for the sensor data to obtain image data; extracting text from the non-sensor data to obtain text data; and filling the image data and text data into the image placeholder and text placeholder in the prompt template, respectively, to obtain structured input data.

[0070] For example, the search results are parsed, and metadata such as field names, numerical types, unit labels, and data source attributes are read for each piece of parsed data. Each piece of data is then classified according to preset judgment rules. For instance, if a field contains features such as time series, sampling frequency, sensor ID (identification), or physical quantity (e.g., temperature, partial discharge, current, voltage), it is classified as sensor data; if the field content is text description, enumerated labels, inspection records, event types, or switch operation instructions, it is classified as non-sensor data. After classification, the data is stored as a sensor data list and a non-sensor data list, respectively, while retaining the mapping information between fields and search nodes for subsequent template filling.

[0071] For example, the sensor data list is traversed, and formatting is performed on each time series data point, including sorting by time order, removing invalid points, and smoothing or limiting the data. A suitable visualization method (such as a line chart, area chart, or multi-channel overlay plot) is selected, and image attributes are set, such as image resolution, axis units, title, and label format. A plotting module (such as Matplotlib) is invoked to generate an image object, and the image is saved as a Portable Network Graphics (PNG) or Joint Photographic Experts Group (JPG / JPEG) file. Finally, the generated image path or binary data is added to the image data list, forming the "image data" that will be used to fill the prompt template.

[0072] For example, the non-sensor data list is traversed, and each non-sensor field is semantically parsed and formatted. If the field is a natural language description, it is stored directly as a text entry. If the field contains structured key-value pairs, it is converted into a standardized sentence, such as "Equipment Model: XM-220kV" is converted into "Equipment Model is XM-220kV". If the text field contains redundant symbols, invalid placeholders, or non-standard expressions, it is cleaned and optimized, such as completing units and standardizing expression formats. Finally, all cleaned text entries are organized into a text dataset according to field priority or template requirements, so that they can be directly filled into the text placeholder positions of the prompt template later.

[0073] For example, a preset prompt template is read, and the image and text placeholders in the template are parsed into replaceable nodes. Images from the image data list are replaced sequentially or according to field correspondences at the corresponding image placeholder positions, while text entries from the text data list are filled into the corresponding text placeholders. If some placeholders have no corresponding data, default values ​​or empty fields are inserted according to the template rules. Finally, a complete structured input data object containing image references, text content, and field labels is output for subsequent generation of detection reports using the visual language large model.

[0074] According to the above implementation method, the structured data in the retrieval results is first divided into sensor data and non-sensor data based on data type. For the sensor data, corresponding sensor data curve images are generated to form image data suitable for visual model understanding. Simultaneously, the non-sensor data is extracted into text data that can be directly used for semantic parsing. Then, the image data and text data are respectively filled into the corresponding image placeholders and text placeholders in the prompt template to construct structured input data with standardized content organization and clear semantic structure. In this way, two types of key information—images and text—can be provided to the large visual language model in a unified format, making it easier for the model to understand the semantic relationships between different data, improving the contextual consistency of the generated content and the accuracy of inference.

[0075] In one implementation, after generating a detection report for the power equipment to be detected by using structured input data as input to the visual language big language model, the method further includes: correcting the detection report to obtain a corrected detection report; comparing the detection report with the corrected detection report to obtain the corrected content, and combining the corrected content with the structured input data to obtain an optimized training dataset; and optimizing the training of the visual language big language model based on the optimized training dataset.

[0076] For example, the test report automatically generated by the Visual Language Model (VLM) is first presented to a qualified electrical engineer or reviewer through an interactive front-end interface. This interface provides functions such as text editing, numerical modification, and option reselection, allowing reviewers to directly modify, delete, or add content to the report, just like editing a regular document, until the report content is completely accurate and professional. After the reviewer completes all modifications and clicks "Submit," the system backend receives and saves this manually reviewed and finalized report as the revised test report.

[0077] For example, existing text difference algorithms (such as operation-transformation-based algorithms or sequence alignment algorithms) are used to perform a fine-grained line-by-line and word-by-word comparison of the content of the detection report and the corrected detection report. The algorithm accurately identifies the deleted, inserted, and replaced text segments. The system structures these differing segments, along with their position information in the report (such as chapters and paragraphs), operation type (add, delete, modify), and contextual information, into a series of "correction operation" records. This set of "correction operations" is the "corrected content."

[0078] For example, each sample in the optimized training dataset is input into the current VLM. The model generates its own predicted output based on the input, and then calculates the loss between the predicted output and the "correct output" (i.e., corrected content) provided by the expert in the sample. Finally, through backpropagation, the model's internal parameters (e.g., weights) are slightly adjusted based on this loss value. In this way, by traversing the entire optimized training dataset multiple times, the model learns the expert's diagnostic logic and language style, making the content quality of the model's output detection report more in line with the expert's requirements.

[0079] According to the above implementation method, the corrected content is combined with the corresponding structured input data to form an optimized training dataset that can be directly used for model learning; the visual language large language model is continuously optimized and trained based on this optimized training dataset. In this way, the model can not only absorb the high-quality knowledge brought by expert corrections, but also continuously reduce the probability of generating errors or omitting professional information, thereby significantly improving the accuracy of subsequent automatically generated detection reports and achieving the effect of continuous model evolution with use.

[0080] Figure 2 This is a structural block diagram of a power equipment testing report generation device according to an embodiment of the present invention.

[0081] like Figure 2 As shown, the power equipment inspection report generation device may include: The time alignment module 510 is used to perform time alignment between the multimodal event data of the power equipment and the sensor test data stream to obtain the sensor data segment corresponding to the power equipment. The graph construction module 520 is used to extract features from the multimodal event data, the sensor data segments, and the preset report template to construct a heterogeneous graph. The retrieval module 530 is used to respond to a detection request from the power equipment to be detected, and to perform a retrieval in the heterogeneous graph based on the data of the equipment to be detected in the detection request, so as to obtain retrieval results; The structured input data construction module 540 is used to construct structured input data based on the search results and the preset prompt template; The inspection report generation module 550 is used to take the structured input data as input to the visual language large language model and generate an inspection report for the power equipment to be inspected. In one embodiment, the time alignment module includes: The time window expansion unit is used to expand the time window of the device information record event in the multimodal event data as the reference anchor point to obtain a reference time series. A standardization processing unit is used to standardize the sensor test data stream to obtain a standard test sequence. The local cost matrix construction unit is used to construct a local cost matrix based on the Euclidean distance between each first time point in the reference time series and each second time point in the standard test sequence; The cumulative cost matrix calculation unit is used to calculate the local cost matrix according to the dynamic time warping algorithm and by setting global path constraints and local monotonic constraints, so as to obtain the cumulative cost matrix. The backtracking unit is used to backtrack from the end of the cumulative cost matrix along the direction of minimum cumulative cost to obtain the target curved path connecting the reference time series and the standard test sequence; The interception unit is used to extract the start and end mapping points in the target curved path to obtain the start and end times, and based on the start and end times, to intercept the standard test sequence to obtain the sensor data segment.

[0082] In one implementation, the graph construction module includes: The first encoding unit is used to perform visualization processing on the sensor data segment, generate a sensor data curve image, and encode the sensor data curve image through a pre-trained visual encoder to obtain the feature vector of the sensor data node. The second encoding unit is used to encode the field description text in the report template, the environment description text in the multimodal event data, and the detection item description text respectively using a pre-trained text encoder, so as to obtain the feature vectors of the report template field nodes, environment description nodes, and detection item nodes respectively. The device node unit is used to extract device nameplate information from the multimodal event data and structure the device nameplate information into numerical parameters to obtain the feature vector of the device node. The semantic association edge establishment unit is used to calculate the cross-modal semantic similarity between the sensor data node, the environment description node, the detection item node and the device node and the report template field node respectively based on each of the feature vectors, and to establish semantic association edges between node pairs with similarity greater than a preset threshold. The connection edge establishment unit is used to establish connection edges between the sensor data node, the environment description node, the detection item node, the report template field node, and the device node; The heterogeneous graph construction unit is used to construct the heterogeneous graph based on the sensor data node, the environment description node, the detection item node, the report template field node, the device node, the semantic association edge, and the connection edge.

[0083] In one embodiment, the connection edge establishment unit includes: A temporal correlation edge establishment subunit is used to establish edges between the sensor data nodes and the environment description nodes within the same time window to obtain temporal correlation edges. Membership edge establishment sub-units are used to establish membership edges connecting the device node with the detection item node, sensor data node and environment description node corresponding to the device node based on the inherent structure of the multimodal event data; A logical edge establishment subunit is used to establish logical edges between node pairs that conform to the preset power business rule base based on the preset power business rule base; wherein, the node pair consists of any two nodes between the sensor data node, the environment description node, the detection item node, the report template field node and the device node; The connection edge determination subunit is used to determine the connection edge based on the temporal association edge, the membership edge, and the logical edge.

[0084] In one embodiment, the retrieval module includes: The task subgraph construction unit is used to take the power equipment to be detected as the target node, and search for nodes and edges that are associated with the target node in the heterogeneous graph to construct a task subgraph, wherein the task subgraph includes each report template field node corresponding to the first node type and each node corresponding to the second node type. The aggregation and update unit is used to aggregate and update the task subgraph through a graph neural network to obtain the updated feature representation of each node; The confidence calculation unit is used to calculate the confidence between the node corresponding to each second node type and the report template field node based on the updated feature representation of each node, and to obtain the confidence scores under each second node type. The first unit is used to select the node corresponding to the maximum value in each confidence score for each of the second node types as the target matching node, thereby obtaining each target matching node. The second unit is used to take the structured data of each target matching node as the retrieval result.

[0085] In one implementation, the structured input data construction module includes: The classification unit is used to classify the structured data in the search results into sensor data and non-sensor data according to the data type. An image data generation unit is used to generate at least one sensor data curve image based on the sensor data, thereby obtaining image data; A text extraction unit is used to extract text from the non-sensor data to obtain text data; The filling unit is used to fill the image data and the text data into the image placeholder and text placeholder in the prompt template, respectively, to obtain the structured input data.

[0086] In one embodiment, after the test report generation module, the device further includes: A correction unit is used to correct the test report to obtain a corrected test report; The comparison unit is used to compare the detection report with the corrected detection report to obtain the correction content, and combine the correction content with the structured input data to obtain the optimized training dataset; An optimized training unit is used to optimize the training of the visual language large language model based on the optimized training dataset.

[0087] The specific functions and examples of each module and submodule of the system in this embodiment of the invention can be found in the relevant descriptions of the corresponding steps in the above method embodiments, and will not be repeated here.

[0088] The acquisition, storage, and application of user personal information involved in the technical solution of this invention all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0089] This invention also provides a power equipment testing report generation system, comprising: At least one processor; and a memory communicatively connected to said at least one processor; The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the method described in any one of the embodiments of the present invention.

[0090] The beneficial effects of the power equipment testing report generation system of this invention are equivalent to the beneficial effects of the power equipment testing report generation method described above, and will not be repeated here.

[0091] This invention also provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to perform the method described in any one of the embodiments of this invention.

[0092] The beneficial effects of the storage medium of the present invention are equivalent to the beneficial effects of the above-described method for generating power equipment inspection reports, and will not be elaborated upon here.

[0093] Figure 3 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present invention is shown. Electronic device 800 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic device 800 may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0094] like Figure 3 As shown, the electronic device 800 includes a computing unit 801, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. The RAM 803 may also store various programs and data required for the operation of the electronic device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0095] Multiple components in electronic device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of displays, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows electronic device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0096] The computing unit 801 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as the power equipment inspection report generation method. For example, in some embodiments, the power equipment inspection report generation method can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the power equipment inspection report generation method described above can be performed. Alternatively, in other embodiments, the computing unit 801 can be configured to perform the power equipment inspection report generation method by any other suitable means (e.g., by means of firmware).

[0097] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0098] The program code used to implement the methods of the present invention can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0099] In the context of this invention, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0100] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0101] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0102] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0103] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this invention can be achieved, and this is not limited herein.

[0104] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for generating a power equipment inspection report, characterized in that, include: Time alignment is performed between the multimodal event data of the power equipment and the sensor test data stream to obtain the sensor data segment corresponding to the power equipment; Feature extraction is performed on the multimodal event data, the sensor data segments, and the preset report template to construct a heterogeneous graph; In response to a testing request from a power device to be tested, a search is performed on the heterogeneous graph based on the data of the device to be tested in the testing request to obtain the search results. Based on the search results and the preset prompt template, structured input data is constructed; The structured input data is used as input to a visual language large language model to generate a detection report for the power equipment to be detected.

2. The method according to claim 1, characterized in that, The step of time-aligning the multimodal event data of the power equipment with the sensor test data stream to obtain the sensor data segment corresponding to the power equipment includes: Using the timestamp of the device information record event in the multimodal event data as the reference anchor point, and expanding the reference anchor point by a time window, a reference time series is obtained; The sensor test data stream is standardized to obtain a standard test sequence; A local cost matrix is ​​constructed based on the Euclidean distance between each first time point in the reference time series and each second time point in the standard test series; The local cost matrix is ​​calculated based on the dynamic time warping algorithm and by setting global path constraints and local monotonic constraints, to obtain the cumulative cost matrix; Starting from the end of the cumulative cost matrix, backtracking is performed along the direction of minimum cumulative cost to obtain the target curved path connecting the reference time series and the standard test sequence; The start and end mapping points in the target curved path are extracted to obtain the start and end times. Based on the start and end times, the standard test sequence is truncated to obtain the sensor data segment.

3. The method according to claim 1, characterized in that, The step of extracting features from the multimodal event data, the sensor data segments, and the preset report template to construct a heterogeneous graph includes: The sensor data segment is visualized to generate a sensor data curve image, and the sensor data curve image is encoded by a pre-trained visual encoder to obtain the feature vector of the sensor data node. The pre-trained text encoder encodes the field description text in the report template, the environment description text in the multimodal event data, and the detection item description text, respectively, to obtain the feature vectors of the report template field nodes, environment description nodes, and detection item nodes. The device nameplate information is extracted from the multimodal event data, and the device nameplate information is structured into numerical parameters to obtain the feature vector of the device node; Based on each of the feature vectors, calculate the cross-modal semantic similarity between the sensor data node, the environment description node, the detection item node, and the device node and the report template field node, and establish semantic association edges between node pairs with similarity greater than a preset threshold. Establish connection edges between the sensor data node, the environment description node, the detection item node, the report template field node, and the device node; The heterogeneous graph is constructed based on the sensor data node, the environment description node, the detection item node, the report template field node, the device node, the semantic association edge, and the connection edge.

4. The method according to claim 3, characterized in that, The process of establishing connections between the sensor data node, the environment description node, the detection item node, the report template field node, and the device node includes: Establish edges between the sensor data nodes and the environment description nodes within the same time window to obtain time-series associated edges; Based on the inherent structure of the multimodal event data, establish membership edges connecting the device node with the corresponding detection item node, sensor data node, and environmental description node; Based on a preset power business rule base, logical edges are established between node pairs that conform to the power business rule base; wherein, the node pair consists of any two nodes between the sensor data node, the environment description node, the detection item node, the report template field node, and the device node; The connecting edge is determined based on the temporal correlation edge, the membership edge, and the logical edge.

5. The method according to claim 1, characterized in that, In response to a detection request from the power equipment to be tested, the method performs a search on the heterogeneous graph based on the data of the equipment to be tested in the detection request to obtain search results, including: The power equipment to be tested is taken as the target node. Nodes and edges associated with the target node are retrieved in the heterogeneous graph to construct a task subgraph. The task subgraph includes each report template field node corresponding to the first node type and each node corresponding to the second node type. The task subgraph is aggregated and updated using a graph neural network to obtain the updated feature representation of each node. Based on the updated feature representation of each node, for each report template field node, the confidence scores between the nodes corresponding to each second node type and the report template field nodes are calculated to obtain each confidence score under each second node type. For each of the second node types, the node corresponding to the maximum value in each confidence score is taken as the target matching node, thus obtaining each target matching node; The structured data of each target matching node is used as the search result.

6. The method according to claim 1, characterized in that, The step of constructing structured input data based on the search results and preset prompt templates includes: The structured data in the search results are classified into sensor data and non-sensor data according to data type. Based on the sensor data, at least one sensor data curve image is generated to obtain image data; Text data is obtained by extracting text from the non-sensor data; The image data and text data are respectively filled into the image placeholder and text placeholder in the prompt template to obtain the structured input data.

7. The method according to claim 1, characterized in that, After using the structured input data as input to the visual language large language model to generate a detection report for the power equipment to be detected, the method further includes: The test report is corrected to obtain a revised test report; The detection report is compared with the corrected detection report to obtain the correction content, and the correction content is combined with the structured input data to obtain the optimized training dataset; Based on the optimized training dataset, the visual language large language model is optimized and trained.

8. A device for generating power equipment inspection reports, characterized in that, include: The time alignment module is used to perform time alignment between the multimodal event data of the power equipment and the sensor test data stream to obtain the sensor data segment corresponding to the power equipment. The graph construction module is used to extract features from the multimodal event data, the sensor data segments, and the preset report template to construct a heterogeneous graph. The retrieval module is used to respond to a detection request from the power equipment to be detected, and to perform a retrieval in the heterogeneous graph based on the data of the equipment to be detected in the detection request, so as to obtain retrieval results; A structured input data construction module is used to construct structured input data based on the search results and preset prompt templates; The inspection report generation module is used to take the structured input data as input to the visual language large language model and generate an inspection report for the power equipment to be inspected.

9. A power equipment testing report generation system, characterized in that, include: At least one processor; and a memory communicatively connected to the at least one processor; The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-7.

Images