Intelligent inspection method and system for power equipment based on augmented reality

Abstract

The invention belongs to the technical field of augmented reality and nuclear power station electric power inspection, and relates to an intelligent inspection method and system for electric power equipment based on augmented reality, and the method comprises the steps: collecting multi-modal data, such as images, postures and radiation, of the electric power equipment; identifying an abnormal region based on the target detection neural network and generating space-time risk data in combination with the operation data; constructing a three-dimensional relationship of the equipment through a spatial positioning algorithm, and converting the risk data into inspection information; performing alignment display in the augmented reality scene and guiding inspection operation; collecting inspection data, associating the inspection data with the risk information, uploading the data to a remote terminal, generating a label, returning and superposing the label, and finally forming a structured inspection record. Through multi-modal data unified processing, spatial mapping and augmented reality superposition, spatial correspondence of the inspection state along with the visual angle is realized, and the method is suitable for equipment-intensive and dynamic inspection scenes, and has the capabilities of centralized positioning, continuous expression and association consistency.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of augmented reality and nuclear power plant power inspection, and particularly relates to an intelligent power equipment inspection method and system based on augmented reality. BACKGROUND

[0002] In the prior art, power equipment on-site inspection usually relies on operation and maintenance personnel to confirm the equipment operating state through image acquisition, instrument reading comparison and manual recording, etc., to meet the equipment safety management requirements. However, the correlation between image recording and operating parameters is low, and the state presentation is mostly limited to the static information level. The inspection results are highly dependent on personnel experience, and the state expression is significantly delayed when the operating conditions change rapidly. The information feedback rhythm is relatively lagging.

[0003] With the gradual application of augmented reality technology in the power field, some systems attempt to superimpose equipment information in the visual scene to assist in inspection, but the source of the superimposed data is single, the continuity and positioning consistency of the operating state expression are weak, the abnormal area is unstable in the visual angle change, and the correlation accuracy between multi-source data is low, which makes the on-site identification of potential risks decline, and cannot effectively match the dynamic safety management needs of key equipment.

[0004] In high-voltage equipment, high-temperature components or complex arrangement scenes, the inspection environment has large light fluctuation, the equipment is densely arranged, the image recognition result is prone to spatial indication deviation, and the inspection record is mostly stored discretely, with low data structuring degree, making it difficult to form a time-evolving state analysis link. The utilization efficiency of inspection data in fault review and experience sedimentation process is relatively low.

[0005] Therefore, the prior art often has problems such as low inspection state presentation accuracy, weak dynamic risk expression stability, and low inspection data correlation structuring degree. This is the deficiency of the prior art.

[0006] Therefore, the present application provides an intelligent power equipment inspection method and system based on augmented reality to solve the above-mentioned defects in the prior art, which is very necessary. SUMMARY

[0007] The purpose of the present application is to provide an intelligent power equipment inspection method and system based on augmented reality to solve the above-mentioned technical problems.

[0008] To achieve the above-mentioned purpose, the present application provides the following technical solution: In a first aspect, the present application provides an intelligent power equipment inspection method based on augmented reality, comprising: Collect multi-modal data of the power equipment, the multi-modal data comprising image data, pose data and radiation data; Based on the multi-modal data, identify abnormal areas of the power equipment by using a target detection neural network, generate a defect area identification result, and generate spatio-temporal risk data by a filtering algorithm in combination with power operation data; Based on the pose data and a spatial positioning algorithm, establish a three-dimensional spatial relationship of the power equipment, and convert the spatio-temporal risk data into inspection information based on the three-dimensional spatial relationship; Display the inspection information and the power equipment in spatial alignment in an augmented reality view, and guide an inspection personnel to perform an inspection operation on the power equipment based on the inspection information; Collect inspection process data, and send the inspection process data, the spatio-temporal risk data and the inspection information to a remote collaboration terminal through a communication network, so as to generate labeled information for the power equipment by the remote collaboration terminal; Receive the labeled information, superimpose the labeled information on the corresponding spatial position of the power equipment in the augmented reality view, and generate a structured inspection record based on the inspection information.

[0009] Preferably, based on the multi-modal data, the step of identifying abnormal areas of the power equipment by using a target detection neural network comprises: Jointly construct features of the image data and the radiation data, and combine the radiation data and the image data into multi-channel input data according to a time correspondence relationship; Input the multi-channel input data into the target detection neural network and identify the abnormal areas.

[0010] Preferably, after generating the defect area identification result, perform cross-frame spatial position stabilization processing on the defect area identification result, merge spatially adjacent positions of the defect area identification result belonging to consecutive frames, and form a stable defect area.

[0011] Preferably, based on the pose data and the spatial positioning algorithm, the step of establishing a three-dimensional spatial relationship of the power equipment and converting the spatio-temporal risk data into inspection information based on the three-dimensional spatial relationship comprises: Determine a spatial orientation of the augmented reality view according to the pose data, and determine a positional relationship of the power equipment in the three-dimensional space by using the spatial positioning algorithm; Perform spatial positioning processing on the spatio-temporal risk data based on the spatial orientation and the positional relationship of the power equipment in the three-dimensional space, and convert the spatio-temporal risk data into the inspection information.

[0012] Preferably, when establishing the three-dimensional spatial relationship of the power equipment, perform continuity verification on a pose sequence constituted by the pose data, and when a spatial change of adjacent poses in the pose sequence exceeds a preset change threshold, re-establish the three-dimensional spatial relationship of the power equipment.

[0013] As preferred, the step of guiding the inspection personnel to perform the inspection operation on the power equipment based on the inspection information comprises: determining an inspection level based on the corresponding abnormal type in the inspection information, and determining an execution order of the inspection operation according to the inspection level, and the inspection personnel performing the inspection operation on the power equipment according to the execution order.

[0014] As preferred, the step of sending the inspection process data, the spatio-temporal risk data and the inspection information to the remote collaboration terminal through the communication network comprises: recording the collection time of the inspection process data and generating a time index; selecting records corresponding to the time from the spatio-temporal risk data and the inspection information based on the time index; determining the corresponding position of the power equipment in space according to the observation posture of the inspection process data at the collection time, and matching the selected records with the inspection process data based on the corresponding position in space to form associated data in sequence of the time index; sending the associated data to the remote collaboration terminal through the communication network in the order corresponding to the time index.

[0015] In a second aspect, the embodiments of the present application further provide an intelligent power equipment inspection system based on augmented reality, comprising: a collection module configured to collect multi-modal data of the power equipment, the multi-modal data comprising image data, posture data and radiation data; an identification module configured to identify abnormal areas of the power equipment based on the multi-modal data using a target detection neural network, generate a defect area identification result, and generate spatio-temporal risk data by a filtering algorithm in combination with power operation data; a modeling module configured to establish a three-dimensional spatial relationship of the power equipment based on the posture data and a spatial positioning algorithm, and convert the spatio-temporal risk data into inspection information based on the three-dimensional spatial relationship; a display module configured to display the inspection information and the power equipment in space alignment in an augmented reality view, and guide an inspection personnel to perform an inspection operation on the power equipment based on the inspection information; a communication module configured to collect inspection process data, and send the inspection process data, the spatio-temporal risk data and the inspection information to a remote collaboration terminal through a communication network, and generate labeled information for the power equipment by the remote collaboration terminal; a superimposition module configured to receive the labeled information, superimpose the labeled information at the corresponding spatial position of the power equipment in the augmented reality view, and generate a structured inspection record based on the inspection information.

[0016] As preferred, the identification module comprises: The feature construction unit is configured to jointly construct features of the image data and the radiation data, and combine the radiation data and the image data into multi-channel input data according to a time-of-acquisition correspondence relationship. The anomaly detection unit is configured to input the multi-channel input data into a target detection neural network and identify an abnormal region.

[0017] Preferably, the modeling module comprises: The spatial orientation unit is configured to determine a spatial orientation of the augmented reality view according to the pose data; The spatial positioning unit is configured to determine a positional relationship of the power equipment in the three-dimensional space by using a spatial positioning algorithm; The mapping processing unit is configured to perform spatial positioning processing on the spatio-temporal risk data based on the spatial orientation and the positional relationship of the power equipment in the three-dimensional space, and convert the spatio-temporal risk data into inspection information.

[0018] As can be seen from the above technical solutions, the present application has the following advantages: In the power equipment intelligent inspection method and system based on augmented reality provided by the present application, the image data, pose data and radiation data of the power equipment are uniformly collected, the abnormal region is identified based on multi-modal data, the spatio-temporal risk data is generated in combination with power operation data, the spatio-temporal risk data is mapped according to the structural relationship of the equipment in the three-dimensional space and is superimposed and presented in the augmented reality view, the inspection process data is associated and recorded according to the observation time and the spatial position, and is sent to a remote collaboration terminal through a communication network, the remote collaboration terminal returns the labeled data, which is superimposed and presented in the augmented reality view corresponding to the actual position of the equipment, and a structured inspection record is generated, the inspection state is kept corresponding to the equipment entity in space with the change of the viewing angle during the inspection process, the abnormal region positioning is concentrated, the operation state expression is continuous, and the inspection data is highly structured in the time and space dimensions in the dense and dynamic inspection scene of the equipment, thereby meeting the needs of high precision of the inspection state presentation, strong stability of the dynamic risk expression and high degree of association and structuring of the inspection data in the background technology.

[0019] In addition, the design principle of the present application is reliable, the structure is simple, and it has a very wide application prospect.

[0020] Therefore, compared with the prior art, the present application has outstanding substantial characteristics and significant progress, and the beneficial effects of its implementation are also obvious. BRIEF DESCRIPTION OF DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the present application, the drawings required to be used in the description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0022] Figure 1 is a flow chart of an intelligent power equipment inspection method based on augmented reality provided by the present application; Figure 2 is a principle block diagram of an intelligent power equipment inspection system based on augmented reality provided by the present application.

[0023] Among them, 1. Acquisition module, 2. Recognition module, 3. Modeling module, 4. Display module, 5. Communication module, 6. Superimposition module. DETAILED DESCRIPTION

[0024] Various embodiments of the present disclosure are described more fully below with reference to the accompanying drawings. The present disclosure can have various embodiments, and adjustments and changes can be made therein. However, it should be understood that there is no intention to limit various embodiments of the present disclosure to the specific embodiments disclosed herein, but the present disclosure should be understood to cover all adjustments, equivalents and / or alternatives falling within the spirit and scope of various embodiments of the present disclosure.

[0025] Hereinafter, the term "include" or "may include" used in various embodiments of the present disclosure indicates the presence of the disclosed functions, operations or elements, and does not limit the addition of one or more functions, operations or elements. In addition, as used in various embodiments of the present disclosure, the terms "include", "have" and their synonyms only mean to indicate the presence of a specific feature, number, step, operation, element, component or combination of the foregoing, and should not be understood as first excluding the presence or addition of one or more other features, numbers, steps, operations, elements, components or combinations of the foregoing.

[0026] It also needs to be explained that, in various embodiments of the present disclosure, in order to facilitate the description of data objects, coordinate quantities and state quantities, some quantities are represented in symbolic form. The specific meaning of the symbol is limited by the context in which it appears, and when the same symbol is repeatedly used in different embodiments, different processing stages or different data sources, it does not necessarily mean that the symbol has exactly the same physical meaning or data source in all cases. For cases where there may be a coincidence of meaning or ambiguity, the definition given in the current paragraph or the meaning last explicitly stated before the current paragraph should be understood, and the use in other contexts appearing earlier is not restricted. Unless otherwise explicitly stated, the symbols used in various embodiments of the present disclosure are only used to assist in describing the logical relationship and data flow of the technical solution, and do not limit the naming method of the variables in the actual implementation.

[0027] For the convenience of clearly describing the technical solutions of the embodiments of the present application, the following briefly introduces some terms and technologies involved in the embodiments of the present application. It should be understood by those skilled in the art that the description of these terms is only for the convenience of understanding the present application and does not constitute a limitation on the protection scope of the present application. 1、Augmented Reality Augmented Reality is usually abbreviated as AR (Augmented Reality), which superimposes virtual information on the real scene observed by the user through camera acquisition, spatial positioning and graphics rendering technology, so that the digital content and the physical object maintain a spatial correspondence, and is suitable for scenes such as on-site inspection, remote collaboration and equipment maintenance.

[0028] 2、Object Detection Neural Network The object detection neural network is used to identify objects in the input image and output the location and classification results of the objects. This type of model is composed of feature extraction, feature fusion and prediction output, and can adopt a single-stage detection structure such as YOLO (You Only Look Once) or a two-stage detection structure such as Faster R-CNN (Region-based Convolutional Neural Network).

[0029] 3、Filtering Algorithm The filtering algorithm is used for smoothing and estimating the noisy observation data, and can use methods such as Kalman Filter to achieve recursive calculation through state prediction and observation update. The filtering algorithm can be used for dynamic trend inference and time series noise suppression.

[0030] 4、Spatial Positioning Algorithm The spatial positioning algorithm is used to determine the three-dimensional position relationship of the observation device relative to the scene target, and can use methods such as SLAM (Simultaneous Localization and Mapping) or ICP (Iterative Closest Point) to obtain the position relationship through feature matching or depth geometry alignment.

[0031] 5、Communication Network The communication network is used for data transmission between the terminal and the remote system, and can include 5G (5th Generation Mobile Network), Wi-Fi (Wireless Fidelity) and Bluetooth (Bluetooth, a short-range wireless communication technology), etc. for transmitting image data, state information and control instructions.

[0032] In view of the problems of low presentation accuracy of inspection state, weak stability of dynamic risk expression, and low structured degree of inspection data correlation in existing power equipment inspection, it is difficult to meet the demand for high accuracy, real-time and structured data processing in intelligent power equipment inspection. The application discloses a power equipment intelligent inspection method and system based on augmented reality. By introducing multi-modal data acquisition, target detection neural network, three-dimensional space relationship modeling and spatio-temporal risk data processing technology, high-precision display of inspection information, stable expression of dynamic risk can be realized, and data correlation can be improved through structured inspection records, thereby effectively improving inspection efficiency and accuracy, reducing human operation errors, improving traceability and collaboration of the inspection process, and further improving the intelligent level of power equipment inspection.

[0033] The technical solutions in the embodiments of the application will be described clearly and completely below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the application.

[0034] As shown in Figure 1 The power equipment intelligent inspection method based on augmented reality provided by the embodiment includes: Step S1: collecting multi-modal data of the power equipment, the multi-modal data including image data, attitude data and radiation data; Step S2: based on the multi-modal data, identifying the abnormal area of the power equipment by using the target detection neural network, generating the defect area identification result, and generating the spatio-temporal risk data by a filtering algorithm in combination with the power operation data; Step S3: establishing the three-dimensional space relationship of the power equipment based on the attitude data and the space positioning algorithm, and converting the spatio-temporal risk data into the inspection information based on the three-dimensional space relationship; Step S4: displaying the inspection information and the power equipment in space alignment in the augmented reality view, and guiding the inspection personnel to perform the inspection operation on the power equipment based on the inspection information; Step S5: collecting the inspection process data, and sending the inspection process data, the spatio-temporal risk data and the inspection information to the remote collaboration terminal through a communication network, so as to generate the labeling information for the power equipment by the remote collaboration terminal; Step S6: receiving the labeling information, superimposing and displaying the labeling information at the corresponding space position of the power equipment in the augmented reality view, and generating the structured inspection record based on the inspection information.

[0035] The embodiment identifies an abnormal area based on multi-source data such as power equipment images, postures, and radiation, and generates a space-time risk expression in combination with operating parameters, so that the inspection result has continuity and time sequence reference value; a three-dimensional corresponding relationship between the actual equipment and the risk information is established by a spatial positioning algorithm, so that the risk indication remains stable display under a mobile perspective, and is suitable for a dense equipment environment; relying on an augmented reality view to realize inspection guidance and label superposition, risk identification, position confirmation, and recording in the inspection operation process form integrated expression, reducing the workload of manual searching, comparison, and input; a remote collaboration terminal generates label information based on the inspection process data and returns it, improving the efficiency of on-site processing, supporting remote review and indication by experts; and finally outputting a structured record containing spatial position association, facilitating subsequent event tracking and state comparison.

[0036] Hereinafter, according to the embodiments of the present application, the above steps S1 to S6 are specifically described.

[0037] In the embodiments of the present application, before the inspection method is executed, an inspection task can be generated by a management terminal. The task generation method includes manually configuring an inspection route and equipment information, or creating a task at a predetermined time according to a preset strategy by a server. After the task is generated, the inspection personnel receive the task in the augmented reality equipment and load the equipment data, position identifier, and execution instruction associated with the task.

[0038] In step S1, the core task is to synchronously and dynamically collect various physical parameters existing in the power equipment operating environment. The collection objects cover multi-modal data capable of representing changes in the appearance, posture state, and radiation field of the equipment, to form a unified data input basis required for subsequent anomaly detection and three-dimensional space relationship construction. Among them, the data types of the multi-modal data can include image data, posture data, and radiation data, and maintain timestamp consistency and spatial coordinate uniformity during the collection process.

[0039] Specifically, the image data is used to express the surface structure, component contour, and potential defect features of the equipment, and is the input basis for the target detection neural network to construct the defect area recognition result. The image data can be denoted as a two-dimensional pixel matrix , wherein represents the collection timestamp, represents the pixel coordinates, represents the channel index, and for an RGB image, ; and for a structure light camera with infrared fusion, , to enhance the high-temperature area recognition capability. Illustratively, image collection can be collected by a forward camera sensor integrated in a wearable AR device at a fixed frame rate during the inspection path advancing process, and the exposure gain and focal length are dynamically adjusted according to the equipment position to maintain a relatively high definition of close-range detail features.

[0040] In some embodiments, to improve the recognition robustness under different lighting conditions, an adaptive brightness adjustment function can be introduced in the acquisition stage wherein, is an ambient brightness estimation value, is a nonlinear lighting correction function, and the image data after lighting correction can reduce feature loss caused by indoor strobe light and highlight reflection in power plant, as subsequent input features.

[0041] In some embodiments of the present application, the pose data is used to represent the motion state of the inspection device in the three-dimensional space, including position and spatial orientation information, forming the mapping basis between the local coordinate system and the global scene coordinate system. The pose data can be denoted as wherein, represents a three-dimensional coordinate, represents the visual angle orientation in the form of Euler angle or quaternion. In some specific embodiments, an accelerometer and a gyroscope can be used for sensor fusion, and the IMU (Inertial Measurement Unit) outputs the pose estimation result, and drift calibration is performed based on image feature points, so that the pose information remains continuous during the inspection process.

[0042] According to another embodiment of the present application, the radiation data is used to express the change of the radiation field around the power equipment, which is closely related to the operation state of the equipment. After sampling by the radiation detector, the radiation data can be denoted as wherein, is the pulse count value output by the counter tube, is the energy spectrum calibration parameter at the sampling time, is a dose rate conversion function. The radiation data is collected in real time, and the time stamp is unified with the image and pose data to allow the subsequent correspondence of the radiation value and the recognized equipment position. Exemplarily, in the power transmission line inspection scene, the operator can wear AR glasses to perform close-range scanning at a position less than 5 m away from the power tower, and the video images and radiation data collected are transmitted in real time to the edge inference device for detection processing through a wireless link.

[0043] Further, the synchronization mechanism of the multi-modal data can be established based on a unified time source, and the time stamp sequence is used to ensure that the image, pose and radiation data at the same time constitute an input sample vector , and then form a training and analysis data structure that can be directly input into the neural network, realizing the consistent expression of each modal information in space and time.

[0044] In some embodiments, the data after multi-modal data acquisition can be stored in a continuous frame buffer area , for subsequent cross-frame stability analysis and dynamic risk generation. Illustratively, the cache area can be set with a maximum capacity to limit memory occupancy, and a sliding window structure can be used to automatically discard the earliest timestamped data when the inspection path is long.

[0045] Illustratively, in the high-radiation inspection scenario of a nuclear power plant, an AR glasses such as HoloLens 2 can be selected, the image acquisition frame rate is 30 fps, the pose sampling frequency is 100 Hz, the radiation data sampling frequency is 1 Hz, the camera resolution is 3840x2160 pixels, the transmission bandwidth is >100 Mbps, the battery endurance is >4h, the radiation counter is a Geiger counter, and the dose rate conversion coefficient is 0.002 The above parameters can be adjusted according to the field environment.

[0046] So far, step S1 provides a directly usable data basis for abnormal area identification, spatial positioning modeling, and dynamic risk mapping by collecting image data, pose data, and radiation data on a unified time reference and forming a multi-modal data structure, so that the inspection information can be expressed in a time-spatial consistent manner, and accurate data input is ensured for subsequent steps.

[0047] In step S2, the core task is to identify abnormal areas of power equipment using a target detection neural network based on the multi-modal data obtained in step S1, to form a detection result for power equipment appearance defects on a continuous time sequence, to generate a defect area identification result based on this, and to further generate time-space risk data by filtering algorithm in combination with power operation data, so as to express the intensity and evolution trend of defects in time and space dimensions, and to provide a risk quantification basis for subsequent three-dimensional spatial relationship modeling and inspection information generation.

[0048] In some embodiments of the present application, joint feature construction can be performed on image data and radiation data, and the radiation data and the image data can be combined into multi-channel input data according to the correspondence relationship of the acquisition time. Specifically, the image data obtained at the same timestamp in step S1 can be denoted as , where represents the time index, represents the pixel position, represents the image channel index; and the synchronously collected radiation data is denoted as a scalar dose rate or a spatial distribution function In the case of only obtaining a global dose rate, the radiation channel can be constructed by broadcasting:

[0049] Thus, the multi-channel input tensor is obtained:

[0050] wherein is the number of original image channels, is the index of the fused channel. The above construction method realizes the joint encoding of radiation intensity and image texture information at the pixel level, so that the image data and radiation data at the same timestamp remain consistent in spatial resolution and time index. For radiation measurement results with spatial distribution, can be directly set as is the two-dimensional radiation field data obtained by interpolation or gridding, to improve the spatial correspondence accuracy between abnormal areas and radiation hotspots.

[0051] Further, the multi-channel input tensor is input into a pre-trained target detection neural network (such as YOLOv5), denoted as function:

[0052] wherein represents the network parameters, and the output is the detection result set at time . Each detection result can be represented as a three-tuple , wherein, represents the rectangular bounding box center position and width-height parameters in the image coordinate system, represents the abnormal class label, such as surface cracks, corrosion points, insulation damage, hotspot areas, etc., represents the confidence of the detection result. Through this process, the multi-channel input data is input into the target detection neural network and the abnormal area is recognized, so as to obtain the initial defect area recognition result for each frame of picture. In specific implementation, the target detection neural network can adopt a single-stage detection structure or a two-stage detection structure, the former directly outputs the bounding box and class probability at each position through dense prediction, and the latter completes detection through the cascade structure of candidate region generation and region classification regression, both of which can adapt to multi-channel input through convolution feature extraction, feature pyramid, and multi-scale prediction mechanisms. Exemplarily, the target detection neural network can be based on the YOLOv5 structure, and a radiation channel is added at the input end to realize joint feature extraction of radiation field and image information.

[0053] In an exemplary embodiment, in order to enable the target detection neural network to fully utilize the joint characteristics of image data and radiation data, a training sample set with multi-channel input can be constructed in the offline training phase, each sample in the training sample set contains an input tensor and a set of artificial labeled targets . The total loss function is defined during training:

[0054] wherein, The classification loss is used to constrain the predicted categories. Compared to the real category Consistency can be achieved through methods such as cross-entropy or focus loss. The bounding box regression loss is used to constrain the predicted bounding boxes. With the true bounding box The degree of geometric overlap can be determined using a loss function based on the cross-union ratio; This is a balancing coefficient used to control the weight ratio between the classification and regression losses. Through the training process, the network parameters... Gradual convergence enables the network to stably identify abnormal regions under varying radiation levels, complex backgrounds, and diverse device configurations, achieving feature fusion learning from multimodal inputs. During the runtime phase, only the forward inference process is retained, and parameters are no longer updated, thus ensuring real-time performance during online inspections.

[0055] In some embodiments of this application, to avoid positional fluctuations in single-frame detection results during actual inspection due to local occlusion, sudden changes in illumination, or transient noise, cross-frame spatial position stabilization processing can be performed on each generated defect region identification result. This process merges spatially adjacent positions belonging to consecutive frames in the defect region identification results to form stable defect regions. Specifically, detection results with adjacent timestamps can be merged. and Perform matching for any two detection boxes. and Calculate its intersection-union ratio:

[0056] And the Euclidean distance between the center points of the bounding box:

[0057] When the intersection-to-union ratio is higher than a preset threshold and the center distance is lower than a preset position offset threshold, it can be... and These defects are considered as projections of the same physical defect across different frames, thus grouping them into the same cross-frame trajectory. After matching multiple frame sequences, the bounding box positions in the trajectory can be averaged or weighted to obtain stable defect regions. It also aggregates the category labels and confidence scores in the trajectories, for example, by using average, maximum, or weighted strategies to generate stable categories. With stable confidence This method integrates the originally independent detection results frame by frame into a temporally continuous and spatially stable defect representation, improving the coherence and reliability of abnormal area indication.

[0058] In some embodiments of the present application, power operation data is generally provided in the form of time series, such as equipment current, bus voltage, winding temperature, cooling medium pressure, etc., which can be denoted as a vector wherein, represents the observation value of the th operation parameter at time .

[0059] At the same time, due to the presence of measurement noise, short-term disturbance or transient fluctuations introduced by control actions in the field acquisition process, if the original operation data is directly used for risk assessment, it is easy to cause the risk quantification results to appear jitter on the time axis. Therefore, the operation data can be smoothed and estimated by a filtering algorithm to construct a smoothed operation state vector:

[0060] wherein, represents the smoothed operation state at time , is a smoothing coefficient for controlling the weight distribution between the historical state and the current observation. A larger makes the smoothed sequence insensitive to short-term fluctuations, and a smaller can respond more quickly to operation state mutations.

[0061] In some specific embodiments, the above first-order exponential smoothing can also be extended to a multi-dimensional Kalman filter form, and a dynamic model of the operation parameter is established through a state transition equation and an observation equation to realize recursive estimation of the operation state and further reduce the noise influence. Illustratively, when there is cross-frame jitter in the target detection output in the inspection process, a Kalman filter can be used to smooth the defect bounding box trajectory to reduce the position instability caused by visual noise and short-term jitter.

[0062] In an exemplary embodiment, multi-dimensional operation data can come from a power grid monitoring system and be updated in real time during the inspection process, and the operation data can include temperature, pressure, current, voltage and radiation intensity parameters. The operation data is organized into a sequence wherein, represents the temperature parameter, represents the pressure parameter, represents the radiation dose rate parameter. In the presence of sampling noise and short-period disturbance, a Kalman filter can be used to estimate the state of the above operation parameters to construct an update equation:

[0063] wherein, is the operation state estimate value, is the sensor measurement value, , , respectively represent state transition, noise input and observation mapping matrix, and represent process noise and observation noise. After completing state estimation, the estimation result is mapped to a two-dimensional field of the device surface or the surrounding area according to the inspection visual angle:

[0064] wherein is a dynamic thermal map corresponding to the power equipment area, represents the mapping function of the running state to the visual coordinate system based on the pose data . In this way, a dynamic thermal map that is updated over time can be formed during the inspection process, which is used to express the trend of the change in the spatial distribution of the running state.

[0065] In other embodiments, a risk mapping relationship can be constructed between the stable defect area and the smooth running state to form spatio-temporal risk data. For example, for each stable defect area , its stable bounding box is , the stable confidence is , and the defect category is , a risk score function can be defined as:

[0066] wherein is a nonlinear function that compresses the input to the interval , such as a Sigmoid function; is a function of normalizing and weighting the smooth running state, which is used to reflect the deviation between the current equipment working condition and the rated working condition; is a function of assigning a basic risk weight according to the defect category, which is used to reflect the severity difference of different defect types in engineering; is a configurable non-negative weight, which is used to balance the contribution of visual confidence, running state deviation and defect category in the overall risk score.

[0067] Through the above risk score function, combined with the timestamp and the spatial position (mapped to three-dimensional coordinates through pose data later), each stable defect area can be bound with a risk value between 0 and 1 , forming spatio-temporal risk data containing the three elements of "time, space, risk intensity" . Exemplarily, in a typical inspection task, the risk score threshold can be set to 0.7, when ​When the threshold is exceeded, the corresponding region is marked as a high-risk region to trigger a higher level of inspection or operation response.

[0068] So far, step S2 completes abnormal region recognition by inputting multi-modal data into the target detection neural network through constructing multi-channel input based on image data and radiation data, and forms stable defect regions through cross-frame spatial position stabilization processing, and then generates spatio-temporal risk data with time continuity and spatial locatability characteristics by combining power operation data smoothed by filtering algorithm, which makes the defect recognition result expand from single-frame static detection to risk expression dynamically evolving with working condition changes, and lays a foundation for risk quantification and time series analysis for subsequent three-dimensional space mapping and inspection information presentation in augmented reality view.

[0069] In step S3, the core task is to construct the spatial orientation and coordinate reference of the augmented reality view relative to the inspection environment based on the posture data collected during the inspection process, and determine the positional relationship of the power equipment in the three-dimensional space by combining the spatial positioning algorithm, thereby forming the mapping structure between the equipment topology and the inspection perspective, and converting the spatio-temporal risk data into inspection information so that the risk information can be presented in a spatial position corresponding manner and used to guide the inspection operation.

[0070] Specifically, the posture data comes from the augmented reality glasses worn by the inspection equipment, which outputs position and angle information in real time through an inertial measurement unit, which can be denoted as a six-dimensional vector wherein, represents the position of the device in the global coordinate system, represents the view orientation angle.

[0071] Based on this, the Euler angle is configured as a rotation matrix and combined with the position vector to form a transformation matrix:

[0072] This matrix is used to determine the spatial orientation of the augmented reality view, thereby realizing the determination of the spatial orientation of the augmented reality view according to the posture data. To improve the stability of nuclear power plant inspection, the sampling frequency can be taken as 100 Hz, and the image resolution can be taken as 3840x2160 pixels to maintain the stability of the view in a low exposure environment in a high radiation area.

[0073] In some embodiments, the positional relationship of the power equipment in the three-dimensional space can be determined based on a spatial positioning algorithm. If a set of feature points is extracted from the image sequence, the topology structure can be initialized by using a device geometric model The position of each component of the computing device in the global coordinate system. Exemplarily, in the nuclear reactor pipeline inspection scene, the positioning process can adopt a visual-inertial SLAM fusion algorithm to construct the spatial topology, and the registration error can be controlled within 1 cm in the radiation interference environment, which can meet the high-precision leakage positioning requirements.

[0074] In some embodiments of the present application, the spatio-temporal risk data can be spatially positioned based on the spatial orientation and the positional relationship of the power equipment in the three-dimensional space, thereby converting it into inspection information. Specifically, the spatio-temporal risk data generated in the previous step can be denoted as , and the center point of the bounding box in the image coordinate system is . In order to map the spatio-temporal risk data to the three-dimensional space, the projection relationship from the pixel coordinates to the three-dimensional space coordinates can be constructed by the camera intrinsic matrix :

[0075] In embodiments of the present application, the inspection information is structured information used to express the running state of the target equipment in the three-dimensional space, expressed in vector form as , wherein represents the risk value, which is derived from the spatio-temporal risk data of the previous step; represents the abnormal type, such as crack, corrosion or radiation hotspot; represents the data acquisition time; is the inspection level determined according to the risk value and the abnormal type, for example, when it can be used as a task point that needs to be executed in priority.

[0076] The inspection information can further bind display attributes such as color or transparency in the augmented reality view for visual prompting of the inspection personnel. Color mapping can be used to represent the risk level, and the risk value can be mapped to color or brightness coded visual markers, for example, the color mapping function , so that the area with higher risk value presents higher brightness or reddish display effect, thereby intuitively prompting the inspection personnel. Exemplarily, when , if the risk value is 0.7, a dynamic high-brightness dark red marker is formed in the view at this time; at the same time, when the risk value exceeds the risk threshold (such as 0.7), the corresponding position area in the augmented reality view can also be displayed dynamically with flashing to prompt the inspection personnel to pay attention to the area.

[0077] Further, due to the attitude jump caused by fast turning, shielding, radiation interference and the like in the inspection process, in order to keep the spatial relationship stable, the continuity of the attitude sequence formed by the attitude data can be checked when establishing the three-dimensional spatial relationship of the power equipment. When the spatial change of adjacent attitudes in the attitude sequence exceeds the preset change threshold, the three-dimensional spatial relationship of the power equipment is re-established. Specifically, the angle change between adjacent attitudes can be calculated: and the position change .

[0078] When both of them exceed the preset threshold, the spatial mapping relationship is re-established. Exemplarily, in the high-radiation environment inspection, the position threshold can be set to 0.8 m, and the angle threshold can be set to 25°; in the nuclear power plant night inspection scene, due to the enhanced stability of infrared imaging, the angle threshold can be reduced to 15°, and the infrared-radiation dual-channel positioning mode is adopted to improve the identification efficiency, and the identification accuracy can reach 92% in the tritium-containing water mist scene, and the leakage point positioning error can be less than 2 cm.

[0079] In an exemplary embodiment, the inspection personnel wear AR glasses to move and scan within a range of less than 5 m from the power transmission tower, the radiation threshold is set to be below 0.5 mSv / h, the risk graph generated during operation is spatially registered through the three-dimensional structure of the tower body reconstructed by SLAM, the positioning error can be less than 0.5 m, and the radiation response delay is less than 2 s; in the nuclear reactor primary loop inspection, the radiation sampling frequency is 1 Hz, and the risk mapping is established based on the reactor pipeline section model, and the positioning error can be controlled within 1 cm.

[0080] At this point, step S3 establishes a three-dimensional mapping relationship between the augmented reality view and the power equipment, and converts the spatio-temporal risk data into inspection information that can directly guide the inspection operation in the spatial level, so that the risk expression has the characteristics of visualization, positioning and traceability, and lays a foundation for subsequent augmented reality view superimposed display and inspection path guidance.

[0081] In step S4, the core task is to correspond the inspection information obtained in step S3 with the three-dimensional position relationship of the power equipment, and display it in a visualized manner in the augmented reality view, while providing operation guidance for the inspection personnel based on the abnormal type indicated by the inspection information.

[0082] In the embodiment of the present application, the inspection information can be superimposed on the augmented reality view based on projection mapping. If the risk point in the three-dimensional coordinate system is , its projection in the rendering coordinate system is , wherein, is the camera intrinsic matrix, is the inverse matrix from the global coordinate system to the view coordinate system. The risk value The mapping is color-coded for rendering the marker wherein, is a visual rendering mapping function, which can be mapped to a gradient red color, a flashing animation or a directional arrow according to the risk level.

[0083] In some embodiments, the inspection level can be determined based on the corresponding abnormal type in the inspection information, and the execution order of the inspection operation can be determined according to the inspection level, and the inspection personnel can perform the inspection operation on the power equipment according to the execution order. For example, the inspection level can be determined as wherein, 0 represents no need for processing, and 3 represents the highest priority. The inspection path can be determined by determining the optimal execution order, wherein, is a three-dimensional distance, is a level-based weight.

[0084] In some specific embodiments, in the power transmission tower inspection scene, the distance between the inspection personnel and the tower can be less than 5 m, and when the risk value exceeds 0.7, the substation component position is marked with a red flashing outline in the visual scene, the SCADA (Supervisory Control And Data Acquisition, data acquisition and monitoring system) transmission voltage and radiation data are superimposed in the visual interface, the SLAM positioning error can be kept within 0.5 cm, and the operator verifies in the order of “high-risk corrosion-insulation damage-low temperature hot spot”; while in the nuclear power plant reactor pipeline night inspection scene, the infrared and radiation dual-channel can be used to construct the visual guidance, the frame rate can reach 60 fps, and the leakage hot spot positioning error can be less than 2 cm.

[0085] In some embodiments, a device model can be constructed based on a three-dimensional engine, and the detected defect area can be superimposed on the outer surface of the device in a color highlighting manner, and the operation data from the external system can be displayed in the form of a text label or a thermal map at the corresponding position, for assisting the inspection personnel to judge the risk degree.

[0086] Exemplarily, in the nuclear power plant piping inspection scene, the corrosion area can be superimposed on the surface of the pipeline model in a red box selection manner, and the radiation monitoring data can be superimposed on the corresponding position.

[0087] So far, step S4 realizes the visual expression of the risk information in the three-dimensional coordinates, so that the inspection guidance has the positioning property and the priority order, and provides a spatial basis for the subsequent remote collaboration and label generation.

[0088] In step S5, the core task is to collect and time-index mark the data generated in the inspection process, and after associating the inspection process data with the aforementioned spatio-temporal risk data and inspection information, the data is sent to the remote collaboration terminal through the communication network link, so that the remote personnel can generate the label information for the device based on the associated data.

[0089] In the embodiments of the present application, the collection time of the inspection process data needs to be indexed by time. The inspection process data can be defined as , wherein, is the attitude information, is the image frame, is the running data. At the same time, in order to establish the time correspondence, a time index can be generated for each item of data at the collection end .

[0090] In some embodiments, the corresponding records can be extracted from the spatio-temporal risk data and the inspection information by time index matching. If the risk data set is , the inspection information set is , and the matching form is .

[0091] In further processing, the corresponding position of the power equipment in space can be determined according to the observation attitude of the inspection process data at the time of collection, and the selected records are matched with the inspection process data based on the corresponding position in space to form the associated data in the order of time index .

[0092] Finally, the sequence is sent to the remote collaboration terminal through the communication network in the order corresponding to the time index. Exemplarily, the communication mode can support 5G, Wi-Fi6 or private network optical fiber link.

[0093] In an exemplary embodiment, in the pipeline inspection of a nuclear power plant, the transmission bandwidth can be greater than 100 Mbps, the end-to-end delay is less than 50 ms, the radiation sampling rate is 1 Hz, the risk curve is synchronized with the on-site view, the remote expert can complete the labeling through cloud virtual superposition by gesture / voice interaction, the labeling generation delay can be less than 2s, and multiple experts can collaborate at the same time, and different experts can be distinguished by color.

[0094] In some embodiments, the remote collaboration terminal can perform real-time labeling on the uploaded inspection data, the inspection personnel display the text labels or graphic labels from the remote end in the augmented reality view, and support instruction feedback, which is suitable for maintenance scenes that need expert participation.

[0095] So far, step S5 matches the spatio-temporal risk data and the inspection data in position and time by establishing a time index for the inspection process data, and transmits the serialized associated data to the remote collaboration terminal, realizes the unification of the on-site inspection data in space, time and semantic level, enables the remote labeling to be accurate based on the real scene, and provides a reliable input for the final superposition presentation and record generation.

[0096] In step S6, the core task is to receive the annotation information generated by the remote collaboration terminal based on the uploaded data, and accurately superimpose the annotation information in the augmented reality view to the corresponding spatial position of the power equipment, while generating a structured inspection record based on the inspection information after the inspection is completed.

[0097] In the embodiments of the present application, the annotation information can include three-dimensional spatial positioning points, text prompts, maintenance guidance paths, risk level adjustments, and the like, denoted as , wherein, represents the position of the annotation point, represents the annotation content. The is mapped to the augmented reality view through the aforementioned projection model for superimposed display, which can make the operation intention of the remote expert consistent with the perspective of the inspection personnel.

[0098] In some embodiments, a structured inspection record can be generated based on the inspection information, and an output format can be constructed as , wherein, represents the inspection time, represents the execution result record.

[0099] In nuclear power plant inspection, the remote expert can add a three-dimensional virtual marker to the high-radiation hot spot of the reactor pipeline and set the maintenance sequence, the rendering frame rate can be maintained at 60 fps or higher, the structured report can be automatically generated for the safety approval process, and the remote expert can completely avoid exposure to the radiation environment. Exemplarily, further support for off-site mode can also be provided, such as simulating a radiation scene with AR glasses and superimposing historical failure cases, while supporting multi-person collaborative training, and the virtual radiation threshold parameter can also be adjusted according to actual needs, such as 0.1-5 mSv / h.

[0100] So far, step S6 receives the annotation information generated by the remote terminal and maps it to the corresponding spatial position in the augmented reality view, while outputting a structured inspection record based on the inspection information, so that the inspection result is transformed from visual perception and risk expression to an archivable, reproducible, and traceable form, providing data support for power equipment operation and maintenance strategy optimization and closed-loop management.

[0101] In one embodiment, for remote inspection of high radiation area in the periphery of a nuclear power plant, augmented reality glasses are used to control an industrial drone to perform an inspection route, and the drone image is linked in real time in the augmented reality view. When the inspection starts, the inspector wears the augmented reality glasses, selects the reactor periphery structure, power transmission tower or shielding wall area that needs to be inspected in the field of view, and the augmented reality glasses sends the spatial position and current radiation distribution information of the target area to the drone control end. The flight path is generated by the path planning program running in the background of the drone based on the A* algorithm. In order to avoid the drone entering the high dose area, the radiation weight is applied to the position with high radiation level in the path search process, and the cost of path node . .

[0102] wherein, represents the spatial distance from the current position to the target area, represents the radiation intensity at the current position, is used to adjust the influence of radiation on the path cost. When the radiation intensity exceeds the on-site safety control standard (for example, 1 mSv / h), the path search automatically increases the cost of the corresponding area, so that the generated optimal path is spatially bypassed around the high radiation area, thereby forming a flight corridor that "avoids areas greater than 1 mSv / h". At the same time, in order to ensure the control experience, the instruction and picture link delay between the augmented reality glasses and the drone is controlled within 50 ms, so that the view angle adjustment instruction issued by the inspector in the view can be reflected in the flight attitude and shooting picture of the drone in a short time.

[0103] In this embodiment, the drone carries a radiation sensor, which collects radiation data at a preset sampling frequency (for example, 1 Hz) during flight, and transmits the radiation data and the corresponding spatial position of the drone to the augmented reality terminal through a wireless link. The augmented reality terminal performs filtering processing on the radiation measurement sequence of continuous time, for example, uses Kalman filtering to suppress transient noise and measurement jitter, binds the filtered radiation estimate value with the three-dimensional coordinates on the flight trajectory of the drone, and gradually accumulates to form a radiation hotspot cloud in three-dimensional space, so that the inspector can view the three-dimensional radiation distribution generated by the drone scanning in the augmented reality view. When the inspector views the site environment on the ground at a first viewing angle, he can switch to the drone on-board camera viewing angle in real time through gestures or line-of-sight interaction, and freely switch between ground viewing angle and air viewing angle in the same augmented reality view framework. For example, when performing an inspection task within a 500m coverage range in the periphery of a nuclear power plant, the drone can always avoid areas with radiation intensity exceeding 1 mSv / h in the path planning stage, and the inspector stays in the safe area throughout the inspection, without entering the high radiation environment, thereby realizing the risk of radiation exposure of on-site personnel close to zero on the premise of completing large-scale inspection and three-dimensional radiation hotspot cloud construction.

[0104] In summary, this method uses multimodal data such as images, posture, and radiation acquired on-site to achieve real-time characterization of equipment status. It identifies abnormal areas from the model as risk triggers, combines time-series filtering to generate dynamic risk representations with location correlation, and aligns the risk data with the equipment's three-dimensional geometry based on spatial positioning. This augmented reality visualization of defect locations and operational guidance, along with the integrated uploading of inspection processes, risk estimation, and location annotation to a remote terminal to generate feedback-enabled annotation information and structured inspection records, transforms the inspection process from being driven by manual experience to being data-driven. This achieves intuitive defect location, standardized inspection execution, and accessibility-free access to high-risk areas, improving anomaly detection efficiency and location accuracy, reducing on-site exposure risks in high-radiation or high-altitude environments, and enhancing the traceability and full-process management capabilities of inspection tasks.

[0105] It should be noted that, although the embodiments in this application are based on... Figure 1 Steps S1 to S6 are described sequentially, but this does not mean that steps S1 to S6 must be performed in a strict order. The reason this embodiment follows this order is... Figure 1 The order in which steps S1 to S6 are described is provided to facilitate understanding of the technical solutions of the embodiments of this application by those skilled in the art. In other words, in the embodiments of this application, the order of steps S1 to S6 can be appropriately adjusted according to actual needs.

[0106] like Figure 2 As shown, the following is an embodiment of an augmented reality-based intelligent inspection system for power equipment provided by this disclosure. This augmented reality-based intelligent inspection system for power equipment belongs to the same inventive concept as the augmented reality-based intelligent inspection methods for power equipment in the above embodiments. For details not described in detail in the embodiments of the augmented reality-based intelligent inspection system for power equipment, please refer to the embodiments of the above-mentioned augmented reality-based intelligent inspection methods for power equipment.

[0107] Based on the same concept, another embodiment of this application provides an augmented reality-based intelligent inspection system for power equipment, comprising: Acquisition module 1 is used to acquire multimodal data of power equipment, including image data, attitude data and radiation data; The identification module 2 is used to identify abnormal areas of power equipment based on multimodal data and target detection neural networks, generate defect area identification results, and generate spatiotemporal risk data by combining power operation data and filtering algorithms. The modeling module 3 is configured to establish a three-dimensional spatial relationship of the power equipment based on the pose data and a spatial positioning algorithm, and convert the spatio-temporal risk data into inspection information based on the three-dimensional spatial relationship; The display module 4 is configured to display the inspection information in spatial alignment with the power equipment in an augmented reality view, and guide the inspection personnel to perform an inspection operation on the power equipment based on the inspection information. The communication module 5 is configured to collect inspection process data, and send the inspection process data, the spatio-temporal risk data and the inspection information to a remote collaboration terminal through a communication network, so that the remote collaboration terminal generates labeling information for the power equipment. The superimposition module 6 is configured to receive the labeling information, superimpose the labeling information at a corresponding spatial position of the power equipment in the augmented reality view, and generate a structured inspection record based on the inspection information.

[0108] Based on the above system architecture, the device running state is mapped to visual inspection information by using the abnormality recognition and spatial modeling capabilities with multi-modal data collection as input on site, and the spatial alignment and interactive display of the risk position are completed in the augmented reality view, so that the inspection personnel can complete defect positioning and task execution without entering high radiation or high operation risk areas. At the same time, the inspection process data and risk estimation results are uploaded and labeling information is generated at a remote end, and then the labeling information is returned to the site for superimposed display and result recording, so as to realize a closed loop from data collection, risk judgment to decision output, and to change the inspection process from an experience-based operation to a real-time, traceable and standardized execution process, which can improve the accuracy and positioning efficiency of defect identification, reduce the risk of manual intervention, and improve the automation degree of the inspection task and the data management capability of the equipment operation and maintenance.

[0109] In one embodiment, the system adopts a hierarchical architecture including a running environment layer, a basic service layer and a core capability layer. The running environment layer is configured to provide computing resources and network connection capabilities, and support local deployment or cloud environment deployment; the basic service layer is configured to provide user management, device access and data access control; and the core capability layer includes spatial positioning processing, visual recognition processing, three-dimensional model rendering and multimedia transmission capability, and is configured to support the data collection, recognition analysis, spatial alignment and information display process in the inspection method.

[0110] In some embodiments of the present application, the identification module 2 can include a feature construction unit and an abnormality detection unit, wherein the feature construction unit is configured to jointly construct features of the image data and the radiation data, and combine the radiation data and the image data into multi-channel input data according to the correspondence relationship of the collection time; and the abnormality detection unit is configured to input the multi-channel input data into a target detection neural network and identify an abnormal area.

[0111] In some embodiments of the present application, the modeling module 3 can include a space orientation unit, a space positioning unit and a mapping processing unit. Among them, the space orientation unit is used to determine the space orientation of the augmented reality view according to the attitude data; the space positioning unit is used to determine the positional relationship of the power equipment in the three-dimensional space by using the space positioning algorithm; the mapping processing unit is used to perform space positioning processing on the space-time risk data based on the space orientation and the positional relationship of the power equipment in the three-dimensional space, and convert it into the inspection information.

[0112] In summary, the system completes the collection of device multi-modal field data, the identification of abnormal areas and risk estimation, the three-dimensional structure mapping based on attitude and space positioning, the spatial alignment of risk information in the augmented reality view, the remote labeling feedback and the generation of structured inspection records through the cooperation of the collection module 1, the identification module 2, the modeling module 3, the display module 4, the communication module 5 and the superposition module 6, realizes the structured expression of the device running state from discrete observation to spatial semantics, can complete the defect positioning and execution guidance in a visual way in high radiation, high altitude and limited environment, maintains the expression consistency and traceability of the inspection process under different field conditions, and supports intervention by using remote collaboration when the risk reaches the set condition, so that the inspection task is completed and managed without entering the dangerous area.

[0113] The above disclosure is only the preferred embodiment of the present application, but the present application is not limited thereto, any non-creative changes that can be thought of by those skilled in the art, and several improvements and refinements made without departing from the principles of the present application, should fall within the scope of protection of the present application.

Claims

1. A method for intelligent inspection of power equipment based on augmented reality, characterized in that, include: Collect multimodal data of power equipment, including image data, attitude data, and radiation data; Based on the multimodal data, an abnormal area of ​​the power equipment is identified using a target detection neural network, generating defect area identification results, and spatiotemporal risk data is generated by combining power operation data with a filtering algorithm. A three-dimensional spatial relationship of power equipment is established based on attitude data and spatial positioning algorithms, and spatiotemporal risk data is converted into inspection information based on the three-dimensional spatial relationship. In augmented reality visuals, inspection information is spatially aligned with power equipment and displayed, and inspection personnel are guided to perform inspection operations on the power equipment based on the inspection information. Collect inspection process data, and send the inspection process data, spatiotemporal risk data and inspection information to the remote collaboration terminal through the communication network. The remote collaboration terminal then generates annotation information for the power equipment. The annotation information is received, and the annotation information is overlaid and displayed at the corresponding spatial location of the power equipment in the augmented reality view. A structured inspection record is generated based on the inspection information.

2. The intelligent inspection method for power equipment based on augmented reality as described in claim 1, characterized in that, The step of identifying abnormal regions of power equipment using a target detection neural network based on the multimodal data includes: Joint features are constructed from image data and radiometric data, and radiometric data and image data are combined into multi-channel input data according to the correspondence of acquisition time. The multi-channel input data is fed into the target detection neural network to identify abnormal regions.

3. The intelligent inspection method for power equipment based on augmented reality as described in claim 1, characterized in that, After generating the defect region identification result, cross-frame spatial position stabilization processing is performed on the defect region identification result, merging the spatially adjacent positions belonging to consecutive frames in the defect region identification result to form a stable defect region.

4. The intelligent inspection method for power equipment based on augmented reality as described in claim 1, characterized in that, The steps of establishing a three-dimensional spatial relationship of power equipment based on attitude data and spatial positioning algorithms, and converting spatiotemporal risk data into inspection information based on the three-dimensional spatial relationship, include: The spatial orientation of the augmented reality scene is determined based on the posture data, and the positional relationship of the power equipment in three-dimensional space is determined using spatial positioning algorithms. Spatial positioning processing of spatiotemporal risk data is performed based on the spatial orientation and the positional relationship of power equipment in three-dimensional space, and then converted into inspection information.

5. The intelligent inspection method for power equipment based on augmented reality as described in claim 1, characterized in that, When establishing the three-dimensional spatial relationship of power equipment, the continuity of the attitude sequence composed of attitude data is checked. When the spatial change of adjacent attitudes in the attitude sequence exceeds the preset change threshold, the three-dimensional spatial relationship of the power equipment is re-established.

6. The intelligent inspection method for power equipment based on augmented reality as described in claim 1, characterized in that, The steps for guiding inspection personnel to perform inspection operations on power equipment based on inspection information include: determining the inspection level based on the anomaly type corresponding to the inspection information, determining the execution order of the inspection operations according to the inspection level, and having the inspection personnel perform the inspection operations on the power equipment according to the execution order.

7. The intelligent inspection method for power equipment based on augmented reality as described in claim 1, characterized in that, The steps for transmitting inspection process data, spatiotemporal risk data, and inspection information to a remote collaboration terminal via a communication network include: Record the data collection time during the inspection process and generate a time index; Records for the corresponding time are selected from the spatiotemporal risk data and inspection information based on the time index; The corresponding position of the power equipment in space is determined based on the observation posture of the inspection process data at the time of collection, and the selected records are matched with the inspection process data based on the corresponding position in space to form associated data ordered by the time index. The associated data is sent to the remote collaboration terminal via a communication network in the order corresponding to the time index.

8. An intelligent inspection system for power equipment based on augmented reality, characterized in that, include: The acquisition module is used to acquire multimodal data of power equipment, including image data, attitude data, and radiation data; The identification module is used to identify abnormal areas of power equipment based on the multimodal data using a target detection neural network, generate defect area identification results, and generate spatiotemporal risk data by combining power operation data with a filtering algorithm. The modeling module is used to establish the three-dimensional spatial relationship of power equipment based on attitude data and spatial positioning algorithms, and to convert spatiotemporal risk data into inspection information based on the three-dimensional spatial relationship. The display module is used to spatially align inspection information with power equipment in an augmented reality view and guide inspection personnel to perform inspection operations on the power equipment based on the inspection information. The communication module is used to collect inspection process data and send the inspection process data, spatiotemporal risk data and inspection information to the remote collaboration terminal through the communication network. The remote collaboration terminal generates annotation information for the power equipment. The overlay module is used to receive the annotation information, overlay the annotation information and display it on the corresponding spatial location of the power equipment in the augmented reality scene, and generate a structured inspection record based on the inspection information.

9. The intelligent power equipment inspection system based on augmented reality as described in claim 8, characterized in that, The identification module includes: The feature construction unit is used to perform joint feature construction on image data and radiometric data, and to combine radiometric data and image data into multi-channel input data according to the correspondence of acquisition time. An anomaly detection unit is used to input the multi-channel input data into the target detection neural network and identify abnormal regions.

10. The intelligent power equipment inspection system based on augmented reality as described in claim 8, characterized in that, The modeling module includes: Spatial orientation unit, used to determine the spatial orientation of the augmented reality scene based on posture data; Spatial positioning unit, used to determine the positional relationship of power equipment in three-dimensional space using spatial positioning algorithms; The mapping processing unit is used to perform spatial positioning processing on spatiotemporal risk data based on the spatial orientation and the positional relationship of power equipment in three-dimensional space, and convert it into inspection information.

Images