Power transformation scene state monitoring method and device, computer device and storage medium

Abstract

This application relates to a method, apparatus, computer equipment, computer-readable storage medium, and computer program product for monitoring the status of a substation scene. The method includes: acquiring multispectral imagery, lidar point cloud data, and real-time terrain monitoring data of a target substation scene; sequentially performing data preprocessing and feature extraction operations on all data to obtain multispectral image features, three-dimensional geometric features, and environmental features; acquiring image recognition results of the target substation scene based on all features; mapping the spectral information of the multispectral imagery to the lidar point cloud to obtain composite data; constructing a three-dimensional terrain model of the target substation scene based on the composite data and image recognition results; performing spatial position correction on the three-dimensional terrain model using a preset technique to obtain a corrected three-dimensional terrain model; and using the corrected three-dimensional terrain model to monitor the equipment and terrain status of the target substation scene. This method enables real-time status monitoring of a substation scene.

Description

Technical Field

[0001] This application relates to the field of digital image processing technology, and in particular to a method, apparatus, computer equipment, computer-readable storage medium, and computer program product for monitoring the status of substation scenarios. Background Technology

[0002] With the development of smart grids and automated monitoring technologies, continuous and accurate monitoring of substation facilities and surrounding terrain has become a necessary prerequisite for ensuring the safe and stable operation of the power grid and preventing major accidents.

[0003] In traditional technologies, substation condition monitoring mainly relies on manual on-site inspections or independent data collection by fixed sensors. However, manual inspections, which rely solely on visual observation and portable devices to record defects and terrain changes, suffer from low inspection efficiency, limited coverage, and delayed data updates, making it difficult to meet the real-time monitoring needs of large-scale, complex substation scenarios. While single sensors can identify surface defects through visible light images or acquire three-dimensional geometric information through lidar, their deployment location and quantity limitations prevent them from comprehensively reflecting the overall condition of the substation scenario, and equipment maintenance costs are high. Furthermore, lidar suffers from severe data loss due to vegetation obstruction, image recognition is susceptible to environmental interference, resulting in high rates of false positives and false negatives for defects. Equipment condition and terrain data are independent of each other, lacking the ability to collaboratively analyze multi-source information, leading to low maintenance efficiency, high safety risks, and difficulty in meeting the high-precision, full-coverage monitoring requirements of substations in complex terrain.

[0004] Therefore, how to achieve real-time and comprehensive monitoring of equipment and terrain conditions in substation scenarios is an urgent problem to be solved. Summary of the Invention

[0005] Therefore, it is necessary to provide a method, device, computer equipment, computer-readable storage medium, and computer program product for monitoring the status of equipment and terrain in substation scenarios, which can realize real-time and comprehensive monitoring of the status of equipment and terrain in substation scenarios, in order to address the above-mentioned technical problems.

[0006] Firstly, this application provides a method for monitoring the state of a substation scenario, including:

[0007] Acquire multispectral images, lidar point clouds, and real-time terrain monitoring data of the target substation scene;

[0008] Data preprocessing and feature extraction operations are performed sequentially on multispectral images, lidar point clouds, and real-time terrain monitoring data to obtain multispectral image features, three-dimensional geometric features, and environmental features.

[0009] Based on multispectral image features, three-dimensional geometric features, and environmental features, image recognition results of the target substation scene are obtained.

[0010] The spectral information of the multispectral image is mapped onto the lidar point cloud to obtain composite data;

[0011] Based on composite data and image recognition results, a three-dimensional terrain model of the target substation scene is constructed; the spatial position of the three-dimensional terrain model is corrected using preset technology to obtain a corrected three-dimensional terrain model; the corrected three-dimensional terrain model is used to monitor the equipment and terrain status of the target substation scene.

[0012] In one embodiment, acquiring multispectral imagery, lidar point cloud data, and real-time terrain monitoring data of the target substation scene includes:

[0013] The system receives multispectral images automatically acquired by a multispectral camera at set time intervals; it also receives lidar point clouds obtained by lidar scanning of a target substation scene; the multispectral camera and lidar are mounted on the drone; the drone flies along a pre-planned safe flight path.

[0014] Receive real-time terrain monitoring data; the real-time terrain monitoring data includes real-time displacement data and real-time meteorological data; the real-time displacement data and real-time meteorological data are collected in real time by displacement sensors and meteorological sensors deployed around the substation tower base.

[0015] In one embodiment, data preprocessing and feature extraction operations are sequentially performed on multispectral imagery, lidar point clouds, and real-time terrain monitoring data to obtain multispectral image features, three-dimensional geometric features, and environmental features, including:

[0016] Radiometric calibration, atmospheric correction, and geometric correction are performed sequentially on the multispectral image to obtain a preprocessed multispectral image. A preset model is then used to classify land cover and extract features from the preprocessed multispectral image to obtain multispectral image features. The multispectral image features include semantic features, texture features, and spectral features.

[0017] The lidar point cloud is preprocessed using at least one preset preprocessing method to obtain the preprocessed lidar point cloud; the three-dimensional geometric features of the preprocessed lidar point cloud are extracted.

[0018] The real-time terrain monitoring data is filtered to obtain preprocessed real-time terrain monitoring data; the environmental features of the preprocessed real-time terrain monitoring data are then extracted.

[0019] In one embodiment, image recognition results are obtained by combining multispectral image features, three-dimensional geometric features, and environmental features, including:

[0020] A preset algorithm is used to calculate the probability of land cover classification for each data point by combining spectral image features and three-dimensional geometric features. Based on the probability of land cover classification, the power equipment, surface type and equipment components of the target substation scene are distinguished, and it is determined whether there are vegetation occlusion areas. If there are vegetation occlusion areas, the equipment or ground information under the vegetation occlusion area is corrected based on the vegetation index of the multispectral image and the echo enhancement information of the lidar point cloud to obtain the equipment identification result.

[0021] The historical defect features of the target substation scenario are obtained; the historical defect features are compared with multispectral image features, three-dimensional geometric features and environmental interference features to identify the defect status of the equipment in the target substation scenario; based on the identification results, the defect identification results are obtained.

[0022] Based on the equipment identification results and the environmental features, a terrain association analysis is obtained; the equipment identification results, defect identification results, and terrain association analysis are combined as the image recognition results.

[0023] In one embodiment, a three-dimensional terrain model of the target substation scene is constructed based on the composite data and the image recognition results; the spatial position of the three-dimensional terrain model is corrected using a preset technique to obtain a corrected three-dimensional terrain model, including:

[0024] Based on composite data, an initial three-dimensional terrain model of the target substation scene is constructed; the image recognition results are then integrated into the initial three-dimensional terrain model to obtain a three-dimensional terrain model of the target substation scene.

[0025] Obtain a standard 3D model of the target substation scenario; match the 3D terrain model with the standard 3D model; calculate the optimal transformation parameters between the 3D terrain model and the standard 3D model using a preset algorithm; based on the optimal transformation parameters, perform spatial position correction on the image recognition results fused in the 3D terrain model to obtain a corrected 3D terrain model.

[0026] In one embodiment, the method further includes:

[0027] Based on the image recognition results of the target substation scenario, a standardized image recognition report is generated.

[0028] Analyze the correlation between environmental features and image recognition results to generate a dynamic feature change report;

[0029] Dynamic feature change reports and standardized image recognition reports are pushed to the user's device.

[0030] Secondly, this application also provides a substation scenario condition monitoring device, comprising:

[0031] The acquisition module is used to acquire multispectral images, lidar point clouds, and real-time terrain monitoring data of the target substation scene.

[0032] The extraction module is used to perform data preprocessing and feature extraction operations on multispectral images, lidar point clouds and real-time terrain monitoring data in sequence to obtain multispectral image features, three-dimensional geometric features and environmental features.

[0033] The recognition module obtains image recognition results based on multispectral image features, three-dimensional geometric features, and environmental features;

[0034] The mapping module is used to map the spectral information of multispectral images to the lidar point cloud to obtain composite data containing multispectral image features and three-dimensional geometric features.

[0035] The correction module is used to construct a three-dimensional terrain model of the target substation scene based on the composite data and the image recognition results; to perform spatial position correction on the three-dimensional terrain model using preset technology to obtain a corrected three-dimensional terrain model; and to use the corrected three-dimensional terrain model to monitor the equipment and terrain status of the target substation scene.

[0036] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0037] Acquire multispectral images, lidar point clouds, and real-time terrain monitoring data of the target substation scene;

[0038] Data preprocessing and feature extraction operations are performed sequentially on multispectral images, lidar point clouds, and real-time terrain monitoring data to obtain multispectral image features, three-dimensional geometric features, and environmental features.

[0039] Based on multispectral image features, three-dimensional geometric features, and environmental features, image recognition results of the target substation scene are obtained.

[0040] The spectral information of the multispectral image is mapped onto the lidar point cloud to obtain composite data;

[0041] Based on composite data and image recognition results, a three-dimensional terrain model of the target substation scene is constructed; the spatial position of the three-dimensional terrain model is corrected using preset technology to obtain a corrected three-dimensional terrain model; the corrected three-dimensional terrain model is used to monitor the equipment and terrain status of the target substation scene.

[0042] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0043] Acquire multispectral images, lidar point clouds, and real-time terrain monitoring data of the target substation scene;

[0044] Data preprocessing and feature extraction operations are performed sequentially on multispectral images, lidar point clouds, and real-time terrain monitoring data to obtain multispectral image features, three-dimensional geometric features, and environmental features.

[0045] Based on multispectral image features, three-dimensional geometric features, and environmental features, image recognition results of the target substation scene are obtained.

[0046] The spectral information of the multispectral image is mapped onto the lidar point cloud to obtain composite data;

[0047] Based on composite data and image recognition results, a three-dimensional terrain model of the target substation scene is constructed; the spatial position of the three-dimensional terrain model is corrected using preset technology to obtain a corrected three-dimensional terrain model; the corrected three-dimensional terrain model is used to monitor the equipment and terrain status of the target substation scene.

[0048] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0049] Acquire multispectral images, lidar point clouds, and real-time terrain monitoring data of the target substation scene;

[0050] Data preprocessing and feature extraction operations are performed sequentially on multispectral images, lidar point clouds, and real-time terrain monitoring data to obtain multispectral image features, three-dimensional geometric features, and environmental features.

[0051] Based on multispectral image features, three-dimensional geometric features, and environmental features, image recognition results of the target substation scene are obtained.

[0052] The spectral information of the multispectral image is mapped onto the lidar point cloud to obtain composite data;

[0053] Based on composite data and image recognition results, a three-dimensional terrain model of the target substation scene is constructed; the spatial position of the three-dimensional terrain model is corrected using preset technology to obtain a corrected three-dimensional terrain model; the corrected three-dimensional terrain model is used to monitor the equipment and terrain status of the target substation scene.

[0054] The aforementioned substation scenario condition monitoring method, device, computer equipment, computer-readable storage medium, and computer program product acquire multispectral images, lidar point clouds, and real-time terrain monitoring data of the target substation scenario; sequentially perform data preprocessing and feature extraction operations on the multispectral images, lidar point clouds, and real-time terrain monitoring data to obtain multispectral image features, three-dimensional geometric features, and environmental features; based on the multispectral image features, three-dimensional geometric features, and environmental features, obtain image recognition results of the target substation scenario; and obtain composite data by mapping the spectral information of the multispectral images to the lidar point cloud; based on the composite data and image recognition results, construct the target substation scenario... A 3D terrain model of the scene is generated. The spatial position of the 3D terrain model is corrected using preset technology to obtain a corrected 3D terrain model. The corrected 3D terrain model is used to monitor the equipment and terrain status of the target substation scene. It can accurately distinguish between equipment and surface types, different equipment components and equipment defects in the substation scene, effectively solve the problem of vegetation or equipment shading, eliminate the interference of weather or terrain changes on image recognition, and ensure that the recognition results are accurately matched with the actual equipment location. It is suitable for application scenarios with dense high-voltage equipment, complex terrain and variable environment in substations, thereby greatly shortening the time spent on equipment monitoring and defect investigation and reducing the understanding cost of the recognition results for operation and maintenance personnel. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0056] Figure 1 This is an application environment diagram of a substation scenario state monitoring method in one embodiment;

[0057] Figure 2 This is a flowchart illustrating a substation scenario status monitoring method in one embodiment;

[0058] Figure 3 This is a flowchart illustrating a substation scenario status monitoring method in another embodiment;

[0059] Figure 4 This is a structural block diagram of a substation scenario status monitoring device in one embodiment;

[0060] Figure 5 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0062] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0063] The substation scenario status monitoring method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or located on a cloud or other network server. Specifically, terminal 102 or server 104 executes a substation scenario state monitoring method, as follows:

[0064] The process involves acquiring multispectral imagery, lidar point cloud data, and real-time terrain monitoring data of the target substation scenario; performing data preprocessing and feature extraction operations on the multispectral imagery, lidar point cloud data, and real-time terrain monitoring data sequentially to obtain multispectral image features, 3D geometric features, and environmental features; obtaining image recognition results of the target substation scenario based on the multispectral imagery, 3D geometric features, and environmental features; mapping the spectral information of the multispectral imagery to the lidar point cloud to obtain composite data; constructing a 3D terrain model of the target substation scenario based on the composite data and image recognition results; performing spatial position correction on the 3D terrain model using preset techniques to obtain a corrected 3D terrain model; and using the corrected 3D terrain model for monitoring the equipment and terrain status of the target substation scenario.

[0065] Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, drones, low-altitude aircraft, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, and projection equipment. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted displays. Head-mounted displays can be virtual reality (VR) devices, augmented reality (AR) devices, and smart glasses. Server 104 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0066] In one exemplary embodiment, such as Figure 2 As shown, a substation scenario condition monitoring method is provided, which is applied to... Figure 1 Taking terminal 102 as an example, the explanation includes the following steps 202 to 210. Wherein:

[0067] Step 202: Acquire multispectral images, lidar point clouds, and real-time terrain monitoring data of the target substation scene.

[0068] Multispectral imagery refers to a set of images obtained by imaging the same ground feature in multiple different electromagnetic wave bands. LiDAR point cloud refers to the set of three-dimensional spatial data acquired by a LiDAR system. When a LiDAR system emits laser pulses and receives the signals reflected from the surface of an object, it can directly calculate the three-dimensional coordinates of each point. This set of points is called a "point cloud." The acquired 3D point cloud data of the equipment and terrain can be used to help eliminate ambiguity in image planar recognition, provide spatial geometric verification and supplementation for image recognition results, and improve the accuracy of equipment positioning and defect location judgment. Real-time terrain monitoring data includes data on subtle changes in terrain such as tower foundation settlement and displacement, as well as environmental parameters that may affect image quality, such as image blurring caused by fog or overexposure caused by strong light.

[0069] Optionally, multispectral images can be acquired using a high-resolution multispectral camera, which can simultaneously acquire multi-band image data such as visible light, near-infrared, and short-wave infrared.

[0070] Optionally, the lidar point cloud can be acquired using multi-line lidar technology (≥16 lines). Specifically, compared to traditional lidar, multi-line lidar technology can effectively increase the point cloud density by scanning with multiple laser beams simultaneously, significantly reducing the problem of missing point clouds caused by vegetation obstruction and intersecting equipment structures in substation scenarios.

[0071] Optionally, real-time terrain monitoring data can be received from displacement sensors and meteorological sensors deployed around the substation tower base. The displacement sensors have an accuracy of less than or equal to 0.1 mm. The meteorological sensors are used to monitor wind speed, humidity, and visibility.

[0072] Step 204: Perform data preprocessing and feature extraction operations sequentially on the multispectral image, lidar point cloud, and real-time terrain monitoring data to obtain multispectral image features, three-dimensional geometric features, and environmental features.

[0073] Multispectral image features include semantic features, texture features, and spectral features. Environmental features refer to environmental characteristics that can interfere with image information, such as visibility reduction parameters caused by fog, which can be used to correct image grayscale deviations; image reflectivity enhancement parameters caused by heavy rainfall, which can be used to eliminate the interference of rainwater on equipment surface features; and the correlation between terrain changes and equipment attitude. Terrain changes include tower base settlement, and the correlation between equipment attitude includes the tilt angle of the equipment caused by settlement.

[0074] Optionally, based on the multispectral image of the target substation scene, a multidimensional spectral feature library of the target substation scene can be constructed by utilizing the reflectivity differences of different targets in the target substation scene in different bands; spectral features include: the characteristic spectrum of insulator skirts in the shortwave infrared band, the reflectivity of metal supports in the visible light band, etc.; different targets in the target substation scene can be equipment such as insulators, supports and tower bases, or surface types such as vegetation, bare soil and rocks.

[0075] For example, the hierarchical point cloud network (Point Net++) algorithm is used to extract the three-dimensional geometric features from the lidar point cloud.

[0076] Step 206: Based on multispectral image features, three-dimensional geometric features, and environmental features, obtain the image recognition results of the target substation scene.

[0077] The image recognition results include equipment recognition results, defect recognition details, terrain correlation analysis, and accuracy verification data.

[0078] For example, by combining multispectral image features and three-dimensional geometric features for analysis, equipment identification results are obtained; by combining the equipment identification results with environmental features for analysis, terrain correlation analysis is obtained; by comparing multispectral image features, three-dimensional geometric features, and environmental features with historical defect features, defect identification details can be obtained. Accuracy verification data can be obtained from the above analysis process.

[0079] Optionally, defect identification details include the development trend of equipment defects. Periodic image comparisons of key equipment, using feature matching algorithms to compare image features from different periods, can identify the development trend of equipment defects. Development trends include increased rust area and deepening damage. The feature matching algorithm can be a Scale Invariant Feature Transform (SIFT) algorithm. Key equipment includes main transformer insulators and transmission tower supports, etc.

[0080] Step 208: Map the spectral information of the multispectral image onto the lidar point cloud to obtain composite data.

[0081] Complex data refers to data that includes spectral features, geometric features, and semantic features. In this case, each LiDAR point cloud data is accompanied by corresponding image pixel features. Spectral information refers to the characteristic spectral curves of the device's material.

[0082] For example, the spectral information of multispectral images is mapped onto LiDAR point clouds to generate composite data containing spectral features, geometric features, and semantic features. At this time, each point cloud data is accompanied by corresponding image pixel features, which is used to achieve dual recognition effects of image visualization and three-dimensional positioning.

[0083] Step 210: Based on the composite data and image recognition results, construct a three-dimensional terrain model of the target substation scene; use preset technology to perform spatial position correction on the three-dimensional terrain model to obtain a corrected three-dimensional terrain model; use the corrected three-dimensional terrain model to monitor the equipment and terrain status of the target substation scene.

[0084] Among them, the preset technique refers to the Iterative Closest Point (ICP) algorithm.

[0085] For example, the Multi-View Stereo Matching (MVS Net) algorithm is used to construct a 3D terrain model based on composite data. The 3D terrain model is presented as an integrated image of the terrain and equipment. Specifically, the terrain section needs to clearly show the surface morphology of the hills and slopes around the substation and mark the areas of terrain change; the equipment section intuitively presents the 3D structural details of equipment such as substation tower bases, insulators, and supports. The equipment features extracted by image recognition are integrated into the model to enable an interactive function where clicking on the equipment in the model allows users to view the corresponding multispectral image and recognition results. This process must conform to the operating habits of substation operation and maintenance personnel.

[0086] For example, a 3D design model of the substation tower base and equipment is obtained; the optimal transformation parameters between the 3D design model and the 3D terrain model are calculated using preset technology; translation or rotation operations are performed based on the optimal transformation parameters to correct the recognition position deviation caused by image shooting angle and terrain changes, ensuring that the recognition result accurately corresponds to the actual physical location of the equipment. Specifically, the model error must be less than or equal to 3cm.

[0087] In the aforementioned substation scenario status monitoring method, multispectral images, lidar point clouds, and real-time terrain monitoring data of the target substation scenario are acquired; data preprocessing and feature extraction operations are sequentially performed on the multispectral images, lidar point clouds, and real-time terrain monitoring data to obtain multispectral image features, three-dimensional geometric features, and environmental features; based on the multispectral image features, three-dimensional geometric features, and environmental features, image recognition results of the target substation scenario are obtained; and composite data is obtained by mapping the spectral information of the multispectral images to the lidar point clouds; based on the composite data and image recognition results, a three-dimensional terrain model of the target substation scenario is constructed; and preprocessing is employed. The technology performs spatial location correction on a 3D terrain model to obtain a corrected 3D terrain model. This corrected 3D terrain model is then used to monitor the equipment and terrain status of a target substation scenario. It can accurately distinguish between equipment and surface types, different equipment components, and equipment defects in the substation scenario. It effectively solves the problem of vegetation or equipment shading, eliminates the interference of weather or terrain changes on image recognition, and ensures that the recognition results are accurately matched with the actual equipment location. It is suitable for application scenarios where substations have dense high-voltage equipment, complex terrain, and variable environments, thereby significantly shortening the time spent on equipment monitoring and defect investigation and reducing the cost for maintenance personnel to understand the recognition results.

[0088] In an exemplary embodiment, acquiring multispectral images, lidar point clouds, and real-time terrain monitoring data of a target substation scene includes: receiving multispectral images automatically acquired by a multispectral camera at set time intervals; receiving lidar point clouds acquired by lidar scanning the target substation scene; mounting the multispectral camera and lidar on a drone; the drone flying along a pre-planned safe flight path; and receiving real-time terrain monitoring data, which includes real-time displacement data and real-time meteorological data, collected in real-time by displacement sensors and meteorological sensors deployed around the substation tower base.

[0089] Among these requirements, the pre-planned safe flight path must avoid areas where the equipment is energized, and the flight path overlap rate must be greater than or equal to 80% to ensure that the image stitching is seamless.

[0090] For example, the data acquisition process must meet the safety specifications for substation scenarios and the image recognition accuracy requirements.

[0091] Optionally, manually collected data can also be used. Data is collected by operators using a portable device integrating a multispectral camera and lidar, which is slowly moved around the tower base. The focus is on vulnerable parts of the equipment (such as insulator skirts and support connections), capturing images from multiple angles (at least three different viewpoints) to improve the coverage and accuracy of equipment defect image identification. Specifically, the portable device must weigh less than or equal to 5 kg for ease of operation, and the moving speed must be less than 0.5 m / s to ensure complete coverage of the scanning area. High-voltage live areas must be strictly avoided, and a safe distance of greater than or equal to 5 m must be maintained.

[0092] Optionally, drones are responsible for scanning large, conventional areas, while human assistance is used to deal with complex terrain (such as steep slopes), equipment blind spots (such as the bottom of tower bases, behind dense equipment), and other areas that are difficult for drones to cover safely or completely. By combining drones and human assistance, multispectral images, lidar point clouds, and real-time terrain monitoring data of the target substation scene can be obtained.

[0093] For example, the shooting interval of the multispectral camera and the flight altitude of the UAV are set, and the consistency of the spectral response of each band is calibrated. Specifically, the shooting interval of the multispectral camera can be set to once every 2 seconds for areas with complex terrain or dense equipment to ensure that no equipment details are missed; the scanning frequency and echo count of the LiDAR are optimized to ensure that sufficient effective point cloud data can be obtained in densely vegetated areas. A high-precision calibration board is used to achieve spatial coordinate calibration of the multispectral camera and the LiDAR, establishing a precise conversion relationship between the two to ensure accurate registration of image pixels and point cloud data.

[0094] For example, each acquired multispectral image is labeled with associated information to ensure that the image recognition results can be accurately traced back to the specific device and scenario, facilitating subsequent operation and maintenance applications. The associated information includes device number, acquisition time, acquisition location coordinates, and meteorological parameters at the time.

[0095] Optionally, displacement data and meteorological data collected in real time by displacement sensors and meteorological sensors can be received wirelessly (4G / 5G private network, in compliance with substation data security requirements) to provide parameters for environmental interference correction in image recognition. The deployment of sensors must comply with substation safety regulations. Displacement sensors are arranged radially at intervals of 5-10 meters, with the tower base as the center. Meteorological sensors are installed in locations with open, unobstructed views and far away from high-voltage equipment (distance ≥10 meters) to monitor meteorological factors that may interfere with image quality, such as fog, strong light, and heavy rain.

[0096] In this embodiment, data is collected by using a drone equipped with a multispectral camera and lidar, combined with manual assistance. This fully leverages the advantages of the drone's wide-area scanning and the precise focusing of the operator, effectively addressing the challenges of complex terrain and equipment blind spots in substation scenarios. Simultaneously, wireless transmission technology is used to receive sensor data in a timely manner, providing accurate parameters for environmental interference correction in subsequent image recognition. This enhances the accuracy and reliability of the overall substation scenario status monitoring, providing strong support for the stable operation and efficient maintenance of substation equipment.

[0097] In one embodiment, data preprocessing and feature extraction operations are sequentially performed on multispectral imagery, lidar point clouds, and real-time terrain monitoring data to obtain multispectral image features, three-dimensional geometric features, and environmental features. This includes: sequentially performing radiometric calibration, atmospheric correction, and geometric correction operations on the multispectral imagery to obtain a preprocessed multispectral image; classifying and extracting land cover features from the preprocessed multispectral imagery using a preset model to obtain multispectral image features; the multispectral image features include semantic features, texture features, and spectral features; performing data preprocessing on the lidar point cloud using at least one preset preprocessing method to obtain a preprocessed lidar point cloud; extracting the three-dimensional geometric features of the preprocessed lidar point cloud; filtering the real-time terrain monitoring data to obtain preprocessed real-time terrain monitoring data; and extracting the environmental features of the preprocessed real-time terrain monitoring data.

[0098] Among these methods, radiometric calibration is used to eliminate sensor response differences; atmospheric correction is used to eliminate image brightness unevenness caused by atmospheric scattering; and geometric correction is used to correct shooting perspective deviations. The preset model is a deep learning model based on a convolutional neural network (CNN) (such as an improved ResNet-50). At least one preset preprocessing method can be a pass-through filter, voxel filter, or radius filter. Pass-through filtering is used to remove invalid points below the ground; voxel filtering is used to reduce the amount of data while retaining key structures; and radius filtering is used to remove outlier noise points.

[0099] Semantic features refer to features that can distinguish between power equipment and surface types, and can be used to generate heat maps for land cover classification. Power equipment includes insulators, supports, and tower bases, while surface types include vegetation, bare soil, and rock. Texture features include extracting texture information from key parts of the equipment. Utilizing the differences in reflectivity of different targets (equipment and surface types) in various wavelengths within the power substation scene, a defect identification feature library is established. Defects include texture breaks caused by damage or rough textures caused by corrosion. Texture information includes the wavy texture of insulator skirts or the smooth reflective texture of metal supports. Three-dimensional geometric features include local surface normals, curvature, and equipment structural dimensions. Local surface normals are used to determine the flatness of the equipment surface, such as abnormal normals caused by support deformation; curvature is used to identify the corner structures of the equipment, such as the connection points of the tower base; equipment structural dimensions can be data such as insulator height or support diameter.

[0100] For example, a three-dimensional geometric feature model of the device can be constructed to complement the multispectral image features.

[0101] For example, the Kalman filter algorithm is used to filter the dynamic terrain monitoring data to remove abnormal data, such as jump values ​​caused by instantaneous sensor failures.

[0102] For example, parameter data such as visibility reduction parameters caused by fog and image reflectivity enhancement parameters caused by heavy rainfall are extracted. Simultaneously, the correlation between terrain changes and equipment attitude is analyzed, and the parameter data and correlation are used as environmental interference features. Visibility reduction parameters caused by fog can be used to correct image grayscale deviations; image reflectivity enhancement parameters caused by heavy rainfall can be used to eliminate the interference of rainwater on equipment surface features; terrain changes include tower base settlement, etc.; the correlation can be the equipment tilt angle caused by settlement, etc.

[0103] In this embodiment, by systematically preprocessing and extracting features from multispectral images, lidar point clouds, and real-time terrain monitoring data, it is possible to comprehensively acquire multispectral image features, three-dimensional geometric features, and environmental features that reflect the state of the substation scene. Furthermore, through a multi-dimensional feature fusion mechanism, not only is the robustness of equipment identification in complex scenes improved, but a high-precision data foundation is also provided for the subsequent construction of three-dimensional terrain models, ensuring that the monitoring system can adapt to the actual application environment of densely packed substation equipment, undulating terrain, and variable weather.

[0104] In one embodiment, the image recognition result of a target substation scene is obtained by combining multispectral image features, three-dimensional geometric features, and environmental features. This includes: using a preset algorithm to calculate the probability of land cover classification for each data point by combining spectral image features and three-dimensional geometric features; based on the land cover classification probability, distinguishing the substation equipment, surface type, and equipment components in the target substation scene, and determining whether there are vegetation-covered areas; if vegetation-covered areas exist, correcting the equipment or ground information under the vegetation-covered areas based on the vegetation index of the multispectral image and the echo enhancement information of the lidar point cloud to obtain equipment recognition results; obtaining historical defect features of the target substation scene; comparing the historical defect features with multispectral image features, three-dimensional geometric features, and environmental interference features to identify the defect status of the equipment in the target substation scene; obtaining defect recognition results based on the recognition results; obtaining terrain correlation analysis based on the equipment recognition results and environmental features; and using the equipment recognition results, defect recognition results, and terrain correlation analysis together as the image recognition result.

[0105] The probability of land cover category classification includes the probability of equipment category identification and the probability of defect identification, which is part of the accuracy verification data of the standardized image recognition report. The preset algorithm is a Bayesian inference fusion algorithm.

[0106] Optionally, the threshold for the vegetation index (NDVI) can be set to 0.3-0.8. Since the vegetation echo intensity is lower than the device echo intensity, it can be determined whether the laser point penetrates the vegetation to reach the ground or device surface, thereby restoring the device image details that are obscured by vegetation. Device image details include, for example, the base of the tower obscured by shrubs.

[0107] For example, the system integrates geographic information data of substation facilities; extracts defect features from the geographic information data to obtain historical defect features; compares these historical defect features with currently collected features to determine whether new defects exist in the equipment and whether the scope and severity of existing defects have expanded; and combines the geometric information provided by 3D point clouds to more accurately quantify defect areas, lengths, etc., forming defect feature descriptions. Based on the development trend and current severity of defects, this assists in defect risk assessment. The geographic information data of substation facilities includes equipment design drawing coordinates and historical maintenance defect records, ultimately yielding defect identification results.

[0108] For example, the extracted environmental features are correlated with the precise location and attitude information of the device obtained through image recognition for correlation modeling and comprehensive analysis to obtain terrain correlation analysis.

[0109] In this embodiment, by fusing multispectral image features, three-dimensional geometric features, and environmental features, and employing a combination of probability calculation and feature comparison, not only is accurate differentiation between equipment and surface types achieved, but also effective identification of equipment information and defect status under vegetation obstruction. Simultaneously, combined with terrain correlation analysis, the impact of the equipment operating environment on defect development can be comprehensively assessed, providing maintenance personnel with multi-dimensional status monitoring results. This identification method based on multi-source data fusion significantly improves the intelligence level of substation scenario monitoring, providing a reliable basis for equipment status assessment and defect prediction.

[0110] In one embodiment, a three-dimensional terrain model of the target substation scene is constructed based on composite data and the image recognition results; the spatial position of the three-dimensional terrain model is corrected using a preset technique to obtain a corrected three-dimensional terrain model, including: constructing an initial three-dimensional terrain model of the target substation scene based on composite data; integrating the image recognition results into the initial three-dimensional terrain model to obtain a three-dimensional terrain model of the target substation scene; obtaining a standard three-dimensional model of the target substation scene; matching the three-dimensional terrain model with the standard three-dimensional model; calculating the optimal transformation parameters between the three-dimensional terrain model and the standard three-dimensional model using a preset algorithm; and performing spatial position correction on the image recognition results fused in the three-dimensional terrain model based on the optimal transformation parameters to obtain a corrected three-dimensional terrain model.

[0111] The standard 3D model is a design 3D model of the substation tower base and equipment, which must include equipment coordinates and dimensional parameters.

[0112] For example, based on composite data, an initial 3D terrain model of the target substation scene is constructed using a multi-view stereo matching algorithm. Image recognition results are then integrated into the initial 3D terrain model to obtain a 3D terrain model of the target substation scene. A standard 3D model of the target substation scene is obtained. The 3D terrain model is matched with the standard 3D model. A preset algorithm is used to calculate the optimal transformation parameters between the 3D terrain model and the standard 3D model. Based on the optimal transformation parameters, the spatial position of the image recognition results fused in the 3D terrain model is corrected. After performing translation or rotation operations, a corrected 3D terrain model is obtained. Equipment features extracted from image recognition are integrated into the model to correct recognition position deviations caused by image shooting angles and terrain changes, ensuring that the recognition results accurately correspond to the actual physical location of the equipment. This enables an interactive function where clicking on the model equipment allows viewing the corresponding multispectral image and recognition results. This process must conform to the operating habits of substation maintenance personnel.

[0113] In this embodiment, by constructing a three-dimensional terrain model of the target substation scenario and correcting the initial three-dimensional terrain model, a corrected three-dimensional terrain model is obtained. This ensures that the constructed three-dimensional terrain model not only has high accuracy but also intuitively presents the terrain changes and equipment status in the substation scenario, providing maintenance personnel with comprehensive and real-time monitoring information.

[0114] In one embodiment, the method further includes: generating a standardized image recognition report based on the image recognition results of the target substation scenario; analyzing the correlation between environmental features and image recognition results to generate a dynamic feature change report; and pushing the dynamic feature change report and the standardized image recognition report to the user terminal.

[0115] The standardized image recognition report includes: equipment identification results, defect identification details, terrain correlation analysis, and accuracy verification data. Equipment identification results include the quantity statistics, three-dimensional coordinates of the location, and status assessment of each piece of equipment within the substation. Equipment includes tower bases, insulators, supports, etc.; the status assessment includes whether the equipment is in normal condition, the type and level of defects it contains, etc. Defect identification details include defect location, defect feature description, and defect risk assessment. Defect locations must be accompanied by multispectral image annotations and three-dimensional model annotations; defect feature descriptions include the damaged area of ​​insulator skirts and the length of corrosion on supports, etc. Accuracy verification data includes equipment identification accuracy, defect identification accuracy, and positioning error analysis. Equipment identification accuracy must be greater than or equal to 95%, defect identification accuracy must be greater than or equal to 90%, and positioning error must be less than or equal to 5cm. Terrain correlation analysis includes the relationship between terrain change areas and equipment location, and an assessment of the impact of terrain changes on equipment image features; terrain changes include settlement and displacement.

[0116] For example, the standardized image recognition report is pushed to the substation operation and maintenance management system, i.e., the user terminal, to provide data support for the user's operation and maintenance decisions, such as: formulating maintenance plans based on defect recognition results, formulating tower foundation reinforcement schemes based on terrain change assessments, and ensuring the stable operation of the substation system.

[0117] Optionally, the device attitude parameters are updated in real time based on the environmental characteristics of the real-time displacement data, and the impact of environmental factors on the device image features is analyzed in conjunction with the environmental characteristics of the real-time meteorological data. The risks that may lead to changes in image recognition features are predicted, and a dynamic feature change report is generated.

[0118] Specifically, the risks of feature changes can include tower base tilt causing a shift in the equipment's image viewpoint, or rain cover altering the spectral characteristics of the equipment surface. Environmental features from real-time meteorological data include equipment swaying caused by strong winds. Equipment attitude parameters include tilt angle and positional offset. Dynamic feature change reports are used for early warning and diagnosis in power operation; these reports continuously record the temporal changes in image features of key equipment components, surrounding terrain, and environmental parameters.

[0119] Optionally, dynamic characteristic change reports can be pushed to the user end, allowing users to analyze trends and receive early warnings of potential risks. For example, if an accelerated settlement rate of the tower base or continuous deterioration of the texture characteristics of specific parts of the equipment is detected, it may indicate that structural defects are imminent or that reinforcement is needed.

[0120] In this embodiment, by generating standardized image recognition reports and dynamic feature change reports and pushing these reports to the user terminal, maintenance personnel can obtain detailed status information of the substation scenario in a timely manner. This helps maintenance personnel to predict potential risks, take early warning and diagnosis measures, significantly improve the accuracy and timeliness of substation scenario status monitoring, reduce maintenance costs, and ensure the stable operation of the substation system.

[0121] The following is for reference. Figure 3 The present application will be illustrated with a specific embodiment of a substation scenario state monitoring method.

[0122] Step 1: Collect data.

[0123] A drone equipped with a high-resolution multispectral camera and lidar operates along a pre-planned flight path. Simultaneously, operators, holding portable devices integrating multispectral cameras and lidar, slowly move around the tower base to scan the substation tower base and equipment area. The terminal receives the acquired multispectral images from the high-resolution multispectral camera and lidar, as well as the lidar point cloud.

[0124] Displacement sensors and meteorological sensors are deployed at the substation tower base and around the equipment area, transmitting real-time terrain monitoring data via a wireless network. The terminal receives the real-time terrain monitoring data.

[0125] Step 2: Perform data preprocessing and feature extraction on the data.

[0126] After performing data preprocessing on multispectral images, lidar point clouds, and real-time terrain monitoring data, the terminal extracts multispectral image features, three-dimensional geometric features, and environmental features.

[0127] Step 3: Perform image analysis based on features to obtain image recognition results.

[0128] The terminal combines multispectral image features and three-dimensional geometric features to calculate the probability of land cover classification for each data point. Based on the land cover classification probability, it distinguishes the substation equipment, surface type and equipment components in the target substation scene, and determines whether there are vegetation occlusion areas. If there are vegetation occlusion areas, it corrects the equipment or ground information under the vegetation occlusion area based on the vegetation index of the multispectral image and the echo enhancement information of the lidar point cloud to obtain the equipment identification result.

[0129] The system acquires historical defect features of the target substation scenario; compares these historical defect features with multispectral image features, three-dimensional geometric features, and environmental interference features to identify the defect status of equipment in the target substation scenario; and finally, based on the identification results, the terminal obtains the defect identification results.

[0130] Based on the device identification results and environmental characteristics, the terminal obtains terrain correlation analysis.

[0131] During the process of acquiring equipment identification results, defect identification results, and terrain correlation analysis, the terminal can obtain accuracy verification data.

[0132] Step 4: Construct a three-dimensional terrain model based on the image recognition results.

[0133] The spectral information of the multispectral image is mapped onto the lidar point cloud to obtain composite data; based on the composite data, an initial three-dimensional terrain model of the target substation scene is constructed; the image recognition results are integrated into the initial three-dimensional terrain model to obtain a three-dimensional terrain model of the target substation scene.

[0134] Step 5: Perform position correction on the 3D terrain model.

[0135] A standard 3D model of the target substation scenario is obtained; a 3D terrain model is matched with the standard 3D model; the optimal transformation parameters between the 3D terrain model and the standard 3D model are calculated; based on the optimal transformation parameters, the spatial position correction is performed on the image recognition results fused in the 3D terrain model to obtain a corrected 3D terrain model. The corrected 3D terrain model is used to monitor the equipment and terrain status of the target substation scenario.

[0136] Step 6: Based on the image recognition results, output a standardized image recognition report.

[0137] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0138] Based on the same inventive concept, this application also provides a substation scenario state monitoring device for implementing the substation scenario state monitoring method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more substation scenario state monitoring device embodiments provided below can be found in the limitations of the substation scenario state monitoring method described above, and will not be repeated here.

[0139] In one exemplary embodiment, such as Figure 4 As shown, a substation scenario state monitoring device 400 is provided, including: an acquisition module 402, an extraction module 404, an identification module 406, a mapping module 408, and a correction module 410, wherein:

[0140] The acquisition module 402 is used to acquire multispectral images, lidar point clouds, and real-time terrain monitoring data of the target substation scene.

[0141] The extraction module 404 is used to perform data preprocessing and feature extraction operations on multispectral images, lidar point clouds and real-time terrain monitoring data in sequence to obtain multispectral image features, three-dimensional geometric features and environmental features.

[0142] The recognition module 406 acquires image recognition results based on multispectral image features, three-dimensional geometric features, and environmental features.

[0143] The mapping module 408 is used to map the spectral information of the multispectral image to the lidar point cloud to obtain composite data containing multispectral image features and three-dimensional geometric features.

[0144] The correction module 410 is used to construct a three-dimensional terrain model of the target substation scene based on the composite data and the image recognition results; to perform spatial position correction on the three-dimensional terrain model using a preset technology to obtain a corrected three-dimensional terrain model; and to use the corrected three-dimensional terrain model to monitor the equipment and terrain status of the target substation scene.

[0145] In one embodiment, the acquisition module is further configured to receive multispectral images automatically acquired by a multispectral camera at set time intervals; receive lidar point clouds acquired by lidar scanning the target substation scene; the multispectral camera and lidar are mounted on a drone; the drone flies along a pre-planned safe route; and receive real-time terrain monitoring data; the real-time terrain monitoring data includes real-time displacement data and real-time meteorological data; the real-time displacement data and real-time meteorological data are collected in real time by displacement sensors and meteorological sensors deployed around the substation tower base.

[0146] In one embodiment, the extraction module is further configured to sequentially perform radiometric calibration, atmospheric correction, and geometric correction operations on the multispectral image to obtain a preprocessed multispectral image; perform land cover classification and feature extraction on the preprocessed multispectral image using a preset model to obtain multispectral image features; the multispectral image features include semantic features, texture features, and spectral features; perform data preprocessing on the lidar point cloud using at least one preset preprocessing method to obtain a preprocessed lidar point cloud; extract the three-dimensional geometric features of the preprocessed lidar point cloud; perform filtering processing on the real-time terrain monitoring data to obtain preprocessed real-time terrain monitoring data; and extract the environmental features of the preprocessed real-time terrain monitoring data.

[0147] In one embodiment, the identification module is further configured to use a preset algorithm to calculate the probability of land cover category attribution for each data point by combining spectral image features and three-dimensional geometric features; based on the land cover category attribution probability, distinguish the substation equipment, surface type, and equipment components in the target substation scene, and determine whether there are vegetation occlusion areas; if there are vegetation occlusion areas, correct the equipment or ground information under the vegetation occlusion area based on the vegetation index of the multispectral image and the echo enhancement information of the lidar point cloud to obtain the equipment identification result; acquire historical defect features of the target substation scene; compare the historical defect features with multispectral image features, three-dimensional geometric features, and environmental interference features to identify the defect status of the equipment in the target substation scene; obtain the defect identification result based on the identification result; obtain the terrain association analysis based on the equipment identification result and environmental features; and use the equipment identification result, defect identification result, and terrain association analysis together as the image identification result.

[0148] In one embodiment, the correction module is further configured to: construct an initial three-dimensional terrain model of the target substation scenario based on composite data; integrate the image recognition results into the initial three-dimensional terrain model to obtain a three-dimensional terrain model of the target substation scenario; obtain a standard three-dimensional model of the target substation scenario; match the three-dimensional terrain model with the standard three-dimensional model; calculate the optimal transformation parameters between the three-dimensional terrain model and the standard three-dimensional model using a preset algorithm; and perform spatial position correction on the image recognition results fused in the three-dimensional terrain model based on the optimal transformation parameters to obtain a corrected three-dimensional terrain model.

[0149] In one embodiment, the substation scenario status monitoring device further includes a push module, which is used to generate a standardized image recognition report based on the image recognition results of the target substation scenario; analyze the correlation between environmental features and image recognition results to generate a dynamic feature change report; and push the dynamic feature change report and the standardized image recognition report to the user terminal.

[0150] Each module in the aforementioned substation scenario status monitoring device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0151] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 5As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements a method for monitoring the state of a substation scenario. The display unit of the computer device is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0152] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0153] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0154] Acquire multispectral images, lidar point clouds, and real-time terrain monitoring data of the target substation scene;

[0155] Data preprocessing and feature extraction operations are performed sequentially on multispectral images, lidar point clouds, and real-time terrain monitoring data to obtain multispectral image features, three-dimensional geometric features, and environmental features.

[0156] Based on multispectral image features, three-dimensional geometric features, and environmental features, image recognition results of the target substation scene are obtained.

[0157] The spectral information of the multispectral image is mapped onto the lidar point cloud to obtain composite data;

[0158] Based on composite data and image recognition results, a three-dimensional terrain model of the target substation scene is constructed; the spatial position of the three-dimensional terrain model is corrected using preset technology to obtain a corrected three-dimensional terrain model; the corrected three-dimensional terrain model is used to monitor the equipment and terrain status of the target substation scene.

[0159] In one embodiment, when the processor executes the computer program, it also performs the following steps: receiving multispectral images automatically acquired by a multispectral camera at set time intervals; receiving lidar point clouds obtained by lidar scanning the target substation scene; the multispectral camera and lidar are mounted on a drone; the drone flies along a pre-planned safe route; receiving real-time terrain monitoring data; the real-time terrain monitoring data includes real-time displacement data and real-time meteorological data; the real-time displacement data and real-time meteorological data are collected in real time by displacement sensors and meteorological sensors deployed around the substation tower base.

[0160] In one embodiment, when the processor executes the computer program, it further performs the following steps: sequentially performing radiometric calibration, atmospheric correction, and geometric correction operations on the multispectral image to obtain a preprocessed multispectral image; classifying and extracting land features from the preprocessed multispectral image using a preset model to obtain multispectral image features; the multispectral image features include semantic features, texture features, and spectral features; performing data preprocessing on the lidar point cloud using at least one preset preprocessing method to obtain a preprocessed lidar point cloud; extracting the three-dimensional geometric features of the preprocessed lidar point cloud; filtering the real-time terrain monitoring data to obtain preprocessed real-time terrain monitoring data; and extracting the environmental features of the preprocessed real-time terrain monitoring data.

[0161] In one embodiment, when the processor executes the computer program, it further performs the following steps: using a preset algorithm, combining spectral image features and three-dimensional geometric features to calculate the probability of land cover category attribution for each data point; based on the probability of land cover category attribution, distinguishing the substation equipment, surface type, and equipment components in the target substation scene, and determining whether there are vegetation occlusion areas; if there are vegetation occlusion areas, correcting the equipment or ground information under the vegetation occlusion area based on the vegetation index of the multispectral image and the echo enhancement information of the lidar point cloud to obtain the equipment identification result; acquiring historical defect features of the target substation scene; comparing the historical defect features with multispectral image features, three-dimensional geometric features, and environmental interference features to identify the defect status of the equipment in the target substation scene; obtaining the defect identification result based on the identification result; obtaining terrain correlation analysis based on the equipment identification result and environmental features; and using the equipment identification result, defect identification result, and terrain correlation analysis together as the image recognition result.

[0162] In one embodiment, when the processor executes the computer program, it further performs the following steps: constructing an initial three-dimensional terrain model of the target substation scene based on composite data; integrating the image recognition results into the initial three-dimensional terrain model to obtain a three-dimensional terrain model of the target substation scene; obtaining a standard three-dimensional model of the target substation scene; matching the three-dimensional terrain model with the standard three-dimensional model; calculating the optimal transformation parameters between the three-dimensional terrain model and the standard three-dimensional model using a preset algorithm; and performing spatial position correction on the image recognition results fused in the three-dimensional terrain model based on the optimal transformation parameters to obtain a corrected three-dimensional terrain model.

[0163] In one embodiment, when the processor executes the computer program, it also performs the following steps: generating a standardized image recognition report based on the image recognition results of the target substation scenario; analyzing the correlation between environmental features and image recognition results to generate a dynamic feature change report; and pushing the dynamic feature change report and the standardized image recognition report to the user terminal.

[0164] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0165] Acquire multispectral images, lidar point clouds, and real-time terrain monitoring data of the target substation scene;

[0166] Data preprocessing and feature extraction operations are performed sequentially on multispectral images, lidar point clouds, and real-time terrain monitoring data to obtain multispectral image features, three-dimensional geometric features, and environmental features.

[0167] Based on multispectral image features, three-dimensional geometric features, and environmental features, image recognition results of the target substation scene are obtained.

[0168] The spectral information of the multispectral image is mapped onto the lidar point cloud to obtain composite data;

[0169] Based on composite data and image recognition results, a three-dimensional terrain model of the target substation scene is constructed; the spatial position of the three-dimensional terrain model is corrected using preset technology to obtain a corrected three-dimensional terrain model; the corrected three-dimensional terrain model is used to monitor the equipment and terrain status of the target substation scene.

[0170] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: receiving multispectral images automatically acquired by a multispectral camera at set time intervals; receiving lidar point clouds obtained by lidar scanning the target substation scene; the multispectral camera and lidar are mounted on a drone; the drone flies along a pre-planned safe route; receiving real-time terrain monitoring data; the real-time terrain monitoring data includes real-time displacement data and real-time meteorological data; the real-time displacement data and real-time meteorological data are collected in real time by displacement sensors and meteorological sensors deployed around the substation tower base.

[0171] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: sequentially performing radiometric calibration, atmospheric correction, and geometric correction operations on the multispectral image to obtain a preprocessed multispectral image; classifying and extracting land cover features from the preprocessed multispectral image using a preset model to obtain multispectral image features; the multispectral image features include semantic features, texture features, and spectral features; performing data preprocessing on the lidar point cloud using at least one preset preprocessing method to obtain a preprocessed lidar point cloud; extracting the three-dimensional geometric features of the preprocessed lidar point cloud; filtering the real-time terrain monitoring data to obtain preprocessed real-time terrain monitoring data; and extracting the environmental features of the preprocessed real-time terrain monitoring data.

[0172] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: using a preset algorithm, combining spectral image features and three-dimensional geometric features to calculate the probability of land cover category attribution for each data point; based on the probability of land cover category attribution, distinguishing the substation equipment, surface type, and equipment components in the target substation scene, and determining whether there are vegetation occlusion areas; if there are vegetation occlusion areas, correcting the equipment or ground information under the vegetation occlusion area based on the vegetation index of the multispectral image and the echo enhancement information of the lidar point cloud to obtain the equipment identification result; acquiring historical defect features of the target substation scene; comparing the historical defect features with multispectral image features, three-dimensional geometric features, and environmental interference features to identify the defect status of the equipment in the target substation scene; obtaining the defect identification result based on the identification result; obtaining terrain correlation analysis based on the equipment identification result and environmental features; and using the equipment identification result, defect identification result, and terrain correlation analysis together as the image recognition result.

[0173] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: constructing an initial three-dimensional terrain model of the target substation scene based on composite data; integrating the image recognition results into the initial three-dimensional terrain model to obtain a three-dimensional terrain model of the target substation scene; obtaining a standard three-dimensional model of the target substation scene; matching the three-dimensional terrain model with the standard three-dimensional model; calculating the optimal transformation parameters between the three-dimensional terrain model and the standard three-dimensional model using a preset algorithm; and performing spatial position correction on the image recognition results fused in the three-dimensional terrain model based on the optimal transformation parameters to obtain a corrected three-dimensional terrain model.

[0174] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: generating a standardized image recognition report based on the image recognition results of the target substation scenario; analyzing the correlation between environmental features and image recognition results to generate a dynamic feature change report; and pushing the dynamic feature change report and the standardized image recognition report to the user terminal.

[0175] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps:

[0176] Acquire multispectral images, lidar point clouds, and real-time terrain monitoring data of the target substation scene;

[0177] Data preprocessing and feature extraction operations are performed sequentially on multispectral images, lidar point clouds, and real-time terrain monitoring data to obtain multispectral image features, three-dimensional geometric features, and environmental features.

[0178] Based on multispectral image features, three-dimensional geometric features, and environmental features, image recognition results of the target substation scene are obtained.

[0179] The spectral information of the multispectral image is mapped onto the lidar point cloud to obtain composite data;

[0180] Based on composite data and image recognition results, a three-dimensional terrain model of the target substation scene is constructed; the spatial position of the three-dimensional terrain model is corrected using preset technology to obtain a corrected three-dimensional terrain model; the corrected three-dimensional terrain model is used to monitor the equipment and terrain status of the target substation scene.

[0181] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: receiving multispectral images automatically acquired by a multispectral camera at set time intervals; receiving lidar point clouds obtained by lidar scanning the target substation scene; the multispectral camera and lidar are mounted on a drone; the drone flies along a pre-planned safe route; receiving real-time terrain monitoring data; the real-time terrain monitoring data includes real-time displacement data and real-time meteorological data; the real-time displacement data and real-time meteorological data are collected in real time by displacement sensors and meteorological sensors deployed around the substation tower base.

[0182] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: sequentially performing radiometric calibration, atmospheric correction, and geometric correction operations on the multispectral image to obtain a preprocessed multispectral image; classifying and extracting land cover features from the preprocessed multispectral image using a preset model to obtain multispectral image features; the multispectral image features include semantic features, texture features, and spectral features; performing data preprocessing on the lidar point cloud using at least one preset preprocessing method to obtain a preprocessed lidar point cloud; extracting the three-dimensional geometric features of the preprocessed lidar point cloud; filtering the real-time terrain monitoring data to obtain preprocessed real-time terrain monitoring data; and extracting the environmental features of the preprocessed real-time terrain monitoring data.

[0183] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: using a preset algorithm, combining spectral image features and three-dimensional geometric features to calculate the probability of land cover category attribution for each data point; based on the probability of land cover category attribution, distinguishing the substation equipment, surface type, and equipment components in the target substation scene, and determining whether there are vegetation occlusion areas; if there are vegetation occlusion areas, correcting the equipment or ground information under the vegetation occlusion area based on the vegetation index of the multispectral image and the echo enhancement information of the lidar point cloud to obtain the equipment identification result; acquiring historical defect features of the target substation scene; comparing the historical defect features with multispectral image features, three-dimensional geometric features, and environmental interference features to identify the defect status of the equipment in the target substation scene; obtaining the defect identification result based on the identification result; obtaining terrain correlation analysis based on the equipment identification result and environmental features; and using the equipment identification result, defect identification result, and terrain correlation analysis together as the image recognition result.

[0184] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: constructing an initial three-dimensional terrain model of the target substation scene based on composite data; integrating the image recognition results into the initial three-dimensional terrain model to obtain a three-dimensional terrain model of the target substation scene; obtaining a standard three-dimensional model of the target substation scene; matching the three-dimensional terrain model with the standard three-dimensional model; calculating the optimal transformation parameters between the three-dimensional terrain model and the standard three-dimensional model using a preset algorithm; and performing spatial position correction on the image recognition results fused in the three-dimensional terrain model based on the optimal transformation parameters to obtain a corrected three-dimensional terrain model.

[0185] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: generating a standardized image recognition report based on the image recognition results of the target substation scenario; analyzing the correlation between environmental features and image recognition results to generate a dynamic feature change report; and pushing the dynamic feature change report and the standardized image recognition report to the user terminal.

[0186] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0187] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0188] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0189] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for monitoring the condition of a substation scenario, characterized in that, The method includes: Acquire multispectral images, lidar point clouds, and real-time terrain monitoring data of the target substation scene; Data preprocessing and feature extraction operations are performed sequentially on the multispectral images, lidar point clouds, and real-time terrain monitoring data to obtain multispectral image features, three-dimensional geometric features, and environmental features. Based on the multispectral image features, three-dimensional geometric features, and environmental features, the image recognition results of the target substation scene are obtained; The spectral information of the multispectral image is mapped onto the lidar point cloud to obtain composite data; Based on the composite data and the image recognition results, a three-dimensional terrain model of the target substation scene is constructed; the spatial position of the three-dimensional terrain model is corrected using a preset technology to obtain a corrected three-dimensional terrain model; the corrected three-dimensional terrain model is used to monitor the equipment and terrain status of the target substation scene.

2. The method according to claim 1, characterized in that, The acquisition of multispectral imagery, lidar point cloud data, and real-time terrain monitoring data of the target substation scene includes: The system receives multispectral images automatically acquired by a multispectral camera at set time intervals; it also receives lidar point clouds obtained by lidar scanning the target substation scene; the multispectral camera and the lidar are mounted on a drone; the drone flies along a pre-planned safe route. Receive real-time terrain monitoring data; the real-time terrain monitoring data includes real-time displacement data and real-time meteorological data; the real-time displacement data and real-time meteorological data are collected in real time by displacement sensors and meteorological sensors deployed around the substation tower base.

3. The method according to claim 1, characterized in that, The process involves sequentially performing data preprocessing and feature extraction operations on the multispectral imagery, lidar point cloud data, and real-time terrain monitoring data to obtain multispectral image features, three-dimensional geometric features, and environmental features, including: Radiometric calibration, atmospheric correction, and geometric correction are sequentially performed on the multispectral image to obtain a preprocessed multispectral image. A preset model is then used to classify land cover and extract features from the preprocessed multispectral image to obtain multispectral image features. The multispectral image features include semantic features, texture features, and spectral features. The lidar point cloud is preprocessed using at least one preset preprocessing method to obtain a preprocessed lidar point cloud; the three-dimensional geometric features of the preprocessed lidar point cloud are then extracted. The real-time terrain monitoring data is filtered to obtain preprocessed real-time terrain monitoring data; environmental features of the preprocessed real-time terrain monitoring data are then extracted.

4. The method according to claim 1, characterized in that, The process of obtaining image recognition results based on the multispectral image features, three-dimensional geometric features, and environmental features includes: A preset algorithm is used to calculate the probability of land cover category attribution for each data point by combining the spectral image features and the three-dimensional geometric features. Based on the land cover category attribution probability, the power equipment, surface type and equipment components of the target substation scene are distinguished, and it is determined whether there are vegetation occlusion areas. If there are vegetation occlusion areas, the equipment or ground information under the vegetation occlusion areas is corrected based on the vegetation index of the multispectral image and the echo enhancement information of the lidar point cloud to obtain the equipment identification result. The historical defect features of the target substation scenario are obtained; the historical defect features are compared with the multispectral image features, three-dimensional geometric features and environmental interference features to identify the defect status of the equipment in the target substation scenario; based on the identification results, the defect identification results are obtained. Based on the device identification results and the environmental features, a terrain association analysis is obtained; the device identification results, defect identification results, and terrain association analysis are combined as the image recognition results.

5. The method according to claim 1, characterized in that, Based on the composite data and the image recognition results, a three-dimensional terrain model of the target substation scene is constructed. The spatial position of the three-dimensional terrain model is corrected using a preset technique to obtain a corrected three-dimensional terrain model, including: Based on the composite data, an initial three-dimensional terrain model of the target substation scenario is constructed; The image recognition results are integrated into the initial three-dimensional terrain model to obtain the three-dimensional terrain model of the target substation scene; Obtain a standard 3D model of the target substation scenario; match the 3D terrain model with the standard 3D model; calculate the optimal transformation parameters between the 3D terrain model and the standard 3D model using a preset algorithm; and perform spatial position correction on the image recognition results fused in the 3D terrain model based on the optimal transformation parameters to obtain a corrected 3D terrain model.

6. The method according to claim 1, characterized in that, The method further includes: Based on the image recognition results of the target substation scenario, a standardized image recognition report is generated; Analyze the correlation between the environmental features and the image recognition results to generate a dynamic feature change report; The dynamic feature change report and the standardized image recognition report are pushed to the user terminal.

7. A substation scenario condition monitoring device, characterized in that, The device includes: The acquisition module is used to acquire multispectral images, lidar point clouds, and real-time terrain monitoring data of the target substation scene; The extraction module is used to sequentially perform data preprocessing and feature extraction operations on the multispectral image, lidar point cloud and real-time terrain monitoring data to obtain multispectral image features, three-dimensional geometric features and environmental features; The recognition module obtains image recognition results based on the multispectral image features, three-dimensional geometric features, and environmental features; The mapping module is used to map the spectral information of the multispectral image to the lidar point cloud to obtain composite data containing the multispectral image features and the three-dimensional geometric features. The correction module is used to construct a three-dimensional terrain model of the target substation scene based on the composite data and the image recognition results; to perform spatial position correction on the three-dimensional terrain model using a preset technology to obtain a corrected three-dimensional terrain model; the corrected three-dimensional terrain model is used to monitor the equipment and terrain status of the target substation scene.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images