Energy infrastructure condition assessment system, energy infrastructure condition assessment method, and energy infrastructure condition assessment program

Abstract

This system eliminates the need to add equipment to energy infrastructure, shortens inspection time, facilitates frequent inspections, and enables early detection of abnormalities and deterioration, thereby preventing large-scale accidents and service disruptions. [Solution] The system includes a point cloud data acquisition unit that acquires 3D point cloud data generated by 3D scanning the energy infrastructure, an image acquisition unit that acquires image data generated by imaging the energy infrastructure, a spectral image acquisition unit that acquires multispectral image data based on specific wavelengths reflected from the energy infrastructure, a 3D position acquisition unit that acquires position and attitude data equipped with at least a GPS receiver and an inertial measuring instrument, and a mobile body on which these acquisition units are mounted. The health of the energy infrastructure is evaluated in real time based on the 3D point cloud data, image data, multispectral image data, and position and attitude data acquired by moving or stationary the mobile body.

Description

Technical Field

[0001] The present invention relates to an energy infrastructure condition evaluation system, an energy infrastructure condition evaluation method, and an energy infrastructure condition evaluation program that can appropriately perform condition evaluations such as abnormalities and aging by monitoring energy infrastructures such as power lines and pipelines.

Background Art

[0002] Conventional monitoring of energy infrastructures such as power lines and pipelines has relied on physical inspections where specialized workers go to the site to directly check the infrastructure. Also, conventional maintenance has mostly been "reactive maintenance" that responds after damage or failure occurs. However, with such an approach, since repairs or replacements are carried out after a problem occurs, there are disadvantages such as unexpected downtime and high repair costs. For this reason, specialized workers go to the site to check the condition of the energy infrastructure by visual inspection and manual measurement, but in human condition confirmation, there are limits to the confirmation accuracy and work ability. Therefore, in overhead transmission lines, self-propelled devices that inspect the transmission line while self-propelling along the transmission line (see Patent Documents 1, 2, etc.) have been proposed.

[0003] Also, in pipelines, techniques have been proposed such as arranging inspection metal blocks that can be moved within the pipeline, transmitting ultrasonic waves toward the inner wall while moving this inspection metal block in the gas flow direction to measure the thickness of the pipe and any defective parts (see Patent Document 3), and a technique that includes a signal source that applies an alternating current or a pulse signal to the pipeline and a signal detector that detects changes in the signal applied to the pipeline, and the signal detector detects changes in the electrical quantity of the signal due to damage occurring in the pipeline to detect the location of the damage (see Patent Document 4).

Prior Art Documents

Patent Documents

[0004] [Patent Document 1] Japanese Patent Publication No. 2006-254567 [Patent Document 2] Japanese Patent Publication No. 2015-177727 [Patent Document 3] Special Publication No. 11-504110 [Patent Document 4] Japanese Patent Application Publication No. 62-081557 [Overview of the Initiative] [Problems that the invention aims to solve]

[0005] However, all of the above technologies require the addition of equipment to the energy infrastructure, necessitating the installation of equipment for each power line or pipeline. This makes inspections time-consuming and difficult to perform at short intervals. Furthermore, monitoring and maintaining energy infrastructure, especially in remote or mountainous areas, is difficult and time-consuming. Moreover, in these hard-to-access areas, physical inspections are only conducted periodically, making it difficult to detect damage in its early stages, often resulting in the need for extensive repairs. These traditional approaches involve significant human resources and financial costs, placing a heavy burden on companies and organizations, especially those with large-scale energy infrastructure.

[0006] Furthermore, conventional methods were unable to optimize energy supply in real time in accordance with the state of the energy infrastructure, making it difficult to address energy waste and shortages. In other words, conventional methods for evaluating the state of energy infrastructure made it difficult to accurately grasp the state of the infrastructure and the balance of energy supply and demand, which prevented effective energy supply planning and decision-making.

[0007] This invention has been made in view of the above circumstances, and its main objective is to provide an energy infrastructure condition evaluation system, energy infrastructure condition evaluation method, and energy infrastructure condition evaluation program that do not require the addition of equipment to the infrastructure, shorten the time required for inspections, facilitate frequent inspections, and enable early detection of abnormalities and deterioration, thereby avoiding large-scale accidents and service interruptions, as well as being able to detect signs of abnormalities and deterioration. Furthermore, it aims to provide a technology that enables real-time optimization of energy supply. [Means for solving the problem]

[0008] To achieve the above objectives, the energy infrastructure condition evaluation system according to the present invention is An energy infrastructure condition evaluation system for evaluating the state of energy infrastructure to be evaluated, A point cloud data acquisition unit that acquires 3D point cloud data generated by 3D scanning the aforementioned energy infrastructure, The image acquisition unit acquires image data generated by imaging the aforementioned energy infrastructure, A spectral image acquisition unit acquires multispectral image data, including infrared image data, based on specific wavelengths reflected from the aforementioned energy infrastructure. A 3D position acquisition unit that acquires position and attitude data, comprising at least a GPS receiver and an inertial measuring instrument, A mobile body equipped with the point cloud data acquisition unit, the image acquisition unit, the spectral image acquisition unit, and the 3D position measurement unit, An acquired data storage unit stores the 3D point cloud data, image data, multispectral image data, and position / orientation information of the energy infrastructure acquired while the moving body is moving or while the moving body is stopped. Based on the acquired data, an infrastructure evaluation unit evaluates the health of the energy infrastructure in real time, It is characterized by possessing the following features.

[0009] Therefore, the health of energy infrastructure can be evaluated in real time based on 3D point cloud data obtained by a point cloud data acquisition unit mounted on a mobile device, image data obtained by an image acquisition unit, and multispectral image data obtained by a spectral image acquisition unit. This allows for monitoring and maintenance of energy infrastructure in remote or mountainous areas that are difficult to access, without requiring additional equipment to be added to the infrastructure. It also shortens the time required for inspections and allows for inspections to be performed at shorter intervals, enabling early detection of abnormalities, which can prevent large-scale accidents and service interruptions, as well as detecting signs of abnormalities and deterioration.

[0010] Here, the mobile entity may be a vehicle equipped with the aforementioned acquisition units, an aircraft including an unmanned aerial vehicle such as a drone, or both. The choice of entity should be appropriate to the environment of the access area, such as remote locations or mountainous regions. Energy infrastructure also includes power lines, transmission grids, power plants, oil and gas pipelines, and renewable energy facilities (such as wind and solar power plants). The assessment of the infrastructure's integrity by the Infrastructure Evaluation Department includes determining whether there are any abnormalities in the energy infrastructure and assessing the degree of deterioration. Here, abnormalities include not only damage but also deterioration, wear, corrosion, overload, malfunction, and leakage. Deterioration can be determined using indicators such as age-related deterioration, cracking and corrosion, deformation, electrical leakage, and leakage.

[0011] Furthermore, this energy infrastructure condition evaluation system may be further equipped with a warning issuing unit that issues a warning when the infrastructure evaluation unit determines that there is an abnormality in the energy infrastructure or that the degree of deterioration is higher than a predetermined threshold. In this case, location information of the abnormal area or the area where the degree of deterioration is higher than a predetermined threshold may be issued along with the warning.

[0012] Furthermore, if the energy infrastructure is power lines, the system may also include a power line load adjustment unit that adjusts the load distribution of power lines, taking into account the soundness of the energy infrastructure determined by the infrastructure evaluation unit.

[0013] Furthermore, in order to address cases where the energy infrastructure is buried underground, the mobile unit may be further equipped with a magnetic field data acquisition unit that acquires magnetic field data generated from the energy infrastructure using a magnetic sensor, and the infrastructure evaluation unit may evaluate the health of the energy infrastructure in real time based on the magnetic field data as well.

[0014] Furthermore, in evaluating the soundness of energy infrastructure, the infrastructure evaluation unit includes a learning model storage unit that stores a learning model that has been machine-trained to determine the correlation between input data, which includes object identification information that identifies the energy infrastructure to be evaluated, the 3D point cloud data of the energy infrastructure, the imaged image data, and the multispectral image data, and output data, which includes the presence or absence of abnormalities and the degree of deterioration. An energy infrastructure state estimation unit that uses the learning model to estimate the presence or absence of abnormalities and the degree of deterioration from the object identification information that identifies the energy infrastructure to be evaluated, the 3D point cloud data of the energy infrastructure, the imaged image data and the multispectral image data obtained from the energy infrastructure state estimation unit, It may also be included. This will make it possible to more accurately assess the health of the energy infrastructure. [Effects of the Invention]

[0015] As described above, according to the energy infrastructure condition evaluation system, energy infrastructure condition evaluation method, and energy infrastructure condition evaluation program according to the present invention, based on the 3D point cloud data obtained by the point cloud data acquisition unit mounted on the moving body, the imaging image data obtained by the imaging image acquisition unit mounted on the moving body, the multi-spectral image data obtained by the spectral image acquisition unit mounted on the moving body, and the position and orientation data obtained by the three-dimensional position acquisition unit mounted on the moving body, the soundness of the energy infrastructure can be evaluated in real time. Therefore, it is not necessary to add equipment to the infrastructure, the time required for inspection can be shortened, and inspection can be easily performed even in a short cycle. By detecting signs of abnormalities and aging at an early stage, it is possible to avoid large-scale accidents and service interruptions.

Brief Description of the Drawings

[0016] [Figure 1] FIG. shows an example of applying the energy infrastructure condition evaluation system according to the present invention to an energy infrastructure laid in a mountainous area. [Figure 2] FIG. is a block diagram showing a configuration example of the energy infrastructure condition evaluation system according to the present invention. [Figure 3] FIG. is a block diagram showing the configuration of a machine learning device used for evaluating the soundness of an energy infrastructure. [Figure 4] FIG. is a flowchart showing an example of an operation process for evaluating the soundness of an energy infrastructure using a learning model. [Figure 5] FIG. is a flowchart showing an example of a control process for automatically distributing the load of a power line. [Figure 6] FIG. is a flowchart showing an example of an operation process for automatically scheduling the repair and maintenance of an energy infrastructure.

Modes for Carrying Out the Invention

[0017] Hereinafter, embodiments according to the present invention will be described with reference to the accompanying drawings.

[0018] As an example of the application of the energy infrastructure condition evaluation system S according to the present invention, Figures 1 and 2 show an example of evaluating the condition of power lines P and gas pipelines G laid in mountainous areas. As shown in Figure 2, this energy infrastructure condition assessment system S includes a mobile unit M equipped with various sensors such as a LiDAR (Light Detection and Ranging) 1, an RGB camera 2, a multispectral camera 3, and an infrared camera 4.

[0019] The mobile device M is envisioned to be a manned or unmanned vehicle C, or an unmanned aerial vehicle F such as a drone, and each mobile device M is equipped with the LiDAR 1, RGB camera 2, multispectral camera 3, and infrared camera 4. Ground-based vehicle mapping is known as MMS (Mobile Mapping System), and it collects data while moving using the aforementioned sensors (LiDAR1, RGB camera2, multispectral camera3, infrared camera4) mounted on the vehicle. Aerial mapping systems using aircraft are known as Aerial Mapping Systems (UAV Mapping Systems), and they collect data while moving using the aforementioned sensors mounted on aircraft such as drones and aircraft.

[0020] Each of the above sensors is installed, for example, on the top of a vehicle C (on the roof or cargo bed), or, for example, on the bottom of an aircraft M. These LiDAR 1 and various cameras 2, 3, and 4 should be rotatable 360 ​​degrees around a vertical axis perpendicular to the support base of the mobile body M, and should also be installed so that their depression angle can rotate up and down within a predetermined range.

[0021] LiDAR1 works by shining laser light onto energy infrastructure objects, measuring the time it takes for the laser to reflect back, and using this data to determine the distance to the objects, thereby generating a 3D point cloud of the infrastructure. In other words, it acquires 3D point cloud data generated by 3D scanning the energy infrastructure using laser light. Based on this data, a detailed 3D model (3D map) of the infrastructure is created.

[0022] The RGB camera (imaging device) 2 acquires image data generated by imaging energy infrastructure. It decomposes light reflected from the subject into three components—red, green, and blue—through an RGB filter. The camera's image sensor (CMOS or CCD) measures the intensity of each color of this decomposed light, converts it into a digital signal, and records it as the RGB value of each pixel. By combining the recorded RGB values, a full-color image is generated. This makes it possible to visually check for physical damage, deformation, and color changes. For example, it becomes possible to visually check even the fine details of cracks, corrosion, and damage on the surface of the infrastructure.

[0023] The multispectral camera 3 can capture light across multiple wavelength bands, simultaneously acquiring information across a wide range of wavelengths, including not only visible light but also near-infrared and ultraviolet light. It also functions as an infrared sensor, acquiring multispectral image data (including infrared image data) based on specific wavelengths reflected from energy infrastructure. This camera makes it possible to identify invisible anomalies (anomalies that can be detected by temperature changes) such as overheating (overheating of power lines), cooling issues, and gas leaks in infrastructure.

[0024] Furthermore, the mobile body (vehicle or aircraft) M is equipped with a point cloud data acquisition unit 11 that acquires 3D point cloud data generated by 3D scanning the energy infrastructure with LiDAR 1, an image acquisition unit 12 that acquires image data generated by imaging the energy infrastructure with an RGB camera (imaging device) 2, and a spectral image acquisition unit 13 that acquires multispectral image data (including infrared image data) based on specific wavelengths reflected from the energy infrastructure with a multispectral camera 3 (including an infrared camera 4). The 3D point cloud data, image data, and multispectral image data of the energy infrastructure are acquired while the mobile body M is moving or while the mobile body M is stopped.

[0025] Furthermore, the mobile body M is equipped with a 3D position information acquisition unit 15. This 3D position information acquisition unit 15 includes a GPS (Global Positioning System) and an IMU (Inertial Measurement Unit). The GPS (Global Positioning System) receives signals from satellites to acquire accurate position information of the mobile body. The IMU (Internal Measurement Unit) measures the movement of the mobile body M, such as acceleration, angular velocity, and direction of travel, using an accelerometer and a gyroscope.

[0026] The acquired data is then sent via the data acquisition transmission unit 16 to the data center's management server 20 or cloud server via the communication network 10, either in real time or periodically. Hereafter, the server to which the data is sent will be referred to as the management server 20. The management server 20 receives the transmitted acquired data with the acquired data receiving unit 21, and stores the acquired data received by the acquired data receiving unit 21 in the acquired data storage unit 22, associating it with the motion parameters and position information of the moving object M identified by the 3D position information acquisition unit 15. Then, based on the data stored in the acquired data storage unit 22, the infrastructure evaluation unit 23 evaluates the health of the energy infrastructure in real time.

[0027] With the above configuration, the evaluation of the health of the energy infrastructure by the infrastructure evaluation unit 23 will now be explained. While various methods are possible for assessing structural integrity (determining the presence or absence of abnormalities and the degree of deterioration), automated evaluation is possible using the following procedure.

[0028] (Regarding the determination of whether or not there is an abnormality) 1. Determination of abnormalities using LiDAR1 (1) Data acquisition: Use LiDAR1 to 3D scan the energy infrastructure and generate detailed point cloud data. (2) Data analysis: The acquired point cloud data is analyzed to detect abnormalities in the shape and structure of the infrastructure (e.g., deformation, strain, cracks, damage, defects, etc.). Specifically, the following distance measurement and shape analysis methods are recommended. Distance measurement (detection of unnatural shapes): Distance data from each point is compared to detect abnormal displacements and missing values. In other words, areas in the LiDAR point cloud data where planes or lines are unnaturally distorted are identified. Shape analysis (comparison with reference data): This involves comparing the data with existing 3D models to identify changes in shape. Specifically, it involves comparing a 3D model of healthy infrastructure with real-time acquired LiDAR data and calculating the difference to identify changes in shape (cracks and chips).

[0029] 2. Determination of abnormalities using RGB camera 2 (1) Data acquisition: Color images of the energy infrastructure are captured with RGB camera 2, and the RGB values ​​of each pixel are obtained. (2) Data analysis: Image processing techniques are used to detect physical damage (cracks and defects), discoloration, and corrosion on the infrastructure surface. Specifically, the following edge detection and color analysis methods are recommended. Edge detection: Detects edges in an image to identify cracks and signs of corrosion. Color analysis: This analyzes color changes to detect corrosion, paint peeling, cracks, and other abnormalities in texture and color.

[0030] 3. Determination of abnormalities using a multispectral camera 3 (including an infrared camera 4) (1) Data acquisition: The multispectral camera 3 captures light in multiple wavelength bands to acquire detailed spectral data of the energy infrastructure. Because the multispectral camera 3 acquires data over a wide range of wavelengths, it can detect anomalies that cannot be captured by the RGB camera 2 (e.g., changes in surface temperature, signs of corrosion, etc.). (2) Data analysis: The acquired spectral data is analyzed to detect invisible anomalies (e.g., overheating, cooling, gas leaks). Specifically, the following temperature analysis and spectral analysis methods are recommended. Temperature analysis: Using infrared data to detect abnormal temperature changes (using infrared sensors to measure the surface temperature of infrastructure and identify areas that are abnormally hot or cold compared to the surrounding area. For example, if power lines are overheating or not cooling properly, this could be a sign of damage or other abnormalities). Spectral analysis: Analyzes the reflection properties at specific wavelengths to identify material degradation or anomalies (detecting areas that exhibit unusual reflection properties, which can be signs of corrosion or foreign matter accumulation).

[0031] 4. Data Integration and Evaluation Data Integration: Data acquired from the above LiDAR1, RGB camera2, multispectral camera3, and infrared camera4 will be integrated and a comprehensive evaluation will be performed. Automated evaluation: Each dataset is associated with location information to construct a 3D model of the entire infrastructure and identify anomalies. Anomaly detection algorithms are applied to determine whether or not anomalies are present.

[0032] (Regarding the assessment of the degree of deterioration) 1. Determination of the degree of deterioration using LiDAR1 Data Acquisition: Energy infrastructure is 3D scanned using LiDAR1 to generate detailed point cloud data. Deterioration is assessed by monitoring changes in the infrastructure's shape and structural deformation over the long term. Data Analysis: The acquired point cloud data is analyzed to detect changes in the shape and structure of the infrastructure. Specifically, the following distance measurement and shape analysis methods are used. Distance measurement: Distance data from each point is compared to detect abnormal displacements or defects. For example, multiple LiDAR point cloud data acquired over time are compared to detect minute deformations or settlements. If a slow, long-term change is observed, it is judged to be a sign of deterioration. Shape analysis: This involves comparing the actual shape with an existing 3D model to identify changes in shape. For example, it analyzes deformation and degradation patterns over time. For instance, surface wear can reduce its smoothness and increase the number of fine irregularities. These subtle shape changes are detected using LiDAR data.

[0033] 2. Determination of the degree of deterioration using RGB camera 2 Data acquisition: Images of the energy infrastructure are captured using RGB camera 2, and the RGB values ​​of each pixel are obtained. Data Analysis: Since deterioration often manifests as changes in the color and texture of infrastructure surfaces, image processing techniques are used to detect physical damage and discoloration of the infrastructure surface. Specifically, the following edge detection and color analysis techniques will be used. Edge detection: Detects edges within an image to identify signs of cracks and corrosion. As deterioration progresses, changes appear in the surface texture, and these changes are detected through image analysis. Color analysis: This analyzes color changes to detect corrosion and paint peeling. This allows for an assessment of the surface's deterioration state (tracking color changes using RGB data to estimate the progression of aging).

[0034] 3. Determination of the degree of deterioration using a multispectral camera. (1) Data acquisition: A multispectral camera captures light across multiple wavelength bands to acquire detailed spectral data of the energy infrastructure. (2) Data Analysis: Based on characteristic changes for each wavelength, signs of deterioration can be identified. By analyzing the acquired spectral data, invisible deterioration (e.g., material degradation or internal changes) can be detected. Specifically, the following temperature analysis and spectral analysis methods are used: Temperature analysis: Using infrared data, abnormal temperature changes are detected. As infrastructure ages, changes may appear in its temperature distribution. By monitoring the accumulation of long-term temperature anomalies, signs of deterioration can be identified. Spectral analysis: This involves analyzing the reflection characteristics at specific wavelengths to identify material degradation or anomalies. Specifically, it involves comparing data captured at wavelengths other than visible light (e.g., near-infrared and ultraviolet) to identify material degradation (e.g., changes in paints or insulators degraded by ultraviolet light).

[0035] 4. Data Integration and Evaluation Data Integration: Data acquired from LiDAR, RGB cameras, and multispectral cameras is integrated for a comprehensive evaluation. Automated evaluation: Each dataset is associated with location information to construct a 3D model of the entire infrastructure and identify anomalies. Anomaly detection algorithms are applied to determine the degree of deterioration. By combining these methods, it is possible to automatically and accurately determine the degree of aging of energy infrastructure without using learning models.

[0036] Therefore, according to the above energy infrastructure condition evaluation system, the health of energy infrastructure can be evaluated in real time based on 3D point cloud data obtained by a point cloud data acquisition unit mounted on a mobile unit, image data obtained by an image acquisition unit mounted on a mobile unit, multispectral image data obtained by a spectral image acquisition unit mounted on a mobile unit, and position and attitude data obtained by a 3D position acquisition unit mounted on a mobile unit. As a result, even when monitoring and maintaining energy infrastructure in remote or mountainous areas that are difficult to access, it is not necessary to add equipment to the infrastructure, the time required for inspection can be shortened, and inspections can be performed at shorter intervals (especially in mountainous areas that are difficult to access, it is effective to use unmanned aerial vehicles such as drones as the mobile unit M). In addition, early detection of abnormalities, deterioration, and signs thereof can be avoided, thereby preventing large-scale accidents and service interruptions.

[0037] The above methods did not utilize a learning model, but it is also possible to combine data acquired from LiDAR1, RGB camera2, and multispectral camera3 (infrared camera4) and use a machine learning model to determine the presence or absence of anomalies. The following describes the evaluation of the health of energy infrastructure using a learning model. As shown in Figure 3, a machine learning device 30 may be provided, comprising: an input data acquisition unit 31 that acquires data (information to be evaluated) obtained from 3D point cloud data generated by 3D scanning the energy infrastructure with LiDAR 1, image data generated by imaging the energy infrastructure with RGB camera 2, and multispectral image data including infrared image data based on specific wavelengths reflected from the energy infrastructure by multispectral camera 3 (infrared camera 4), as input data; a label acquisition unit 32 that acquires data including the presence or absence of abnormalities in the energy infrastructure (binary classification) or the degree of deterioration of the energy infrastructure (deterioration level) as labels; and a learning model construction unit 33 that constructs a learning model 35 by performing supervised learning using the input data and label pairs as training data. Here, the input data input to the input data acquisition unit 31 may be directly input from each of the sensors via the communication network 10, or data stored in the acquired data storage unit 22 of the management server 20 may be used as input data.

[0038] Furthermore, based on the 3D point cloud data, image data, and multispectral image data of the current energy infrastructure acquired by various sensors mounted on the mobile unit M, a constructed learning model 35 is used to estimate whether there are any abnormalities in the energy infrastructure or the degree of deterioration (energy infrastructure state estimation unit). The estimated results (presence or absence of abnormalities, degree of deterioration) may then be displayed on a display unit (not shown) of the management server 20. A more detailed explanation follows.

[0039] (Regarding the determination of whether or not there is an abnormality) To form a learning model for determining the presence or absence of anomalies, the following data will be used as input data for the learning phase. LiDAR1 3D point cloud data: Data used to detect changes in the physical structure and shape of infrastructure (cracks, distortion, defects, etc.). Image data from RGB camera 2: Data used to capture visually recognizable anomalies such as surface cracks, corrosion, and paint deterioration. Spectral image data for each wavelength band of the multispectral camera 3: Data used to detect temperature changes and anomalies in the infrared region (such as overheating of power lines and electrical leakage).

[0040] Next, data preprocessing and feature extraction are performed. LiDAR data: Point cloud data is converted into a 3D mesh, and geometric features indicating cracks or surface anomalies (e.g., abrupt changes in distance, surface irregularities) are extracted. RGB data: Features such as edges, cracks, and corrosion patterns are extracted from images using convolutional neural networks (CNNs), etc. Multispectral data: Detects abnormal reflection patterns or temperature anomalies at specific wavelengths at the pixel level and uses them as features. Feature integration: Features obtained from each sensor are combined to generate a feature vector for ultimately classifying the presence or absence of anomalies.

[0041] The labels used to determine the presence or absence of anomalies can be binary labels, such as "1" for an anomaly and "0" for no anomaly. In other words, a classification model such as a Support Vector Machine (SVM), Random Forest, or Deep Learning model (e.g., a multilayer perceptron or an end-to-end CNN+RNN model) is used to perform a two-class classification of normal and abnormal. Then, datasets of normal infrastructure and abnormal infrastructure are prepared, and the learning model is trained.

[0042] In the above, the presence or absence of abnormalities was determined using a two-class classification, but the type of abnormality may also be a label indicating the specific nature of the abnormality, such as cracks, corrosion, or deformation. Furthermore, in determining whether or not an anomaly exists, threshold-based detection may be used: Anomalies may be detected using a pre-set threshold. For example, anomalies may be identified by comparing changes in the shape of LiDAR data, color changes in RGB images, or temperature anomalies in thermal images with the threshold. Alternatively, a rule-based algorithm may be used to identify anomaly locations based on pre-defined rules. For example, an anomaly may be determined when a specific shape change or temperature increase is detected.

[0043] (Regarding the assessment of the degree of deterioration) Because the aging of energy infrastructure progresses slowly, with physical and superficial changes appearing gradually, it is necessary to acquire both temporal and quantitative data on changes. To form a learning model for determining the degree of aging, the following data will be used as input data for the learning phase. LiDAR 3D point cloud data: Data used to capture changes in 3D point cloud data over long periods, i.e., subtle changes in shape over time (e.g., deflection, settlement, surface wear, etc.). RGB camera data: Data used to observe changes in full-color images over long periods, i.e., visual signs of aging such as changes in surface color, progression of corrosion, and deterioration of paint. Multispectral camera data: Data that captures changes in spectral images over long periods, i.e., material degradation and temperature changes in the infrared and ultraviolet regions, in order to understand the progression of aging.

[0044] For data collection, the infrastructure is periodically scanned using LiDAR1, RGB camera2, and multispectral camera3 (infrared camera4) to collect data. Record actual cases of anomalies and deterioration and add them to the dataset. Labeling to determine the degree of deterioration should be done using expert evaluation (infrastructure experts evaluate collected data and label anomalies and deterioration) or automated labeling tools (initial labels are generated using existing anomaly detection algorithms and then reviewed and corrected by experts). The degree of deterioration should be expressed using numerical labels (e.g., a scale from 0 to 10) that indicate the progression of deterioration. Alternatively, labels may be used that indicate the type of deterioration, such as corrosion, discoloration, or surface deterioration. Using this data and labels, we will train a machine learning model to automatically detect anomalies and deterioration in energy infrastructure.

[0045] More specifically, an approach to forming a learning model may be as follows: Data preprocessing and feature extraction LiDAR data: By comparing point cloud data from multiple measurements over a long period, cumulative changes in shape and minute deformations are quantitatively extracted as features. RGB data: Automatically detects the degree of surface discoloration and corrosion, and extracts features that indicate the progression of deterioration. Multispectral data: This quantifies reflectance and temperature changes at specific wavelengths and uses them as indicators of deterioration. Handling of temporal data Time series models: Since infrastructure deterioration progresses over time, it is advisable to use time series models such as LSTM (Long Short-Term Memory) or GRU (Gated Recurrent Unit) to learn how data changes over time. Data accumulation and comparison: Newly acquired data is compared with past data to evaluate the degree of deterioration.

[0046] Classification of deterioration levels Regression Model: A regression model is used to numerically predict the rate of deterioration. This quantitatively represents the level of infrastructure deterioration, predicting a score from, for example, 0 (new) to 10 (extremely deteriorated). Hierarchical classification: To categorize deterioration into categories such as "new," "moderate," and "severely deteriorated," classification models such as logistic regression, decision trees, and random forests are applied. Training and evaluation Using infrastructure data spanning multiple years, the model is trained to understand how deterioration progresses over time. The model compares newly acquired data with historical data to predict the rate and extent of deterioration.

[0047] Flowchart for determining the presence or absence of abnormalities and for determining deterioration. To determine the presence or absence of anomalies and assess deterioration using the learning model formed as described above, as shown in Figure 4, a mobile device M (vehicle or drone) is moved while periodically acquiring 3D point cloud data from LiDAR1, image data from RGB camera2, and spectral image data in each wavelength band from multispectral camera3 (including infrared camera4) from various positions and angles from above or from the ground to the energy infrastructure (step S41). The acquired data is input into the trained learning model to automatically determine whether or not there are anomalies, and the degree of deterioration of the infrastructure is quantified by comparing it with past data, and a prediction result is output (step S42). Then, the estimation results are displayed on the display unit of the management server 20 (step S43). If the estimation results indicate an abnormality (damage, etc.), or if the quantified deterioration assessment result reaches a predetermined threshold (step S44), a warning (notification of abnormality / deterioration detection) is issued to the administrator or person in charge who has been registered in advance, along with the location of the abnormality or deterioration (step S45: warning issuing unit 24).

[0048] The approach using the learning models described above integrates characteristic data from LiDAR1, RGB camera2, multispectral camera3, and infrared camera4, enabling highly accurate automatic detection of anomalies and deterioration. A classification model is suitable for determining the presence or absence of anomalies, while a regression model or classification model capable of handling time-series data is suitable for determining the degree of deterioration.

[0049] In the above, data acquired from LiDAR1, RGB camera2, multispectral camera3, and infrared camera4 mounted on a mobile device was used to evaluate the presence or absence of abnormalities and the degree of deterioration of the energy infrastructure. However, in addition to the above sensors, a magnetic sensor7 may be mounted on the mobile device (especially a vehicle) to acquire magnetic field data 17 that acquires magnetic field data generated from buried energy infrastructure such as underground pipes and cables (see Figure 2). This magnetic sensor 7 is best installed on the underside of vehicle C, and by detecting the magnetic field around buried pipes and cables close to the ground and capturing changes in the magnetic field, it becomes possible to determine whether there are any abnormalities or deterioration in these buried infrastructure.

[0050] Furthermore, the energy supply status may be optimized according to the assessment of the health of the energy infrastructure (presence or absence of abnormalities and degree of deterioration). In other words, data collected by mobile mapping and UAV mapping can be used not only to monitor the state of energy infrastructure but also to optimize energy supply. Here, energy supply optimization refers to providing appropriate energy supply based on demand forecasting, load balancing, failure risk assessment, etc. Damage or deterioration of power lines can cause overloading. Specifically, as power lines deteriorate and their electrical resistance increases, more energy is required to carry the same current, resulting in overloading. Furthermore, deterioration of the insulation increases the risk of leakage and short circuits, which can also cause overloading. In addition, deteriorated power lines cannot effectively dissipate heat, causing their temperature to rise easily, which further contributes to overloading. Due to these factors, damage or deterioration of power lines can cause overloading and affect the stability of the power supply. Therefore, in order to achieve optimal energy supply, for example, as shown in Figure 5, future energy demand is predicted based on energy consumption information such as past consumption data, weather data, and event information (refer to Energy Consumption Information DB51) (Step S51), and the failure risk of the power supply system is evaluated based on the aforementioned infrastructure health data (presence or absence of abnormalities and degree of deterioration) (refer to Energy Infrastructure Health DB52) (Step S52), and control is performed to automatically distribute the load on power lines P to areas and times with high risk (Step S53; Power Line Load Adjustment Unit 25). Furthermore, by closely monitoring the condition of gas pipeline G, it becomes possible to minimize the risk of gas leaks and ensure the stability of the supply.

[0051] Furthermore, based on the aforementioned assessment of the health of the energy infrastructure (presence or absence of abnormalities and degree of deterioration), the system may be automated to schedule repairs and maintenance of the infrastructure. The specific process is carried out as shown in Figure 6, for example. [Step 61: Data Collection and Anomaly Detection] Based on data collected from various sensors mounted on the mobile unit M, the infrastructure evaluation unit 23 automatically detects abnormalities and the degree of deterioration (indications) of the energy infrastructure. [Step 62: Determining Priorities] Next, the severity and urgency of the detected problems are assessed. For example, if a major anomaly or deterioration is detected, or if a problem is detected in a critical energy supply line, repair and maintenance will be given high priority. [Step 63: Scheduling] A schedule is automatically created based on the priority of each repair or maintenance task, taking into account the available time of the repair team, the necessary materials and equipment, and other constraints. [Step 64: Execution and Feedback] Repair and maintenance work is carried out according to the created schedule. The condition is monitored again after the work is completed, and the results are fed back into the system. This allows the system to evaluate the effectiveness of the repairs and maintenance and schedule additional work as needed. This will enable early detection and rapid response to problems, thereby maintaining the stability and reliability of the energy supply.

[0052] Furthermore, the energy infrastructure condition assessment system S described above can also be provided in the form of a program (energy infrastructure condition assessment program) that causes a computer to execute each step of the energy infrastructure condition assessment method described above. [Explanation of Symbols]

[0053] 11. Point cloud data acquisition unit 12 Image acquisition unit 13. Spectral image acquisition unit 17 Magnetic field data acquisition unit 51 Input data acquisition unit 52 Label acquisition unit 53 Learning Model Construction Department S Energy Infrastructure Condition Assessment System M Mobile object C Vehicle F flying object P power line G Gas Pipeline

Claims

1. An energy infrastructure condition evaluation system for evaluating the state of energy infrastructure to be evaluated, A point cloud data acquisition unit that acquires 3D point cloud data generated by 3D scanning the aforementioned energy infrastructure, The image acquisition unit acquires image data generated by imaging the aforementioned energy infrastructure, A spectral image acquisition unit acquires multispectral image data, including infrared image data, based on specific wavelengths reflected from the aforementioned energy infrastructure. A 3D position information acquisition unit that acquires position and attitude data, comprising at least a GPS receiver and an inertial measuring instrument, A mobile body equipped with the point cloud data acquisition unit, the image acquisition unit, the spectral image acquisition unit, and the three-dimensional position information acquisition unit, An acquired data storage unit stores the 3D point cloud data, image data, multispectral image data, and position / orientation data of the energy infrastructure acquired while the moving body is moving or while the moving body is stopped. Based on the acquired data, an infrastructure evaluation unit evaluates the health of the energy infrastructure in real time, An energy infrastructure condition evaluation system characterized by comprising the following:

2. The mobile entity includes at least one of a vehicle or an aircraft, the energy infrastructure includes power lines or gas pipelines, and the assessment of integrity by the infrastructure evaluation unit includes determining whether there are any abnormalities in the energy infrastructure or the degree of deterioration. The system further includes a warning issuing unit that issues a warning if the infrastructure evaluation unit determines that there is an abnormality in the energy infrastructure, or if the degree of deterioration is determined to be higher than a predetermined threshold. The energy infrastructure condition evaluation system according to claim 1.

3. The aforementioned energy infrastructure is power lines, The system further includes a power line load adjustment unit that adjusts the load distribution of the power lines, taking into account the soundness of the energy infrastructure determined by the infrastructure evaluation unit. The energy infrastructure condition evaluation system according to claim 1, characterized in that it is the same as described in claim 1.

4. The aforementioned mobile body is further equipped with a magnetic field data acquisition unit that acquires magnetic field data generated from the energy infrastructure using a magnetic sensor. The energy infrastructure condition evaluation system according to claim 1, wherein the infrastructure evaluation unit evaluates the health of the energy infrastructure in real time based on the magnetic field data.

5. The aforementioned infrastructure evaluation unit, A learning model storage unit stores a learning model that has been machine-trained to store the correlation between input data, which includes object identification information that identifies the energy infrastructure to be evaluated, the 3D point cloud data of the energy infrastructure, the imaged image data, and the information to be evaluated obtained from the multispectral image data, and output data, which includes the presence or absence of abnormalities and the degree of deterioration. An energy infrastructure state estimation unit that uses the learning model to estimate the presence or absence of abnormalities and the degree of deterioration from the object identification information that identifies the energy infrastructure to be evaluated, the 3D point cloud data of the energy infrastructure, the imaged image data and the multispectral image data obtained from the energy infrastructure state estimation unit, The energy infrastructure condition evaluation system according to claim 1, which includes the following:

6. An energy infrastructure condition evaluation method for evaluating the state of energy infrastructure to be evaluated, A mobile body equipped with a point cloud data acquisition unit that acquires 3D point cloud data generated by 3D scanning the energy infrastructure, an image acquisition unit that acquires image data generated by imaging the energy infrastructure, a spectral image acquisition unit that acquires multispectral image data including infrared image data based on specific wavelengths reflected from the energy infrastructure, and a 3D position acquisition unit that acquires position and attitude data, comprising at least a GPS receiver and an inertial measuring instrument, is driven while the acquired data is stored in each acquisition unit, and an acquired data storage step is performed. Based on the acquired data, an infrastructure evaluation step is performed to evaluate the health of the energy infrastructure in real time, A method for evaluating the state of energy infrastructure, characterized by comprising the following:

7. The assessment of soundness in the aforementioned infrastructure assessment step includes determining whether there are any abnormalities in the energy infrastructure or the degree of deterioration. If the infrastructure evaluation step determines that there is an abnormality in the energy infrastructure, or that the degree of deterioration is higher than a predetermined threshold, a warning is issued step, The energy infrastructure condition evaluation method according to claim 6, further comprising the features described above.

8. The aforementioned energy infrastructure is power lines, The system further comprises a power line load balancing step that adjusts the load distribution of the power lines, taking into account the soundness of the energy infrastructure determined in the infrastructure evaluation step. The method for evaluating the state of energy infrastructure according to claim 6.

9. The aforementioned mobile body is further equipped with a magnetic field data acquisition unit that acquires magnetic field data generated from the energy infrastructure using a magnetic sensor. The energy infrastructure condition evaluation method according to claim 6, wherein the infrastructure evaluation step evaluates the health of the energy infrastructure in real time based on the magnetic field data.

10. The aforementioned infrastructure evaluation step is, A learning model storage step involves storing a learning model that has been machine-trained to store the correlation between input data, which includes object identification information that identifies the energy infrastructure to be evaluated, the 3D point cloud data of the energy infrastructure, the imaged image data and the multispectral image data, and output data, which includes the presence or absence of abnormalities and the degree of deterioration. A step of estimating the state of energy infrastructure, using the learning model, to estimate the presence or absence of abnormalities and the degree of deterioration, based on the object identification information that identifies the energy infrastructure to be evaluated, the 3D point cloud data of the energy infrastructure, the imaged image data and the multispectral image data obtained from the information to be evaluated, The energy infrastructure condition evaluation method according to claim 6, comprising the above.

11. An energy infrastructure condition assessment program for causing a computer to perform each step of the energy infrastructure condition assessment method according to any one of claims 6 to 10.

Images