Unmanned aerial vehicle infrared inspection target abnormity diagnosis and judgment method and equipment

Abstract

The invention discloses an unmanned aerial vehicle infrared inspection target anomaly diagnosis and judgment method and equipment, and the method comprises the steps: screening historical data with the similarity higher than a threshold value as similar historical data to train a reference value calculation model, and taking multi-source data (environment data, working condition data and equipment characteristic data) as input; a reference temperature value output by the model is not dependent on a fixed environment temperature or a historical average value any more, but is dynamically generated according to a current load, environment parameters (such as wind speed and sunlight) and an equipment type; by performing dynamic temperature compensation on the infrared ROI, the problem of insufficient environmental interference compensation in the temperature measurement process is solved; through the technical thought of data fusion, dynamic modeling, intelligent compensation and accurate traceability, the inspection precision and efficiency of the unmanned aerial vehicle are improved, and reliable technical support is provided for state monitoring of power equipment.

Description

Technical Field

[0001] The present invention belongs to the field of infrared inspection technology, and in particular relates to a method and equipment for diagnosing and determining abnormalities of infrared inspection targets of unmanned aerial vehicles. Background Art

[0002] During the drone inspection of overhead collector lines, using an infrared thermal imager to obtain infrared images of the equipment is an important means of detecting abnormal heating of the equipment and judging the equipment's operating status.

[0003] The existing technology has the following problems: the benchmark value is single and fixed. The traditional method uses a fixed ambient temperature or historical average value as the benchmark, without considering the equipment type, load changes and dynamic environmental interference; spatial registration error. The change in the drone's posture causes the infrared image and the physical position of the equipment to shift, affecting the accurate extraction of the benchmark area; insufficient compensation for environmental interference. The existing method lacks a quantitative compensation model for external factors such as wind speed and sunlight intensity; there is a lack of equipment differentiation processing. The normal temperature rise characteristics of different equipment (such as insulators and wire joints) are not differentiated and modeled.

[0004] Related patents propose infrared detection methods based on ambient temperature correction, but they do not address the issue of dynamic changes in equipment load. Related papers use image registration algorithms, but lack effective solutions for adaptively updating reference values. Summary of the Invention

[0005] The purpose of the present invention is to provide a method and device for diagnosing and judging abnormalities of targets inspected by unmanned aerial vehicle infrared inspection, so as to solve the technical problem of single and solid reference value in existing unmanned aerial vehicle infrared inspection methods.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions: A method for diagnosing and determining abnormalities of a target inspected by an unmanned aerial vehicle (UAV) using infrared inspection, comprising the following steps: Obtain visible light images, infrared images, environmental data, working condition data and equipment characteristic data of the equipment to be inspected; Based on the environmental data, working condition data and equipment characteristic data of the equipment to be inspected, historical data with a similarity higher than a threshold is selected as similar historical data; The environmental data, working condition data and equipment characteristic data in similar historical data are used as input, and the temperature values ​​of key parts of the equipment in similar historical data are used as output to train the benchmark value calculation model; Input the environmental data, working condition data and equipment characteristic data of the equipment to be tested into the reference value calculation model, output the temperature value of the key parts of the equipment to be tested under the current working condition, and combine the temperature distribution law of the key parts of the equipment to be tested to obtain the reference temperature value range; Based on the visible light image and infrared image of the device to be inspected, the infrared ROI area is calibrated and the temperature data of the infrared ROI area is dynamically compensated; The reference temperature value range is compared with the characteristic points selected from the infrared ROI area temperature data after dynamic compensation to determine whether the current temperature of the device under test is abnormal.

[0007] Furthermore, the working condition data and equipment characteristic data of the equipment to be inspected are based on the visible light image, infrared image, and environmental data of the equipment to be inspected, and are obtained in combination with the visible light image YOLO model and the equipment ledger database.

[0008] Furthermore, the process of selecting similar historical data is as follows: Use environmental data, working condition data, and equipment characteristic data as weights to calculate the similarity between the current data and historical data of the equipment under test; Select historical data with similarity higher than the set threshold as similar historical data.

[0009] Furthermore, the similarity between the current data of the device under test and the historical data is calculated using a Euclidean distance or cosine similarity algorithm.

[0010] Furthermore, the temperature distribution pattern of the key parts of the device under test is obtained based on the temperature fluctuation range of similar historical data statistics; The reference temperature value range is centered on the temperature values ​​of key parts of the equipment under current working conditions, and the upper and lower limits are determined based on the standard deviation of temperature fluctuations calculated from similar historical data.

[0011] Furthermore, the dynamic compensation process of the infrared ROI area temperature data is as follows: Eliminate outlier temperature points in the ROI area based on the 3σ criterion; The LSTM network is used to combine historical temperature deviation data and current operating condition data to predict the current temperature supplement value and obtain the temperature compensation value.

[0012] Furthermore, based on the difference in characteristic points between the reference temperature value interval and the temperature data of the infrared ROI area after dynamic compensation, the operating condition data of the device under test and similar historical data are compared and analyzed to determine the cause of the abnormal temperature of the device under test.

[0013] In a second aspect, the present invention provides a UAV infrared inspection target abnormality diagnosis and judgment system, comprising an acquisition module, a selection module, a model training module, a reference temperature range determination module, a compensation module, and a comparison module, wherein: Acquisition module: used to obtain visible light images, infrared images, environmental data, working condition data and equipment characteristic data of the equipment to be inspected; Selection module: Based on the environmental data, working condition data and equipment characteristic data of the equipment to be inspected, historical data with a similarity higher than a threshold is selected as similar historical data; Model training module: used to take environmental data, operating condition data, and equipment characteristic data from similar historical data as input, and the temperature values ​​of key parts of the equipment from similar historical data as output, to train the benchmark value calculation model; Reference temperature interval determination module: used to input the environmental data, working condition data and equipment characteristic data of the equipment to be tested into the reference value calculation model, output the temperature value of the key parts of the equipment to be tested under the current working conditions, and combine the temperature distribution law of the key parts of the equipment to be tested to obtain the reference temperature value interval; Compensation module: Based on the visible light image and infrared image of the device to be inspected, the infrared ROI area is calibrated and the temperature data of the infrared ROI area is dynamically compensated; Comparison module: used to compare the reference temperature value interval with the characteristic points selected from the infrared ROI area temperature data after dynamic compensation to determine whether the current temperature of the device under test is abnormal.

[0014] According to a third aspect, a terminal device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0015] In a fourth aspect, a computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0016] Compared with the prior art, the present invention has the following beneficial technical effects: The present invention provides a method for diagnosing and judging abnormalities of targets inspected by unmanned aerial vehicle infrared inspections. The method screens historical data with a similarity higher than a threshold as similar historical data to train a reference value calculation model, and uses multi-source data (environmental data, operating condition data, and equipment characteristic data) as input, so that the reference temperature value output by the model no longer depends on a fixed ambient temperature or a historical average value, but is dynamically generated according to the current load, environmental parameters (such as wind speed and sunshine), and equipment type. Dynamic temperature compensation is performed on the infrared ROI area, thereby solving the problem of insufficient compensation for environmental interference during temperature measurement. The present invention improves the inspection accuracy and efficiency of unmanned aerial vehicle inspections through the technical approach of "data fusion-dynamic modeling-intelligent compensation-precise tracing", providing reliable technical support for power equipment status monitoring.

[0017] The present invention focuses on the temperature values ​​of key parts of the equipment to be tested under the current working conditions, determines the upper and lower limits based on the standard deviation of temperature fluctuations statistically obtained from similar historical data, and obtains the infrared image reference value temperature interval. The reference value temperature interval is used as a comparison benchmark, taking into account the temperature fluctuations during normal operation of the equipment, and the data selection is more reasonable.

[0018] Preferably, the ROI area is extracted by fusing visible light and infrared images, which solves the problem of infrared image offset caused by changes in the drone's posture and avoids temperature measurement deviation caused by misjudgment of the target area. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a flow chart of a method for diagnosing and determining abnormalities in infrared inspection targets of UAVs; Figure 2 The figure is a flow chart of a method for diagnosing and determining abnormalities in infrared inspection targets of a UAV in an embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0021] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.

[0022] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0023] The present invention is described in further detail below with reference to the accompanying drawings: like Figure 1As shown, a method for diagnosing and determining abnormalities of a target during infrared inspection by a UAV includes the following steps: Step 1: Obtain visible light images, infrared images, environmental data, working condition data, and equipment characteristic data of the equipment to be inspected; Specifically, the visible light images, infrared images and environmental data of the equipment to be inspected are obtained through the onboard equipment of the drone. The visible light images, infrared images and environmental data of the equipment to be inspected are used, and the visible light image YOLO model and the equipment ledger database are combined to obtain the working condition data and equipment characteristic data of the equipment to be inspected.

[0024] Step 2: Based on the environmental data, working condition data and equipment characteristic data of the equipment to be inspected, historical data with a similarity higher than a threshold is selected as similar historical data; The selection of similar historical data includes the following steps: Using the similarity calculation method, the weights of each parameter in the environmental data, working condition data and equipment characteristic data of the equipment under test and the historical data are used to calculate the similarity between the current data and the historical data through the Euclidean distance or cosine similarity algorithm; Select historical data with similarity higher than the set threshold as similar historical data.

[0025] Among them, environmental data includes at least ambient temperature, humidity, wind speed and sunshine intensity, operating condition data includes at least load size and operating time, and equipment characteristic data includes at least equipment type and model.

[0026] Step 3: Use the environmental data, operating condition data, and equipment characteristic data in similar historical data as input, and the temperature values ​​of key parts of the equipment in similar historical data as output to train the benchmark value calculation model; The environmental data, working condition data and equipment characteristic data in similar historical data are used as input, and the temperature values ​​of key parts of the equipment in the corresponding infrared images in similar historical data are used as output to train the benchmark value calculation model. During the training process, the model parameters are optimized through cross-validation method to improve the prediction accuracy of the model.

[0027] Step 4: Input the environmental data, operating condition data, and equipment characteristic data of the equipment to be tested into the reference value calculation model, output the temperature value of the key parts of the equipment to be tested under the current operating conditions, and combine the temperature distribution law of the key parts of the equipment to be tested to obtain the reference temperature value range; The environmental data, operating condition data, and equipment characteristic data obtained during the current inspection are input into a trained benchmark value calculation model. The model outputs the temperature value of the key parts of the equipment under the current operating conditions as the infrared image benchmark temperature value of the equipment under the current operating conditions. Then, based on the temperature distribution law of the key parts of the equipment, the fluctuation range of the benchmark temperature value is determined to form a benchmark temperature value interval.

[0028] Step 5: Based on the visible light image and infrared image of the device to be inspected, the infrared ROI area is calibrated and the temperature data of the infrared ROI area is dynamically compensated; After eliminating outlier temperature points in the ROI area of ​​the equipment under test based on the 3σ criterion, the LSTM network is used to combine historical temperature deviation data and current operating condition data to predict the current temperature supplement value and obtain the temperature compensation value; The temperature measurement value of the device under test is obtained based on the temperature data of the infrared ROI area of ​​the device under test and the predicted temperature compensation value.

[0029] Step 6: Compare the reference temperature range with the characteristic points of the infrared ROI area temperature data after dynamic compensation to determine whether the current temperature of the device under test is abnormal: The reference temperature value range of the equipment to be tested obtained by using the reference value calculation model and the temperature distribution law of the key parts of the equipment is compared with the infrared ROI area temperature data after dynamic compensation to determine whether the temperature of the current inspection object of the drone is abnormal.

[0030] Example A method for diagnosing abnormalities of infrared inspection targets of UAVs, the specific steps are as follows: Figure 2 As shown: First, multi-source data acquisition and fusion modeling: UAVs are used to acquire visible light and infrared images of equipment while simultaneously collecting environmental data. The visible light image YOLO model is combined with the equipment inventory database to automatically identify the target equipment type and number, and load the corresponding temperature rise characteristic curve based on the equipment type (for example, the allowable temperature rise of wire joints is ≤30K).

[0031] The drone connects directly to the power system's data interface via built-in data communication modules (such as network interfaces and power-specific communication protocol interfaces), obtaining current equipment load data and operating hours and operating condition data. The equipment ledger database is a pre-established database that stores equipment type, model, specifications, and historical operating data and equipment characteristics.

[0032] Second, screening of similar historical data: Infrared image data of similar working conditions are screened out from historical data. Specifically, a similarity calculation method is used to set the weights of each parameter in environmental data (ambient temperature, humidity, wind speed, and sunshine intensity), working condition data (load size and operating time), and equipment characteristic data (equipment type and model). The similarity between current data and historical data is calculated using the Euclidean distance and cosine similarity algorithms, and historical data with a similarity higher than the set threshold is selected as similar historical data.

[0033] Third, build a baseline value calculation model: Using machine learning algorithms, such as support vector regression (SVR) or random forest regression, this model is trained using environmental data, operating condition data, and equipment characteristic data from similar historical data as input features and the temperature values ​​of key equipment parts in corresponding infrared images as output labels. During training, cross-validation is used to optimize model parameters and improve prediction accuracy.

[0034] Fourth, determine the baseline value. The environmental data, operating condition data, and equipment characteristic data collected during the current inspection are input into the trained baseline value calculation model. The model then outputs the baseline temperature value for the infrared image of the equipment under the current operating conditions. Furthermore, the temperature distribution patterns of key parts of the equipment are combined to determine the fluctuation range of the baseline temperature value. For example, using the baseline temperature value as the center, the upper and lower limits are determined based on the standard deviation of temperature fluctuations calculated from historical data to form the infrared image baseline value range.

[0035] Fifth, the target value is determined and the dual-light fusion ROI area of ​​the target to be measured is extracted: based on the fusion of visible light and infrared images, the target area to be identified is accurately cut, the infrared ROI area is recalibrated, and the ROI outlier temperature points are eliminated based on the 3σ criterion.

[0036] The LSTM network is used to combine historical temperature deviation data and current working condition data to predict the current possible temperature supplement value, obtain the temperature compensation value, and dynamically compensate the current ROI area according to the temperature compensation value.

[0037] Sixth, abnormality judgment: By comparing the infrared image reference value interval with the current compensated measurement value, the characteristic points are selected to determine whether the current temperature of the device under test is abnormal.

[0038] After comparing and analyzing the differences in characteristic points and current data with historical operating condition data, the possible probability factors of the abnormal cause are determined.

[0039] In yet another embodiment of the present invention, a system for diagnosing and determining abnormalities in target inspections by an unmanned aerial vehicle (UAV) is provided, comprising an acquisition module, a selection module, a model training module, a reference temperature interval determination module, a compensation module, and a comparison module, wherein: Acquisition module: used to obtain visible light images, infrared images, environmental data, working condition data and equipment characteristic data of the equipment to be inspected; Selection module: Based on the environmental data, working condition data and equipment characteristic data of the equipment to be inspected, historical data with a similarity higher than a threshold is selected as similar historical data; Model training module: used to take environmental data, operating condition data, and equipment characteristic data from similar historical data as input, and the temperature values ​​of key parts of the equipment from similar historical data as output, to train the benchmark value calculation model; Reference temperature interval determination module: used to input the environmental data, working condition data and equipment characteristic data of the equipment to be tested into the reference value calculation model, output the temperature value of the key parts of the equipment to be tested under the current working conditions, and combine the temperature distribution law of the key parts of the equipment to be tested to obtain the reference temperature value interval; Compensation module: Based on the visible light image and infrared image of the device to be inspected, the infrared ROI area is calibrated and the temperature data of the infrared ROI area is dynamically compensated; Comparison module: used to compare the reference temperature value interval with the characteristic points selected from the infrared ROI area temperature data after dynamic compensation to determine whether the current temperature of the device under test is abnormal.

[0040] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0041] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0042] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0043] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0044] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit its scope of protection. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the field should understand that after reading the present invention, those skilled in the art may still make various changes, modifications or equivalent substitutions to the specific implementation methods of the invention, but these changes, modifications or equivalent substitutions are all within the scope of protection of the pending claims of the invention.

Claims

1. A method for diagnosing and judging abnormalities of a target during an infrared inspection by a drone, characterized in that: The following steps are involved: Obtain visible light images, infrared images, environmental data, working condition data and equipment characteristic data of the equipment to be inspected; Based on the environmental data, working condition data and equipment characteristic data of the equipment to be inspected, historical data with a similarity higher than a threshold is selected as similar historical data; The environmental data, working condition data and equipment characteristic data in similar historical data are used as input, and the temperature values ​​of key parts of the equipment in similar historical data are used as output to train the benchmark value calculation model; Input the environmental data, working condition data and equipment characteristic data of the equipment to be tested into the reference value calculation model, output the temperature value of the key parts of the equipment to be tested under the current working condition, and combine the temperature distribution law of the key parts of the equipment to be tested to obtain the reference temperature value range; Based on the visible light image and infrared image of the device to be inspected, the infrared ROI area is calibrated and the temperature data of the infrared ROI area is dynamically compensated; The reference temperature value range is compared with the characteristic points selected from the infrared ROI area temperature data after dynamic compensation to determine whether the current temperature of the device under test is abnormal.

2. The method for diagnosing and judging abnormalities of a target inspected by an unmanned aerial vehicle (UAV) in infrared according to claim 1, characterized in that: The working condition data and equipment characteristic data of the equipment to be inspected are based on the visible light image, infrared image, and environmental data of the equipment to be inspected, and are obtained by combining the visible light image YOLO model and the equipment ledger database.

3. The method for diagnosing and judging abnormalities of a target inspected by an unmanned aerial vehicle (UAV) in infrared according to claim 1, characterized in that: The process of selecting similar historical data is as follows: Use environmental data, working condition data, and equipment characteristic data as weights to calculate the similarity between the current data and historical data of the equipment under test; Select historical data with similarity higher than the set threshold as similar historical data.

4. The method for diagnosing and judging abnormalities of a target inspected by an unmanned aerial vehicle (UAV) in infrared according to claim 3, characterized in that: The similarity between the current data and the historical data of the device under test is calculated using a Euclidean distance or cosine similarity algorithm.

5. The method for diagnosing and judging abnormalities of a target inspected by an unmanned aerial vehicle (UAV) in infrared according to claim 1, characterized in that: The temperature distribution law of the key parts of the device under test is obtained based on the temperature fluctuation range of similar historical data statistics; The reference temperature value range is centered on the temperature values ​​of key parts of the equipment under current working conditions, and the upper and lower limits are determined based on the standard deviation of temperature fluctuations calculated from similar historical data.

6. The method for diagnosing and judging abnormalities of a target inspected by an unmanned aerial vehicle (UAV) in infrared according to claim 1, characterized in that: The dynamic compensation process of the infrared ROI area temperature data is as follows: Eliminate outlier temperature points in the ROI area based on the 3σ criterion; The LSTM network is used to combine historical temperature deviation data and current operating condition data to predict the current temperature supplement value and obtain the temperature compensation value.

7. The method for diagnosing and judging abnormalities of a target inspected by an unmanned aerial vehicle (UAV) in infrared according to claim 1, characterized in that: Based on the difference in characteristic points between the reference temperature value range and the temperature data of the infrared ROI area after dynamic compensation, and by comparing and analyzing the operating data of the device under test and similar historical data, the cause of the abnormal temperature of the device under test is determined.

8. A UAV infrared inspection target abnormality diagnosis and judgment system, characterized by: The method for diagnosing and determining abnormalities of a target inspected by an infrared drone according to any one of claims 1 to 7 comprises an acquisition module, a selection module, a model training module, a reference temperature interval determination module, a compensation module, and a comparison module, wherein: Acquisition module: used to obtain visible light images, infrared images, environmental data, working condition data and equipment characteristic data of the equipment to be inspected; Selection module: Based on the environmental data, working condition data and equipment characteristic data of the equipment to be inspected, historical data with a similarity higher than a threshold is selected as similar historical data; Model training module: used to take environmental data, operating condition data, and equipment characteristic data from similar historical data as input, and the temperature values ​​of key parts of the equipment from similar historical data as output, to train the benchmark value calculation model; Reference temperature interval determination module: used to input the environmental data, working condition data and equipment characteristic data of the equipment to be tested into the reference value calculation model, output the temperature value of the key parts of the equipment to be tested under the current working conditions, and combine the temperature distribution law of the key parts of the equipment to be tested to obtain the reference temperature value interval; Compensation module: Based on the visible light image and infrared image of the device to be inspected, the infrared ROI area is calibrated and the temperature data of the infrared ROI area is dynamically compensated; Comparison module: used to compare the reference temperature value interval with the characteristic points selected from the infrared ROI area temperature data after dynamic compensation to determine whether the current temperature of the device under test is abnormal.

9. A terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images