An unmanned aerial vehicle inspection data processing method and device based on FPGA

Abstract

The present application relates to a kind of unmanned aerial vehicle inspection data processing method based on FPGA, comprising the following steps: radio inspection data is converted to AD, and is transmitted to FPGA;FPGA reads identification data, reads corresponding historical image data from server;FPGA converts the pixel value corresponding to each pixel point in current color image data, generates pixel value with quality code;FPGA carries out image gray scale processing to current color image data and historical image data;The current position of monitoring target and the historical position of monitoring target are identified from the image data after gray scale processing respectively by edge detection algorithm;Compare whether the current position of monitoring target and the historical position of monitoring target exist difference, if there is, then artificial secondary check.The present application also relates to a kind of unmanned aerial vehicle inspection data processing device based on FPGA.The present application has the advantages that by setting quality code to each pixel point, improve image recognition accuracy and abnormal determination accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of unmanned aerial vehicle inspection data processing, and particularly relates to an unmanned aerial vehicle inspection data processing method and device based on FPGA. BACKGROUND

[0002] A large part of the inspection of the existing power system is through unmanned aerial vehicles to collect images of power monitoring targets from multiple angles, and then return the images to the receiving station together with azimuth information, time information, etc. The receiving station intelligently identifies and compares the inspection data to assist manual judgment of whether the power system in the inspection area has risks. However, in the power system, there are many targets that need to be monitored in real time, such as ground wires, insulators, anti-vibration hammers, small hardware, wire clamps and grading rings, infrastructure, auxiliary facilities, channel environment, etc. When collecting data, multiple unmanned aerial vehicles work simultaneously and take images from multiple angles. Not only is the data returned quickly, but also the amount of data returned is large. The inspection images of only one power transmission line reach more than 200. The system in the receiving station uses an identification system based on FPGA intelligent algorithm, which can automatically identify possible defects in the images. Team personnel only need to quickly check the results again, which greatly improves the work efficiency of the team. However, the system still has the following problems in operation: 1. The images received by the inspection have poor image quality such as overexposure. The system does not perform any detection, screening or marking, and all of them are used for identification and comparison, which not only wastes computing resources but also leads to incorrect comparison results, affecting the accuracy of abnormal judgment. 2. The FPGA internal cache space is insufficient due to the large amount of inspection data received in a short time, which can easily lead to data loss; 3. The received inspection data is preprocessed and calculated according to the first-in first-out principle, which leads to idle recognition and comparison algorithms when preprocessing image data, resulting in slow data processing speed; 4. When a large amount of inspection data is received, there is a lack of timing control of the inspection data, which leads to comparison errors. SUMMARY

[0003] In order to solve the above problems, the purpose of the present application is to provide an unmanned aerial vehicle inspection data processing method based on FPGA, which intelligently, efficiently and accurately evaluates the quality of each pixel point in the image, thereby improving the accuracy of abnormal judgment, reducing the calculation amount of FPGA, reducing system overhead, and improving data processing speed.

[0004] To achieve the above purpose, the present application adopts the following technical scheme:

[0005] Technical scheme one

[0006] The application discloses an unmanned aerial vehicle (UAV) inspection data processing method based on FPGA, which comprises the following steps: step 1, performing A / D conversion on radio inspection data returned by the UAV, and transmitting the converted data including current color image data and identification data of the current color image data to FPGA; step 2, reading the identification data in the FPGA, and then reading historical image data corresponding to the identification data in a server through a PCIE interface; step 3, converting a pixel value corresponding to each pixel point in the current color image data into a pixel value with a quality code according to a pre-defined data format with a quality code; step 4, performing image gray scale processing on the current color image data and the historical image data to obtain current gray scale image data and historical gray scale image data, wherein a gray scale value of each pixel point in the current gray scale image data has the quality code; step 5, identifying a historical position of a monitoring target from the historical gray scale image data through an edge detection algorithm, meanwhile, identifying a current position of the monitoring target from the current gray scale image data through the edge detection algorithm, and detecting whether the current position is correctly identified according to the quality code, if the current position is not correctly identified, repeating step 5 to continue identifying the monitoring target, if the current position is correctly identified, further detecting whether image quality of the identified current position meets a requirement according to the quality code, if yes, entering step 6, if no, determining that the current position is abnormal, and then sending a notification to perform manual secondary verification; and step 6, comparing whether there is a difference between the current position and the historical position of the monitoring target, if yes, notifying manual secondary verification, and sending the current color image data to the server for storage.

[0007] More preferably, in step 3, the FPGA converts the pixel value of each pixel point from a floating point number into a fixed point number and adds a quality code of at least 2 bits to generate a pixel value with a quality code, wherein the 2-bit quality code respectively represents whether the pixel value exceeds an upper limit and whether the pixel value exceeds a lower limit; the upper limit and the lower limit are both preset values, and the value before the decimal point in the fixed point number is compared with the upper limit and the lower limit to determine whether the pixel value of the point is abnormal, and the pixel point corresponding to the pixel value exceeding the upper limit or the lower limit is determined as a bad point.

[0008] More preferably, the pixel value with a quality code is a pixel value plus a quality code of 4 bits, wherein the 4-bit quality code respectively represents whether the pixel value exceeds an upper limit, whether the pixel value exceeds a lower limit, whether the quality of the pixel point is bad, and whether the quality of the pixel point is good, when the pixel value neither exceeds the upper limit nor exceeds the lower limit, the quality of the pixel point is good, and when the pixel value exceeds the upper limit or the lower limit, the quality of the pixel point is bad.

[0009] More preferably, in the step 5, whether the current position is correctly identified is detected according to the quality code, and the specific process is as follows: the bad point rate of all edge pixel points in the identified current position is calculated, when the bad point rate exceeds a preset first threshold, it is determined that the edge of the current position of the monitored target is incorrectly identified, and the current position of the monitored target needs to be detected again, when the bad point rate does not exceed the preset first threshold, the image quality detection of the current position is continued.

[0010] More preferably, in the step 5, whether the image quality of the identified current position meets the requirements is detected according to the quality code, and the specific process is as follows: the bad point rate of all pixel points in the identified current position is calculated, when the bad point rate exceeds a preset second threshold, it is determined that the image of the current position is abnormal, and manual verification needs to be notified, when the bad point rate does not exceed the preset second threshold, the step 6 is executed.

[0011] More preferably, the FPGA is further in communication connection with a memory A and a memory B, the FPGA adopts a ping-pong pipeline mechanism to process the received data, and the specific process includes the following steps: the FPGA alternately stores the inspection data from different unmanned aerial vehicles between the memory A and the memory B, and the FPGA pre-processes the data in the memory B while reading the data in the memory A for calculation; when the FPGA completes the calculation of the data in the memory A, the pre-processed data in the memory B is calculated, and the data in the memory A is pre-processed; the pre-processing is to execute steps 3 and 4, and the calculation of the data is to execute steps 5 and 6.

[0012] More preferably, the identification data includes acquisition time, latitude and longitude, and height.

[0013] Technical solution two

[0014] Based on the same inventive concept, the application also provides an unmanned aerial vehicle inspection data processing device based on FPGA.

[0015] An unmanned aerial vehicle inspection data processing device based on FPGA, comprising an A / D converter, an FPGA, a memory and a server, the A / D converter converts the received radio data into a digital signal, the PFGA stores it into the memory and executes the method steps as described in technical solution one.

[0016] The application has the following beneficial effects:

[0017] 1、The present application defines a new pixel data format, and adds a quality code to each pixel point, so that when the edge detection algorithm identifies the current position of the monitoring target, the quality of the edge pixel point at the current position can be detected first, and the algorithm is verified whether it finds the correct edge of the monitoring target, and the confidence of the identification result is evaluated, and the image of the monitoring target with edge detection error is filtered out, which helps to improve the identification accuracy of the monitoring target, and then the image quality at the current position is detected whether it meets the requirements, and the image with quality abnormality can be immediately notified to the artificial verification, and the current position image with quality meeting the requirements can be identified whether there is a difference through comparison algorithm, which greatly improves the accuracy of the monitoring target abnormality judgment.

[0018] 2、The present application introduces pixel values with quality codes, optimizes data calculation process, and improves data processing speed.

[0019] 3、The present application greatly accelerates the algorithm processing speed of FPGA through the ping-pong pipeline mechanism, so that the algorithm is not idle, and the algorithm overflow is also avoided, which causes data loss.

[0020] 4、The present application is especially suitable for processing unmanned aerial vehicle inspection data of power facilities such as mountain top and bridge. DETAILED DESCRIPTION

[0021] Figure 1 The flowchart of the present application;

[0022] Figure 2 The hardware deployment schematic diagram in the return data receiving station of the present application. CONCRETE EMBODIMENT

[0023] The present application will be further described in detail below in combination with the drawings and specific embodiments.

[0024] Embodiment one

[0025] The present application is a kind of unmanned aerial vehicle inspection data processing method based on FPGA, and the data processing steps are executed based on the following hardware deployment. Please refer to Figure 2 In the receiving station, the radio data returned by the unmanned aerial vehicle is received by ADRV9009, and then sent to the AD converter, and then the converted digital signal data is processed by the FPGA chip. The FPGA is connected with two memories, memory A and memory B, and the FPGA is also connected with the server through the PCIE interface. The AD converter can also be replaced by the FPGA chip with A / D function.

[0026] Please refer to Figure 1The application discloses a kind of unmanned aerial vehicle inspection data processing method based on FPGA, comprising the following steps: step 1, the radio inspection data returned by unmanned aerial vehicle is converted by A / D, the data after conversion includes current color image data and its identification data, then is transmitted to FPGA;Step 2, FPGA reads the identification data therein, the identification data includes acquisition time, latitude and longitude and height, then the historical image data corresponding to the identification data in the server is read by PCIE interface;Step 3, FPGA converts the pixel value corresponding to each pixel point in current color image data according to the data format with quality code defined in advance, generates pixel value with quality code;Step 4, FPGA carries out image gray scale processing to current color image data and historical image data, obtains current gray scale image data and historical gray scale image data, wherein the gray scale value of each pixel point in current gray scale image data has the quality code;Step 5, the historical position of monitoring target is identified from historical gray scale image data by edge detection algorithm, simultaneously, the current position of monitoring target is identified from current gray scale image data by edge detection algorithm, and whether the current position is identified correctly is detected according to quality code, if not identified correctly, then step 5 is repeatedly executed, and monitoring target is continuously identified, if identified correctly, then whether the image quality of the current position identified meets the requirement is further detected according to quality code, if yes, then enter step 6, if no, it is judged that the current position is abnormal, then send notice, and artificial secondary verification is carried out;Step 6, whether there is difference between the current position and the historical position of monitoring target, if yes, then notify artificial secondary check, and current color image data is sent to server storage.

[0027] In step 3, the pixel value received by the FPGA from the A / D converter is a floating point number. The FPGA first converts the pixel value of each pixel point from a floating point number to a fixed point number and adds at least 2 bits of quality code to generate a pixel value with quality code. In this embodiment, the original pixel value is converted using a pixel value format with 4-bit quality code, wherein the 4-bit quality code respectively represents whether the pixel value exceeds the upper limit, whether the pixel value exceeds the lower limit, whether the pixel point quality is bad, and whether the pixel point quality is good. The upper limit and the lower limit are both preset values. The value before the decimal point in the fixed point number is compared with the upper limit and the lower limit to determine whether the pixel value of the point is abnormal. The pixel point corresponding to the pixel value exceeding the upper limit or the lower limit is determined as a bad point. When the pixel value neither exceeds the upper limit nor exceeds the lower limit, the pixel point quality is good. When the pixel value exceeds the upper limit or the lower limit, the pixel point quality is bad. For example, the data type of the pixel value with quality code is 32-bit data and 4-bit quality code, wherein the 32-bit data includes 13-bit integer and 19-bit decimal. The value of the 13-bit integer is compared with the upper limit and the lower limit to determine whether the pixel value of the point is abnormal. The meanings of the bit positions of the 4-bit quality code are as follows: bit 4 represents that the data exceeds the upper limit, bit 3 represents that the data exceeds the lower limit, bit 2 represents that the data quality is bad, and bit 1 represents that the data quality is good.

[0028] The 4-bit quality code can improve the reliability of the quality code itself and can timely find errors or abnormalities of the quality code in the data processing process.

[0029] In step 5, whether the current position is correctly identified is detected according to the quality code, and the specific process is as follows: the bad point rate of all edge pixel points in the identified current position is calculated, when the bad point rate exceeds a preset first threshold, it is determined that the edge of the current position of the monitored target is incorrectly identified, and the current position of the monitored target needs to be detected again, when the bad point rate does not exceed the preset first threshold, the image quality detection of the current position is continued. Whether the image quality of the identified current position meets the requirements is detected according to the quality code, and the specific process is as follows: the bad point rate of all pixel points in the identified current position is calculated, when the bad point rate exceeds a preset second threshold, it is determined that the image of the current position is abnormal, and manual verification needs to be notified, when the bad point rate does not exceed the preset second threshold, step 6 is executed. For example, the first threshold and the second threshold are both set to 35%, in the edge detection process, the pixel point bad probability of the edge region of the current position exceeds 35%, then the region is abandoned, which to some extent avoids that the picture with bad quality is incorrectly identified as the position of the monitored target, and the correct identification rate is improved. Then, the quality code is introduced to count the overall bad point probability of the identified position of the monitored target, once the picture bad point probability exceeds 35%, manual verification needs to be introduced to avoid false alarm or missed alarm. Therefore, the pixel point quality code is introduced, the identification result confidence is first evaluated, the correct identification rate can be improved, and then image comparison is performed, the accuracy of the abnormal judgment of the monitored target can be improved.

[0030] The FPGA processes the data from the A / D converter in a ping-pong pipeline mechanism, specifically including the following steps: the FPGA alternately stores the inspection data from different unmanned aerial vehicles between memory A and memory B, and the FPGA pre-processes the data in memory B while calculating the data in memory A; when the FPGA completes the calculation of the data in memory A, the pre-processed data in memory B is calculated, and the data in memory A is pre-processed. For example, when multiple unmanned aerial vehicles simultaneously transmit the inspection data of multiple towers, the data of the first unmanned aerial vehicle is cached to memory A, and the data of the second unmanned aerial vehicle is cached to memory B, the algorithm pre-processes the data in memory A, and calculates the data in memory B at the same time; when the data in memory A is pre-processed, the data of the next unmanned aerial vehicle in memory B is pre-processed, and when the data in memory B is calculated, the pre-processed data in memory A is calculated. The pre-processing is to execute steps 2 to 4, and the calculation of the data is to execute steps 5 and 6. The ping-pong pipeline mechanism ensures that the algorithm can work in full pipeline, and the unmanned aerial vehicles will not be queued, and the data will not be overlapped or wrong.

[0031] In step 4, image gray scale processing is performed, that is, the image is binarized, and various binarization methods in the prior art can be applied to the embodiment, which will not be described here. In step 6, the image at the current position is compared with the image at the historical position for similarity, and the traditional Hash algorithm can be used, or the method of judging whether the gray scale values of the corresponding pixel points in the two position images are the same can be simply used, and then the probability of the same gray scale values is counted to determine whether it is abnormal, and the convolutional neural network can also be trained using historical data, and the current position image is input into the convolutional neural network to determine whether it is abnormal. The image similarity comparison algorithm is prior art and will not be described in detail.

[0032] Embodiment two

[0033] Please refer to Figures 1 to 2 Based on the same inventive concept, the embodiment also provides a UAV inspection data processing device based on FPGA, which comprises an A / D converter, a FPGA, a memory and a server, the A / D converter converts the received radio data into digital signals, the PFGA stores them into the memory and executes the method steps as described in embodiment one.

[0034] The UAV inspection data processing method and device based on FPGA improve the image recognition accuracy and the accuracy of abnormal judgment of the monitoring target by performing the format conversion of the pixel points with the quality code and applying it to the recognition of whether the detection and monitoring target is correct and the evaluation of the image quality.

[0035] The above is only a specific embodiment of the present application, and does not limit the patent scope of the present application, and any equivalent structural transformation using the content of the specification and drawings, or direct or indirect application in other related technical fields, is also included in the patent protection scope of the present application.

Claims

1. A UAV inspection data processing method based on FPGA, characterized in that: The method comprises the following steps: Step 1, A / D conversion is performed on radio inspection data returned by the unmanned aerial vehicle, the converted data comprising current color image data and identification data thereof, and then the data is transmitted to the FPGA; Step 2, the FPGA reads the identification data therein, and then reads historical image data corresponding to the identification data in the server through the PCIE interface; Step 3, the FPGA converts a pixel value corresponding to each pixel point in the current color image data into a pixel value with a quality code according to a pre-defined data format with a quality code; specifically, the FPGA converts the pixel value of each pixel point from a floating-point number into a fixed-point number and adds at least 2 bits of quality code to generate a pixel value with a quality code, wherein the 2 bits of quality code respectively represent whether the pixel value exceeds an upper limit and whether the pixel value exceeds a lower limit; the upper limit and the lower limit are both preset values, and the value before the decimal point in the fixed-point number is compared with the upper limit and the lower limit respectively to determine whether the pixel value of the point is abnormal, and the pixel point corresponding to the pixel value exceeding the upper limit or the lower limit is determined as a bad point; Step 4, the FPGA performs image gray scale processing on the current color image data and the historical image data to obtain current gray scale image data and historical gray scale image data, wherein the gray scale value of each pixel point in the current gray scale image data has the quality code; Step 5, a historical position of the monitoring target is identified from the historical gray scale image data through an edge detection algorithm, and a current position of the monitoring target is identified from the current gray scale image data through the edge detection algorithm, and whether the current position is identified correctly is detected according to the quality code, if not, step 5 is repeatedly executed to continue identifying the monitoring target, and if yes, further detection is performed on the image quality of the identified current position according to the quality code, if the image quality meets the requirements, step 6 is entered, if not, it is determined that the current position is abnormal, a notification is sent, and manual secondary verification is performed; the specific process of detecting whether the current position is identified correctly according to the quality code is that a bad point rate of all edge pixel points in the identified current position is calculated, when the bad point rate exceeds a pre-set first threshold, it is determined that the edge of the current position of the monitoring target is identified incorrectly, and the current position of the monitoring target needs to be detected again, and when the bad point rate does not exceed the pre-set first threshold, the image quality detection of the current position is continued; Step 6, whether there is a difference between the current position and the historical position of the monitoring target is compared, if yes, manual secondary verification is notified, and the current color image data is sent to the server for storage. 2.The FPGA-based unmanned aerial vehicle inspection data processing method of claim 1, wherein: The pixel value with a quality code is a pixel value plus a 4-bit quality code, wherein the 4-bit quality code respectively represents whether the pixel value exceeds the upper limit, whether the pixel value exceeds the lower limit, whether the quality of the pixel point is bad, and whether the quality of the pixel point is good, when the pixel value neither exceeds the upper limit nor the lower limit, the quality of the pixel point is good, and when the pixel value exceeds the upper limit or the lower limit, the quality of the pixel point is bad. 3.The FPGA-based unmanned aerial vehicle inspection data processing method of claim 1, wherein: In step 5, whether the image quality of the recognized current position meets the requirements is detected according to the quality code, and the specific process is as follows: the bad pixel rate of all pixel points in the recognized current position is calculated, when the bad pixel rate exceeds a preset second threshold, it is determined that the image of the current position is abnormal, and manual verification is required, when the bad pixel rate does not exceed the preset second threshold, step 6 is executed.

4. The FPGA-based unmanned aerial vehicle inspection data processing method of claim 1, wherein: The FPGA is also in communication connection with the memory A and the memory B, the FPGA adopts a ping-pong flow mechanism to process the received data, and specifically includes the following steps: the FPGA alternately stores the inspection data from different unmanned aerial vehicles between the memory A and the memory B, and the FPGA pre-processes the data in the memory B while reading the data in the memory A for calculation; When the FPGA completes the calculation of the data in the memory A, the pre-processed data in the memory B is calculated, and the data in the memory A is pre-processed; The pre-processing is step 3 and step 4, and the calculation of the data is step 5 and step 6.

5. The FPGA-based unmanned aerial vehicle inspection data processing method of claim 1, wherein: The identification data includes acquisition time, latitude and longitude, and height.

6. An FPGA-based unmanned aerial vehicle inspection data processing device, characterized in that: It comprises an A / D converter, an FPGA, a memory and a server, the A / D converter converts the received radio data into a digital signal, the FPGA stores it into the memory and executes the method steps of any one of claims 1 to 5.

Images