Unmanned aerial vehicle front-end image recognition equipment and method based on lightweight edge calculation

Abstract

The invention belongs to the technical field of power grid digitization, and particularly relates to an unmanned aerial vehicle front-end image recognition device and method based on lightweight edge calculation. A damping rubber mat and a heat dissipation system are arranged in the device shell, a computing mainboard is installed, a storage, a battery, a keyboard and a display screen are embedded in the device shell, and a USB interface, an HDMI interface and an Ethernet interface are formed in one side of the device shell. In the power equipment inspection process, a large amount of manpower and material resources can be saved, the defects of various equipment and facilities can be quickly and accurately recognized, the problems of data transmission delay, misjudgment, missed judgment and the like are avoided, potential safety hazards are eliminated, the operation and maintenance cost is reduced, meanwhile, stable operation of a power system is guaranteed, and the reliability and safety of the system are improved; and the inspection efficiency and quality are obviously improved. The method is suitable for industries such as electric power industry, railways, petroleum, buildings and the like which have high requirements on unmanned aerial vehicle inspection and defect identification accuracy.

Description

Technical Field

[0001] The present invention belongs to the technical field of power grid digitization, and particularly relates to an unmanned aerial vehicle (UAV) front-end image recognition device and method based on lightweight edge computing. Background Art

[0002] With the rapid development of power grid construction, the inspection work of power equipment has become increasingly important. Traditional power inspections mainly rely on manual methods, which are inefficient and have certain safety hazards. In recent years, with the development of UAV technology, UAV inspections have gradually become an efficient and safe alternative. UAVs can quickly reach the inspection site, collect image and video data of power equipment, and provide a basis for subsequent defect detection and analysis.

[0003] However, existing UAV inspection systems use a centralized cloud computing model for data processing. The image data collected by UAVs needs to be transmitted to a cloud server for analysis and processing. This method has the following technical problems: 1. Data transmission delay: Under the existing centralized cloud computing model, due to the huge amount of image data collected by UAVs, it takes a long time to transmit to the cloud server, resulting in a large time delay and poor real-time performance of data processing, making it difficult to meet the real-time requirements of power equipment inspections.

[0004] 2. Computing resource occupancy: The centralized cloud computing model requires a large amount of computing resources to process massive image data, which not only increases the operation and maintenance costs but also may cause performance bottlenecks in the system under high load.

[0005] 3. Network dependence: Data transmission depends on the network environment. In areas with unstable network signals or insufficient coverage, the efficiency and reliability of data transmission will be affected, thus affecting the normal progress of inspection work.

[0006] 4. Security and privacy: Data transmission in the power grid system is usually carried out in an intranet environment, with high security. However, under the centralized processing mode, the centralized storage and management of data may still face some potential security and privacy issues.

[0007] 5. Algorithm adaptability issues: Most existing intelligent inspection algorithms rely on high-performance and high-power-consuming computing resources and are difficult to be directly deployed on edge computing devices, especially in scenarios such as UAVs where there are strict restrictions on power consumption and weight.

[0008] 6. Balance between model accuracy and efficiency: Existing lightweight models often struggle to achieve an ideal balance between accuracy and efficiency and are difficult to achieve high-precision image recognition with limited computing resources.

[0009] 7. System integration complexity: When integrating the edge computing module with the UAV system, multiple aspects such as hardware adaptation, software compatibility, and communication protocols need to be considered, resulting in a relatively high system integration complexity.

[0010] In the field of UAV inspection, although edge computing technology has significant advantages, there is currently no mature solution that can effectively apply edge computing technology to UAV front-end image recognition. When dealing with image data collected by UAVs, the existing edge computing technology still faces the following challenges: 1. Algorithm adaptability issue: Most existing intelligent inspection algorithms rely on high-performance and high-power-consuming computing resources, making it difficult to directly deploy them on edge computing devices, especially in scenarios such as UAVs where there are strict restrictions on power consumption and weight.

[0011] 2. Balance between model accuracy and efficiency: Although edge computing can improve the real-time performance of data processing, how to achieve high-precision image recognition with limited computing resources remains a challenge. Existing lightweight models often struggle to achieve an ideal balance between accuracy and efficiency.

[0012] 3. System integration complexity: When integrating the edge computing module with the UAV system, multiple aspects such as hardware adaptation, software compatibility, and communication protocols need to be considered, resulting in a relatively high system integration complexity.

[0013] Therefore, there are still many problems and deficiencies in the application of existing technologies in UAV inspection. Especially in the application of edge computing technology, there is currently no mature solution. Summary of the Invention

[0014] In view of the above deficiencies in the existing technology, the present invention provides a UAV front-end image recognition device and method based on lightweight edge computing. Its purpose is to achieve the immediate inspection and intelligent recognition of power grid equipment through the UAV front-end defect recognition device, and to improve the efficiency and accuracy of power grid inspection.

[0015] The technical solution adopted by the present invention to achieve the above object is: A UAV front-end image recognition device based on lightweight edge computing is provided. Inside the device housing, there are shock-absorbing rubber pads and a heat dissipation system, and a computing mainboard, a memory, a battery are installed. The keyboard and the display screen are embedded on the device housing. On one side of the device housing, there are a USB interface, an HDMI interface, and an Ethernet interface.

[0016] Furthermore, the computing mainboard: is used for processing and analyzing image data; The memory: is used for storing image processing algorithms, machine learning models, and recognition results; The display screen: for real-time display of image processing and recognition results; The keyboard: a keyboard with a touchpad, used for operating and controlling the device; The battery: a 30000mAh battery, charged and discharged through a USB interface.

[0017] Furthermore, the front-end image recognition device of the drone based on lightweight edge computing includes an edge computing module, a communication module, and a power module; The edge computing module is equipped with a high-performance computing mainboard, equipped with a CPU chip with more than four cores and a GPU chip with 128 CUDA cores, supporting multi-threaded concurrent processing, providing powerful computing capabilities for image recognition; the built-in memory is used to store image processing algorithms, machine learning models, and recognition results, ensuring fast data reading, writing, and storage; The communication module has high-speed wireless communication interfaces and wired communication interfaces, used to transmit the recognition results to the system backend server of the power grid in real time; The power module is equipped with a high-capacity battery, charged and discharged through a USB interface.

[0018] The front-end image recognition method of the drone based on lightweight edge computing includes the following steps: Step 1. Image recognition method, including: Image preprocessing: performing enhancement and denoising preprocessing on the collected images; Image recognition: using a lightweight deep learning model to perform object detection and classification on the preprocessed images, and identifying defects in power equipment; Transmission of image recognition results: transmitting the image recognition results and warning information to the system backend server of the power grid through the communication module for further analysis and management; Step 2. Model optimization, including: Lightweight model design: adopting a lightweight deep learning model architecture for lightweight model design; Knowledge distillation model training: through model distillation, using the prediction results of the complex model as a guide to train a lightweight model; referring to the real label data and the output of the complex model, inheriting the key features and knowledge of the complex model, and at the same time adapting to the resource limitations of edge computing devices; Training of a dynamically pruned deep neural network model: removing redundant neurons and connections through pruning techniques to further optimize the performance of the model; adopting a quantized model design to convert the weights and activation functions of the model from floating-point numbers to low-precision representations to achieve model optimization.

[0019] Furthermore, the front-end image recognition method of the drone based on lightweight edge computing includes the following steps: Step 1. Device initialization; Start the defect recognition device: Turn on the defect recognition device, ensure that the device starts up normally, check the battery power of the device, and check the network connection of the device; Check the hardware modules: including: check whether the computing mainboard, memory, display screen, keyboard with touchpad, and battery hardware modules are working properly; check the protection function of the device shell; Load the operating system: including starting the Linux operating system; Step 2. Image acquisition; Start the drone: Ensure that the drone has sufficient battery power; check whether the camera of the drone is working properly; Connect the drone to the defect recognition device: Connect the drone to the industrial control-level operation tablet through the USB interface or wireless communication module; start the drone control software on the defect recognition device; Collect images: According to the preset inspection route, control the drone to fly and collect images of power equipment; ensure that the collected images are clear and complete, covering all equipment and areas that need to be inspected; Step 3. Image preprocessing; Image transmission: Transmit the collected images to the defect recognition device through the communication module; Image enhancement and denoising: Run the image preprocessing algorithm on the operation tablet to enhance and denoise the collected images; Use the denoising algorithm to remove the noise in the images and improve the image quality; Step 4. Image recognition; Load the lightweight model: Load the pre-trained lightweight deep learning model on the operation tablet; Object detection and classification: Run the image recognition algorithm to perform object detection and classification on the preprocessed images; identify the defects of power equipment; generate recognition results, including the location, type, and confidence information of the defects; Step 5. Result transmission and storage; Result transmission: Transmit the recognition results to the system backend server of the power grid through the communication module; Result storage: Store the recognition results on the power grid backend server for subsequent analysis and management; Step 6. Inspection report generation; Report generation: Generate an inspection report on the power grid backend server; Report storage and backup: Store the generated inspection report in the database of the server and perform a backup.

[0020] Furthermore, the lightweight deep learning model includes: MobileNet, Tiny-YOLO.

[0021] An unmanned aerial vehicle (UAV) front-end image recognition device based on lightweight edge computing, comprising: An image recognition module for image recognition, performing enhancement and denoising preprocessing on the acquired images; using a lightweight deep learning model to perform object detection and classification on the preprocessed images, identifying defects in power equipment; transmitting the image recognition results and warning information to the system back-end server of the power grid through a communication module for further analysis and management; A model optimization module for model optimization; Including lightweight model design: adopting a lightweight deep learning model architecture for lightweight model design; Knowledge distillation model training: through model distillation, using the prediction results of a complex model as a guide to train a lightweight model; referring to real label data and the output of the complex model, inheriting the key features and knowledge of the complex model while adapting to the resource limitations of edge computing devices; Training of a dynamically pruned deep neural network model: removing redundant neurons and connections through pruning techniques to further optimize the performance of the model; adopting a quantized model design to convert the weights and activation functions of the model from floating-point numbers to low-precision representations to achieve model optimization.

[0022] Furthermore, the image recognition module includes: Image acquisition: controlling the UAV to fly and acquire images of power equipment according to a preset inspection route; ensuring that the acquired images are clear and complete, covering all equipment and areas that need to be inspected; Image transmission: transmitting the acquired images to the defect recognition device through a communication module; Image enhancement and denoising: running an image preprocessing algorithm on an operation tablet to perform enhancement and denoising processing on the acquired images; using a denoising algorithm to remove noise in the images and improve image quality; Image recognition; loading a lightweight model: loading a pre-trained lightweight deep learning model on an operation tablet.

[0023] A computer device, comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein when the processor executes the computer program, the steps of any one of the above-mentioned UAV front-end image recognition methods based on lightweight edge computing are implemented.

[0024] A computer storage medium, having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps of any one of the above-mentioned UAV front-end image recognition methods based on lightweight edge computing are implemented.

[0025] The present invention has the following beneficial effects and advantages: The purpose of the present invention is the research and development and application of a UAV front-end defect recognition device based on lightweight edge computing. By innovatively combining lightweight edge computing technology with an industrial control-level operation tablet, a UAV front-end defect recognition device based on lightweight edge computing is realized, enabling the immediate inspection and intelligent recognition of power grid equipment, and improving the efficiency, accuracy, and safety of power grid inspection.

[0026] The specific advantages of the present invention are as follows: 1. Strong real-time performance: Through the edge computing recognition algorithm carried by the industrial control-level operation tablet, the immediate inspection and intelligent recognition of power grid equipment are realized, reducing data transmission delay and improving real-time performance.

[0027] 2. High accuracy: Advanced image processing algorithms and lightweight deep learning models are adopted to improve the accuracy of power grid equipment recognition.

[0028] 3. High security: The edge computing device directly processes and analyzes data on the industrial control-level operation tablet, reducing the risk of data transmission and improving the security of the system.

[0029] 4. Strong portability and adaptability: The device of the present invention is small in size and light in weight. The device shell is designed considering the complex outdoor working environment, with functions such as waterproof, dustproof, and anti-collision, adapting to the complex outdoor environment. At the same time, a handle is equipped for easy carrying, and inspection work can be carried out anytime and anywhere.

[0030] 5. High efficiency: It supports lightweight defect recognition algorithms, significantly improving the speed and accuracy of defect recognition, and reducing the possibility of misjudgment and missed judgment. Through model optimization techniques such as model pruning, quantization, and distillation, it ensures efficient operation on low-power, high-performance edge computing devices. By adopting a lightweight deep learning model, it operates efficiently on low-power, high-performance edge computing devices, significantly improving the inspection efficiency.

[0031] 6. Simplicity of system integration: The optimized system design and modular architecture reduce the integration complexity of the system and improve the scalability and maintainability of the system.

[0032] 7. Easy to operate: The device of the present invention is easy to operate, with a user-friendly interface, and can be started without professional training. Through the intuitive display screen and touchpad operation, users can easily control the UAV and image recognition device to complete the inspection task.

[0033] During the inspection process of power equipment, the device of the present invention can save a large amount of manpower and material resources, and quickly and accurately identify defects in various equipment and facilities, improve the inspection efficiency and quality, avoid problems such as data transmission delay, misjudgment and missed judgment, eliminate potential safety hazards, and also provide an efficient, reliable and safe solution for the intelligent inspection of power equipment. The application of this innovative technology will greatly improve the efficiency and quality of power inspection work, reduce operation and maintenance costs by reducing the demand for data transmission and centralized processing, ensure the stable operation of the power system at the same time, and improve the reliability and safety of the system. Through lightweight edge computing technology, optimized algorithms and system design, the blank of edge computing technology in the application of drone inspection is filled, and the inspection efficiency and quality are significantly improved.

[0034] The present invention is not limited to the power industry, but also applicable to industries such as railways, petroleum, and construction that have a large demand for drone inspection and high requirements for the accuracy of defect identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The above and / or additional aspects and advantages of the present invention will become obvious and easy to understand from the description of the embodiments in conjunction with the following drawings, wherein: Figure 1 is a schematic structural diagram of the front-end defect identification device of the drone of the present invention; Figure 2 is a schematic structural diagram of the image recognition device of the present invention; Figure 3 is an implementation schematic diagram of the image recognition device of the present invention; Figure 4 is a module diagram of the present invention; Figure 5 is a method flow chart of the present invention; In the figure: computing main board 1, memory 2, display screen 3, keyboard 4, battery 5, device housing 6, USB interface 7, HDMI interface 8, Ethernet interface 9. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0036] In order to more clearly understand the above objects, features and advantages of the present invention, the present invention will be further described in detail below in conjunction with the drawings and specific embodiments. It should be noted that, without conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other.

[0037] Many specific details are set forth in the following description in order to fully understand the present invention. However, the present invention may be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited by the specific embodiments disclosed below.

[0038] The following refers to Figures 1 - 5 Describe the technical solutions of some embodiments of the present invention.

[0039] Example 1

[0040] The present invention provides an embodiment, which is a front-end image recognition device for drones based on lightweight edge computing. As Figures 1 - 3 shown, Figure 1 is a schematic structural diagram of the front-end defect recognition device for drones of the present invention, Figure 2 is a schematic structural diagram of the image recognition device of the present invention, Figure 3 is an implementation schematic diagram of the image recognition device of the present invention.

[0041] The figure shows the internal structure of the front-end defect recognition device for drones and the layout of each component. The figure includes the positions and connection relationships of the computing main board 1, memory 2, display screen 3, keyboard 4, battery 5, device housing 6, USB interface 7, HDMI interface 8, and Ethernet interface 9.

[0042] The image recognition device of the present invention adopts a modular structure, and the spatial layout and connection relationships of each component within the device housing are as follows: (1) Positioning of the computing main board.

[0043] Position: Embedded as the core module on the central axis of the device, in the golden heat dissipation interval 10 cm from the top and 4 cm from the bottom.

[0044] Connection: Realize the three-element link of GPU-CPU-Memory through the PCIe 4.0 interface.

[0045] (2) Memory configuration.

[0046] Position: Close to the back side of the computing main board, forming a heat conduction path with the aluminum heat dissipation fins of the housing through thermal conductive silicone.

[0047] Connection: Adopt a dual-channel DDR4 3200MHz SODIMM interface, and reserve an M.2 2280 expansion slot.

[0048] (3) Human-computer interaction unit.

[0049] Display screen 3: A 7-inch IPS touch screen (resolution 1920×1200) is embedded on the device housing 6 at an inclination angle of 30°, and is directly connected to the GPU through the HDMI interface.

[0050] Keyboard 4: An industrial-grade keyboard module with a touchpad function is connected to the computing main board through the USB bus, and the mechanical structure design with a key travel of 3.0 mm is suitable for outdoor operations.

[0051] (4) Energy system.

[0052] Battery pack: A 30000mAh lithium polymer battery is distributed at the bottom of the device and adopts a bidirectional fast charging protocol. It includes battery 5.

[0053] Connection: Dynamic power distribution is achieved through the intelligent power management unit, supporting power supply for the computing motherboard + peripherals.

[0054] (5) Interface topology.

[0055] Data transmission: The full-function USB interface 7 and the HDMI interface 8 are misaligned and arranged on the right side of the device. The HDMI interface 8 is an HDMI 2.0b interface.

[0056] Network expansion: The 2.5G Ethernet interface 9 is directly connected to the CPU through a PCIe transfer chip.

[0057] Including: USB interface 7, HDMI interface 8, and Ethernet interface 9.

[0058] (6) Structural innovation.

[0059] Adopt a structure with a composite material 3D printed shell, and 12 groups of shock-absorbing rubber pads are arranged inside.

[0060] The cooling system includes 3 groups of 4020 ultra-thin turbo fans with a thickness of only 8 mm, forming a three-dimensional air duct with air flowing in from the front and out from the back.

[0061] The computing motherboard 1: is used for processing and analyzing image data. In order to provide sufficient computing power for the lightweight defect recognition algorithm, the computing motherboard needs to be equipped with a GPU chip with 128 CUDA cores and a CPU chip with more than four cores to support multi-threaded concurrent processing.

[0062] The memory 2: is used for storing image processing algorithms, machine learning models, and recognition results.

[0063] The display screen 3: is used for real-time display of image processing and recognition results.

[0064] The keyboard 4: is a keyboard with a touchpad, used for operating and controlling the device.

[0065] The battery 5: In order to adapt to the lack of power at outdoor operation sites and the battery life issue, a 30000 mAh battery is selected and can be charged and discharged through the USB interface.

[0066] The device shell 6: has functions such as waterproof, dustproof, and anti-collision.

[0067] The interface: includes USB interface 7, HDMI interface 8, and Ethernet interface 9, and has wireless communication interfaces such as Type-C interface, WiFi, and Bluetooth.

[0068] Operating system: It is equipped with an industrial-grade Linux operating system, supporting the deployment and operation of lightweight image recognition models.

[0069] The present invention relates to a UAV front-end image recognition device based on edge computing, which includes an edge computing module, a communication module, and a power module.

[0070] Among them, the edge computing module is equipped with a high-performance computing mainboard, with a CPU chip with more than four cores and a GPU chip with 128 CUDA cores, supporting multi-threaded concurrent processing, providing powerful computing capabilities for image recognition. The built-in memory is used to store image processing algorithms, machine learning models, and recognition results, ensuring fast data reading, writing, and storage.

[0071] The communication module is equipped with high-speed wireless communication interfaces such as 5G and WiFi, and wired communication interfaces such as Ethernet interfaces, which are used to transmit the recognition results to the system backend server of the power grid in real time, ensuring data transmission stability in complex environments.

[0072] The power module is equipped with a high-capacity battery, at least 30000 mAh, ensuring that the device can run for a long time without an external power supply. It is charged and discharged through a USB interface, facilitating users to charge in different scenarios. The protective shell has functions such as waterproof, dustproof, and anti-collision, adapting to complex outdoor working environments. The shell design takes portability into consideration and is equipped with a handle for easy carrying and use.

[0073] As Figure 4 shown, Figure 4 is the module diagram of the present invention.

[0074] The present invention includes three major modules: a sample model library, an edge computing AI platform, and an instant report.

[0075] (1) The sample model library includes: Sample data set: It is the basic data source for the operation of the entire system. It collects a large amount of data related to specific tasks. These data are sorted and labeled, providing rich materials for subsequent model training, ensuring that the model can learn enough features and patterns to achieve functions such as accurate image recognition.

[0076] Sample model standard: It stipulates the specifications and criteria for model construction and evaluation. It clarifies the performance indicators, accuracy requirements, data processing methods, and other aspects of the standards that the model should meet, providing a unified measurement scale for the model during training and application, ensuring the consistency and comparability between different models, and helping to improve the quality and reliability of the model.

[0077] Model training: It is the process of using the data in the sample data set, according to certain algorithms and optimization strategies, to let the model learn the characteristics and rules in the data. By continuously adjusting the parameters of the model, the model can produce more accurate output results for the input data, thereby improving the performance and accuracy of the model and preparing for tasks such as image recognition in practical applications.

[0078] Model algorithm compression: aims to reduce the amount of computation, storage requirements, and running time of the model while maintaining the performance of the model as much as possible. By adopting technologies such as pruning, quantization, and knowledge distillation, the trained model is optimized so that it can run efficiently in resource-constrained environments such as front-end devices, improving the overall efficiency and deployability of the system.

[0079] Front-end image recognition: It is mainly responsible for the recognition and processing of input images on the front-end device. It uses trained and optimized models to analyze and judge the real-time acquired images, extract key information from them, and provide a basis for subsequent decision-making and processing. It is the key link for the entire system to directly interact with actual image data.

[0080] (2) The edge computing AI platform includes: GPU / FPGA computing: GPU graphics processors and FPGA field programmable gate arrays are powerful computing hardware. In edge computing AI platforms, they are used to accelerate complex computing tasks, especially those related to artificial intelligence, such as reasoning and training of deep learning models. GPU has a highly parallel computing architecture that can quickly process large-scale data, while FPGA can perform flexible hardware configuration according to specific needs. The combination of the two can significantly improve the computing efficiency of the system, reduce processing time, and meet real-time requirements.

[0081] LiDAR recognition: LiDAR is a sensor used to obtain three-dimensional spatial information of target objects. In this platform, the LiDAR recognition function uses the point cloud data collected by LiDAR to identify and analyze the shape, position, size and other characteristics of the target object through specific algorithms and models. This helps to understand the environmental information more comprehensively, complements the data of front-end image recognition, and improves the system's ability to perceive and understand the scene.

[0082] Real-time data interaction: Ensure that data can be transmitted and shared between various parts of the system in a timely and accurate manner. In the edge computing AI platform, different modules, such as GPU / FPGA computing modules, LiDAR recognition modules, etc., as well as other external devices or systems, need to exchange data in real time in order to work together. This real-time data interaction can ensure the response speed and overall performance of the system, allowing the system to make timely decisions and adjustments based on the latest data.

[0083] (3) The instant report includes: Patrol inspection report settings: Users can customize the format, content, and generation rules of the patrol inspection report according to actual needs. For example, set what information should be included in the report, such as device status, recognition results, abnormal situations, etc., the typesetting style of the report, the time interval for report generation, etc. Through flexible settings, meet the personalized needs of different users in different scenarios for the patrol inspection report.

[0084] Feedback during the patrol inspection process: During the patrol inspection process, the system collects and feedbacks various information in real time, including the operating status of the device, intermediate results of image recognition, problems encountered, etc. These feedback information helps the operators to understand the progress of the patrol inspection in a timely manner, discover potential problems or abnormalities, so as to take measures for adjustment and processing in a timely manner to ensure the smooth progress of the patrol inspection work.

[0085] Generation of the patrol inspection report: According to the data and information collected during the patrol inspection process, as well as the pre-set report rules, a detailed patrol inspection report is automatically generated. The report content usually includes the inspection situation of the device, summary of image recognition results, discovered abnormal situations and handling suggestions, etc. The generated report is presented in a clear and standardized format, providing a comprehensive and accurate summary of the patrol inspection for the management personnel, facilitating their further analysis and decision-making.

[0086] Embodiment 2

[0087] The present invention further provides an embodiment, which is a method for front-end image recognition of an unmanned aerial vehicle based on lightweight edge computing, and is implemented by using the front-end image recognition device of the unmanned aerial vehicle based on lightweight edge computing described in Embodiment 1, as Figure 5 shown, Figure 5 is the flowchart of the method of the present invention.

[0088] A method for front-end image recognition of an unmanned aerial vehicle based on lightweight edge computing is that the unmanned aerial vehicle transmits the captured picture through a real-time video stream, and after a series of processing and recognition steps, finally realizes the recognition of defects in the picture and the acquisition and storage of relevant information.

[0089] Specifically, it includes the following steps: Step 1. Image recognition method: Image preprocessing: Perform preprocessing operations such as enhancement and denoising on the collected images to improve the image quality.

[0090] Image recognition: Use a lightweight deep learning model, that is, the optimized Tiny-YOLO algorithm, to perform target detection and classification on the preprocessed image, recognize the defects of power equipment, and generate early warning information.

[0091] The warning information is generated in the defect determination link of the image recognition stage and is specifically triggered after object detection and classification are completed. The system performs three-level dynamic grading on the recognition results according to the "Power Equipment Defect Classification Standard": when the defect confidence reaches 85%, a yellow warning is triggered, which is a general defect; when the confidence is 90% and it belongs to type B defects, such as insulator cracks, an orange warning is triggered; when the confidence is 95% and it belongs to type A critical defects, such as wire breaks, a red warning is triggered. The warning information includes the level, equipment code, geographical coordinates, timestamp, and defect type code, and is encapsulated in JSON format and transmitted to the operation platform for data display. The accuracy of the geographical coordinates is ±0.5m.

[0092] Transmission of the image recognition result: The image recognition result and the warning information are transmitted to the system backend server of the power grid through the communication module for further analysis and management by the management personnel.

[0093] Step 2. Model optimization: Lightweight model design: Adopt a lightweight deep learning model architecture (Tiny-YOLO). These models significantly reduce the computational complexity and resource consumption while maintaining a high recognition accuracy.

[0094] Knowledge distillation model training: Through model distillation, use the prediction results of the complex model as a guide to train a lightweight model. The lightweight model not only refers to the real label data during training but also refers to the output of the complex model, thereby inheriting the key features and knowledge of the complex model and adapting to the resource limitations of edge computing devices.

[0095] Training of a dynamically pruned deep neural network model: Remove redundant neurons and connections through pruning techniques to further optimize the performance of the model. At the same time, use quantization techniques to convert the weights and activation functions of the model from floating-point numbers to low-precision representations, such as INT8, to reduce the storage requirements and computational amount of the model and improve the inference speed of the model.

[0096] Embodiment 3

[0097] The present invention further provides an embodiment, which is a method for front-end image recognition of an unmanned aerial vehicle based on lightweight edge computing, and is implemented by using the device for front-end image recognition of an unmanned aerial vehicle based on lightweight edge computing described in Embodiment 1, as Figure 5 shown Figure 5 is the flowchart of the method of the present invention.

[0098] In specific implementation, the present invention includes the following steps: Step 1. Device initialization; Step 1.1. Start the defect recognition device: (1) Turn on the defect recognition device to ensure that the device starts normally.

[0099] (2)Check the battery level of the device to ensure that the battery is fully charged, at least reaching 30% or more.

[0100] (3)Check the network connection of the device to ensure that the device can connect to the network via WiFi or Ethernet.

[0101] Step 1.2. Check the hardware modules: (1)Check whether the hardware modules such as the computing motherboard, memory, display screen, keyboard with touchpad, and battery are working properly.

[0102] (2)Check the protection function of the device shell to ensure that the device has functions such as waterproof, dustproof, and anti-collision.

[0103] Step 1.3. Load the operating system: (1)Start the Linux operating system to ensure that the system is running properly.

[0104] (2)Check whether the software and tools required for image recognition have been installed and configured in the system.

[0105] Step 2. Image acquisition; Step 2.1. Start the drone: (1)Start the drone to ensure that the drone's battery is fully charged and can complete the inspection task.

[0106] (2)Check whether the drone's camera is working properly to ensure that clear images can be captured.

[0107] Step 2.2. Connect the drone to the defect recognition device: (1)Connect the drone to the industrial control-level operation tablet via a USB interface or a wireless communication module such as WiFi or Bluetooth.

[0108] (2)Start the drone control software on the defect recognition device to ensure that the flight and image acquisition of the drone can be controlled.

[0109] Step 2.3. Acquire images: (1)Control the drone to fly and acquire images of the power equipment according to the preset inspection route.

[0110] (2)Ensure that the acquired images are clear and complete, covering all the equipment and areas that need to be inspected.

[0111] Step 3. Image preprocessing; Step 3.1. Image transmission: (1)Transmit the acquired images to the defect recognition device through the communication module.

[0112] (2)Ensure that no images are lost or damaged during the transmission process.

[0113] Step 3.2. Image Enhancement and Denoising: (1) Run the image preprocessing algorithm on the operation tablet to enhance and denoise the acquired image.

[0114] (2) Improve the clarity of the image by adjusting parameters such as contrast and brightness.

[0115] Step 3.3. Use a denoising algorithm to remove noise from the image and improve image quality.

[0116] Step 4. Image Recognition; Step 4.1. Load a lightweight model: (1) Load a pre-trained lightweight deep learning model, such as MobileNet, Tiny-YOLO, etc. on the operation tablet.

[0117] (2) Ensure that the model has been correctly loaded into the device's memory and can run properly.

[0118] Step 4.2. Object Detection and Classification: (1) Run the image recognition algorithm to perform object detection and classification on the preprocessed image.

[0119] (2) Identify defects in power equipment, such as damaged insulators, broken conductors, etc.

[0120] (3) Generate recognition results, including information such as the location, type, and confidence of the defect.

[0121] Step 5. Result Transmission and Storage; Step 5.1. Result Transmission: (1) Transmit the recognition results to the system backend server of the power grid through the communication module.

[0122] (2) Ensure that there is no data loss or error during the data transmission process.

[0123] Step 5.2. Result Storage: (1) Store the recognition results on the power grid backend server for subsequent analysis and management.

[0124] (2) Ensure that the stored data is complete and accurate and can support the generation of subsequent inspection reports and defect analysis.

[0125] Step 6. Inspection Report Generation; Step 6.1. Report Generation: (1) Generate an inspection report on the power grid backend server, including information such as inspection time, inspected equipment, and discovered defects.

[0126] (2) Ensure that the report content is clear and accurate, providing valuable references for management personnel.

[0127] Step 6.2. Report storage and backup: (1) Store the generated inspection report in the database of the server and perform a backup.

[0128] (2) Ensure the secure and reliable storage of the report, enabling it to be accessed and analyzed at any time.

[0129] Embodiment 4

[0130] The present invention further provides an embodiment, which is a drone front-end image recognition device based on lightweight edge computing, including: An image recognition module for image recognition, performing enhancement and denoising preprocessing on the collected images; using a lightweight deep learning model to perform object detection and classification on the preprocessed images, identifying defects in power equipment; transmitting the image recognition results and warning information to the system back-end server of the power grid through a communication module for further analysis and management; A model optimization module for model optimization; Including lightweight model design: adopting a lightweight deep learning model architecture for lightweight model design; Knowledge distillation model training: through model distillation, using the prediction results of a complex model as a guide to train a lightweight model; referring to real label data and the output of the complex model, inheriting the key features and knowledge of the complex model while adapting to the resource limitations of edge computing devices; Dynamic pruning of deep neural network model training: removing redundant neurons and connections through pruning techniques to further optimize the performance of the model; adopting a quantized model design to convert the weights and activation functions of the model from floating-point numbers to low-precision representations to achieve model optimization.

[0131] The drone front-end image recognition device based on lightweight edge computing is used to implement the drone front-end image recognition method based on lightweight edge computing described in Embodiment 2 or Embodiment 4.

[0132] Embodiment 5

[0133] Based on the same inventive concept, an embodiment of the present invention further provides a computer device, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor. When the processor executes the computer program, it implements the steps of the drone front-end image recognition method based on lightweight edge computing described in any one of Embodiments 2 or 3.

[0134] Embodiment 6

[0135] Based on the same inventive concept, an embodiment of the present invention further provides a computer storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method for front-end image recognition of an unmanned aerial vehicle based on lightweight edge computing according to any one of Embodiment 2 or 3 are implemented.

[0136] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0137] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be realized by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate means for realizing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0138] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including instruction means, and the instruction means realizes the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0139] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for realizing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0140] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that: modifications or equivalent substitutions can still be made to the specific implementation manners of the present invention, and any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the protection scope of the claims of the present invention.

Claims

1. The front-end image recognition device for drones based on lightweight edge computing is characterized in that: in Inside the device housing (6), there are shock-absorbing rubber pads and a heat dissipation system, and a computing mainboard (1), a memory (2), a battery (5) are installed. The keyboard (4) and the display screen (3) are embedded on the device housing (6). On one side of the device housing (6), there are a USB interface (7), an HDMI interface (8) and an Ethernet interface (9).

2. The front-end image recognition device for unmanned aerial vehicles based on lightweight edge computing according to claim 1, wherein: The computing mainboard (1): is used for processing and analyzing image data; The memory (2): is used for storing image processing algorithms, machine learning models and recognition results; The display screen (3): is used for real-time displaying image processing and recognition results; The keyboard (4): is a keyboard with a touchpad, and is used for operating and controlling the device; The battery (5): is a 30000mAh battery, and is charged and discharged through the USB interface.

3. The front-end image recognition device for unmanned aerial vehicles based on lightweight edge computing according to claim 1, characterized in that: It includes an edge computing module, a communication module and a power module; The edge computing module is equipped with a high-performance computing mainboard, and is equipped with a CPU chip with more than four cores and a GPU chip with 128 CUDA cores, supporting multi-threaded concurrent processing, providing powerful computing capabilities for image recognition; the built-in memory is used for storing image processing algorithms, machine learning models and recognition results, ensuring fast reading, writing and storage of data; The communication module has high-speed wireless communication interfaces and wired communication interfaces, and is used for transmitting the recognition results to the system backend server of the power grid in real time; The power module is equipped with a high-capacity battery, and is charged and discharged through the USB interface.

4. A method for front-end image recognition of unmanned aerial vehicles based on lightweight edge computing, characterized in that: It includes the following steps: Step 1. The image recognition method includes: Image preprocessing: performing enhancement and denoising preprocessing on the collected images; Image recognition: using a lightweight deep learning model to perform target detection and classification on the preprocessed images, and identifying defects of power equipment; Transmission of the image recognition result: transmitting the image recognition result and the warning information to the system backend server of the power grid through the communication module for further analysis and management; Step 2. Model optimization includes: Lightweight model design: adopting a lightweight deep learning model architecture for lightweight model design; Knowledge distillation model training: through model distillation, using the prediction results of the complex model as a guide to train a lightweight model; referring to the real label data and the output of the complex model, inheriting the key features and knowledge of the complex model, and at the same time adapting to the resource limitations of the edge computing device; Training of the dynamically pruned deep neural network model: removing redundant neurons and connections through pruning technology to further optimize the performance of the model; adopting quantization model design to convert the weights and activation functions of the model from floating-point numbers to low-precision representations to achieve model optimization.

5. The method for front-end image recognition of an unmanned aerial vehicle based on lightweight edge computing according to claim 4, characterized in that: It includes the following steps: Step 1. Device initialization; Starting the defect recognition device: turning on the defect recognition device, ensuring that the device starts normally, checking the battery power of the device, and checking the network connection of the device; Checking the hardware modules: including: checking whether the computing mainboard, memory, display screen, keyboard with a touchpad, and battery hardware modules are working properly; checking the protection function of the device housing; Loading the operating system: including starting the Linux operating system; Step 2. Image acquisition; Starting the drone: ensuring that the battery of the drone is fully charged; checking whether the camera of the drone is working properly; Connect the drone to the defect recognition device: Connect the drone to the industrial control-level operation tablet through a USB interface or a wireless communication module; Start the drone control software on the defect recognition device; Collect images: Control the drone to fly and collect images of power equipment according to the preset inspection route; Ensure that the collected images are clear and complete, covering all equipment and areas that need to be inspected; Step 3. Image preprocessing; Image transmission: Transmit the collected images to the defect recognition device through the communication module; Image enhancement and denoising: Run the image preprocessing algorithm on the operation tablet to enhance and denoise the collected images; Use the denoising algorithm to remove the noise in the images and improve the image quality; Step 4. Image recognition; Load the lightweight model: Load the pre-trained lightweight deep learning model on the operation tablet; Object detection and classification: Run the image recognition algorithm to perform object detection and classification on the preprocessed images; Identify the defects of power equipment; Generate recognition results, including the location, type, and confidence information of the defects; Step 5. Result transmission and storage; Result transmission: Transmit the recognition results to the system backend server of the power grid through the communication module; Result storage: Store the recognition results on the power grid backend server for subsequent analysis and management; Step 6. Inspection report generation; Report generation: Generate an inspection report on the power grid backend server; Report storage and backup: Store the generated inspection report in the database of the server and perform a backup.

6. The method for front-end image recognition of an unmanned aerial vehicle based on lightweight edge computing according to claim 5, characterized in that: The lightweight deep learning model includes: MobileNet, Tiny-YOLO.

7. An unmanned aerial vehicle front-end image recognition device based on lightweight edge computing, characterized in that: Including: An image recognition module for image recognition, performing enhancement and denoising preprocessing on the collected images; Using a lightweight deep learning model to perform object detection and classification on the preprocessed images, identifying the defects of power equipment; Transmitting the image recognition results and warning information to the system backend server of the power grid through the communication module for further analysis and management; A model optimization module for model optimization; Including lightweight model design: Adopt a lightweight deep learning model architecture for lightweight model design; Knowledge distillation model training: Through model distillation, use the prediction results of the complex model as a guide to train a lightweight model; Refer to the real label data and the output of the complex model, inherit the key features and knowledge of the complex model, and at the same time adapt to the resource limitations of edge computing devices; Dynamic pruning of the deep neural network model training: Remove redundant neurons and connections through pruning techniques to further optimize the performance of the model; Adopt a quantized model design to convert the weights and activation functions of the model from floating-point numbers to low-precision representations to achieve model optimization.

8. The front-end image recognition device for unmanned aerial vehicles based on lightweight edge computing according to claim 7, characterized in that: The image recognition module includes: Collect images: Control the drone to fly and collect images of power equipment according to the preset inspection route; Ensure that the collected images are clear and complete, covering all equipment and areas that need to be inspected; Image transmission: Transmit the collected images to the defect recognition device through the communication module; Image enhancement and denoising: Run image preprocessing algorithms on the operation tablet to enhance and denoise the collected images; Use denoising algorithms to remove noise in the images and improve image quality; Image recognition; Loading a lightweight model: Load a pre-trained lightweight deep learning model on the operation tablet.

9. A computer device, comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method for front-end image recognition of an unmanned aerial vehicle based on lightweight edge computing according to any one of claims 4-6.

10. A computer storage medium, characterized in that: The computer storage medium stores a computer program, and when the computer program is executed by a processor, it implements the steps of the method for front-end image recognition of an unmanned aerial vehicle based on lightweight edge computing according to any one of claims 4-6.

Images