Inter-plant safety-oriented multi-source target inspection method

Abstract

The invention provides a multi-source target inspection method for inter-plant safety. According to the method, an unmanned aerial vehicle carries high-definition camera shooting to carry out target identification, and the target is transmitted to a ground station cloud platform to carry out component defect diagnosis, so that the problems of high labor intensity, low efficiency and easiness in being influenced by the environment in a traditional inspection mode are solved, and the problems of error detection and missing detection caused by inspection personnel through pictures on a remote controller are solved; and unmanned aerial vehicle inspection can enter a high-risk area, so that personnel are prevented from directly contacting a dangerous environment, and the working safety is remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) inspection technology, and in particular to a multi-source target inspection method for inter-plant security. Background Technology

[0002] With the rapid development of the power industry, the safety of personnel working in power plants is of paramount importance. Traditional inspection methods suffer from drawbacks such as high labor intensity, low efficiency, and susceptibility to environmental influences, making it difficult to meet the efficient and safe operation and maintenance needs of modern power plants. The introduction of drone technology provides a new approach to power plant inspection. It can quickly cover the power plant perimeter and equipment, and by carrying high-definition cameras, promptly detect safety hazards and violations, reducing labor costs and improving inspection efficiency. Simultaneously, drone inspections can access high-risk areas, avoiding direct contact between personnel and hazardous environments, significantly improving work safety.

[0003] However, drone inspections also face some challenges. Traditionally, drone-captured videos are transmitted back to the remote controller, where inspectors judge violations based on the images displayed. Due to the high speed of drones, it's difficult for inspectors to detect violations promptly, often resulting in missed inspections. Furthermore, the diverse types and states of violations also complicate inspections, leading to false positives. All of these factors pose potential safety hazards to power plant operations, necessitating improvements to inspection methods and solutions to these problems. Summary of the Invention

[0004] This invention proposes a multi-source target inspection method for inter-plant safety. The method targets routine perimeter inspections at plant boundaries, utilizing drones equipped with cameras to efficiently identify safety violations. The method uses high-definition cameras on drones for target identification, transmitting the data to a ground-based cloud platform for component defect diagnosis. This solves the problems of high labor intensity, low efficiency, and susceptibility to environmental influences in traditional inspection methods, as well as the problem of misidentification and missed detections due to images viewed by inspection personnel via remote controls. Furthermore, drone inspections can access high-risk areas, avoiding direct contact with hazardous environments for personnel, significantly improving work safety.

[0005] This invention is achieved through the following technical solution: This invention proposes a multi-source target inspection method for inter-plant safety, establishing a functional architecture for an unmanned inter-plant machine inspection intelligent platform, which includes a cloud platform and an edge layer. Data is collected by intelligent data acquisition equipment at the edge layer, the raw data is aggregated and processed, packaged, and transmitted to the cloud platform for storage. The cloud platform then processes the data using its algorithms, and finally, the processing results are distributed to various intelligent inspection equipment to assist in guiding the operation and maintenance of photovoltaic power plants. The specific process of data processing by the cloud platform includes: Step 1: Use the data acquisition intelligent equipment carried by the drone to acquire images and perform data annotation to form a dataset for subsequent model training. Step 2: Build an object detection model and train it using the dataset; Step 3: Use the trained target detection model to perform fault classification and detection on the actual image.

[0006] Furthermore, trajectory planning is performed on the drone to enable autonomous flight. Specifically, first, waypoints are added. Using the touchscreen on the remote controller, the locations the drone needs to fly over are selected on the map. These points are the waypoints. Relevant parameters are set for each waypoint. After all waypoints are added, their order is adjusted according to actual needs. Then, the flight path mode is set. The loop mode is selected to make the drone continuously fly in a loop between the set waypoints. Finally, before starting the flight path, the preview function is used to view the drone's flight path and attitude changes.

[0007] Furthermore, the target detection model employs data layer fusion, feature layer fusion, and decision layer fusion to fuse the results. In data layer fusion, pixel data from infrared and visible light images are directly concatenated as input to the model. In feature layer fusion, features from infrared and visible light images are extracted separately, and then these features are fused and input into the detection model. Decision layer fusion involves having the two modalities perform detection separately, and then making a fusion decision based on their respective detection results.

[0008] Furthermore, the fault classification detection includes safety helmet detection, smoking detection, personnel detection, vehicle detection, photovoltaic panel damage detection, and fire detection.

[0009] Furthermore, during personnel detection, the drone flies along a planned route, collects images in real time, and transmits them to the cloud platform. The cloud platform uses a trained model to process the images, quickly identify personnel, and determine whether their behavior is compliant. Once an abnormality is detected, an alarm is immediately issued, and the specific location of the personnel is displayed, facilitating timely handling by staff.

[0010] Furthermore, the drone is equipped with a voice broadcasting device to issue information or instructions to specific objects or areas through voice prompts.

[0011] Furthermore, the cloud platform visualizes big data, allowing maintenance personnel to monitor the real-time status of each piece of intelligent inspection equipment during photovoltaic power station inspections by using the cloud platform's visual interface.

[0012] Furthermore, the target detection model is the YOLO model.

[0013] The present invention also proposes an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the multi-source target inspection method for inter-plant security.

[0014] The present invention also proposes a computer-readable storage medium for storing computer instructions, which, when executed by a processor, implement the steps of the multi-source target inspection method for inter-plant security.

[0015] Compared with existing technologies, the beneficial effects of this invention are as follows: the method of this invention reduces labor costs, improves inspection efficiency, reduces safety risks, improves fault diagnosis capabilities, extends equipment life, reduces environmental impact, promotes technological innovation, saves resources, and promotes sustainable development. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0017] Figure 1 This is a schematic diagram of the intelligent inspection process using drones.

[0018] Figure 2 This is a flowchart of a multi-source target inspection method for inter-plant safety as described in this invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] This invention proposes a multi-source target inspection method for inter-plant safety. It utilizes unmanned aerial vehicles (UAVs) to conduct inspections of plant boundaries along efficient inspection paths. The UAVs are equipped with infrared cameras, visible light cameras, and high-definition cameras to inspect photovoltaic modules in power plants, as well as detect and alarm on violations and safety hazards within the plant. They can also carry fire-fighting equipment and broadcasting equipment to perform various tasks. A ground station cloud platform uses the data collected by the UAVs to diagnose component defects and schedule UAV missions, displaying offline flight path planning and monitoring flight status.

[0021] Specifically, in combination Figures 1-2 This invention proposes a multi-source target inspection method for inter-plant safety, establishing a functional architecture for an unmanned inter-plant inspection intelligent platform, which includes a cloud platform and an edge layer. Data is collected by intelligent data acquisition equipment at the edge layer, the raw data is aggregated and processed, packaged, and transmitted to the cloud platform for storage. The cloud platform then processes the data using its algorithms, and finally, the processing results are distributed to various intelligent inspection equipment to assist in guiding the operation and maintenance of photovoltaic power plants. The specific process of data processing by the cloud platform includes: Step 1: Use the data acquisition intelligent equipment carried by the drone to acquire images and perform data annotation to form a dataset for subsequent model training. Step 2: Construct an object detection model and train it using a dataset; the object detection model is the YOLO model.

[0022] Step 3: Use the trained target detection model to perform fault classification and detection on the actual image.

[0023] To enable autonomous flight of a drone, the following steps are performed: First, add waypoints. Using the touchscreen on the remote controller, select the locations the drone will fly over on the map. These points are the waypoints. Set the relevant parameters for each waypoint. After adding all waypoints, adjust their order as needed, and carefully check that the parameters for each waypoint are correct to ensure the drone flies along the desired route and under the desired flight conditions. Next, set the flight path mode. Select the loop mode to allow the drone to continuously cycle between the set waypoints. Finally, before starting the flight path, use the preview function to check the drone's flight path and attitude changes. After confirming the flight path is correct, move the drone to the starting position to prepare for the flight path mission.

[0024] The target detection model employs data layer fusion, feature layer fusion, and decision layer fusion for result fusion. It fuses infrared and visible light images, utilizing the rich texture and color information provided by visible light images and the characteristics of infrared images—sensitivity to temperature differences and ability to highlight target contours—to fuse the two. In data layer fusion, pixel data from infrared and visible light images are directly concatenated as input to the model. In feature layer fusion, features are extracted from both infrared and visible light images, then fused and input into the detection model. Decision layer fusion allows the two modalities to perform detection separately, and then makes a fusion decision based on their respective detection results. This fusion method overcomes the limitations of single-modality images, improving the accuracy and robustness of UAV detection under different lighting conditions and environments.

[0025] To significantly improve the performance and efficiency of UAV detection, the target detection model described in this invention has undergone key algorithm optimizations based on the YOLOv8n model. Core improvements include: replacing the original C2f module with the FasterViT module in the backbone network, effectively enhancing the model's feature representation ability and multi-scale feature map balancing ability, thereby significantly improving the capture effect of long-range dependencies; introducing a Bidirectional Feature Pyramid Network (BiFPN) to replace the original Path Aggregation Network (PANet), greatly reducing the redundancy and computational complexity of the network structure; proposing a box similarity comparison metric RIOU, and using a bounding box regression loss function based on RIOU for optimization, effectively improving the accuracy and robustness of bounding box localization. These improvements work synergistically to drive a comprehensive improvement in the accuracy, efficiency, and generalization ability of the UAV target detection model.

[0026] The fault classification detection includes safety helmet detection, smoking detection, personnel detection, vehicle detection, photovoltaic panel damage detection, and fire detection.

[0027] Helmet Detection: Drones utilize computer vision and image processing technology to detect helmets at construction sites. The drone's onboard camera acquires images of the site and then identifies whether individuals are wearing helmets. A target detection model processes and analyzes the acquired images. The images are then transmitted to a cloud platform for further processing. The drone, equipped with a high-definition camera, uses image acquisition technology to capture images of personnel at the construction site from multiple aerial angles. A deep learning-based YOLO target detection model processes and analyzes the acquired images. The model is trained using a large amount of labeled images of both helmet-wearing and helmet-less individuals, enabling it to accurately identify people in images and determine whether they are wearing helmets. If a worker is found not wearing a helmet, the drone can issue a real-time warning via a voice broadcast system, reminding them to put on a helmet immediately.

[0028] Smoking Detection: Drones equipped with high-resolution cameras fly along a preset route above the construction site, collecting real-time ground images and video data to ensure clear capture of personnel's movements. First, a large number of images and videos containing both smoking and non-smoking scenes are collected, and key features of smoking behavior are labeled, such as hand posture while holding a cigarette and smoke generation. This information is then used to train a YOLO object detection model, allowing the model to learn and recognize smoking behavior patterns. During detection, the model quickly analyzes the images transmitted back by the drone, and once suspected smoking behavior is detected, it is immediately marked and an alert is triggered.

[0029] Personnel Detection: During personnel detection, image data of power plant personnel under different time periods, weather conditions, and work scenarios are collected and labeled to construct a dataset. Information such as the location, behavior, and clothing of personnel in the images is labeled, including whether they are wearing work clothes and whether they are near dangerous areas, providing accurate data for model training. The YOLO deep learning-based object detection model is trained on the labeled dataset. The model learns the feature patterns of personnel in the images, and its performance is optimized by continuously adjusting parameters to improve detection accuracy and recall. Drones fly along planned routes, collecting images in real time and transmitting them to a cloud platform. The cloud platform uses the trained model to process the images, quickly identify personnel, and determine whether their behavior is compliant. If any abnormality is detected, an alarm is immediately issued, and the specific location of the personnel is displayed, facilitating timely handling by staff.

[0030] Vehicle Detection: Through data collection and annotation, images and point cloud data of various vehicles driving and parked within the power plant are collected under different time periods and weather conditions. The data is meticulously annotated to clearly identify vehicle type, location, and direction of travel, constructing a high-quality training dataset. The YOLO target detection model is used to train on the annotated data. The model learns the appearance, size, and structural features of different vehicles to master vehicle detection and recognition capabilities, and continuously optimizes parameters during training to improve detection accuracy and efficiency. Drones fly along preset routes, with high-definition cameras collecting data in real time and transmitting it to a cloud platform. The cloud platform uses the trained model to process the data, quickly identifying vehicle targets and determining their type, location, and driving status. If any abnormalities are detected, such as illegal parking, speeding, or entering restricted areas, the system immediately issues an alarm and records the relevant information.

[0031] Photovoltaic panel damage detection: Drones equipped with high-resolution optical cameras can clearly capture surface details of photovoltaic panels. For internal defects that are difficult to identify with visible light, an infrared thermal imager is also used to detect potential problems by detecting abnormal surface temperatures. Following a pre-planned flight path, the drone flies over the photovoltaic power station at an appropriate altitude and speed, capturing multi-angle, all-around images of the photovoltaic panels to obtain clear image data. The acquired images are then labeled, marking various defects in the photovoltaic panels, such as cracks, hot spots, microcracks, broken grids, and fragments. Detailed information such as the location, type, and severity of the defects is recorded, constructing a high-quality labeled dataset to provide accurate samples for model training. Using the labeled dataset, the model is trained using the YOLO deep learning algorithm. This enables the model to accurately identify and distinguish different types of photovoltaic panel defects. Newly acquired images are input into the trained model, which automatically analyzes the images, quickly identifies defects in the photovoltaic panels, and outputs information such as the location, type, and severity of the defects.

[0032] Fire Detection: Drones equipped with infrared cameras detect potential high-temperature fire sources by measuring the surface temperature of objects. The drones fly along a preset route over the power plant, with the infrared cameras acquiring images in real time to ensure timely capture of dynamic changes in the fire situation. A large amount of images and data on both fire and normal conditions at the power plant are collected to build a dataset. A model is trained using the YOLO deep learning algorithm to learn the characteristic patterns of fire points, smoke color, shape, and temperature distribution. The collected data is transmitted to a ground-based cloud platform, where the trained model automatically analyzes it. Once a suspected fire is detected, the system immediately verifies it using multiple datasets. Upon confirmation of a fire, an audible and visual alarm is quickly issued, the fire location is marked on an electronic map, and detailed information is pushed to management terminals for rapid response.

[0033] The drone is equipped with a voice broadcasting device, which delivers information or instructions to specific objects or areas via voice prompts, improving communication efficiency and accuracy. In the field of drone-based photovoltaic module inspection, the voice broadcasting device is integrated into the drone, taking into account its weight and size to ensure it does not affect the drone's flight performance. Drone voice broadcasting software is developed to enable connection and control between the drone and the voice broadcasting device.

[0034] The cloud platform visualizes big data, allowing maintenance personnel to monitor the real-time status of various intelligent inspection equipment during photovoltaic power plant inspections through its visual interface. This includes tracking drone flight paths, the on-site status of photovoltaic modules, and the battery level of the drone. By enabling platform openness, data sharing, and resource integration, a photovoltaic power plant maintenance resource ecosystem is built, standardizing the accuracy, safety, and completeness of maintenance personnel's inspection operations, thereby achieving low-cost and high-efficiency manual maintenance.

[0035] The present invention also proposes an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the multi-source target inspection method for inter-plant security.

[0036] The present invention also proposes a computer-readable storage medium for storing computer instructions, which, when executed by a processor, implement the steps of the multi-source target inspection method for inter-plant security.

[0037] The memory in this application embodiment can be volatile memory or non-volatile memory, or it can include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM). It should be noted that the memory used in the methods described in this invention is intended to include, but is not limited to, these and any other suitable types of memory.

[0038] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., high-density digital video discs (DVDs)), or semiconductor media (e.g., solid-state disks (SSDs)).

[0039] In implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software. The steps of the method disclosed in the embodiments of this application can be directly implemented by a hardware processor, or by a combination of hardware and software modules in the processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, detailed descriptions are omitted here.

[0040] It should be noted that the processor in the embodiments of this application can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method embodiments can be completed by the integrated logic circuitry in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied as execution by a hardware decoding processor, or as a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above methods.

[0041] The above provides a detailed description of the multi-source target inspection method for inter-plant safety proposed in this invention. Specific examples have been used to illustrate the principles and implementation methods of this invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this invention. Therefore, the content of this specification should not be construed as a limitation of this invention.

Claims

1. A multi-source target inspection method for inter-plant safety, characterized in that, Establish a functional architecture for an intelligent inspection platform for unmanned inter-factory machines, which includes a cloud platform and an edge layer. Data is collected by intelligent equipment in the edge layer, the raw data is aggregated and processed, packaged and transmitted to the cloud platform for storage, the data is then processed by the cloud platform's algorithm, and finally the processing results are sent to each intelligent inspection equipment to assist and guide the operation and maintenance of photovoltaic power plants. The specific process by which the cloud platform processes data includes: Step 1: Use the data acquisition intelligent equipment carried by the drone to acquire images and perform data annotation to form a dataset for subsequent model training. Step 2: Build an object detection model and train it using the dataset; Step 3: Use the trained target detection model to perform fault classification and detection on the actual image.

2. The method according to claim 1, characterized in that, To enable autonomous flight of a drone, the following steps are performed: First, add waypoints. Using the touchscreen on the remote controller, select the locations the drone needs to fly over on the map. These points are the waypoints. Set the relevant parameters for each waypoint. After all waypoints are added, adjust their order as needed. Then, set the flight path mode. Select the loop mode to make the drone continuously fly in a loop between the set waypoints. Finally, before starting the flight path, use the preview function to check the drone's flight path and attitude changes.

3. The method according to claim 1, characterized in that, The target detection model employs data layer fusion, feature layer fusion, and decision layer fusion to fuse the results. In the data layer fusion, pixel data from infrared and visible light images are directly concatenated as input to the model. In feature layer fusion, features are extracted from infrared and visible light images respectively, and then these features are fused and input into the detection model; decision layer fusion allows the two modal models to perform detection separately, and then makes a fusion decision based on their respective detection results.

4. The method according to claim 1, characterized in that, The fault classification detection includes safety helmet detection, smoking detection, personnel detection, vehicle detection, photovoltaic panel damage detection, and fire detection.

5. The method according to claim 4, characterized in that, During personnel detection, the drone flies along a planned route, collects images in real time, and transmits them to the cloud platform. The cloud platform uses a trained model to process the images, quickly identify personnel, and determine whether their behavior is compliant. Once an abnormality is detected, an alarm is immediately issued, and the specific location of the personnel is displayed, so that staff can handle the situation in a timely manner.

6. The method according to claim 1, characterized in that, The drone is equipped with a voice broadcasting device, which can issue information or instructions to specific objects or areas through voice prompts.

7. The method according to claim 1, characterized in that, The cloud platform visualizes big data, allowing maintenance personnel to monitor the real-time status of each piece of intelligent inspection equipment during photovoltaic power station inspections using the platform's visual interface.

8. The method according to claim 1, characterized in that, The target detection model is the YOLO model.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1-8.

10. A computer-readable storage medium for storing computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the steps of the method according to any one of claims 1-8.

Images