Airtight space unmanned aerial vehicle intelligent inspection method and device based on AI visual identification

Abstract

The invention provides a closed space unmanned aerial vehicle intelligent inspection method and device based on AI visual identification, and relates to the technical field of power grids. The unmanned aerial vehicle is used for replacing manual inspection of the closed space, and inspection safety is improved. Then unmanned aerial vehicle inspection data is identified through an AI visual identification model, surface temperature distribution characteristics, three-dimensional point cloud characteristics and equipment parameter characteristics of the closed space are obtained, equipment division is carried out, and a fault detection model is combined from the three aspects of the three-dimensional structure, the surface temperature and the equipment parameters of single equipment to obtain a fault detection result; and the operation state of each device is efficiently and comprehensively evaluated. And finally, the operation state and the inspection characteristics of each device are integrated to generate an intelligent inspection report, and the report is visually displayed to inspection personnel, so that efficient, comprehensive and safe intelligent inspection in the closed space is realized, and the inspection efficiency and safety of the closed space are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of power grids, and particularly to an intelligent inspection method and device for drones in confined spaces based on AI visual recognition. Background Art

[0002] In many key fields such as industrial production and infrastructure maintenance, the inspection work of confined spaces (such as pipeline systems, large storage tanks, underground facilities, etc.) has always occupied a crucial position. However, with the continuous expansion of the scale of confined spaces, the inspection work has become increasingly huge and intricate, facing unprecedented challenges.

[0003] When dealing with these complex inspection tasks, the traditional manual inspection method seems inadequate. The inspection process is time-consuming and inefficient, making it difficult to detect faults and potential problems in confined spaces in a timely manner. More troublesome is that due to the complex structure of confined spaces, some areas are often difficult to reach or observe, resulting in the inability of manual inspection to fully cover all key areas. This not only causes omissions and blind spots in the inspection results, but also makes it difficult to detect and properly handle potential problems in a timely manner.

[0004] The publication number is CN116161251A, and the name is a drone inspection method, which includes the following steps: installing a robotic arm assembly in the to-be-inspected area where the drone needs to monitor and stay; obtaining a hover flight instruction for the expected position of the drone in the to-be-inspected area where it needs to monitor and stay; obtaining the real-time position and attitude information of the drone; generating a control instruction according to the hover flight instruction, real-time position and attitude information; realizing the hover of the drone at the expected position through the control instruction and sending the position information of the drone after hovering to the robotic arm assembly; the robotic arm assembly obtains the hover position of the drone and performs corresponding stretching actions to dock with the drone so that the drone is suspended on the robotic arm assembly; the rotors of the drone stop rotating and stay for monitoring.

[0005] The publication number is CN112214032A, and the name is a drone inspection system and a drone inspection method. The drone receives an inspection instruction from the base station of the drone inspection system to perform an inspection task on the target area. The inspection task includes: flying at a first height according to a cruise path and obtaining a first thermal induction image of the target area with a first field of view; in response to determining that there is an abnormal point with a temperature higher than a temperature threshold and located on one of a plurality of target objects in the first thermal induction image, pausing to fly on the cruise path, changing to fly at a second height, capturing an abnormal image of the abnormal point with a second field of view, and storing and marking the abnormal image, where the second field of view is smaller than the first field of view and the second height is lower than the first height.

[0006] In addition, there are still huge potential safety hazards during manual inspection. The confined space may be filled with dangerous factors such as toxic gases, high temperature and high pressure, which pose a serious threat to the lives of inspection personnel. Despite various safety measures, accidents still occur from time to time, bringing immeasurable losses to enterprises and individuals.

[0007] Therefore, the traditional manual inspection method has obvious deficiencies in terms of efficiency, comprehensiveness and safety. Summary of the Invention

[0008] The present invention provides an intelligent inspection method for drones in confined spaces based on AI visual recognition, which can achieve efficient, comprehensive and safe intelligent inspection in confined spaces, and improve the inspection efficiency and safety of confined spaces.

[0009] In a first aspect, the present invention provides an intelligent inspection method for drones in confined spaces based on AI visual recognition. The method includes: obtaining inspection data of the drone in the confined space, where the inspection data includes video data and infrared imaging data; based on the inspection data and the AI visual recognition model, obtaining an identification result, where the identification result includes surface temperature distribution characteristics, three-dimensional point cloud characteristics and equipment parameter characteristics; based on the identification result, performing equipment division to obtain the inspection characteristics of each equipment; based on the inspection characteristics of each equipment and the fault detection model, determining the operating state of each equipment; based on the operating state of each equipment and the inspection characteristics of each equipment, generating an intelligent inspection report to achieve intelligent inspection of the confined space.

[0010] In a possible implementation manner, obtaining the identification result based on the inspection data and the AI visual recognition model includes: performing frame division on the inspection data to obtain multiple frames of video images and multiple frames of infrared images; based on the multiple frames of video images and multiple frames of infrared images, and the feature extraction module of the AI visual model, performing feature extraction to obtain the visual features corresponding to each frame of video image and the temperature distribution features corresponding to each frame of infrared image; the visual features include target contour, target shape and texture features; based on the visual features corresponding to each frame of video image and the target recognition and tracking module of the AI visual model, determining the three-dimensional point cloud characteristics of multiple targets in the confined space; based on the three-dimensional point cloud characteristics and the temperature distribution characteristics corresponding to each frame of infrared image, performing target fusion to obtain the surface temperature distribution characteristics; based on the multiple frames of video images and the image recognition module of the AI visual model, extracting the meter measurement data in the video images to obtain the equipment parameter characteristics of each equipment; based on the surface temperature characteristics, three-dimensional point cloud characteristics and equipment parameter characteristics, the equipment type and equipment location of each equipment, and the feature fusion module of the AI visual model, performing feature fusion to obtain the identification result.

[0011] In a possible implementation, before obtaining the recognition result based on the patrol inspection data and the AI visual recognition model, it further includes: acquiring multiple video images of known meter measurement data and the corresponding meter measurement data for each video image; using each video image as an input and the corresponding meter measurement data for each video image as an output to establish multiple first training samples; performing machine learning based on the multiple first training samples to obtain an image recognition module.

[0012] In a possible implementation, before obtaining the recognition result based on the patrol inspection data and the AI visual recognition model, it further includes: acquiring the patrol inspection data within a historical period and the three-dimensional point cloud features of multiple targets at each moment within the historical period; performing frame division processing on the patrol inspection data within the historical period to obtain multiple frames of video images and the corresponding visual features for each frame of video image; determining the target and three-dimensional point cloud features corresponding to each frame of video image based on the three-dimensional point cloud features of multiple targets at each moment within the historical period and the correspondence between each frame of video image and the target; using the visual features corresponding to each frame of video image as an input and the target and three-dimensional point cloud features corresponding to each frame of video image as an output to construct multiple second training samples; performing neural network training based on the multiple second training samples to obtain a target recognition and tracking module.

[0013] In a possible implementation, before obtaining the recognition result based on the patrol inspection data and the AI visual recognition model, it further includes: acquiring the patrol inspection data of a confined space within a historical period, the visual features of multiple targets in the confined space, and the temperature distribution features at multiple moments within the historical period; performing frame division processing on the patrol inspection data of the confined space within the historical period to obtain multiple frames of video images and multiple frames of infrared images; performing matching based on the multiple frames of video images and the visual features of multiple targets in the confined space to obtain the visual features corresponding to each frame of video image; performing matching based on the multiple frames of infrared images and the temperature distribution features at multiple moments within the historical period to obtain the temperature distribution features corresponding to each frame of infrared image; constructing multiple third training samples based on the multiple frames of video images and the visual features corresponding to each frame of video image; constructing multiple fourth training samples based on the multiple frames of infrared images and the temperature distribution features corresponding to each frame of infrared image; performing neural network training based on the multiple third training samples to obtain the visual feature extraction sub-module of the feature extraction module; performing neural network training based on the multiple fourth training samples to obtain the infrared feature extraction sub-module of the feature extraction module; constructing the feature extraction module of the AI visual model based on the visual feature extraction sub-module, the infrared feature extraction sub-module, and a preset image category classifier.

[0014] In a possible implementation, based on the recognition results, device division is performed to obtain the inspection features of each device, including: matching based on the device type, device location, and recognition results to obtain the recognition results of each device; determining the related devices of each device based on the architecture information of the confined space; and performing feature fusion based on the recognition results of each device and the recognition results of the related devices of each device to obtain the inspection features of each device.

[0015] In a possible implementation, based on the inspection features of each device and the fault detection model, the operating status of each device is determined, including: inputting the inspection features of each device into the fault detection model to obtain the model output results of each device; the model output results include the fault probability, fault type, and fault cause; and determining the operating status of each device based on the model output results of each device, where the operating status includes the real-time status and fault information; the real-time status includes normal operation, fault operation, or fault shutdown.

[0016] In a possible implementation, before determining the operating status of each device based on the inspection features of each device and the fault detection model, it further includes: obtaining the inspection features of multiple devices with known operating statuses and the operating statuses of the multiple devices; constructing multiple fifth training samples with the inspection features of each device as the input and the operating status of each device as the output; and performing neural network training based on the multiple fifth training samples to obtain the fault detection model.

[0017] In a possible implementation, based on the operating status of each device and the inspection features of each device, an intelligent inspection report is generated to implement intelligent inspection of the confined space, including: determining the severity and impact range of the fault points based on the operating status of each device; determining the abnormal data of the fault points and the change trend of the abnormal data based on the inspection features of each device; sorting each fault point based on the severity, impact range, abnormal data, and change trend of the abnormal data of the fault points to obtain the inspection sequence; and generating an intelligent inspection report based on the template of the intelligent inspection report, the operating status of each device, the inspection features of each device, the severity, impact range, abnormal data, and change trend of the abnormal data of the fault points, and the inspection sequence.

[0018] In a possible implementation, the method further includes: receiving a re-inspection instruction input by the user, where the re-inspection instruction is used to instruct the human-machine to perform a secondary inspection on the target fault point; forwarding the re-inspection instruction to the unmanned aerial vehicle (UAV); receiving the secondary inspection data transmitted back by the UAV; generating the secondary inspection features of the target fault point based on the secondary inspection data; determining the operating status of the target fault point based on the secondary inspection features of the target fault point and the fault detection model; and generating a secondary inspection report based on the operating status of the target fault point and the inspection features of the target fault point.

[0019] In a second aspect, an embodiment of the present invention provides an intelligent inspection device for drones in a confined space based on AI visual recognition, including: a communication module for obtaining inspection data of drones in the confined space, where the inspection data includes video data and infrared imaging data; a processing module for obtaining an identification result based on the inspection data and an AI visual recognition model, where the identification result includes surface temperature distribution characteristics, three-dimensional point cloud characteristics, and equipment parameter characteristics; dividing the equipment based on the identification result to obtain the inspection characteristics of each equipment; determining the operating status of each equipment based on the inspection characteristics of each equipment and a fault detection model; and generating an intelligent inspection report based on the operating status of each equipment and the inspection characteristics of each equipment to achieve intelligent inspection of the confined space.

[0020] In a third aspect, an embodiment of the present invention provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor is configured to call and run the computer program stored in the memory to execute the method described in the first aspect and any possible implementation manner in the first aspect above.

[0021] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium storing a computer program, characterized in that when the computer program is executed by a processor, the steps of the method described in the first aspect and any possible implementation manner in the first aspect above are implemented.

[0022] The present invention provides an intelligent inspection method and device for drones in a confined space based on AI visual recognition. The present invention replaces manual inspection of the confined space with drones, improving the safety of inspection. Then, the AI visual recognition model is used to identify the inspection data of the drones to obtain the surface temperature distribution characteristics, three-dimensional point cloud characteristics, and equipment parameter characteristics of the confined space, and the equipment is divided. From three aspects of the three-dimensional structure, surface temperature, and equipment parameters of a single device, combined with the fault detection model, the operating status of each device is evaluated efficiently and comprehensively. Finally, the operating status and inspection characteristics of each device are integrated to generate an intelligent inspection report, which is intuitively displayed to the inspection personnel, realizing efficient, comprehensive, and safe intelligent inspection in the confined space, and improving the inspection efficiency and safety of the confined space. Description of the Drawings

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0024] Figure 1It is a schematic flowchart of an intelligent inspection method for drones in a confined space based on AI visual recognition provided by an embodiment of the present invention;

[0025] Figure 2 It is a schematic structural diagram of an intelligent inspection device for drones in a confined space based on AI visual recognition provided by an embodiment of the present invention. Detailed implementation manners

[0026] In the following description, specific details such as specific system structures and technologies are presented for the purpose of illustration rather than limitation, so as to thoroughly understand the embodiments of the present invention. However, those skilled in the art should clearly understand that the present invention can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from interfering with the description of the present invention.

[0027] In the description of the present invention, unless otherwise specified, " / " means "or". For example, A / B can represent A or B. The "and / or" herein is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, "at least one" and "a plurality of" mean two or more. The words such as "first" and "second" do not limit the quantity and execution order, and the words such as "first" and "second" do not necessarily limit to be different.

[0028] In the embodiments of the present application, words such as "exemplary" or "for example" are used to represent examples, illustrations, or explanations. Any embodiment or design solution described as "exemplary" or "for example" in the embodiments of the present application should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Exactly speaking, using words such as "exemplary" or "for example" aims to present relevant concepts in a specific way for easy understanding.

[0029] In addition, the terms "including" and "having" mentioned in the description of the present application and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device including a series of steps or modules is not limited to the listed steps or modules, but optionally further includes other steps or modules not listed, or optionally further includes other steps or modules inherent to these processes, methods, products, or devices.

[0030] To make the purpose, technical solutions, and advantages of the present invention clearer, the following will be described through specific embodiments in combination with the drawings of the present invention.

[0031] Such as Figure 1As shown in the figure, an embodiment of the present invention provides an intelligent inspection method for drones in a confined space based on AI visual recognition. This method includes steps S101 - S105.

[0032] S101. Obtain the inspection data of the drone in the confined space.

[0033] In an embodiment of the present application, the inspection data includes video data and infrared imaging data.

[0034] Exemplarily, the drone is equipped with a high - definition camera and enters the confined space for flight inspection. The camera captures the video images in the confined space in real - time to ensure that the images are clear and stable. The video data is transmitted back to the ground control station or the cloud server in real - time through a wireless transmission method.

[0035] Exemplarily, the drone is also equipped with an infrared thermal imager to perform infrared imaging detection on the equipment in the confined space. The infrared thermal imager captures the temperature distribution information on the surface of the equipment and generates infrared imaging data. The infrared imaging data is also transmitted back to the ground control station or the cloud server in real - time through a wireless transmission method.

[0036] S102. Based on the inspection data and the AI visual recognition model, obtain the recognition result.

[0037] In an embodiment of the present application, the recognition result includes surface temperature distribution characteristics, three - dimensional point cloud characteristics, and equipment parameter characteristics.

[0038] It should be noted that the embodiment of the present invention can perform pre - processing operations such as denoising and enhancement on the acquired video data and infrared imaging data to improve the data quality. Frame extraction is performed on the video data, and key frames are selected for subsequent analysis. The embodiment of the present invention can input the pre - processed video data and infrared imaging data into the AI visual recognition model. The model uses deep - learning algorithms to extract features and classify the input data. The recognition result includes surface temperature distribution characteristics (such as hot spots, temperature - abnormal areas, etc.), three - dimensional point cloud characteristics (such as equipment shape, size, position, etc.), and equipment parameter characteristics (such as equipment model, manufacturer, etc.).

[0039] As a possible implementation, step S102 can be specifically implemented as steps S1021 - S1026.

[0040] S1021. Perform frame - by - frame processing on the inspection data to obtain multiple frames of video images and multiple frames of infrared images.

[0041] Exemplarily, the embodiment of the present invention can use video - processing software or libraries (such as OpenCV) to extract frames from the acquired video data frame by frame. Set an appropriate frame rate to ensure that the extracted video frames can fully reflect the dynamic changes of the equipment. Save the extracted video frames as image files or process them directly in memory.

[0042] Exemplarily, for infrared imaging data, frame-by-frame extraction is also performed. Since infrared imaging data is usually a temperature distribution image based on a time series, attention should be paid to maintaining the consistency of timestamps during frame processing. Synchronize the extracted infrared frames with the corresponding video frames for subsequent analysis.

[0043] S1022. Based on multi-frame video images, multi-frame infrared images, and the feature extraction module of the AI vision model, perform feature extraction to obtain the visual features corresponding to each frame of the video image and the temperature distribution features corresponding to each frame of the infrared image.

[0044] In some embodiments, the visual features include target contours, target shapes, and texture features.

[0045] Exemplarily, in the embodiments of the present invention, the feature extraction module of the AI vision model can be used to process each frame of the video image. Identify the target boundaries in the image through an edge detection algorithm (such as Canny edge detection). Use a shape analysis algorithm (such as the Hough transform) to identify the geometric shape of the target. Use a texture analysis algorithm (such as local binary pattern LBP) to describe the texture information of the target surface.

[0046] Exemplarily, in the embodiments of the present invention, for each frame of the infrared image, the feature extraction module can be used to extract temperature information. Calculate the temperature value of each pixel point in the image to generate a temperature distribution matrix. Analyze the temperature distribution matrix to identify temperature anomaly regions (such as high-temperature or low-temperature regions).

[0047] S1023. Based on the visual features corresponding to each frame of the video image and the target recognition and tracking module of the AI vision model, determine the three-dimensional point cloud features of multiple targets in the enclosed space.

[0048] Exemplarily, in the embodiments of the present invention, the target recognition and tracking module of the AI vision model can be used to process multi-frame video images. Identify and track the same target in consecutive frames to obtain its position information from different perspectives. Through triangulation or stereo vision algorithms, combine multi-perspective information to calculate the three-dimensional coordinates of the target. Convert the three-dimensional coordinates of the identified target into point cloud data. Perform preprocessing operations such as denoising and filtering on the point cloud data to improve the data quality. Use point cloud processing algorithms (such as the PCL library) to analyze the shape, size, and other features of the point cloud.

[0049] S1024. Based on the three-dimensional point cloud features and the temperature distribution features corresponding to each frame of the infrared image, perform target fusion to obtain the surface temperature distribution features.

[0050] Exemplarily, embodiments of the present invention can fuse the extracted 3D point cloud features with the temperature distribution features corresponding to each frame of infrared image. According to the target position information in the 3D point cloud, find the corresponding temperature distribution area in the infrared image. Combine the temperature distribution information with the 3D point cloud features to generate the surface temperature distribution features of the target.

[0051] S1025. Based on multiple frames of video images and the image recognition module of the AI vision model, extract the meter measurement data in the video images to obtain the device parameter features of each device.

[0052] Exemplarily, embodiments of the present invention can use the image recognition module of the AI vision model to process multiple frames of video images. Identify the meters in the images (such as pressure gauges, thermometers, etc.) and read their readings. Calibrate and verify the read meter measurement data to ensure the accuracy of the data. Generate the parameter features of the device according to the extracted meter measurement data and other relevant information (such as device model, manufacturer, etc.). Save the parameter features to the database for subsequent analysis and processing.

[0053] S1026. Based on the surface temperature features, 3D point cloud features, and device parameter features, the device type and device location of each device, and the feature fusion module of the AI vision model, perform feature fusion to obtain the recognition result.

[0054] Exemplarily, embodiments of the present invention can use the feature fusion module of the AI vision model to fuse the extracted surface temperature features, 3D point cloud features, and device parameter features. According to information such as the type and location of the device, further analyze and process the fused features. Use machine learning algorithms (such as support vector machine SVM, random forest RF, etc.) to classify and identify the fused features. Generate the final recognition result according to the classification and recognition results. The recognition result includes the status of the device (such as normal, abnormal, faulty, etc.), location information, temperature distribution features, etc. Save the recognition result to the database or output it to a report for subsequent analysis and processing.

[0055] In this way, the present invention can more clearly understand the processing and feature extraction process of the inspection data, providing strong support for subsequent fault detection and equipment maintenance.

[0056] S103. Based on the recognition result, perform device division to obtain the inspection features of each device.

[0057] It should be noted that the present invention can divide the equipment in the enclosed space according to the equipment parameter characteristics and three-dimensional point cloud characteristics in the recognition result. Similar or identical equipment is classified into the same group, or related equipment is classified into a group, which is convenient for subsequent analysis and processing. For each equipment group, the corresponding inspection characteristics are extracted. The inspection characteristics include surface temperature distribution characteristics, three-dimensional point cloud characteristics, and possible abnormal information (such as cracks, corrosion, etc.).

[0058] As a possible implementation manner, step S103 can be specifically implemented as steps S1031 - S1033.

[0059] S1031: Based on the equipment type, equipment location, and recognition result, perform matching to obtain the recognition result of each equipment.

[0060] Exemplarily, the embodiments of the present invention can collect the basic information of the equipment, including the equipment type (such as motors, transformers, switches, etc.) and the equipment location (such as floor, room, specific position coordinates, etc.). At the same time, obtain the equipment recognition results obtained through image recognition, sensor detection, or other technical means, and these results may include detailed information such as the model, serial number, manufacturer, etc. of the equipment. According to the equipment type and location information, formulate a matching strategy. For example, for the same type of equipment, preliminary screening can be performed according to the proximity of the location information; for equipment with a unique identifier (such as a serial number), direct exact matching can be performed. Match the recognition result with the equipment information. For the initially screened equipment list, further verification can be performed through other characteristics of the equipment (such as appearance, size, etc.) to ensure the accuracy of the matching result. For equipment that cannot be directly matched, it can be marked as an unknown equipment and recorded for subsequent processing. Output the matching result of each equipment, including detailed information such as equipment name, model, serial number, etc., as well as the confidence level or probability of successful matching.

[0061] S1032: Based on the architecture information of the enclosed space, determine the related equipment of each equipment.

[0062] Exemplarily, the embodiments of the present invention can obtain the architecture information of the enclosed space (such as a machine room, a power distribution room, etc.), including the layout of the equipment, connection relationships, power supply lines, etc. This information can be obtained through drawings, CAD files, on-site inspections, etc. According to the architecture information of the enclosed space, define other equipment related to each equipment. For example, for a motor, its related equipment may include the switch, transformer that powers it, and the transmission device connected to it, etc. According to the definition, combined with the architecture information, determine the related equipment of each equipment. This may require analyzing the connection relationships, power supply lines, etc. of the equipment to ensure the accuracy and integrity of the related equipment.

[0063] S1033. Based on the recognition results of each device and the recognition results of the related devices of each device, perform feature fusion to obtain the inspection features of each device.

[0064] Exemplarily, for each device and its related devices, extract the key features in their recognition results. These features may include the device model, serial number, manufacturer, operating status (such as temperature, vibration, etc.), connection relationship, etc. Develop a feature fusion strategy to determine how to fuse the features of the device itself with the features of its related devices. For example, the operating status features of the device can be combined with the operating status features of its related devices to form more comprehensive inspection features. According to the fusion strategy, execute the feature fusion process. This may involve steps such as data preprocessing, feature selection and combination, data normalization or standardization. Output the inspection features of each device, which will be used for subsequent tasks such as fault detection, early warning, or status evaluation.

[0065] S104. Based on the inspection features of each device and the fault detection model, determine the operating status of each device.

[0066] It should be noted that in the embodiments of the present invention, the extracted inspection features can be input into the fault detection model. The model uses machine learning algorithms to analyze and judge the input features. Output the operating status of each device, including normal, abnormal, or faulty, etc. For devices identified as abnormal or faulty, further analysis and processing are carried out. Possible processing measures include issuing an alarm, generating a maintenance work order, arranging on-site inspection by personnel, etc.

[0067] As a possible implementation manner, step S104 can be specifically implemented as steps S1041 - S1042.

[0068] S1041. Input the inspection features of each device into the fault detection model to obtain the model output results of each device.

[0069] In some embodiments, the model output results include the fault probability, fault type, and fault cause.

[0070] Exemplarily, embodiments of the present invention can ensure that the fault detection model has been fully trained and has the ability to accurately predict new data. Check the input format of the model to ensure that the inspection features are consistent with the input format required by the model. Perform necessary preprocessing on the inspection features, such as data cleaning, missing value handling, normalization, etc., to improve the prediction accuracy of the model. Input the preprocessed inspection features into the fault detection model. This is usually done through a programming interface (such as an API) or a data loading tool. After receiving the input features, the model performs internal calculations, including feature extraction, feature transformation, model matching, etc. According to the internal structure and algorithm of the model, output the fault probability, fault type, and fault cause of the device. Analyze the results output by the model and extract key information such as fault probability, fault type, and fault cause. Ensure that the output results are easy to understand and use, and appropriate format conversion or explanation may be required.

[0071] S1042. Based on the model output results of each device, determine the operating status of each device.

[0072] In some embodiments, the operating status includes real-time status and fault information; the real-time status includes normal operation, fault operation, or fault shutdown.

[0073] Exemplarily, embodiments of the present invention can set a reasonable fault probability threshold according to the actual situation and the performance of the fault detection model. When the fault probability exceeds the threshold, it is considered that the device may have a fault. Based on the fault probability, fault type, and fault cause output by the model, judge the operating status of the device. If the fault probability is lower than the threshold, the device is in a normal operating state. If the fault probability exceeds the threshold, further judge whether the device is in a fault operation state or has fault shutdown according to the fault type and fault cause. Update the operating status (normal operation, fault operation, or fault shutdown) of the device to the device management system or monitoring platform in real time. This helps to detect and handle device faults in a timely manner, and improve the reliability and safety of the device. For devices in a fault operation state or that have fault shutdown, record detailed fault information, including fault type, fault cause, occurrence time, etc. This information can be used for subsequent fault analysis, maintenance planning, and device improvement work.

[0074] S105. Based on the operating status of each device and the inspection features of each device, generate an intelligent inspection report to achieve intelligent inspection of the confined space.

[0075] It should be noted that the embodiments of the present invention can generate intelligent inspection reports according to the operating status and inspection characteristics of each device. The report content includes device name, operating status, inspection characteristics, abnormal information (if any), treatment suggestions, etc. Optimize and adjust the generated inspection report to ensure its accuracy, clarity, and ease of understanding. Supplementary information such as charts and photos can be added as needed to improve the readability and practicality of the report. Provide the generated intelligent inspection report to relevant personnel or departments for equipment maintenance, fault troubleshooting, safety management, etc. Corresponding measures can be taken according to the suggestions in the report to improve the reliability and safety of the equipment.

[0076] As a possible implementation, step S105 can be specifically implemented as steps S1051 - S1054.

[0077] S1051. Determine the severity and impact range of the fault point based on the operating status of each device.

[0078] Exemplarily, the embodiments of the present invention can evaluate the severity of the fault point according to the operating status of the device (such as normal operation, fault operation, fault shutdown, etc.), combined with the fault probability, fault type, and fault cause output by the fault detection model. The severity can be divided into multiple levels, such as minor, medium, severe, and critical, etc., and each level corresponds to different treatment priorities and measures. Analyze the impact of the fault point on the device itself, related devices, production line, or the entire system. Consider factors such as the location of the fault point, connected devices, power supply lines, and process flow to evaluate the possible consequences of the fault, such as production interruption, quality decline, and safety hazards. The impact range can include direct impact (such as device shutdown) and indirect impact (such as production efficiency decline).

[0079] S1052. Determine the abnormal data of the fault point and the change trend of the abnormal data based on the inspection characteristics of each device.

[0080] Exemplarily, the embodiments of the present invention can identify data with obvious deviation compared with the normal state according to the inspection characteristics of the device, which is the abnormal data. The abnormal data may include abnormal temperature, abnormal vibration, current and voltage fluctuations, etc. Conduct time - series analysis on the identified abnormal data to observe its change trend. The change trend can include rising, falling, fluctuating, etc., which helps to judge whether the fault is deteriorating or alleviating. Historical data can also be combined to analyze whether the change of abnormal data is consistent with the historical pattern to assist in judging the cause and trend of the fault.

[0081] S1053. Sort each fault point based on the severity, impact range, abnormal data, and change trend of the abnormal data of the fault point to obtain an inspection sequence.

[0082] Exemplarily, embodiments of the present invention can formulate criteria for sorting fault points based on factors such as severity, impact range, abnormal data, and change trends. The weights of multiple factors can be considered. For example, severity may have a relatively high weight because it directly affects the operation and safety of the equipment. According to the formulated criteria, each fault point is scored or sorted. The sorting result should reflect the urgency and importance of the fault point, so as to prioritize the treatment of severe fault points with a wide impact range. According to the sorting result, the inspection sequence of each fault point is determined. The inspection sequence should guide the inspectors to conduct inspections according to the priority, ensuring that urgent and important fault points are processed first.

[0083] S1054. Generate an intelligent inspection report based on the template of the intelligent inspection report, the operating status of each device, the inspection characteristics of each device, the severity, impact range, abnormal data, and change trend of the abnormal data of the fault point, and the inspection sequence.

[0084] Exemplarily, embodiments of the present invention can prepare a template for the intelligent inspection report, including the format, structure, content, etc. of the report. The template should clearly display the operating status of each device, inspection characteristics, fault point information, inspection sequence, and other contents. Fill in the data such as the operating status of each device, inspection characteristics, severity, impact range, abnormal data, and change trend of the fault point into the report template. Ensure the accuracy and integrity of the data, avoiding omission or error. Generate an intelligent inspection report according to the filled data and the template. Review the report to ensure the accuracy of the content and the standardization of the format. Distribute the generated intelligent inspection report to relevant personnel or departments so that they can understand the operating status and fault conditions of the equipment. At the same time, archive and backup the report for subsequent query and analysis.

[0085] The present invention provides an intelligent inspection method for an unmanned aerial vehicle in a confined space based on AI visual recognition. By using the unmanned aerial vehicle to replace manual inspection of the confined space, the inspection safety is improved. Then, the AI visual recognition model is used to identify the inspection data of the unmanned aerial vehicle to obtain the surface temperature distribution characteristics, three-dimensional point cloud characteristics, and equipment parameter characteristics of the confined space, and conduct equipment division. From the three aspects of the three-dimensional structure, surface temperature, and equipment parameters of a single device, combined with the fault detection model, the operating status of each device is evaluated efficiently and comprehensively. Finally, the operating status and inspection characteristics of each device are integrated to generate an intelligent inspection report, which is intuitively displayed to the inspectors, realizing efficient, comprehensive, and safe intelligent inspection in the confined space, and improving the inspection efficiency and safety of the confined space.

[0086] Optionally, for the intelligent inspection method for an unmanned aerial vehicle in a confined space based on AI visual recognition provided by embodiments of the present invention, before step S102, steps S201-S203 are further included.

[0087] S201. Obtain multiple video images of known meter measurement data and the corresponding meter measurement data for each video image.

[0088] Exemplarily, embodiments of the present invention can collect video images containing meter measurement data from actual patrol inspection processes or historical records. Ensure that the collected video images are clear, stable, and can accurately reflect the readings of the meters. Label each video image and record the corresponding meter measurement data. The labeled data should include information such as the accurate readings of the meters, the types of the meters, the positions, etc. Manual labeling tools or automated labeling software can be used to complete the labeling work. Preprocess the collected video images, such as denoising, enhancing contrast, etc., to improve the image quality. Convert the video images into a format suitable for machine learning models, such as JPEG, PNG, etc.

[0089] S202. Use each video image as an input and the corresponding meter measurement data for each video image as an output to establish multiple first training samples.

[0090] Exemplarily, embodiments of the present invention can use each preprocessed video image as an input sample. Use the corresponding meter measurement data as an output label or target value. Combine the input sample and the output label into a training sample pair. Divide the training sample pairs into a training set, a validation set, and a test set. The training set is used to train the machine learning model; the validation set is used to adjust the model parameters to prevent overfitting; the test set is used to evaluate the performance of the model. Perform data augmentation operations on the training set, such as rotation, scaling, cropping, etc., to increase the generalization ability of the model. Data augmentation can generate more training samples and improve the training effect of the model.

[0091] S203. Perform machine learning based on multiple first training samples to obtain an image recognition module.

[0092] Exemplarily, for image recognition tasks, commonly used models include convolutional neural networks (CNNs), support vector machines (SVMs), random forests (RFs), etc. In embodiments of the present invention, since it is necessary to recognize the meters in the images and read their readings, it is appropriate to select a convolutional neural network as the model.

[0093] Exemplarily, embodiments of the present invention can use a training set to train a selected model. During the training process, the model learns to extract useful features from the input images and predicts the output labels based on these features. Through optimization methods such as backpropagation algorithm and gradient descent, the parameters of the model are continuously adjusted to minimize the prediction error. The trained model is verified using a validation set. The performance of the model is evaluated by calculating metrics such as accuracy, recall, and F1-score. The model is adjusted and optimized according to the verification results to improve its generalization ability. The finally optimized model is tested using a test set. The test results can reflect the performance of the model in actual applications. If the test results meet the requirements, the model can be deployed to actual applications; if not, the model needs to be further optimized and improved.

[0094] Embodiments of the present invention can encapsulate the trained machine learning model into an image recognition module. During the encapsulation process, it is necessary to ensure that the module can receive the input image and output the corresponding meter measurement data. Encapsulation code can be written using programming languages such as Python and Java, and an executable module file is generated. The encapsulated image recognition module is deployed to actual applications. During the deployment process, it is necessary to ensure that the module can be seamlessly integrated and communicate with other system components (such as data acquisition modules, data analysis modules, etc.). Containerization technologies such as Docker and Kubernetes can be used to simplify the deployment and management of the module.

[0095] In this way, embodiments of the present invention can machine-learn an image recognition module, and then build an AI recognition model to achieve image recognition of confined spaces.

[0096] Optionally, the method for intelligent inspection of drones in confined spaces based on AI vision recognition provided by embodiments of the present invention further includes steps S301-S306 before step S102.

[0097] S301. Obtain the inspection data during the historical period and the three-dimensional point cloud features of multiple targets at each moment during the historical period.

[0098] Exemplarily, embodiments of the present invention can collect inspection data containing multiple targets (such as equipment, personnel, etc.) from historical inspection records. The data should include basic information such as inspection time, location, target type, and quantity. At the same time, collect the three-dimensional point cloud features of multiple targets at each moment, which can be obtained through devices such as lidar (LiDAR) and depth cameras. Clean the inspection data to remove duplicate, invalid, or abnormal data. Preprocess the three-dimensional point cloud data, such as denoising, filtering, registration, etc., to improve the data quality. Ensure that the time stamps of the inspection data and the three-dimensional point cloud features are consistent for subsequent matching.

[0099] S302. Perform frame processing on the inspection data within the historical period to obtain multiple frames of video images and the visual features corresponding to each frame of video image.

[0100] Exemplarily, in the embodiments of the present invention, the video data captured during the inspection process can be frame-processed in chronological order to obtain multiple frames of video images. Ensure that the timestamp of each frame image corresponds to the timestamp of the inspection data. Use image processing algorithms (such as edge detection, shape analysis, texture extraction, etc.) to extract features from each frame of video image. The extracted features should include object contours, shapes, textures, etc., which are helpful for subsequent object recognition and tracking.

[0101] S303. Based on the three-dimensional point cloud features of multiple objects at each moment within the historical period, and the correspondence between each frame of video image and the object, determine the object and three-dimensional point cloud features corresponding to each frame of video image.

[0102] Exemplarily, in the embodiments of the present invention, the object type and quantity in the image can be determined according to the visual features in each frame of video image and the inspection data within the historical period. Methods such as template matching and feature point matching can be used for object recognition. According to the object matching result, associate the object in each frame of video image with the three-dimensional point cloud features at the corresponding moment. This requires ensuring that the timestamps of the video image and the three-dimensional point cloud data are consistent, and the correspondence between the objects in the two is accurate.

[0103] S304. Using the visual features corresponding to each frame of video image as input and the object and three-dimensional point cloud features corresponding to each frame of video image as output, construct multiple second training samples.

[0104] Exemplarily, in the embodiments of the present invention, the visual features corresponding to each frame of video image can be used as input samples. The object type and three-dimensional point cloud features corresponding to each frame of video image are used as output labels or target values. Combine the input samples and output labels into training sample pairs. Divide the training sample pairs into a training set, a validation set, and a test set. The training set is used to train the neural network model; the validation set is used to adjust the model parameters to prevent overfitting; the test set is used to evaluate the performance of the model.

[0105] S305. Based on multiple second training samples, perform neural network training to obtain an object recognition and tracking module.

[0106] Exemplarily, for object recognition and tracking tasks, common models include convolutional neural networks (CNNs), recurrent neural networks (RNNs), long short-term memory networks (LSTMs), etc.

[0107] Since it is necessary to process visual features and 3D point cloud features simultaneously, a neural network architecture that can fuse multiple features can be selected. The selected neural network model is trained using the training set. During the training process, the model learns to extract useful information from the input visual features and predicts the output labels (i.e., the target type and 3D point cloud features) based on this information. Through optimization methods such as backpropagation algorithm and gradient descent, the parameters of the model are continuously adjusted to minimize the prediction error. The trained model is validated using the validation set to evaluate its performance. The model is adjusted and optimized according to the validation results to improve its generalization ability. The finally optimized model is tested using the test set to ensure its performance in actual applications. The trained neural network model is encapsulated into a target recognition and tracking module. During the encapsulation process, it is necessary to ensure that the module can receive the input visual features and output the corresponding target type and 3D point cloud features. Encapsulation code can be written using programming languages such as Python and C++, and an executable module file can be generated.

[0108] In this way, the embodiment of the present invention can obtain a target tracking and recognition module through neural network training, realize the recognition and tracking of each target during the inspection of the drone flight process, and then construct a 3D point cloud of the enclosed space, and further realize the 3D reconstruction during the inspection of the enclosed space, and intuitively display the inspection results.

[0109] Optionally, the method for intelligent inspection of an enclosed space drone based on AI vision recognition provided by the embodiment of the present invention further includes steps S401-S409 before step S102.

[0110] S401. Obtain the inspection data of the enclosed space in the historical period, as well as the visual features of multiple targets in the enclosed space and the temperature distribution features at multiple moments in the historical period.

[0111] Exemplarily, the embodiment of the present invention can collect the inspection data of the enclosed space (such as basements, tunnels, storage tanks, etc.) from historical inspection records. The data should include inspection time, location, target type (such as equipment, pipelines, personnel, etc.), quantity, as well as videos and infrared images during the inspection process. Clean the inspection data to remove duplicate, invalid or abnormal data. Preprocess the videos and infrared images, such as denoising, enhancing contrast, correcting colors, etc., to improve the image quality. Extract the visual features of multiple targets from the preprocessed video images, such as edges, shapes, textures, colors, etc. Extract the temperature distribution features from the infrared images, which usually involves converting the infrared images into temperature maps and calculating temperature gradients, hot spots, cold spots, etc.

[0112] S402. Perform frame processing on the inspection data of the enclosed space in the historical period to obtain multiple frames of video images and multiple frames of infrared images.

[0113] Exemplarily, embodiments of the present invention can perform frame-by-frame processing on video data during the inspection process in chronological order to obtain multiple frames of video images. Ensure that the timestamp of each frame image corresponds to the timestamp of the inspection data. Extract infrared images corresponding to the timestamps of the video images from the inspection data. The infrared images should be taken at the same time or within a short time of the video images to ensure data consistency.

[0114] S403. Based on multiple frames of video images and the visual features of multiple targets in the confined space, perform matching to obtain the visual features corresponding to each frame of video image.

[0115] Exemplarily, embodiments of the present invention can use image processing algorithms (such as template matching, feature point matching, etc.) to match the targets in each frame of video image with predefined visual features. The matching result should be able to accurately identify the target type, position, and quantity in each frame of image.

[0116] S404. Based on multiple frames of infrared images and the temperature distribution features at multiple moments in the historical period, perform matching to obtain the temperature distribution features corresponding to each frame of infrared image.

[0117] Exemplarily, embodiments of the present invention can match each frame of infrared image with the temperature distribution features in the historical period. This may involve converting the infrared image into a temperature map and comparing it with historical temperature data to identify areas or targets with abnormal temperatures.

[0118] S405. Based on multiple frames of video images and the visual features corresponding to each frame of video image, construct multiple third training samples.

[0119] Exemplarily, embodiments of the present invention can use each frame of video image as input and the corresponding visual features as output to construct multiple third training samples. The samples should include images of different types of targets under different lighting, angles, and occlusion conditions.

[0120] S406. Based on multiple frames of infrared images and the temperature distribution features corresponding to each frame of infrared image, construct multiple fourth training samples.

[0121] Exemplarily, embodiments of the present invention can use each frame of infrared image as input and the corresponding temperature distribution features as output to construct multiple fourth training samples. The samples should include infrared images with different temperature distributions, different times (such as day and night, seasons), and different environmental conditions.

[0122] S407. Based on multiple third training samples, perform neural network training to obtain the visual feature extraction sub-module of the feature extraction module.

[0123] S408. Based on multiple fourth training samples, perform neural network training to obtain the infrared feature extraction sub-module of the feature extraction module.

[0124] S409. Construct a feature extraction module of the AI vision model based on the visual feature extraction sub-module, the infrared feature extraction sub-module, and a preset image category classifier.

[0125] Exemplarily, an embodiment of the present invention can use a third training sample to train a neural network to obtain a sub-module capable of extracting visual features in video images. The sub-module should be able to accurately identify the target type, location, and quantity in the image and extract useful visual features.

[0126] Exemplarily, an embodiment of the present invention can use a fourth training sample to train another neural network to obtain a sub-module capable of extracting temperature distribution features in infrared images. The sub-module should be able to accurately identify temperature anomaly regions or targets in the infrared image and extract useful temperature distribution features.

[0127] An embodiment of the present invention can integrate the visual feature extraction sub-module and the infrared feature extraction sub-module into a feature extraction module. The module should be able to process video images and infrared images simultaneously and extract useful visual and temperature distribution features.

[0128] An embodiment of the present invention can integrate a preset image category classifier (such as a support vector machine, a random forest, a convolutional neural network, etc.) into the feature extraction module. The classifier should be able to classify or identify images using the extracted visual and temperature distribution features.

[0129] In this way, an embodiment of the present invention can separately construct a visual feature extraction sub-module and an infrared feature extraction sub-module to extract features from video data and infrared data in the inspection data, realize feature recognition and fusion in the inspection data, and improve the inspection efficiency.

[0130] Optionally, the intelligent inspection method for an unmanned aerial vehicle in a confined space based on AI vision recognition provided by an embodiment of the present invention further includes steps S501 - S503 before step S102.

[0131] S501. Obtain the inspection features of multiple devices with known operating states and the operating states of the multiple devices.

[0132] Exemplarily, embodiments of the present invention can collect inspection data of multiple devices from a device management system or inspection records. Ensure that this data contains the operating status information of the devices, such as normal operation, fault warning, faulty, etc. Inspection features may include physical parameters of the devices, such as temperature, vibration, sound, current, voltage, etc., as well as visual features such as the appearance and wear condition of the devices. Preprocess the collected inspection data to extract features that indicate the operating status of the devices. The preprocessing may include steps such as data cleaning (removing outliers, filling missing values), data transformation (such as normalization, standardization), etc. Feature extraction methods may include statistical analysis (such as mean, variance, peak value, etc.), signal processing (such as Fourier transform, wavelet transform, etc.), machine learning algorithms (such as principal component analysis, linear discriminant analysis, etc.), etc. According to the operating status information of the devices, label the corresponding operating status labels for the inspection feature data of each device. The labels should accurately reflect the current status of the devices, such as "normal", "warning", "fault", etc.

[0133] S502. Using the inspection features of each device as input and the operating status of each device as output, construct multiple fifth training samples.

[0134] Exemplarily, embodiments of the present invention can combine the inspection feature data of each device with its corresponding operating status label into a training sample. Ensure that each training sample contains a complete feature vector and label. If the data volume is large, the data can be divided into multiple subsets, and each subset contains a certain number of training samples. To improve the generalization ability of the model, data augmentation can be performed on the training samples. Data augmentation methods may include random rotation, scaling, translation, adding noise, etc. It should be noted that data augmentation should maintain the consistency between the feature vector and the label, that is, the enhanced feature vector should still correspond to the original label.

[0135] S503. Based on multiple fifth training samples, perform neural network training to obtain a fault detection model.

[0136] Exemplarily, for fault detection tasks, commonly used models include convolutional neural network (CNN), recurrent neural network (RNN), long short-term memory network (LSTM), fully connected neural network (FNN), etc. When selecting a model, factors such as the complexity of the model, training speed, and generalization ability need to be considered.

[0137] Embodiments of the present invention can use training samples to train a selected neural network model. During the training process, the model learns to extract useful information from the input feature vectors and predicts the operating state of the device based on this information. Through optimization methods such as backpropagation algorithm and gradient descent, the parameters of the model are continuously adjusted to minimize the prediction error. During the training process, the hyperparameters of the model (such as learning rate, batch size, number of iterations, etc.) need to be adjusted to optimize the performance of the model. Methods such as cross-validation can be used to evaluate the performance of the model under different hyperparameter combinations and select the optimal hyperparameter combination. The trained model is validated using a validation set to evaluate its performance. The validation set should be independent of the training set and contain the same types of devices and operating states. The model is adjusted and optimized according to the validation results to improve its generalization ability. Finally, the final optimized model is tested using a test set to ensure its performance in actual applications.

[0138] Exemplarily, embodiments of the present invention can save the trained fault detection model to a disk or a database for subsequent use. When saving the model, information such as the parameters of the model, the structure, and the dimension of the input feature vectors needs to be included. The fault detection model is deployed to a device management system or an inspection system to implement the real-time fault detection function. During the deployment process, it is necessary to ensure the compatibility between the model and the system, as well as the real-time performance and accuracy of the model.

[0139] In this way, the present invention can construct a fault detection model before the inspection. After the inspection data is transmitted back by the unmanned aerial vehicle, automatic fault determination is performed for the user to confirm, reducing the manual analysis burden of the inspection personnel and improving the inspection efficiency of the confined space.

[0140] Optionally, the method for intelligent inspection of an unmanned aerial vehicle in a confined space based on AI vision recognition provided by embodiments of the present invention further includes steps S601 - S606 after step S105.

[0141] S601. Receive a re-inspection instruction input by the user.

[0142] In some embodiments, the re-inspection instruction is used to instruct the unmanned aerial vehicle to perform a secondary inspection on the target fault point.

[0143] Exemplarily, embodiments of the present invention can receive a re-inspection instruction input by a user through a user interface (such as a mobile device application, a web page, or a console). The user interface should provide clear and understandable options for the user to easily select the target fault points that need to be reinspected. Parse the received re-inspection instruction to extract key information such as the location, number, or identifier of the target fault point. Ensure that the parsed instruction information is accurate and error-free so that the subsequent steps can be correctly executed. Verify the legality and validity of the re-inspection instruction. For example, check whether the instruction is from an authorized user and whether the target fault point exists. If the instruction is invalid or illegal, an error message should be fed back to the user and the subsequent steps should be refused to execute.

[0144] S602. Forward the re-inspection instruction to the unmanned aerial vehicle.

[0145] Exemplarily, embodiments of the present invention can forward the parsed and verified re-inspection instruction to a designated unmanned aerial vehicle through a wireless communication method (such as Wi-Fi, 4G / 5G, etc.). Ensure that the instruction is not interfered with during the transmission process and can accurately reach the unmanned aerial vehicle. Receive the confirmation reply of the unmanned aerial vehicle to the re-inspection instruction to ensure that the instruction has been correctly received and understood. If the unmanned aerial vehicle does not confirm the instruction, the instruction should be resent or the communication connection should be checked for normality.

[0146] S603. Receive the secondary inspection data transmitted back by the unmanned aerial vehicle.

[0147] Exemplarily, embodiments of the present invention can receive the secondary inspection data transmitted back by the unmanned aerial vehicle through a wireless communication method. The data may include images, videos, sensor readings, etc. of the target fault point. Check the received secondary inspection data to ensure the integrity and accuracy of the data. If the data is incomplete or there are errors, a request should be sent to the unmanned aerial vehicle to request retransmission of the data.

[0148] S604. Generate secondary inspection features of the target fault point based on the secondary inspection data.

[0149] Exemplarily, embodiments of the present invention can preprocess the secondary inspection data to extract features related to the operating state of the target fault point. Feature extraction methods may include image processing (such as edge detection, image segmentation, etc.), signal processing (such as filtering, spectrum analysis, etc.), etc. Integrate the extracted features into a complete feature vector for subsequent fault detection. Ensure that the feature vector can comprehensively and accurately reflect the operating state of the target fault point.

[0150] S605. Determine the operating state of the target fault point based on the secondary inspection features of the target fault point and the fault detection model.

[0151] Exemplarily, embodiments of the present invention may use the secondary inspection features of the target fault point as input and feed them into a previously trained fault detection model. Ensure that the format and dimension of the input data are consistent with the requirements of the model. Use the fault detection model to predict the input feature vector to obtain the operating status of the target fault point. The prediction results may include states such as normal operation, early warning, and fault. Verify the prediction results to ensure their accuracy and reliability. If the prediction results do not match the actual situation, it may be necessary to retrain the model or adjust the feature extraction method.

[0152] S606. Generate a secondary inspection report based on the operating status of the target fault point and the inspection features of the target fault point.

[0153] Exemplarily, embodiments of the present invention may write the content of the secondary inspection report according to the operating status and inspection features of the target fault point. The report should include information such as the location, number, operating status, and description of inspection features of the target fault point. Adjust the format of the report to make it conform to the reading habit and report requirements. Charts, images, etc. can be used for auxiliary explanation to make the report more intuitive and understandable. Store the generated secondary inspection report in a specified location or database. Distribute the report to relevant users or departments as needed so that they can understand the operating status of the target fault point and take corresponding measures.

[0154] In this way, embodiments of the present invention can perform secondary inspections on fault points according to user requirements, accurately determine the fault conditions of the fault points, and improve the accuracy of inspection results in confined spaces.

[0155] It should be understood that the magnitudes of the sequence numbers of the steps in the above embodiments do not mean the order of execution. The order of execution of each process should be determined according to its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.

[0156] The following is an apparatus embodiment of the present invention. For details not described in detail herein, reference may be made to the corresponding method embodiments above.

[0157] Figure 2 The structural schematic diagram of an intelligent inspection unmanned aerial vehicle for confined spaces based on AI visual recognition provided by an embodiment of the present invention is shown. The intelligent inspection device 700 includes a communication module 701 and a processing module 702.

[0158] The communication module 701 is used to obtain the inspection data of the unmanned aerial vehicle in the confined space, and the inspection data includes video data and infrared imaging data.

[0159] The processing module 702 is configured to obtain an identification result based on the inspection data and the AI vision recognition model. The identification result includes surface temperature distribution features, three-dimensional point cloud features, and device parameter features. Based on the identification result, device division is performed to obtain the inspection features of each device. Based on the inspection features of each device and the fault detection model, the operating status of each device is determined. Based on the operating status of each device and the inspection features of each device, an intelligent inspection report is generated to achieve intelligent inspection of the confined space.

[0160] The above-described embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features. These modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A confined space drone intelligent inspection method based on AI visual recognition, characterized in that: include: Acquire inspection data of a drone in a confined space, wherein the inspection data includes video data and infrared imaging data; Based on the inspection data and the AI ​​visual recognition model, a recognition result is obtained, wherein the recognition result includes surface temperature distribution characteristics, three-dimensional point cloud characteristics, and equipment parameter characteristics; Based on the identification results, the equipment is divided to obtain the inspection characteristics of each equipment; Determine the operating status of each device based on the inspection characteristics of each device and the fault detection model; Based on the operating status of each device and the inspection characteristics of each device, an intelligent inspection report is generated to achieve intelligent inspection of confined spaces.

2. The method for intelligent inspection of confined space drones based on AI visual recognition according to claim 1 is characterized in that: The recognition result obtained based on the inspection data and the AI ​​visual recognition model includes: Performing frame processing on the inspection data to obtain multiple frames of video images and multiple frames of infrared images; Based on the multiple frames of video images and the multiple frames of infrared images, and the feature extraction module of the AI ​​visual model, feature extraction is performed to obtain visual features corresponding to each frame of video image and temperature distribution features corresponding to each frame of infrared image; the visual features include target contour, target shape and texture features; Determine the three-dimensional point cloud features of multiple targets in the confined space based on the visual features corresponding to each frame of the video image and the target recognition and tracking module of the AI ​​visual model; Based on the three-dimensional point cloud features and the temperature distribution features corresponding to each frame of infrared image, target fusion is performed to obtain the surface temperature distribution features; Based on the multiple frames of video images and the image recognition module of the AI ​​visual model, the meter measurement data in the video images are extracted to obtain device parameter characteristics of each device; Based on the surface temperature features, three-dimensional point cloud features and device parameter features, the device type and device location of each device, and the feature fusion module of the AI ​​vision model, feature fusion is performed to obtain the recognition result.

3. The method for intelligent inspection of confined space drones based on AI visual recognition according to claim 1 is characterized in that: Before obtaining the recognition result based on the inspection data and the AI ​​visual recognition model, the method further includes: Acquire multiple video images of known meter measurement data, and the meter measurement data corresponding to each video image; Taking each video image as input and the meter measurement data corresponding to each video image as output, a plurality of first training samples are established; Based on the multiple first training samples, machine learning is performed to obtain the image recognition module.

4. The method for intelligent inspection of confined space by unmanned aerial vehicles based on AI visual recognition according to claim 1 is characterized in that: Before obtaining the recognition result based on the inspection data and the AI ​​visual recognition model, the method further includes: Obtaining inspection data within a historical period and three-dimensional point cloud features of multiple targets at each time within the historical period; Performing frame processing on the inspection data in the historical period to obtain multiple frames of video images and visual features corresponding to each frame of video image; Based on the three-dimensional point cloud features of multiple targets at each moment in the historical period and the corresponding relationship between each frame of video image and the target, determine the target and three-dimensional point cloud features corresponding to each frame of video image; Taking the visual features corresponding to each frame of the video image as input and the target and three-dimensional point cloud features corresponding to each frame of the video image as output, a plurality of second training samples are constructed; Based on the multiple second training samples, neural network training is performed to obtain the target recognition and tracking module.

5. The method for intelligent inspection of confined space by unmanned aerial vehicles based on AI visual recognition according to claim 1 is characterized in that: Before obtaining the recognition result based on the inspection data and the AI ​​visual recognition model, the method further includes: Obtaining inspection data of the confined space during a historical period, as well as visual features of multiple targets in the confined space and temperature distribution features at multiple times during the historical period; Performing frame processing on the inspection data of the confined space during the historical period to obtain multiple frames of video images and multiple frames of infrared images; Matching the multiple video frames with visual features of multiple targets in the confined space to obtain visual features corresponding to each video frame; Based on the multiple frames of infrared images and the temperature distribution characteristics of multiple moments in the historical period, matching is performed to obtain the temperature distribution characteristics corresponding to each frame of infrared image; Constructing a plurality of third training samples based on the plurality of video frames and the visual features corresponding to each video frame; constructing a plurality of fourth training samples based on the plurality of infrared images and the temperature distribution characteristics corresponding to each infrared image; Based on the plurality of third training samples, neural network training is performed to obtain a visual feature extraction submodule of the feature extraction module; Based on the plurality of fourth training samples, neural network training is performed to obtain an infrared feature extraction submodule of the feature extraction module; Based on the visual feature extraction submodule, the infrared feature extraction submodule and the preset image category classifier, a feature extraction module of the AI ​​vision model is constructed.

6. The method for intelligent inspection of confined space by unmanned aerial vehicles based on AI visual recognition according to claim 1 is characterized in that: Based on the identification result, the equipment is divided to obtain the inspection characteristics of each equipment, including: Based on the device type and device location, and the identification result, matching is performed to obtain identification results of each device; Based on the architectural information of the confined space, determine the relevant equipment for each device; Based on the recognition results of each device and the recognition results of each device's related devices, feature fusion is performed to obtain the inspection features of each device.

7. The method for intelligent inspection of confined space by unmanned aerial vehicles based on AI visual recognition according to claim 1 is characterized in that: The operation status of each device is determined based on the inspection characteristics of each device and the fault detection model, including: Inputting the inspection characteristics of each device into the fault detection model to obtain the model output results of each device; the model output results include fault probability, fault type and fault cause; Based on the model output results of each device, the operating status of each device is determined, and the operating status includes real-time status and fault information; the real-time status includes normal operation, fault operation or fault shutdown.

8. The confined space drone intelligent inspection method based on AI visual recognition according to any one of claims 1 to 7, characterized in that: The method generates an intelligent inspection report based on the operating status of each device and the inspection characteristics of each device to realize intelligent inspection of the confined space, including: Based on the operating status of each device, determine the severity and impact scope of the fault point; Based on the inspection characteristics of each device, determine the abnormal data of the fault point and the change trend of the abnormal data; Based on the severity of the fault point, the scope of impact, the abnormal data and the change trend of the abnormal data, the fault points are sorted to obtain an inspection sequence; The intelligent inspection report is generated based on the template of the intelligent inspection report, the operating status of each device, the inspection characteristics of each device, the severity of the fault point, the scope of impact, the abnormal data and the changing trend of the abnormal data, and the inspection sequence.

9. The confined space drone intelligent inspection method based on AI visual recognition according to any one of claims 1 to 8, characterized in that: The method further comprises: Receiving a re-inspection instruction input by a user, wherein the re-inspection instruction is used to instruct the human-machine to perform a second inspection on the target fault point; forwarding the heavy inspection instruction to the drone; Receiving secondary inspection data sent back by the drone; Based on the secondary inspection data, generating secondary inspection features of the target fault point; Determine the operating state of the target fault point based on the secondary inspection characteristics of the target fault point and the fault detection model; A secondary inspection report is generated based on the operating status of the target fault point and the inspection characteristics of the target fault point.

10. A confined space drone intelligent inspection device based on AI visual recognition, characterized in that: include: A communication module, used to obtain inspection data of the drone in the confined space, wherein the inspection data includes video data and infrared imaging data; A processing module, used to obtain a recognition result based on the inspection data and an AI visual recognition model, wherein the recognition result includes surface temperature distribution characteristics, three-dimensional point cloud characteristics, and equipment parameter characteristics; Based on the identification results, the equipment is divided to obtain the inspection characteristics of each equipment; Determine the operating status of each device based on the inspection characteristics of each device and the fault detection model; Based on the operating status of each device and the inspection characteristics of each device, an intelligent inspection report is generated to achieve intelligent inspection of confined spaces.

Images