Fire-fighting equipment inspection method and equipment based on AI identification and digital twin positioning, and medium

Abstract

The invention discloses a fire-fighting equipment inspection method and equipment based on AI identification and digital twin positioning, and a medium, and relates to the technical field of fire-fighting equipment operation and maintenance. The method comprises the steps of collecting multi-source monitoring data, and performing feature extraction on the multi-source monitoring data to obtain a first feature; carrying out attention enhancement on the first feature to obtain a second feature, carrying out fault identification based on the second feature, and determining a fault fire-fighting facility; based on the appearance characteristics of the fault fire-fighting equipment, fire-fighting equipment identifiers matched with the appearance characteristics are determined from the digital twin model; and determining a fault position from the digital twin model based on the fire-fighting facility identifier. Therefore, the fault fire-fighting equipment can be accurately identified and positioned, and the inspection efficiency of the fire-fighting equipment is improved.

Description

Technical Field

[0001] This application relates to the field of fire protection facility operation and maintenance technology, and in particular to a fire protection facility inspection method, equipment and medium based on AI recognition and digital twin positioning. Background Technology

[0002] Current fire protection facility inspections generally rely on manual point-by-point checks, which consumes a significant amount of manpower and time, resulting in low inspection efficiency. Furthermore, inspection results are affected by the inspectors' experience, sense of responsibility, and surrounding environmental factors, especially in special scenarios such as nighttime or inclement weather, leading to a higher rate of missed faults. In addition, after a fault is discovered during an inspection, its location is usually recorded only through natural language descriptions, lacking precise spatial coordinate positioning. Maintenance personnel arriving on-site must then conduct a point-by-point check to confirm the fault, significantly extending the fault-handling time. Summary of the Invention

[0003] This application provides a method, equipment, and medium for inspecting fire protection facilities based on AI recognition and digital twin positioning, in order to solve the following technical problem: how to accurately identify and locate faulty fire protection facilities and improve the efficiency of fire protection facility inspection.

[0004] In a first aspect, embodiments of this application provide a method for inspecting fire protection facilities based on AI recognition and digital twin positioning, the method comprising: Collect multi-source monitoring data and extract features from the multi-source monitoring data to obtain the first feature; Attention enhancement is applied to the first feature to obtain a second feature, and fault identification is performed based on the second feature to determine the faulty fire protection facility; Based on the appearance characteristics of the faulty fire protection facilities, fire protection facility identifiers that match the appearance characteristics are determined from the digital twin model; Based on the fire protection facility identification, the fault location is determined from the digital twin model.

[0005] Secondly, embodiments of this application also provide a fire protection facility inspection device based on AI recognition and digital twin positioning. The device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the method as described above.

[0006] Thirdly, embodiments of this application also provide a computer storage medium storing computer-executable instructions, which, when executed, implement the method described in any of the above claims.

[0007] The fire protection facility inspection method, equipment, and medium based on AI recognition and digital twin positioning provided in this application have the following beneficial effects: First, multi-source monitoring data is collected and feature extracted to obtain the first feature. By integrating the status information of fire protection facilities obtained from multiple dimensions by multi-source monitoring devices, blind spots of single data sources can be avoided, improving the accuracy of subsequent fault identification. Furthermore, feature extraction of the collected multi-source monitoring data can transform the raw data into standardized features that can be processed by the algorithm, eliminating data inconsistencies caused by equipment differences and improving the reliability of subsequent processing. Subsequently, attention enhancement is applied to the first feature to obtain the second feature, and fault identification is performed based on the second feature to determine the faulty fire protection facility. Attention enhancement of the first feature can enhance specific parts of the first feature, enabling the algorithm to more accurately focus on fault-related areas and improve the accuracy of fault identification. Next, based on the appearance characteristics of the faulty fire protection facility, the fire protection facility identifier matching the appearance characteristics is determined from the digital twin model. This can automatically complete the automatic association between the faulty facility and the fire protection facility identifier in the twin model, significantly reducing manual operation and improving processing efficiency. Finally, based on the fire protection facility identifier, the fault location is determined from the digital twin model, enabling precise fault location and effectively improving the efficiency of fire protection facility inspection. Attached Figure Description

[0008] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 A flowchart illustrating a fire protection facility inspection method based on AI recognition and digital twin positioning, provided for an embodiment of this application; Figure 2 This is a schematic diagram of the internal structure of a fire protection facility inspection equipment based on AI recognition and digital twin positioning, provided as an embodiment of this application. Detailed Implementation

[0009] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0010] The technical solutions proposed in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0011] Figure 1This document presents a flowchart illustrating a fire protection facility inspection method based on AI recognition and digital twin positioning, provided for one or more embodiments within this specification. This method can be applied to various fire protection facility inspection scenarios, such as large buildings, industrial parks, commercial complexes, and explosion-proof chemical plants where high inspection accuracy and efficiency are required. Certain input parameters or intermediate results in the process can be manually adjusted to help improve accuracy.

[0012] This application provides a method for inspecting fire protection facilities based on AI recognition and digital twin positioning. It should be noted that the executing entity in this embodiment can be a server or any terminal device with data processing capabilities. For example, the server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms. The terminal device can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, vehicle terminal, etc., but is not limited to these.

[0013] like Figure 1 As shown in the figure, the fire protection facility inspection method based on AI recognition and digital twin positioning provided in this application embodiment specifically includes the following steps: Step 101: Collect multi-source monitoring data.

[0014] It should be noted that multi-source monitoring data refers to the real-time status data of fire protection facilities collected by detection devices of different types, locations, and functions. Multi-source monitoring data can include data from fixed acquisition devices (such as data collected by high-definition infrared cameras, pressure sensors, temperature and humidity sensors, etc.), data from mobile acquisition devices (such as data collected by explosion-proof inspection robots, handheld inspection terminals, etc.), and environmental and location data (such as timestamps, equipment status logs, Global Positioning System (GPS) coordinates, etc.).

[0015] As an example, in the scenario of fire extinguisher pressure monitoring in a commercial park, the CAM-01-03-001 camera at the entrance of office 301 on the 3rd floor of Building 1 periodically collects images, while the pressure sensor on the fire extinguisher monitors the air pressure value in real time. The collected images and air pressure data are then transmitted to the server in real time for subsequent processing. In the scenario of fire hydrant inspection in a chemical industrial park, the explosion-proof inspection robot inspects according to a preset route. When it reaches the vicinity of reactor No. 2, it takes high-definition images of the fire hydrant. At the same time, the explosion-proof pressure sensor on the fire hydrant monitors the water pressure value in real time. The images and pressure data are then transmitted to the server via optical fiber for subsequent processing.

[0016] In some embodiments, before performing step 101 above, the following processing may also be performed: collecting inspection scene data and modeling based on the inspection scene data to obtain an initialized digital twin model; generating a unique identifier for each fire protection facility in the initialized digital twin model and calibrating the coordinates of each fire protection facility to obtain a calibrated digital twin model; configuring the basic attributes, operation and maintenance attributes, and safety attributes of each fire protection facility in the calibrated digital twin model to generate a digital twin model. Thus, through the complete process of high-precision modeling, coordinate calibration, and attribute configuration, the high accuracy and reliability of the digital twin model are ensured, providing a solid foundation for fault location. Furthermore, the standardized processing from scene data collection to attribute configuration ensures data consistency and accuracy, avoiding information silos and data inconsistencies. In addition, by configuring the attributes of the fire protection facilities, the digital twin model not only has a precise spatial representation but also contains rich semantic information, providing comprehensive support for subsequent intelligent identification, fault classification, and handling decisions.

[0017] As an example, in a commercial park scenario, firstly, a Faro Focus S70 laser scanner is used to perform a comprehensive scan of the commercial park, acquiring precise geometric data on building structures, corridors, offices, etc. A 4K resolution drone is then used to conduct aerial photography of the park, obtaining macroscopic scene information such as the overall layout, green spaces, and roads. Existing BIM model data, including design information such as building structures and floor layouts, is also collected. Next, the laser scan data, drone aerial photography data, and BIM model are imported into professional modeling software (such as Revit). Point cloud processing technology is used to convert the point cloud data obtained from the laser scan into a 3D model. Simultaneously, the drone aerial images are used as textures mapped onto the surface of the 3D model to enhance its realism, resulting in a model that includes office buildings, public areas, and fire exits. A complete 3D model of the scene is created and imported into a game engine such as Unity3D to generate an initial digital twin model. Then, all fire-fighting facilities (such as fire extinguishers, fire hydrants, smoke detectors, emergency lights, etc.) are identified and categorized within the initial digital twin model, and a unique identifier is generated for each facility (e.g., "MF-01-03-005" represents the 5th fire extinguisher on the 3rd floor of Building 1, "XS-02-01-001" represents the 1st fire hydrant on the 1st floor of Building 2, and "YG-03-02-012" represents the 12th smoke detector on the 2nd floor of Building 3). Corresponding identification tags are added to each fire-fighting facility in the digital twin model, establishing a relationship between the identifier and the model. The process involves establishing connections to ensure each identifier corresponds to a unique model element. Next, a three-dimensional reference coordinate system (e.g., WGS84 latitude and longitude coordinate system) is established within the digital twin model. The coordinates of each fire-fighting facility in the digital twin model are then calibrated (e.g., using high-precision GPS to measure the coordinates of key control points, calibrating the deviation between laser scanning and GPS coordinates using a Kalman filter algorithm, and randomly selecting several fire-fighting facilities for on-site coordinate measurement, comparing the measured coordinates with the model coordinates to verify calibration accuracy). This yields the calibrated digital twin model. Finally, the basic attributes of each fire-fighting facility in the calibrated digital twin model are defined (e.g., fire extinguisher A's basic attributes are model MFZ / ABC4, manufacturer X...). The system configures the following attributes: X Fire Equipment Factory, installation time January 2024, 5-year warranty, filling date December 2023; operation and maintenance attributes (e.g., the inspection record in the operation and maintenance attribute configuration of fire extinguisher A is the last inspection time October 15, 2024, the inspector is Zhang San, the inspection result is normal, the next inspection time is November 15, 2024, the fault record is historical fault 1 in March 2024 with insufficient pressure and the handling method is filling, historical fault 2 in July 2024 with aging of the sealing component and the handling method is replacing the sealing component); and safety attributes (e.g., the applicable scenario for the fire hydrant safety attribute is office area, the operation requires professional personnel to operate, and the maintenance cycle is once a month). This configuration generates the final digital twin model.

[0018] In some embodiments, the digital twin model in this application can be updated incrementally in real time, that is, the model is fully calibrated monthly by laser scanning; when facilities are added, removed or their location changes, maintenance personnel can upload change information (including location photos and attribute parameters) through the terminal, and the system can automatically complete the incremental update; for chemical explosion-proof scenarios, explosion-proof areas will be marked in the digital twin model and associated with the exclusive parameters of explosion-proof facilities (such as explosion-proof level Exd IIB T3); at the same time, the digital twin model supports local caching at the edge, and the location and attributes of facilities can be viewed through the local model when the network is disconnected, and the model data will be automatically synchronized and updated after the network is connected.

[0019] Step 102: Extract features from the multi-source monitoring data to obtain the first feature.

[0020] It should be noted that feature extraction can be the extraction of visual features such as color, texture, and shape from camera images, or the extraction of statistical features and trend features from sensor data. It can also combine time and spatial information to extract features (such as position change trends), or it can fuse different types of data features to form comprehensive features. The specific method can be determined according to the actual situation, and no specific limitation is made here.

[0021] As an example, in a commercial park scenario, the collected image data undergoes preprocessing (e.g., adjusting brightness and contrast to eliminate the effects of light changes), and color and shape features of the fire extinguisher pressure gauge area are extracted, along with texture and corrosion features of the fire hydrant interface. Simultaneously, the text information of fire facility markings is identified to obtain the image features corresponding to the image data. For the collected pressure data, the current pressure value feature is extracted, the pressure change rate feature is calculated, and pressure fluctuation features are extracted to obtain the pressure features corresponding to the pressure data. For the collected location data, the facility's latitude and longitude coordinates are extracted, the floor and relative location features of the facility are calculated, and the surrounding environment features (e.g., corridors, office entrances, etc.) are extracted to obtain the location features corresponding to the location data. Finally, the features corresponding to the different collected data are fused (e.g., stitching, weighted summation, etc.) to obtain the first feature.

[0022] Step 103: Perform attention enhancement on the first feature to obtain the second feature.

[0023] It should be noted that attention enhancement is a technique that dynamically allocates attention weights through algorithms, enabling the model to focus on key information in the input features and ignore irrelevant interference.

[0024] In some embodiments, step 103 described above can be implemented by: performing channel attention processing on the first feature to obtain a third feature; and performing spatial attention processing on the third feature to obtain the first feature. Thus, the channel attention module can automatically learn the importance of different channel features and assign weights to each channel, thereby strengthening fault-related features and making the model more sensitive to capturing fault signals. The spatial attention module can accurately locate the specific position of the fault region within the features and effectively suppress interference from irrelevant backgrounds, thereby improving the accuracy of fault identification.

[0025] As an example, in a smoke sensor fault detection scenario in a commercial park, assume the first feature includes smoke sensor images from a high-definition camera (containing channels such as indicator light area, sensor housing, and ceiling) and sensor status data. First, channel attention processing is performed on the first feature, resulting in "indicator light color channel" and "sensor status data channel" being of high importance, while "ceiling texture channel" and others are of lower importance. Based on the degree of importance, the indicator light status and sensor status data are enhanced to obtain the channel-enhanced feature (third feature). Subsequently, spatial attention processing is performed on the channel-enhanced feature, resulting in "indicator light area" being of high importance, while "sensor housing area" and "ceiling area" are of lower importance. Based on the degree of importance, the indicator light status is enhanced to obtain the spatially enhanced feature (second feature).

[0026] Step 104: Based on the second feature, identify the faulty fire protection facilities.

[0027] It should be noted that fault identification can be achieved based on deep learning-based object detection and classification methods, such as the improved YOLOv8 algorithm combined with transfer learning strategies; it can also be based on rule engines for fault judgment, such as threshold rules (stress values), trend rules, combination rules, etc.; it can also be based on multi-source data fusion for fault identification, such as combining visual-sensor data fusion, time series data fusion, etc.; it can also be based on knowledge graphs and other common fault identification methods, which are not specifically limited here.

[0028] In some embodiments, step 104 described above can be implemented as follows: based on the second feature, perform fault prediction on the fire protection facility to obtain the predicted probability of each fault type corresponding to the fire protection facility; select the fault type with the highest predicted probability as the candidate fault type corresponding to the fire protection facility; if the predicted probability of the candidate fault type corresponding to the fire protection facility is greater than the first probability threshold, determine the fire protection facility as a faulty fire protection facility. In this way, the probabilities corresponding to different fault types can be evaluated simultaneously based on the prediction results to provide more detailed judgment results. Furthermore, the strictness of fault identification can be flexibly controlled through the first probability threshold to adjust the identification strategy according to the safety requirements of different scenarios, thereby improving the adaptability of the model.

[0029] As an example, in the scenario of fire extinguisher malfunction prediction in a commercial park, the second feature includes visual features, pressure data features, location features, and historical features processed by channel-space dual attention. Subsequently, based on the second feature, the system performs malfunction prediction on the fire extinguisher, calculates the prediction probability of each malfunction type (e.g., insufficient pressure: 0.92, aging seals: 0.15, blurred markings: 0.05), and finds that the malfunction type corresponding to the highest prediction probability is insufficient pressure, thus taking insufficient pressure as the candidate malfunction type corresponding to the fire protection facility. Then, assuming the first probability threshold is 0.9, if the prediction probability (0.92) of the candidate malfunction type corresponding to the fire protection facility is greater than the first probability threshold (0.9), then the fire protection facility is identified as a malfunctioning fire protection facility, and the corresponding malfunction type is insufficient pressure.

[0030] In some embodiments, if the predicted probability of a candidate fault type corresponding to a fire protection facility is less than a first probability threshold but greater than a second probability threshold, multi-dimensional cross-validation is performed on the fire protection facility to obtain a cross-validation result. If the cross-validation result is normal, the fire protection facility is identified as a faulty fire protection facility; if the cross-validation result is abnormal, the fire protection facility is identified as a normal fire protection facility. Thus, multi-dimensional cross-validation can significantly improve the accuracy and reliability of fire protection facility fault identification while reducing the false alarm rate.

[0031] It should be noted that multi-dimensional cross-validation can include multi-source data cross-validation and time-series analysis validation. For multi-source data cross-validation, the camera recognition results can be compared with physical data from pressure sensors, temperature sensors, etc., to eliminate false alarms caused by image interference. For example, if the camera detects an abnormal pressure gauge pointer on a fire extinguisher, but the pressure sensor shows normal pressure, it is determined to be a false alarm. For time-series analysis validation, the fault characteristics of three consecutive frames of images can be compared to eliminate transient interference. For example, if the pressure gauge shows an abnormality only in a single frame, but multiple consecutive frames show normal pressure, it is determined to be transient interference, and false alarms are eliminated.

[0032] As an example, in the scenario of detecting corrosion at fire hydrant interfaces in a chemical explosion-proof area, assuming that the candidate fault type of the fire-fighting facility is interface corrosion obtained through fault detection based on the second feature, the corresponding predicted probability is 0.82, the first probability threshold is 0.9, and the second probability threshold is 0.8. By comparison, it can be seen that the predicted probability (0.82) of the candidate fault type corresponding to the fire-fighting facility is less than the first probability threshold (0.9) and greater than the second probability threshold (0.8). Therefore, multi-dimensional cross-validation is performed on the fire-fighting facility. First, multi-source data cross-validation is performed. Based on the analysis of the explosion-proof camera image, it is judged to be "moderate corrosion". Based on the humidity displayed by the surrounding humidity sensor... The 85% level is relatively high, consistent with the environmental characteristics of a chemical industrial zone. High humidity accelerates metal corrosion. The analysis and image recognition results are consistent, indicating that the multi-source data cross-validation is normal. Next, time series analysis was performed, retrieving five frames of images collected by the explosion-proof camera in the past hour. All five frames showed interface corrosion characteristics, indicating that it was not an instantaneous phenomenon. Furthermore, the historical records of the fire hydrant were checked. The inspection record from three months ago showed slight interface corrosion, while the current inspection showed that the corrosion had worsened, which is consistent with the development pattern. The time series analysis results are normal. The comprehensive analysis results show that the multi-dimensional cross-validation results are normal, thus identifying the fire-fighting facility as a faulty fire-fighting facility.

[0033] In another example, in the scenario of detecting insufficient fire extinguisher pressure in a commercial park, assuming that the candidate fault type of the fire-fighting facility is insufficient pressure based on the second feature, the corresponding prediction probability is 0.85, the first probability threshold is 0.9, and the second probability threshold is 0.8. By comparison, it can be seen that the prediction probability of the candidate fault type corresponding to the fire-fighting facility (0.85) is less than the first probability threshold (0.9) and greater than the second probability threshold (0.8). Therefore, multi-dimensional cross-validation is performed on the fire-fighting facility. First, multi-source data cross-validation is performed. According to the high-definition camera image analysis, it is judged to be "insufficient pressure". According to the real-time reading of the fire extinguisher pressure sensor, it is 1.0MPa (within the normal range of 0.8-1.2MPa), which is inconsistent with the image recognition result. It can be seen that the multi-source data cross-validation is abnormal. Furthermore, according to the time series analysis, it can be seen that only the current frame shows insufficient fire extinguisher pressure, while other adjacent frames show normal fire extinguisher pressure values. In summary, the cross-validation result is abnormal, so the fire-fighting facility is determined to be a normal fire-fighting facility.

[0034] In some embodiments, fault prediction is achieved through a trained fault identification model. Before performing fault prediction on fire protection facilities based on a second feature to obtain the predicted probability for each fault type, the following processing can be performed: pre-training the initialized fault identification model on a general dataset to obtain a pre-trained fault identification model; and specialized training on the pre-trained fault identification model based on a set of fire protection facility fault samples to obtain a trained fault identification model. Thus, by first pre-training on a general dataset, the model can learn a wide range of visual features, and the pre-training process ensures that the network parameters are in a good initial state, avoiding the instability of training from random initialization. Subsequently, through specialized training on a set of fire protection facility fault samples, general visual knowledge is transformed into recognition capabilities specific to the fire protection facility domain. Specific visual patterns of typical faults can be learned, thereby improving the accuracy of fault identification. Furthermore, through two-stage training, the model's basic capabilities and specialized recognition capabilities can be organically combined, significantly improving the accuracy, generalization ability, and training efficiency of fault identification.

[0035] It should be noted that general datasets can be image datasets (such as COCO, ImageNet, PASCAL VOC, etc.), video datasets (such as ActivityNet, Kinetics, etc.), multimodal datasets (such as VGG-Sound, MSR-VTT, etc.), etc., and fault recognition models can be object detection models (such as YOLOv5, YOLOv8, Faster R-CNN, RetinaNet, etc.), image classification models (such as ResNet-50, ResNet-101, etc.), multi-task learning models (such as MmoE, MTI, etc.), time series analysis models (such as 3D CNN, LSTM / GRU, etc.), etc., without specific limitations.

[0036] As an example, images related to the business environment, such as "desk," "chair," and "computer," are selected from the COCO dataset. These object images help the model learn basic visual features, such as edges, textures, and shapes. The model can build an understanding of the basic structure of objects through these general images. First, the COCO dataset is input into the initialized fault recognition model for prediction processing. The model loss is calculated based on the difference between the predicted and actual results, and the model parameters are updated through backpropagation based on the model loss. Through multiple iterations, a pre-trained fault recognition model can be obtained, enabling the model to recognize common object and scene features. After pre-training, the model has basic visual recognition capabilities. Subsequently, 100,000 fire protection facility fault samples are collected, including 40,000 normal samples, which can be used to display various... The system includes 35,000 samples of functional faults in normal fire protection facilities, such as insufficient pressure in fire extinguishers, no water in fire hydrants, and sensor failure, and 25,000 samples of appearance faults, such as corrosion, blurred markings, and obstruction. Each sample is meticulously labeled, including the type of fire protection facility (e.g., fire extinguisher, fire hydrant, smoke sensor) and the type of fault (e.g., insufficient pressure, corrosion, blurred markings). Then, 100,000 samples of fire protection facility faults are input into a pre-trained fault identification model for specialized training. By learning the specific characteristics of fire protection facilities, the model loss is calculated based on the prediction results and sample labels. The model parameters are then updated through backpropagation based on the model loss. Through multiple iterations, a trained fault identification model is obtained. After specialized training, the model is capable of identifying various types of fire protection facility faults.

[0037] Step 105: Based on the appearance characteristics of the faulty fire protection facilities, determine the fire protection facility identification that matches the appearance characteristics from the digital twin model.

[0038] It should be noted that the appearance characteristics of faulty fire protection facilities may include shape characteristics (such as overall outline, local structure, size ratio, etc.), texture characteristics (such as surface texture, rust texture, wear texture, etc.), color characteristics (such as color distribution, color anomaly, etc.), marking characteristics (such as text markings, graphic markings, etc.), and location characteristics (such as relative position of components, status indication, etc.).

[0039] In some embodiments, step 105 described above can be implemented as follows: for each fire protection facility feature in the facility feature library of the digital twin model, a similarity calculation is performed between the fire protection facility feature and the appearance feature of the faulty fire protection facility to obtain a first similarity corresponding to the fire protection facility feature; the fire protection facility corresponding to the fire protection facility feature with the highest first similarity is selected as the target fire protection facility; the fire protection facility identifier of the target fire protection facility is determined from the digital twin model, and the fire protection facility identifier of the target fire protection facility is selected as the fire protection facility identifier that matches the appearance feature. In this way, feature comparison and optimal matching selection can achieve high-precision matching between the faulty fire protection facility and the fire protection facilities in the digital twin model, thereby providing a reliable foundation for subsequent accurate positioning and rapid response.

[0040] It should be noted that similarity calculation can be performed using methods such as cosine similarity or Euclidean distance, and no specific limitation is made here.

[0041] As an example, suppose the appearance of the faulty fire protection facility A is as follows: The facility feature library of the digital twin model includes fire protection facility features. Characteristics of fire protection facilities Characteristics of fire protection facilities Features of fire protection facilities are calculated using the cosine similarity algorithm. The corresponding first similarity score is 0.95, fire protection facility characteristics. The corresponding first similarity is 0.80, fire protection facility characteristics. The corresponding first similarity score is 0.62, and the characteristics of fire protection facilities can be obtained through screening. The corresponding fire protection facility is the target fire protection facility. At the same time, the fire protection facility identifier XF-03-02-11 of the target fire protection facility can be found from the digital twin model. The fire protection facility identifier XF-03-02-11 of the target fire protection facility is used as the fire protection facility identifier that matches the appearance characteristics. That is, the fire protection facility identifier of the faulty fire protection facility A is XF-03-02-11.

[0042] Step 106: Based on the fire protection facility identification, determine the fault location from the digital twin model.

[0043] As an example, the system obtains the twin ID of the smoke sensor, YG-03-02-012, through similarity calculation. This ID represents the 12th smoke sensor on the 2nd floor of Building 3. Subsequently, the system queries the coordinate information of YG-03-02-012 in the digital twin model, which shows that the smoke sensor's location is WGS84 latitude and longitude coordinates of 116.3028°E, 39.9045°N, floor elevation of 22.7 meters, and relative position coordinates X=8.4 meters, Y=12.6 meters (relative to the floor origin). The installation location is in the center of the ceiling, next to the corridor lighting fixture. The fault location can be obtained, and the system can mark the fault at the fault location.

[0044] In some embodiments, the fault type corresponding to the faulty fire protection facility is determined, and the risk level of the faulty fire protection facility is determined based on the fault type; based on the risk level of the faulty fire protection facility, the faulty fire protection facility at the fault location is repaired. In this way, through scientific fault classification and risk grading, accurate identification, rapid response, and efficient handling of fire protection facility faults can be achieved, which can improve the level of fire safety, optimize resource allocation, reduce operation and maintenance costs, and achieve efficient synergy between technology and management.

[0045] As an example, assuming the fault type of the malfunctioning fire protection facility is "insufficient pressure," the risk level of the malfunctioning fire protection facility can be determined as "Level 1 functional failure" based on the mapping relationship. Furthermore, considering that the malfunctioning fire protection facility is located in an office area with high personnel density and a high risk of electrical fires, and based on historical data showing that the fire extinguisher failure frequency in this area is 1.2 times / year (higher than the park average), the analysis determines that the risk level of the malfunctioning fire protection facility is high risk. The system automatically triggers an emergency response, and the work order is automatically dispatched to maintenance personnel within 3 kilometers who hold an explosion-proof operation certificate. Third, the fire control room dashboard triggers an audible and visual alarm, displaying the location and type of the fault. Zhang San can receive a work order via a mobile app, which includes navigation routes and fault evidence. The app navigation shows: "Take elevator No. 1 from the fire control room to the 3rd floor, turn right and walk straight for 50 meters. It is on the left side of the entrance to office 301." Zhang San arrives at the fault site according to the navigation, confirms that the fire extinguisher pressure is insufficient, and replaces it with a new fire extinguisher (model MFZ / ABC4). The old equipment is recycled for refilling. Then, Zhang San uploads before-and-after comparison photos and pressure data (0.5MPa before handling, 1.0MPa after handling) via the app.

[0046] In some embodiments, an analysis model can be built based on historical data to automatically shorten the inspection cycle from once a month to once every half month for facilities with a failure frequency of ≥3 times / year; extend the cycle to once a quarter for facilities with no failures for 6 consecutive months; at the same time, the failure rate of each region is calculated and the density of inspection equipment is increased in high-risk areas (such as around the reactors in chemical industrial parks); in addition, the fault samples after handling are automatically included in the AI ​​training set and the algorithm parameters are fine-tuned every month to continuously improve the recognition accuracy.

[0047] In some embodiments, access permissions can be tiered for different personnel, and sensitive data (such as fault images, personnel information, etc.) can be stored using AES-256 encryption, transmitted using SSL encryption, and all operation logs can be retained for one year to support traceability and auditing.

[0048] In some embodiments, the implementation process of this application will be described in detail in conjunction with a composite scenario of a 500,000㎡ commercial park and a 20,000㎡ chemical explosion-proof zone. The hardware equipment used includes a Faro Focus S70 laser scanner (0.1mm accuracy), a Hikvision high-definition infrared camera (DS-2CD6A20FWD-IZS, 1920×1080 resolution), and a fire-fighting pressure sensor (PTX7517, 0.01MPa accuracy, explosion-proof rating Exd IIB). The system consists of T3, an explosion-proof inspection robot (RXR-MC80D), a cloud server (Huawei Cloud 8-core 16G×10 nodes), and a mobile app (supporting Android / iOS, compatible with explosion-proof phones). First, a laser scanner is used to perform a full-scene scan of the commercial park and the chemical explosion-proof area. Drone aerial photography is used to acquire panoramic data of the park. This data is then imported into Revit software to build a BIM model, and finally integrated with Unity3D to generate a 3D digital twin model. The fire extinguishers in the commercial park are assigned the twin ID "MF-01-03-005", coordinates 116.3024°E, 39.9042°N, floor 3, location: entrance to office 301. The fire hydrants in the chemical explosion-proof area are assigned the ID "XS-02-01-001", coordinates 116.3056°E, 39.9068°N, explosion-proof rating Exd IIB. After calibration using Kalman filtering, the positioning error of T3 is ≤0.4 meters. Next, the attributes of MF-01-03-005 are entered into the digital twin model as follows: Model MFZ / ABC4, Manufacturer XX Fire Equipment Factory, Installation Time January 2024, Last Inspection October 2024. The attributes of XS-02-01-001 are: Model SS100 / 65-1.6, Explosion-proof Rating Exd IIB. T3. Installation time: May 2023. The system automatically cached the model to the edge server, supporting offline viewing. In November 2024, 50 fire extinguishers were added to the commercial park. Maintenance personnel uploaded photos of the new locations and model information via an explosion-proof mobile app. The system completed the incremental update of the digital twin model within 5 minutes, without requiring a full scan. Subsequently, the camera CAM-01-03-001 at the entrance of office 301 on the 3rd floor of the commercial park collected images every 5 minutes, and the pressure sensor uploaded data every 30 seconds. At 10:00 on November 5, 2024, the camera collected an image of MF-01-03-005. The algorithm identified that the pressure gauge pointer was below the normal scale, and the pressure sensor uploaded data of 0.5MPa (normal range 0.8-1).(2MPa), and after image processing using an improved YOLOv8 algorithm, the fault type was determined to be "Level 1 functional fault - insufficient pressure" with a confidence level of 92%. The system automatically correlated pressure sensor data for cross-validation, confirming the pressure abnormality and eliminating false alarms. Subsequently, by matching the camera ID CAM-01-03-001 and the fire extinguisher model characteristics in the image, a twin ID was obtained. MF-01-03-005, in the digital twin model, the location is marked with a flashing red indicator, generating a physical location description "Entrance to Office 301, 3rd Floor, Building 1" and navigation route. An early warning push is completed within 10 seconds, triggering an audible and visual alarm on the fire control room dashboard, displaying the fault location and pressure data. Maintenance personnel Zhang San (within 3 kilometers, holding an explosion-proof operation certificate) simultaneously receive a work order on their app, including navigation route and fault photos. At the same time, the administrator generates a real-time early warning report in the backend. Finally, Zhang San navigates to the site via the app, confirms the fire extinguisher pressure is insufficient, replaces it with a new one (of the same model), uploads pressure data before (pressure 0.5MPa) and after (pressure 1.0MPa) and comparison photos, and fills in the handling description "Replaced MFZ / ABC4". The system automatically verified that the pressure data of the fire extinguisher ("Old equipment to be recycled and refilled") was normal, completing the review within one minute. After review, the system updated the status of MF-01-03-005 to "Normal," and the disposal record was archived to the full lifecycle data, with the last disposal date updated to November 5, 2024, forming a complete data chain. Furthermore, according to data statistics from December 2024, the fire extinguisher malfunction frequency on the 3rd floor of Building 1 reached 4 times per year (higher than the average of 1.2 times per year). The system automatically adjusted the inspection cycle for this area from once a month to once every two weeks and pushed an optimization suggestion to the administrator: "It is recommended to investigate the environmental temperature and humidity in this area, which may affect the sealing performance of the fire extinguishers." After optimization, the malfunction frequency in this area dropped to once in January 2025.

[0049] The above are embodiments of the method proposed in this application. Based on the same inventive concept, embodiments of this application also provide a device, the structure of which is as follows: Figure 2 As shown.

[0050] Figure 2 This is a schematic diagram of the internal structure of a device provided in an embodiment of this application. Figure 2 As shown, the device includes: At least one processor 201; And a memory 202 that is communicatively connected to at least one processor; The memory 202 stores instructions that can be executed by at least one processor. The instructions are executed by at least one processor 201 to enable at least one processor 201 to perform the steps of the method corresponding to any of the above embodiments.

[0051] Some embodiments of this application provide corresponding to Figure 1 A non-volatile computer storage medium stores computer-executable instructions, which are configured to perform the steps of the method corresponding to any of the above embodiments.

[0052] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments for IoT devices and media are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0053] The systems, media, and methods provided in this application are one-to-one correspondences. Therefore, the systems and media also have similar beneficial technical effects as their corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the systems and media will not be repeated here.

[0054] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0055] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0056] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0057] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0058] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0059] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0060] Computer-readable media include both permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0061] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0062] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for inspecting fire protection facilities based on AI recognition and digital twin positioning, characterized in that, The method includes: Collect multi-source monitoring data and extract features from the multi-source monitoring data to obtain the first feature; Attention enhancement is applied to the first feature to obtain a second feature, and fault identification is performed based on the second feature to determine the faulty fire protection facility; Based on the appearance characteristics of the faulty fire protection facilities, fire protection facility identifiers that match the appearance characteristics are determined from the digital twin model; Based on the fire protection facility identification, the fault location is determined from the digital twin model.

2. The method according to claim 1, characterized in that, Prior to collecting the multi-source monitoring data, the method further includes: Collect inspection scenario data and build a model based on the inspection scenario data to obtain an initialized digital twin model; A unique identifier is generated for each fire protection facility in the initialized digital twin model, and the coordinates of each fire protection facility are calibrated to obtain a calibrated digital twin model. Configure the basic attributes, operation and maintenance attributes, and safety attributes of each fire protection facility in the calibrated digital twin model to generate the digital twin model.

3. The method according to claim 1, characterized in that, The process of enhancing the attention of the first feature to obtain the second feature includes: The first feature is subjected to channel attention processing to obtain the third feature; Spatial attention processing is applied to the third feature to obtain the first feature.

4. The method according to claim 1, characterized in that, The fault identification based on the second feature to determine the faulty fire protection facility includes: Based on the second feature, fault prediction is performed on the fire protection facilities to obtain the predicted probability of each fault type of the fire protection facilities. The fault type with the highest predicted probability is taken as the candidate fault type for the fire protection facility. If the predicted probability of the candidate fault type corresponding to the fire protection facility is greater than the first probability threshold, the fire protection facility is identified as the faulty fire protection facility.

5. The method according to claim 4, characterized in that, The method further includes: If the predicted probability of the candidate fault type corresponding to the fire protection facility is less than the first probability threshold and greater than the second probability threshold, the fire protection facility is subjected to multi-dimensional cross-validation to obtain the cross-validation result. If the cross-validation results are normal, the fire protection facility is identified as the faulty fire protection facility. If the cross-validation result is abnormal, the fire protection facility will be identified as a normal fire protection facility.

6. The method according to claim 4, characterized in that, The fault prediction is achieved through a trained fault identification model. Before predicting the faults of the fire protection facilities based on the second feature to obtain the predicted probability of each fault type for the fire protection facilities, the method further includes: The initial fault identification model is pre-trained based on a general dataset to obtain a pre-trained fault identification model. The pre-trained fault identification model is specifically trained based on the fault sample set of fire protection facilities to obtain the trained fault identification model.

7. The method according to claim 1, characterized in that, The step of determining the fire protection facility identifier matching the appearance characteristics from the digital twin model based on the appearance characteristics of the faulty fire protection facility includes: For each fire protection facility feature in the facility feature library of the digital twin model, the similarity between the fire protection facility feature and the appearance feature of the faulty fire protection facility is calculated to obtain the first similarity corresponding to the fire protection facility feature; The fire-fighting facility corresponding to the fire-fighting facility feature with the highest similarity is taken as the target fire-fighting facility; The fire protection facility identifier of the target fire protection facility is determined from the digital twin model, and the fire protection facility identifier of the target fire protection facility is used as the fire protection facility identifier that matches the appearance feature.

8. The method according to claim 1, characterized in that, The method further includes: Determine the fault type corresponding to the faulty fire protection facility, and determine the risk level of the faulty fire protection facility based on the fault type; Based on the risk level of the faulty fire protection facilities, the faulty fire protection facilities at the fault location are repaired.

9. A fire protection facility inspection device based on AI recognition and digital twin positioning, characterized in that, The device includes: At least one processor; And, a memory communicatively connected to the at least one processor; The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method as described in any one of claims 1-8.

10. A computer storage medium storing computer-executable instructions, characterized in that, When the computer-executable instructions are executed, they implement the method as described in any one of claims 1-8.

Images