Fire-fighting hidden danger intelligent inspection system based on AR glasses

Abstract

The invention belongs to the technical field of fire safety and intelligent detection, and provides an intelligent fire hazard inspection system based on AR glasses, and the system comprises a data collection module which is used for capturing on-site images and video data through a high-definition camera of the AR glasses, and providing route planning and key inspection prompts through a display screen of the AR glasses; the data transmission module is used for transmitting the image and video data acquired by the AR glasses in real time; the data processing and analyzing module is used for receiving and processing the data transmitted from the AR glasses, intelligently analyzing the uploaded image or video and automatically identifying the position, the type and the risk level of a hidden danger according to an analysis result; the AR interaction module is used for displaying an analysis result and a hidden danger prompt on the AR glasses in real time; according to the invention, fire-fighting potential safety hazard inspection of a place is carried out through the AR glasses, route planning and key inspection prompts are provided, the inspection efficiency is improved, scenes seen by eyes are intelligently analyzed, and fire-fighting potential safety hazards are automatically detected.

Description

Technical Field

[0001] The present invention belongs to the technical field of fire safety and intelligent detection, and specifically relates to an intelligent fire hazard inspection system based on AR glasses. Background Art

[0002] With the acceleration of the urbanization process and the increase in fixed asset investment, the number of buildings in the country is continuously increasing, and fire safety issues are becoming increasingly prominent. The traditional fire inspection method usually relies on manual inspection, and there are various deficiencies in the process, resulting in the detection accuracy and efficiency of fire hazards unable to meet the safety requirements of modern society.

[0003] During manual inspection, inspectors often work based on personal experience and limited paper materials. This makes the inspection route planning lack scientificity, with a large amount of blind searching, consuming a lot of time and energy of inspectors, resulting in extremely low inspection efficiency. For example, in large commercial complexes or industrial parks, due to the large area and complex building structure, inspectors may need to repeatedly turn back and cannot quickly cover all areas that need to be inspected, easily missing key fire hazard points. In the actual inspection process, personnel often lack systematic route planning, resulting in possible repeated inspections or omission of key areas during the inspection. This blind searching not only wastes time but also reduces the inspection efficiency. In terms of hazard detection, manual inspection mainly relies on visual observation to identify fire hazards, which is greatly affected by subjective factors, and the accuracy and reliability are difficult to guarantee. Some fire hazards with strong concealment, such as aging and short - circuit problems inside electrical lines, or damage to fire facilities blocked by objects, are difficult to be detected in time by manual inspection.

[0004] In addition, traditional inspection technologies lack real - time monitoring and early warning mechanisms. Problems can only be discovered when inspectors arrive at the scene. During the inspection interval, once a sudden fire hazard occurs, such as a fire risk caused by overheating of electrical equipment, it is difficult to detect and handle in time, easily leading to small fires developing into major accidents. With the development of technology, some simple electronic monitoring devices have been introduced in some places, but these devices have single functions, cannot be combined with the actual on - site scenario, and cannot provide intuitive and effective information for inspectors. Moreover, these devices are independent of each other, lacking data fusion and intelligent analysis capabilities, unable to comprehensively process the collected data, and difficult to achieve a comprehensive assessment and accurate early warning of fire hazards.

[0005] Therefore, technicians in this field have proposed an intelligent fire hazard inspection system based on AR glasses, aiming to improve the efficiency, accuracy, and intelligence level of fire hazard inspections through intelligent route planning and real - time environmental risk assessment, realize real - time monitoring and early warning, and ensure the fire safety of various places. Summary of the Invention

[0006] To solve the above technical problems, the present invention provides an intelligent fire hazard inspection system based on AR glasses to solve the problems raised in the background art.

[0007] An intelligent fire hazard inspection system based on AR glasses, comprising:

[0008] A data acquisition module, configured to capture on-site images and video data through the high-definition camera of the AR glasses, and provide route planning and key inspection prompts through the display screen of the AR glasses;

[0009] A data transmission module, configured to transmit the image and video data collected by the AR glasses to the data processing and analysis module in real time;

[0010] A data processing and analysis module, configured to receive and process the data transmitted from the AR glasses, use deep learning algorithms to perform intelligent analysis on the uploaded images or videos, automatically detect fire hazards, and automatically identify the location, type, and risk level of the hazards according to the analysis results;

[0011] An AR interaction module, configured to display the analysis results and hazard prompts on the AR glasses in real time.

[0012] Preferably, the data acquisition module integrates a camera, sensors, and a display screen through the AR glasses. The camera collects on-site images and video data in real time, and the sensors monitor environmental data including temperature, humidity, smoke, and gas concentration in real time. The environmental data collected by the sensors is combined with the data collected by the camera to form comprehensive on-site information, and the real-time on-site information is expressed as:

[0013] I(t) = f(C(t), S(t))

[0014] Wherein, I(t) represents the images and videos output from the camera at time t, C(t) represents the optical information captured by the camera, S(t) represents the data input by the sensors, and f represents a function of the image processing and fusion algorithm, which is used to combine the optical information collected by the camera with the environmental data of the sensors and process them into a final output image.

[0015] Preferably, the data acquisition module displays the images captured by the camera, measurement results, and prompt information on the display screen of the AR glasses in real time, and marks suspicious areas and adds annotations on the display screen through gestures, voice commands, or touch operations. The marking of the suspicious area is expressed as:

[0016] A sus (x, y) = g(I(t), Interactive(x, y))

[0017] Wherein, Asus (x, y) represents a suspicious area marked on the image I(t) based on the interaction between the user and the image. Interactive(x, y) represents the interaction between the user and the image at the coordinates (x, y), and g() represents a function used to process the input coordinates (x, y) of the user and the current image;

[0018] The addition of annotations is expressed as:

[0019] N sus (x, y) = h(A sus (x, y), Input)

[0020] Among them, N sus (x, y) represents the annotation content at the coordinates (x, y), h() represents a processing function that combines the annotation with the annotation information input by the user, and Input represents the annotation information input by the user;

[0021] According to the inspection task, the AR glasses provide a navigation route and prompt the areas that need to be inspected key points, for route planning and key inspection prompts. The route planning is expressed as:

[0022] R(t) = P(L, E)

[0023] Among them, R(t) represents the currently planned route, which is updated with time t. P() represents a path planning function based on location data L and environmental data E, that is, the starting point, ending point of the inspection method and the real-time status of the inspection area;

[0024] The key inspection prompt is expressed as:

[0025] T check (I(t), H) = J(I(t), E, H)

[0026] Among them, T check represents the inspection prompt at the inspection site, H represents the list of items that need to be inspected key points, and J() represents a function that combines the current image, sensor data with the key items to provide inspection prompts.

[0027] Preferably, the data processing and analysis module receives images, videos, and sensor data from the AR glasses through the data transmission module and performs preprocessing; performs hidden danger detection through a deep learning model. The deep learning model is expressed as:

[0028] Y = f θ (X, S)

[0029] Among them, Y is the output of the model, representing the identified hidden danger types and risk scores, f θDenote the deep learning model with parameters θ, where X is the preprocessed image or video data, and S is the sensor data;

[0030] Input the preprocessed image or video data into the trained deep learning model. The model outputs the class probability distribution of fire hazards. Then the probability vector output by the deep learning model is where represents the probability of the i-th type of fire hazard. Select the class with the highest probability as the prediction result;

[0031] For each input data, the deep learning model will generate a prediction result, including the hazard location, type, and risk level. Then the hazard identification is represented as: Z = {z1, z2,..., z n}, where Z is the hazard set, including all potential hazards, and each hazard z i is composed of the elements of hazard location, type, and risk level.

[0032] Preferably, the data processing and analysis module also combines the sensor data to conduct environmental risk assessment and generate intelligent early warnings. It fuses the sensor data with the output result of the deep learning model. Its risk assessment is represented as:

[0033]

[0034] where ω i represents the weight of the hazard type z i , Risklevel(z i , S) represents the actual risk level of the hazard z i under the given environmental condition S, and E represents the comprehensive environmental risk score;

[0035] After the analysis is completed, according to the set early warning threshold E th , when the comprehensive environmental risk score exceeds the early warning threshold, an intelligent early warning is triggered, which is represented as:

[0036]

[0037] where Alert represents the early warning signal.

[0038] Preferably, the AR interaction module uses the real-time image captured by the camera and combines computer vision algorithms to locate the hazard position, enabling the superposition of information and the actual scene; The AR glasses integrate voice recognition and motion sensors, enabling the inspection personnel to quickly receive instructions and perform operations through voice commands or gestures.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] 1. The present invention provides route planning and key inspection prompts through AR glasses, reducing the blind search of inspection personnel, improving the inspection efficiency, and using deep learning algorithms to intelligently analyze images and videos, automatically detecting fire hazards, and improving the accuracy and reliability of detection.

[0041] 2. The present invention conducts environmental risk assessment by combining sensor data. When the comprehensive environmental risk score exceeds the warning threshold, it triggers an intelligent warning to achieve real-time monitoring and warning. By integrating a variety of sensors to collect on-site environmental data, it more comprehensively assesses the risk of potential safety hazards, providing a strong guarantee for fire safety.

[0042] 3. The present invention overlays information with the actual scene through the AR interaction module, quickly receives instructions and performs operations according to voice commands or gestures, enhancing the user experience and convenience. The overall solution realizes the intelligent inspection and warning of fire hazards, promoting the improvement of the intelligent and automated level of fire management. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a block diagram of the intelligent fire hazard inspection system based on AR glasses of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0044] The following further describes in detail the embodiments of the present invention in conjunction with the drawings and embodiments. The following embodiments are used to illustrate the present invention, but cannot be used to limit the scope of the present invention.

[0045] As shown in the Figure 1 drawing:

[0046] Embodiment 1: The present invention provides an intelligent fire hazard inspection system based on AR glasses, including: a data acquisition module, which is used to capture on-site image and video data through the high-definition camera of the AR glasses, and provide route planning and key inspection prompts through the display screen of the AR glasses; a data transmission module, which is used to transmit the image and video data collected by the AR glasses to the data processing and analysis module in real time; a data processing and analysis module, which is used to receive and process the data transmitted from the AR glasses, use deep learning algorithms to intelligently analyze the uploaded images or videos, automatically detect fire hazards, and according to the analysis results, automatically identify the location, type and risk level of the hazards; an AR interaction module, which is used to display the analysis results and hazard prompts in real time on the AR glasses.

[0047] The data acquisition module integrates a camera, sensors and a display screen through the AR glasses. The camera collects on-site image and video data in real time, and the sensors monitor environmental data including temperature, humidity, smoke and gas concentration in real time. The environmental data collected by the sensors is combined with the data collected by the camera to form comprehensive on-site information, and the real-time on-site information is expressed as:

[0048] I(t) = f(C(t), S(t))

[0049] Wherein, I(t) represents the images and videos output from the camera at time t, C(t) represents the optical information captured by the camera, S(t) represents the data input by the sensor, and f represents the function of the image processing and fusion algorithm, which is used to combine the optical information collected by the camera with the environmental data of the sensor and process them into a final output image.

[0050] The data acquisition module also displays the images captured by the camera, measurement results, and prompt information in real time through the display screen in the AR glasses, and marks suspicious areas and adds annotations on the display screen through gestures, voice commands, or touch operations. The marking of suspicious areas is expressed as:

[0051] A sus (x, y) = g(I(t), Interactive(x, y))

[0052] Wherein, A sus (x, y) represents the suspicious area marked on the image I(t) based on the interaction between the user and the image, Interactive(x, y) represents the interaction between the user and the image at the coordinates (x, y), and g() represents the function used to process the input coordinates (x, y) of the user and the current image;

[0053] The addition of annotations is expressed as:

[0054] N sus (x, y) = h(A sus (x, y), Input)

[0055] Wherein, N sus (x, y) represents the annotation content at the coordinates (x, y), h() represents the processing function, which combines the marking with the annotation information input by the user, and Input represents the annotation information input by the user;

[0056] According to the inspection task, the AR glasses provide a navigation route and prompt the areas that need to be inspected key points, for route planning and key inspection prompts. The route planning is expressed as:

[0057] R(t) = P(L, E)

[0058] Wherein, R(t) represents the currently planned route, which is updated with time t, and P() represents the path planning function, which is based on the position data L and environmental data E, that is, the starting point, ending point of the inspection method, and the real-time status of the inspection area;

[0059] The key inspection prompt is expressed as:

[0060] T check (I(t),H)=J(I(t),E,H)

[0061] Among them, T check Represents the inspection prompts at the inspection site, H represents the list of key inspection items, and J() represents the function that combines the current image, sensor data and key items to provide inspection prompts. AR glasses provide route planning and key inspection prompts to reduce blind searches by inspectors and improve inspection efficiency.

[0062] The high-definition camera of AR glasses captures on-site images and video data, and the integrated sensors monitor environmental data such as temperature, humidity, smoke and gas concentration. The two are combined to form comprehensive on-site information to more comprehensively assess the risk of safety hazards. The display screen displays images, measurement results and prompt information in real time, supports gestures, voice commands or touch operations to mark suspicious areas and add annotations, and can also provide navigation routes and key inspection prompts according to inspection tasks to improve the pertinence and efficiency of inspections.

[0063] The data transmission module transmits the images, videos and environmental sensor data acquired by the data acquisition module to the data processing and analysis module in real time to ensure smooth and timely information.

[0064] The data processing and analysis module receives images, videos and sensor data from the AR glasses through the data transmission module and performs preprocessing; hidden danger detection is performed through a deep learning model, and the deep learning model is expressed as:

[0065] Y=f θ (X,S)

[0066] Among them, Y is the output of the model, indicating the type of hidden dangers and risk scores identified, and f θ represents a deep learning model, with parameter θ, X is the preprocessed image or video data, and S is the sensor data;

[0067] The preprocessed image or video data is input into the trained deep learning model, and the model outputs the category probability distribution of fire hazards. The probability vector output by the deep learning model is in represents the probability of the i-th fire hazard, and the category with the largest probability is selected as the prediction result;

[0068] For each input data, the deep learning model will generate a prediction result, including the location, type and risk level of the hidden danger. The hidden danger identification is expressed as: Z = {z1, z2, ..., z n}, where Z is the hidden danger set, including all potential hidden dangers, and each hidden danger z i It is composed of the elements of hidden danger location, type and risk level.

[0069] The data processing and analysis module also combines sensor data to conduct environmental risk assessment and generate intelligent warnings. It fuses the sensor data with the output results of the deep learning model, and its risk assessment is expressed as:

[0070]

[0071] where ω i represents the weight of the hidden danger type z i , Risklevel(z i , S) represents the actual risk level of the hidden danger z i under the given environmental condition S, and E represents the comprehensive environmental risk score;

[0072] After the analysis is completed, according to the set warning threshold E th , when the comprehensive environmental risk score exceeds the warning threshold, an intelligent warning is triggered, which is expressed as:

[0073]

[0074] where Alert represents the warning signal. The data processing and analysis module receives and preprocesses the data from the AR glasses, uses the deep learning algorithm to conduct intelligent analysis on the images or videos, detects fire hazards, and identifies the locations, types, and risk levels of the hazards. It combines the sensor data to conduct environmental risk assessment, and triggers an intelligent warning when the comprehensive environmental risk score exceeds the warning threshold, providing a decision-making basis for the inspection personnel.

[0075] The AR interaction module uses the real-time images captured by the camera in combination with computer vision algorithms to locate the hazard positions, enabling the superposition of information and the actual scene; by displaying the analysis results and hazard prompts in real time on the AR glasses, it uses the real-time images captured by the camera in combination with computer vision algorithms to locate the hazard positions, enabling the superposition of information and the actual scene. The AR glasses integrate voice recognition and motion sensors, enabling the inspection personnel to quickly receive instructions and perform operations through voice commands or gestures.

[0076] This system can be widely applied to various fire protection places, such as shopping malls, schools, factories, etc. Through intelligent inspection and analysis means, it can greatly improve the recognition and handling efficiency of fire hazards and reduce the risk of fire accidents. At the same time, this system can also provide detailed data support and decision-making basis for the fire department, helping it better formulate and implement fire safety strategies.

[0077] Embodiment 2: The present invention also proposes a method for intelligent inspection of fire hazards based on AR glasses. Using the intelligent inspection system in Embodiment 1, it includes the following steps:

[0078] S1. The management personnel issue inspection tasks through the AR interaction module. The AR glasses receive the tasks and display the inspection routes and key inspection areas.

[0079] S2. The inspection personnel wear AR glasses and conduct inspections according to the prompts, taking images, videos and marking suspicious areas.

[0080] S3. The AR glasses upload the collected data to the data processing and analysis module at the back end in real time for analysis.

[0081] S4. The data processing and analysis module analyzes the data using image recognition and deep learning models, identifies potential hazards and generates analysis results.

[0082] S5. The analysis results are displayed to the management personnel and inspection personnel through the AR interaction module, enabling the inspection personnel and management personnel to intuitively see the locations of potential hazards and risk levels.

[0083] The intelligent inspection system and method for fire hazards based on AR glasses can efficiently and accurately identify and handle fire hazards, improve the work efficiency of inspection personnel, strengthen safety management and decision-making support at the same time. It not only improves the effect of on-site inspections, but also reduces the risks caused by insufficient monitoring, thus effectively preventing safety hazards such as fires.

[0084] It is important to note that the construction and arrangement of the present application shown in multiple different exemplary embodiments are merely illustrative. Although only a few embodiments are described in detail in this disclosure, those who refer to this disclosure should easily understand that many modifications are possible on the premise of substantially not deviating from the novel teachings and advantages of the subject matter described in this application. Other substitutions, modifications, changes and omissions can be made in the design, operating conditions and arrangement of the exemplary embodiments without departing from the scope of the present invention. Therefore, the present invention is not limited to specific embodiments, but extends to various modifications that still fall within the scope of the appended claims.

[0085] In addition, in order to provide a concise description of the exemplary embodiments, all features of the actual embodiments may not be described (i.e., those features that are not relevant to the best mode currently considered for implementing the present invention or those features that are not relevant to implementing the present invention).

[0086] It should be understood that in the development process of any actual implementation, in any engineering or design project, a large number of specific implementation decisions can be made. Such development efforts may be complex and time-consuming, but for those ordinary technical personnel who benefit from this disclosure, without excessive experimentation, such development efforts will be a routine work of design, manufacturing and production.

[0087] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention, and they should all be covered within the scope of the claims of the present invention.

Claims

1. An intelligent inspection system for fire hazards based on AR glasses, characterized in that: include: A data acquisition module, which is used to capture on-site images and video data through the high-definition camera of the AR glasses, and provide route planning and key inspection prompts through the display screen of the AR glasses; A data transmission module, used to transmit the image and video data collected by the AR glasses to the data processing and analysis module in real time; The data processing and analysis module is used to receive and process the data transmitted from the AR glasses, use deep learning algorithms to intelligently analyze the uploaded images or videos, automatically detect fire hazards, and automatically identify the location, type and risk level of the hazard based on the analysis results; AR interaction module, used to display analysis results and hidden danger prompts in real time on AR glasses.

2. The fire hazard intelligent inspection system based on AR glasses as claimed in claim 1, characterized in that: The data acquisition module integrates a camera, a sensor and a display screen through AR glasses. The camera collects images and video data of the inspection site in real time. The sensor monitors environmental data including temperature, humidity, smoke and gas concentration in real time. The environmental data collected by the sensor is combined with the data collected by the camera to form comprehensive on-site information. The real-time on-site information is expressed as: I(t)=f(C(t),S(t)) Among them, I(t) represents the image and video output from the camera at time t, C(t) represents the optical information captured by the camera, S(t) represents the data input by the sensor, and f represents the function of the image processing and fusion algorithm, which is used to combine the optical information collected by the camera with the environmental data of the sensor to process it into a final output image.

3. The fire hazard intelligent inspection system based on AR glasses as claimed in claim 2, characterized in that: The data acquisition module displays the images captured by the camera, the measurement results and prompt information in real time through the display screen in the AR glasses, and marks the suspicious areas and adds annotations on the display screen through gestures, voice commands or touch operations. The marked suspicious areas are represented as: A sus (x,y)=g(I(t),Interactive(x,y)) Among them, A sus (x, y) represents the suspicious area marked on the image I(t) based on the interaction between the user and the image, Interactive(x, y) represents the interaction between the user and the image at the coordinate (x, y), and g() represents the coordinate (x, y) and the current image used to process the user input; The added annotation is represented as: N sus (x,y)=h(A sus (x,y),Input) Among them, N sus (x,y) represents the annotation content at the coordinate (x,y), h() represents the processing function, which combines the annotation with the annotation information entered by the user, and Input represents the annotation information entered by the user; According to the inspection task, the AR glasses provide a navigation route and prompt the areas that need to be inspected, perform route planning and key inspection prompts, and the route planning is expressed as: R(t)=P(L,E) Among them, R(t) represents the currently planned route, which is updated with time t, and P() represents the path planning function, which is based on the location data L and the environmental data E, that is, the starting point, end point and real-time status of the inspection area of ​​the inspection mode; The key inspection prompts are expressed as follows: T check (I(t),H)=J(I(t),E,H) Among them, T check It represents the inspection prompt at the inspection site, H represents the list of items that need to be inspected, and J() represents the function of providing inspection prompts by combining the current image, sensor data and key items.

4. The fire hazard intelligent inspection system based on AR glasses as claimed in claim 1, characterized in that: The data processing and analysis module receives images, videos and sensor data from the AR glasses through the data transmission module and performs preprocessing; hidden danger detection is performed through a deep learning model, and the deep learning model is expressed as: Y=f θ (X,S) Among them, Y is the output of the model, indicating the type of hidden dangers and risk scores identified, and f θ represents a deep learning model, with parameter θ, X is the preprocessed image or video data, and S is the sensor data; The preprocessed image or video data is input into the trained deep learning model, and the model outputs the category probability distribution of fire hazards. The probability vector output by the deep learning model is in represents the probability of the i-th fire hazard, and the category with the largest probability is selected as the prediction result; For each input data, the deep learning model will generate a prediction result, including the location, type and risk level of the hidden danger. The hidden danger identification is expressed as: Z = {z1, z2, ..., z n }, where Z is the hidden danger set, including all potential hidden dangers, and each hidden danger z i It is composed of the elements of hidden danger location, type and risk level.

5. The fire hazard intelligent inspection system based on AR glasses as claimed in claim 4, characterized in that: The data processing and analysis module also combines sensor data to conduct environmental risk assessment and generate intelligent warnings, integrating sensor data with the output results of the deep learning model. Its risk assessment is expressed as: Among them, ω i Indicates the type of hidden danger z i The weight of Risklevel(z i ,S) indicates hidden danger i The actual risk level under given environmental conditions S, E represents the comprehensive environmental risk score; After the analysis is completed, according to the set warning threshold E th ,When the comprehensive environmental risk score exceeds the warning threshold, the intelligent warning is triggered, which is expressed as: Among them, Alert represents a warning signal.

6. The fire hazard intelligent inspection system based on AR glasses as claimed in claim 1, characterized in that: The AR interaction module uses real-time images captured by the camera combined with computer vision algorithms to locate hidden dangers, so that the information is superimposed on the actual scene; AR glasses integrate voice recognition and motion sensors to enable inspectors to quickly receive instructions and perform operations through voice commands or gestures.

Images