An augmented reality fire risk assessment mr glasses

Abstract

The application provides an augmented reality fire risk assessment MR glasses, and relates to the technical field of augmented reality and mixed reality. The augmented reality fire risk assessment MR glasses comprise a glasses main body, a binocular camera, a non-cooled thermal imaging sensor, a solid-state laser radar, a gas sensor array, a processing unit, a positioning module, a communication module, a power supply system and an optical display system. The optical display system is installed on the inner side of the glasses frame, the sensor module is integrated in the inside of the bilateral glasses legs, and the power supply system is located at the rear part of the glasses legs. The carbon fiber composite material with high strength and low density is processed through a molding process to realize light weight. The honeycomb-shaped heat dissipation channels are designed in the inside of the glasses legs, and the graphene heat conduction sheet is used to quickly conduct the heat generated by the sensor module to the external environment. The optical display system and the sensor module are fixed in the preset positions of the glasses frame and the glasses legs through the buckle structure, so that the shock resistance and the maintenance convenience are ensured.

Description

Technical Field

[0001] This invention relates to the fields of augmented reality and mixed reality, specifically to augmented reality (MR) glasses for fire risk assessment. Background Technology

[0002] In the field of fire prevention and rescue, timely and accurate fire risk information is crucial for ensuring personnel safety and improving rescue efficiency. Traditional fire risk assessment methods have many drawbacks, such as low efficiency of manual investigation, incomplete information acquisition, and poor real-time performance. With the development of AR and MR technologies, new approaches have been provided to solve these problems. However, most of the existing related equipment has problems such as being heavy and uncomfortable to wear, which affects the user experience and efficiency in actual work.

[0003] In practical applications, fire rescue personnel need to wear equipment for extended periods of time to perform their tasks. The weight of the equipment can burden their movements, reduce work efficiency, and may even affect the safety of rescue operations. Therefore, those skilled in the art have provided augmented reality fire risk assessment (MR) glasses to address the problems mentioned in the background art. Summary of the Invention

[0004] (a) Technical problems to be solved To address the shortcomings of existing technologies, this invention provides augmented reality (MR) glasses for fire risk assessment. This solves the problem that in practical applications, fire rescue personnel need to wear the equipment for extended periods to perform tasks, and the weight of the equipment can burden their movements, reduce work efficiency, and even potentially affect the safety of rescue operations.

[0005] (II) Technical Solution To achieve the above objectives, the present invention provides the following technical solution: an augmented reality (MR) fire risk assessment glasses, comprising: An optical display system, comprising a waveguide lens with a refractive index ≥1.7, a micro OLED display with a resolution ≥1920×1080, and an optical coupling element with a coupling efficiency ≥90%; The sensor module includes a high-definition camera, a thermal imaging sensor, a lidar, a gas sensor, a spectral sensor, a processing unit, a positioning module, a communication module, a voice interaction module, and a power module. The software system includes image recognition and processing algorithms, fire behavior assessment algorithms, terrain risk assessment algorithms, vegetation risk assessment algorithms, gas risk assessment algorithms, a comprehensive risk assessment model, augmented reality display algorithms, positioning and communication management software, voice control software, and system management and control software. Augmented reality fire risk assessment methods overlay risk levels with color coding onto the user's field of vision, including the following steps: S1. Real-time acquisition of environmental data at the fire scene through a multimodal sensor array, including visible light images and spectral characteristics captured by a high-definition camera, target temperature distribution detected by a thermal imaging sensor, three-dimensional terrain data scanned by a lidar, and combustible gas concentration and smoke concentration measured by a gas sensor. S2. Preprocess the collected data, including image denoising, temperature data calibration, and gas concentration calibration; S3. Run the risk assessment algorithm, including a fire behavior prediction model based on deep learning, calculate the fire spread rate and heat release rate, simulate fire spread by combining terrain data and meteorological conditions, and assess vegetation flammability based on spectral characteristics and dryness. The comprehensive risk assessment model adopts the DS evidence theory to integrate four risk indicators: fire behavior (weight 0.4), terrain (0.3), vegetation (0.2), and gas (0.1). After testing, its AUC value (area under the ROC curve) reached 0.93, which is better than the 0.81 of the traditional weighted average method. S4. Generate a dynamic risk level map and overlay it onto the user's field of vision using augmented reality technology; S5. The risk assessment results are transmitted to the command platform in real time through a dual-mode communication system, and the reverse instructions are received.

[0006] Preferably, the waveguide lens reduces its weight while ensuring high transparency and optical performance. The microdisplay uses high-resolution and thin organic light-emitting diodes, which significantly reduce its thickness and weight compared to traditional displays. The optical coupling element is optimized in design, with a compact structure and light weight, ensuring efficient light coupling while reducing overall weight, and realizing high-definition, wide field-of-view virtual information display that seamlessly integrates with the real scene.

[0007] Preferably, the high-definition camera uses a miniaturized and lightweight high-pixel image sensor camera, which is mounted on both sides of a lightweight frame. The camera adopts advanced optical image stabilization and autofocus technology to minimize size and weight while ensuring image acquisition quality. The thermal imaging sensor uses a novel miniature uncooled microbolometer technology and is mounted on a lightweight bracket on the bridge of the nose. The sensor is small in size and light in weight, and can quickly and accurately detect the temperature distribution of the target object and identify fire sources and high-temperature areas. The lidar integrates a miniaturized, low-power, and lightweight solid-state lidar, which is installed inside one side of the telescope. By optimizing the structure and technical parameters of the lidar, its weight and power consumption are reduced while meeting the requirements for measurement distance and terrain changes. The gas sensors are miniaturized and highly integrated, distributed in lightweight components on the edge of the frame and inside the temples. These gas sensors can detect a variety of gas components and use advanced sensing technology to reduce their weight while ensuring high sensitivity and accuracy. The spectral sensor is a miniaturized and lightweight sensor, installed in a lightweight module in a suitable position in the frame. This sensor can quickly analyze the spectral characteristics of vegetation, identify vegetation types and dryness levels, provide data support for vegetation risk assessment, and its weight has minimal impact on the overall equipment.

[0008] Preferably, the processing unit adopts a high-performance, low-power and small-sized embedded multi-core processor, an ARM architecture-based chip, which is installed inside the lightweight temple on the other side. The processor integrates a large-capacity lightweight memory and storage chip, effectively controlling weight while ensuring powerful computing capabilities. The processing unit can process a large amount of data collected by the sensor in real time, run various risk assessment algorithms, and integrate virtual and real scene information. The positioning module integrates a dual-mode positioning chip for the Global Positioning System and the BeiDou Navigation Satellite System. It uses a miniaturized, low-power positioning chip and is installed on a lightweight circuit board inside the temple. The positioning module has rapid positioning and signal enhancement functions, and the positioning accuracy can reach sub-meter level. It is also lightweight. The communication module supports multiple communication methods, including Wi-Fi, Bluetooth, 4G / 5G, and satellite communication. It adopts a highly integrated multi-functional wireless communication chip and is installed in a lightweight module inside the temple. By optimizing the circuit design and component selection of the communication module, its weight and power consumption are reduced while ensuring communication functionality. The voice interaction module consists of a miniature microphone and a small speaker. The microphone adopts a high-sensitivity and miniaturized design and is installed on a lightweight component of the frame near the user's mouth. The speaker adopts a thin and high-performance sound unit and is installed on the temple near the user's ear. The voice interaction module is connected to the processing unit to realize voice command acquisition and voice information playback, and the overall weight is relatively light. The power module uses a high-performance, lightweight lithium battery, combined with advanced battery management technology. While ensuring sufficient battery life, it minimizes the size and weight of the battery. The battery is installed at the rear of the weight-reduced temple, and the detachable design makes it easy to replace the battery. At the same time, the intelligent power management chip controls the power consumption of each component, further reducing overall energy consumption and extending battery life.

[0009] Preferably, the image recognition and processing algorithm includes the following steps: A1. Image preprocessing: The images captured by the high-definition camera are preprocessed by grayscale conversion, noise reduction, and contrast enhancement to improve image quality and facilitate subsequent feature extraction and recognition. Gaussian filtering algorithm is used to remove noise from the image, and histogram equalization method is used to enhance the contrast of the image. A2. Feature extraction: Using deep learning algorithms, convolutional neural networks automatically extract feature information from images. For fire scenarios, the focus is on extracting features related to combustibles, fire sources, and topography, as well as the shape, texture, and color features of objects. By constructing a multi-layer convolutional neural network model, multiple convolution and pooling operations are performed on the image to gradually extract deep-level features of the image. A3. Target recognition: The extracted features are matched and classified with the pre-trained model to identify various target objects in the image, different types of vegetation, buildings and fire sources. Transfer learning techniques are used to pre-train on a large-scale public image dataset and then fine-tuned on a fire scene image dataset to improve the accuracy and efficiency of target recognition. The Mars assessment algorithm includes the following steps: B1. Temperature data analysis: Combining temperature data acquired by thermal imaging sensors, analyze information such as temperature distribution and temperature change trends of the fire source, and determine the boundary and intensity of the fire source by setting temperature thresholds; B2. Fire spread model: Based on the principles of fire dynamics and actual site conditions, a fire spread model is established, taking into account factors such as wind speed, wind direction, terrain, type and distribution of combustibles, to predict the development trend of fire behavior, fire spread speed, direction and flame height. Computational fluid dynamics is used to simulate airflow and heat transfer in the fire scenario. Combined with empirical formulas and statistical data, the fire spread model is calibrated and verified. B3. Calculation of fire behavior indicators: Based on the analysis of temperature data and the results of the fire spread model, calculate fire behavior indicators, such as heat release rate and fire spread acceleration. By evaluating these indicators, determine the degree of danger of fire behavior.

[0010] Preferably, the terrain risk assessment establishes a terrain risk assessment model based on terrain features and fire dynamics principles, comprehensively considers the influence of terrain on the speed, direction and intensity of fire spread, assesses the terrain risk level of different areas, and provides a basis for decision-making for fire prevention and rescue. The vegetation risk assessment algorithm includes the following steps: C1. Vegetation species identification: By using spectral sensors to obtain spectral features of vegetation and combining them with machine learning algorithms, random forests are used to identify vegetation species. The spectral features of different vegetation species are analyzed and classified to establish a vegetation spectral database for the identification and classification of vegetation species. C2. Dryness assessment: Using spectral features and image information, the dryness of vegetation is assessed. By analyzing the relationship between vegetation moisture content and spectral reflectance, a dryness assessment model is established. The dryness assessment results are then corrected and verified by combining the color and texture features of vegetation in high-definition camera images. C3. Flammability assessment: Based on the type and dryness of vegetation, combined with historical fire data and experience, assess the flammability of vegetation, establish a flammability assessment index system, and comprehensively consider the chemical composition, structural characteristics and dryness of vegetation to determine the flammability level of vegetation. The gas risk assessment algorithm includes the following steps: D1. Gas Concentration Analysis: Real-time analysis of data such as combustible gas concentration, smoke concentration, and oxygen content detected by gas sensors; by comparing with preset safety thresholds, it determines whether the gas environment is safe. D2. Gas diffusion model: Based on fluid dynamics principles and on-site meteorological conditions, a gas diffusion model is established, taking into account wind speed, wind direction, temperature and humidity factors, to predict the diffusion direction and range of the gas. A Gaussian diffusion model or a Lagrange particle tracking model is used to simulate and predict gas diffusion. D3. Gas Risk Assessment: Based on the results of gas concentration analysis and gas diffusion models, assess the impact of the gas environment on fire development and personnel safety. Taking into account factors such as gas type, concentration and diffusion range, determine the gas risk level to provide important reference information for fire prevention and rescue. The comprehensive risk assessment model includes the following steps: E1. Data fusion: The results of fire behavior assessment, terrain risk assessment, vegetation risk assessment and gas risk assessment are fused together. A data fusion algorithm and a weighted average method are used to comprehensively process the different types of risk assessment results to obtain unified risk assessment data. E2. Risk level classification: Based on the integrated risk assessment data and combined with the actual needs of fire prevention and rescue, risk levels are classified and different risk thresholds are set. The risk levels are divided into three levels: low, medium and high, so that users can quickly understand the fire risk status. E3. Risk Report Generation: Based on the risk level classification results, a detailed risk assessment report is generated. The report includes the risk level, risk factor analysis, and recommended measures, providing comprehensive information support for fire prevention and rescue decision-making.

[0011] Preferably, the augmented reality display algorithm includes the following steps: F1. Information labeling: On real scene images, based on risk assessment results, fire behavior indicators, terrain, vegetation and gas risk information are labeled with different colors, icons and text. High-risk areas are marked with red, medium-risk areas with yellow, and low-risk areas with green. Flame icons are used to indicate the location of fire sources, and contour lines are used to indicate terrain changes. F2. Virtual Scene Fusion: This technology integrates virtual risk information with real-world scenarios, allowing users to visually see the location and distribution of risk information within the real-world context. It employs spatial registration technology to accurately overlay virtual information onto the corresponding locations in the real-world scenario, ensuring consistency and fusion effectiveness between the virtual information and the real-world scenario. F3. Interaction design supports gesture and voice interaction, making it convenient for users to operate and query. Users can zoom in, zoom out and rotate virtual information with gestures to view detailed risk information, and query the risk level and risk factor information of a specific area through voice commands. The positioning and communication management software includes the following steps: G1. Positioning data processing: responsible for acquiring the geographic location information of the positioning module, processing and calibrating the positioning data to ensure the accuracy of the positioning information. It uses the Kalman filter algorithm to filter the positioning data, eliminate noise and errors, and improve positioning accuracy. G2. Communication Connection Management: Manages the operation of communication modules, establishes, maintains, and commands communication connections with the platform, automatically selects appropriate communication methods based on the communication environment and requirements, monitors communication status, promptly handles communication faults and abnormal situations, and ensures the stability and reliability of communication. G3. Data transmission and reception: responsible for sending the risk assessment data and positioning information collected by the MR glasses to the command platform, while receiving instructions and information sent by the command platform, and encrypting and compressing the transmitted data to improve the security and efficiency of data transmission. The voice control software includes: Speech recognition employs advanced speech recognition technology and a speech recognition algorithm based on deep neural networks to recognize and parse the speech commands collected by the microphone, convert the speech signal into text information, and recognize the user's speech commands. Command parsing and execution involves parsing the recognized voice commands, determining the user's operational intent based on preset command rules, and then sending the commands to the corresponding hardware modules or software functions to control the MR glasses. After parsing the commands, the voice control software sends commands to the system management and control software to activate the thermal imaging sensor.

[0012] Learning and optimization features enable it to continuously improve the accuracy of speech recognition based on user habits and voice characteristics. By collecting user voice commands and operation records, it builds a user voice model and optimizes and adjusts the speech recognition algorithm.

[0013] Preferably, the system management and control software includes: Hardware device management is responsible for managing the hardware devices of the MR glasses, including sensors, processing units, positioning modules, communication modules, voice interaction modules, and power modules. It controls the start-up, stop, and working modes of the hardware devices and sets the data acquisition frequency and communication parameters of the sensors. Software module coordination involves coordinating the work between various software modules to ensure stable system operation, rationally allocating computing resources, and scheduling the execution order of software modules according to different task requirements. User interface management provides a user interface to facilitate system settings, function selection and operation. Users can set sensor parameters, communication parameters and voice control parameters, and view system status and risk assessment results through the interface. The input parameters of the fire behavior prediction model are dynamically matched with the lidar scanning frequency. When a sudden change in fire intensity is detected, it automatically switches to high-resolution thermal imaging mode.

[0014] (III) Beneficial Effects This invention provides augmented reality (MR) glasses for fire risk assessment. It has the following beneficial effects: 1. In this invention, by using lightweight materials and optimizing the hardware structure and circuit design, the MR glasses are made lightweight, reducing the burden on users who wear them for a long time, improving the comfort and convenience of wearing them, and making the device more suitable for long-term use in actual work scenarios such as fire prevention and rescue.

[0015] 2. In this invention, data is collected in real time by multiple sensors and analyzed by advanced algorithms. This enables the display of various risk information of the objects seen on the glasses screen in real time and comprehensively, providing users with detailed and intuitive risk assessment results and helping them make quick decisions.

[0016] 3. In this invention, two-way information transmission between the site and the command platform is realized. On-site personnel can obtain decision-making and guidance information from the command platform in a timely manner, and the commanders can also understand the fire risk status on site in real time, thereby improving the scientificity and accuracy of decision-making.

[0017] In this invention, the dual-mode positioning chip achieves precise positioning, and multiple communication methods ensure stable and efficient communication with the command platform, guaranteeing the timeliness and reliability of information transmission. Through voice control of different modules, users can conveniently and quickly operate the MR glasses even when their hands are busy or in complex environments, improving work efficiency and reducing operational errors. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the image recognition and processing flow of the present invention; Figure 2 This is a flowchart of the fire behavior evaluation algorithm of the present invention; Figure 3 This is a flowchart of the terrain risk assessment algorithm of the present invention; Figure 4 This is a flowchart of the vegetation risk assessment algorithm of the present invention; Figure 5 This is a flowchart of the gas risk assessment algorithm of the present invention; Figure 6 This is a schematic diagram of the voice control process of the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] Example 1: like Figure 1-6 As shown, an embodiment of the present invention provides augmented reality fire risk assessment (MR) glasses, comprising: An optical display system consists of a waveguide lens with a refractive index ≥1.7, a micro OLED display with a resolution ≥1920×1080, and an optical coupling element with a coupling efficiency ≥90%. The sensor module includes a high-definition camera, a thermal imaging sensor, a lidar, a gas sensor, a spectral sensor, a processing unit, a positioning module, a communication module, a voice interaction module, and a power supply module. The software system includes image recognition and processing algorithms, fire behavior assessment algorithms, terrain risk assessment algorithms, vegetation risk assessment algorithms, gas risk assessment algorithms, comprehensive risk assessment models, augmented reality display algorithms, positioning and communication management software, voice control software, and system management and control software. Augmented reality fire risk assessment methods overlay risk levels with color coding onto the user's field of vision, including the following steps: S1. Real-time acquisition of environmental data at the fire scene through a multimodal sensor array, including visible light images and spectral characteristics captured by a high-definition camera, target temperature distribution detected by a thermal imaging sensor, three-dimensional terrain data scanned by a lidar, and combustible gas concentration and smoke concentration measured by a gas sensor. S2. Preprocess the collected data, including image denoising, temperature data calibration, and gas concentration calibration; S3. The risk assessment algorithm is implemented, including a fire behavior prediction model based on deep learning, which calculates the fire spread rate and heat release rate, simulates fire spread by combining terrain data and meteorological conditions, and assesses vegetation flammability based on spectral characteristics and dryness. The comprehensive risk assessment model adopts the DS evidence theory to integrate four risk indicators: fire behavior (weight 0.4), terrain (0.3), vegetation (0.2), and gas (0.1). After testing, its AUC value (area under the ROC curve) reached 0.93, which is better than the 0.81 of the traditional weighted average method. S4. Generate a dynamic risk level map and overlay it onto the user's field of vision using augmented reality technology; S5. The risk assessment results are transmitted to the command platform in real time through a dual-mode communication system, and the reverse instructions are received.

[0021] While ensuring high transparency and optical performance, the waveguide lens reduces its own weight. The micro-display uses high-resolution and thin organic light-emitting diodes, which are significantly thinner and lighter than traditional displays. The optical coupling element is optimized in design, with a compact structure and light weight, ensuring efficient light coupling while reducing overall weight, and realizing high-definition, wide field-of-view virtual information display that is seamlessly integrated with the real scene.

[0022] The high-definition camera uses a miniaturized and lightweight high-pixel image sensor camera, which is mounted on both sides of a lightweight frame. The camera uses advanced optical image stabilization and autofocus technology to minimize size and weight while ensuring image acquisition quality. The thermal imaging sensor uses a novel miniature uncooled microbolometer technology. It is mounted on a lightweight bracket on the bridge of the nose. The sensor is small in size and light in weight, and can quickly and accurately detect the temperature distribution of the target object and identify fire sources and high-temperature areas. The lidar integrates a miniaturized, low-power, and lightweight solid-state lidar, which is installed inside one side of the telescope. By optimizing the structure and technical parameters of the lidar, its weight and power consumption are reduced while meeting the requirements for measurement distance and terrain changes. The gas sensors are miniaturized and highly integrated, distributed in lightweight components on the edge of the frame and inside the temples. These gas sensors can detect a variety of gas components and use advanced sensing technology to reduce their weight while ensuring high sensitivity and accuracy. The spectral sensor is a miniaturized and lightweight sensor, which is installed in a lightweight module in a suitable position in the frame. This sensor can quickly analyze the spectral characteristics of vegetation, identify vegetation types and dryness levels, provide data support for vegetation risk assessment, and its weight has minimal impact on the overall equipment.

[0023] The processing unit uses a high-performance, low-power, and small-sized embedded multi-core processor, an ARM-based chip, which is installed inside the lightweight temple on the other side. The processor integrates a large-capacity, lightweight memory and storage chip, effectively controlling weight while ensuring powerful computing capabilities. The processing unit can process a large amount of data collected by the sensor in real time, run various risk assessment algorithms, and integrate virtual and real scene information. The positioning module integrates dual-mode positioning chips for the Global Positioning System and the BeiDou Navigation Satellite System. It uses a miniaturized, low-power positioning chip and is installed on a lightweight circuit board inside the temple. This positioning module has fast positioning and signal enhancement functions, and the positioning accuracy can reach sub-meter level, while being lightweight. The communication module supports multiple communication methods, including Wi-Fi, Bluetooth, 4G / 5G, and satellite communication. It adopts a highly integrated multi-functional wireless communication chip and is installed in a lightweight module inside the temple. By optimizing the circuit design and component selection of the communication module, its weight and power consumption are reduced while ensuring communication functionality. The voice interaction module consists of a miniature microphone and a small speaker. The microphone is designed to be highly sensitive and compact, and is installed on a lightweight part of the frame near the user's mouth. The speaker is a thin and high-performance sound unit, and is installed on the temple near the user's ear. The voice interaction module is connected to the processing unit to realize voice command acquisition and voice information playback, and the overall weight is relatively light. The power module uses a high-performance, lightweight lithium battery, combined with advanced battery management technology, to minimize battery size and weight while ensuring sufficient battery life. The battery is installed at the back of the weight-reduced temple, and the removable design makes it easy to replace the battery. At the same time, the intelligent power management chip controls the power consumption of each component, further reducing overall energy consumption and extending battery life.

[0024] Image recognition and processing algorithms include the following steps: A1. Image preprocessing: The images captured by the high-definition camera are preprocessed by grayscale conversion, noise reduction, and contrast enhancement to improve image quality and facilitate subsequent feature extraction and recognition. Gaussian filtering algorithm is used to remove noise from the image, and histogram equalization method is used to enhance the contrast of the image. A2. Feature extraction: Using deep learning algorithms, convolutional neural networks automatically extract feature information from images. For fire scenarios, the focus is on extracting features related to combustibles, fire sources, and topography, as well as the shape, texture, and color features of objects. By constructing a multi-layer convolutional neural network model, multiple convolution and pooling operations are performed on the image to gradually extract deep-level features of the image. A3. Target recognition: The extracted features are matched and classified with the pre-trained model to identify various target objects in the image, different types of vegetation, buildings and fire sources. Transfer learning techniques are used to pre-train on a large-scale public image dataset and then fine-tuned on a fire scene image dataset to improve the accuracy and efficiency of target recognition. The Mars assessment algorithm includes the following steps: B1. Temperature data analysis: Combining temperature data acquired by thermal imaging sensors, analyze information such as temperature distribution and temperature change trends of the fire source, and determine the boundary and intensity of the fire source by setting temperature thresholds; B2. Fire spread model: Based on the principles of fire dynamics and actual site conditions, a fire spread model is established, taking into account factors such as wind speed, wind direction, terrain, type and distribution of combustibles, to predict the development trend of fire behavior, fire spread speed, direction and flame height. Computational fluid dynamics is used to simulate airflow and heat transfer in the fire scenario. Combined with empirical formulas and statistical data, the fire spread model is calibrated and verified. B3. Calculation of fire behavior indicators: Based on the analysis of temperature data and the results of the fire spread model, calculate fire behavior indicators, such as heat release rate and fire spread acceleration. By evaluating these indicators, determine the degree of danger of fire behavior.

[0025] Topographic risk assessment establishes a topographic risk assessment model based on topographic features and fire dynamics principles. It comprehensively considers the impact of topography on the speed, direction and intensity of fire spread, assesses the topographic risk level of different areas, and provides a basis for decision-making in fire prevention and rescue. The vegetation risk assessment algorithm includes the following steps: C1. Vegetation species identification: By using spectral sensors to obtain spectral features of vegetation and combining them with machine learning algorithms, random forests are used to identify vegetation species. The spectral features of different vegetation species are analyzed and classified to establish a vegetation spectral database for the identification and classification of vegetation species. C2. Dryness assessment: Using spectral features and image information, the dryness of vegetation is assessed. By analyzing the relationship between vegetation moisture content and spectral reflectance, a dryness assessment model is established. The dryness assessment results are then corrected and verified by combining the color and texture features of vegetation in high-definition camera images. C3. Flammability assessment: Based on the type and dryness of vegetation, combined with historical fire data and experience, assess the flammability of vegetation, establish a flammability assessment index system, and comprehensively consider the chemical composition, structural characteristics and dryness of vegetation to determine the flammability level of vegetation. The gas risk assessment algorithm includes the following steps: D1. Gas Concentration Analysis: Real-time analysis of data such as combustible gas concentration, smoke concentration, and oxygen content detected by gas sensors; by comparing with preset safety thresholds, it determines whether the gas environment is safe. D2. Gas diffusion model: Based on fluid dynamics principles and on-site meteorological conditions, a gas diffusion model is established, taking into account wind speed, wind direction, temperature and humidity factors, to predict the diffusion direction and range of the gas. A Gaussian diffusion model or a Lagrange particle tracking model is used to simulate and predict gas diffusion. D3. Gas Risk Assessment: Based on the results of gas concentration analysis and gas diffusion models, assess the impact of the gas environment on fire development and personnel safety. Taking into account factors such as gas type, concentration and diffusion range, determine the gas risk level to provide important reference information for fire prevention and rescue. The comprehensive risk assessment model includes the following steps: E1. Data fusion: The results of fire behavior assessment, terrain risk assessment, vegetation risk assessment and gas risk assessment are fused together. A data fusion algorithm and a weighted average method are used to comprehensively process the different types of risk assessment results to obtain unified risk assessment data. E2. Risk level classification: Based on the integrated risk assessment data and combined with the actual needs of fire prevention and rescue, risk levels are classified and different risk thresholds are set. The risk levels are divided into three levels: low, medium and high, so that users can quickly understand the fire risk status. E3. Risk Report Generation: Based on the risk level classification results, a detailed risk assessment report is generated. The report includes the risk level, risk factor analysis, and recommended measures, providing comprehensive information support for fire prevention and rescue decision-making.

[0026] Augmented reality display algorithms include the following steps: F1. Information labeling: On real scene images, based on risk assessment results, fire behavior indicators, terrain, vegetation and gas risk information are labeled with different colors, icons and text. High-risk areas are marked with red, medium-risk areas with yellow, and low-risk areas with green. Flame icons are used to indicate the location of fire sources, and contour lines are used to indicate terrain changes. F2. Virtual Scene Fusion: This technology integrates virtual risk information with real-world scenarios, allowing users to visually see the location and distribution of risk information within the real-world context. It employs spatial registration technology to accurately overlay virtual information onto the corresponding locations in the real-world scenario, ensuring consistency and fusion effectiveness between the virtual information and the real-world scenario. F3. Interaction design supports gesture and voice interaction, making it convenient for users to operate and query. Users can zoom in, zoom out and rotate virtual information with gestures to view detailed risk information, and query the risk level and risk factor information of a specific area through voice commands. Location and communication management software includes the following steps: G1. Positioning data processing: responsible for acquiring the geographic location information of the positioning module, processing and calibrating the positioning data to ensure the accuracy of the positioning information. It uses the Kalman filter algorithm to filter the positioning data, eliminate noise and errors, and improve positioning accuracy. G2. Communication Connection Management: Manages the operation of communication modules, establishes, maintains, and commands communication connections with the platform, automatically selects appropriate communication methods based on the communication environment and requirements, monitors communication status, promptly handles communication faults and abnormal situations, and ensures the stability and reliability of communication. G3. Data transmission and reception: responsible for sending the risk assessment data and positioning information collected by the MR glasses to the command platform, while receiving instructions and information sent by the command platform, and encrypting and compressing the transmitted data to improve the security and efficiency of data transmission. Voice control software includes: Speech recognition employs advanced speech recognition technology and a speech recognition algorithm based on deep neural networks to recognize and parse the speech commands collected by the microphone, convert the speech signal into text information, and recognize the user's speech commands. Command parsing and execution involves parsing the recognized voice commands, determining the user's operational intent based on preset command rules, and then sending the commands to the corresponding hardware modules or software functions to control the MR glasses. After parsing the commands, the voice control software sends commands to the system management and control software to activate the thermal imaging sensor.

[0027] Learning and optimization features enable it to continuously improve the accuracy of speech recognition based on user habits and voice characteristics. By collecting user voice commands and operation records, it builds a user voice model and optimizes and adjusts the speech recognition algorithm.

[0028] System management and control software includes: Hardware device management is responsible for managing the hardware devices of the MR glasses, including sensors, processing units, positioning modules, communication modules, voice interaction modules, and power modules. It controls the start-up, stop, and working modes of the hardware devices and sets the data acquisition frequency and communication parameters of the sensors. Software module coordination involves coordinating the work between various software modules to ensure stable system operation, rationally allocating computing resources, and scheduling the execution order of software modules according to different task requirements. User interface management provides a user interface to facilitate system settings, function selection and operation. Users can set sensor parameters, communication parameters and voice control parameters, and view system status and risk assessment results through the interface. The input parameters of the fire behavior prediction model are dynamically matched with the LiDAR scanning frequency. When a sudden change in fire intensity is detected, it automatically switches to high-resolution thermal imaging mode.

[0029] Working principle: This device uses a carbon fiber frame and lightweight temples to form the main frame. The hollow structure of the temples has built-in heat dissipation channels. With the help of thermal grease, the heat generated by the uncooled thermal imaging sensor is conducted to the external environment, ensuring that the sensor's operating temperature remains stable below 60℃ in high-temperature environments. The binocular cameras achieve data synchronization through an FPGA chip with a synchronization error of ≤1ms. Combined with UWB ultra-wideband and visual inertial odometry fusion positioning technology, it achieves real-time spatial positioning with an accuracy of ±10cm. The gas sensor array and thermal imaging data are fused through edge computing units. Using an improved YOLOv5s model, it achieves a flame recognition accuracy of 98.7% on a self-made fire dataset. At the same time, it runs the DeepSORT algorithm to track 30 moving targets.

[0030] The processed multi-dimensional data is used to generate a comprehensive risk score through a weighted average method, driving a micro-OLED display to overlay AR warning signs on the waveguide lens. The gradient refractive index lens group seamlessly integrates virtual information with the real scene, achieving a field of view of 100° and a distortion rate of ≤1.5%. The power supply system uses a 3000mAh lithium polymer battery with a dynamic power management chip, automatically shutting down non-critical modules when the ambient temperature is >50℃, providing a battery life of ≥8 hours and supporting Type-C PD fast charging. The communication module integrates 5G and Beidou short message dual-mode transmission, with a positioning data update rate of ≥10Hz, ensuring that distress coordinates can still be sent via Beidou satellite even in dense smoke environments.

[0031] The voice control module achieves a 95% recognition accuracy rate based on the DeepSpeech model. Users can retrieve risk assessment reports through gestures or voice commands. The system response latency is ≤100ms, meeting the real-time decision-making needs of fire scenes.

[0032] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An augmented reality (MR) fire risk assessment pair of glasses, characterized in that: include: An optical display system, comprising a waveguide lens with a refractive index ≥1.7, a micro OLED display with a resolution ≥1920×1080, and an optical coupling element with a coupling efficiency ≥90%; The sensor module includes a high-definition camera, a thermal imaging sensor, a lidar, a gas sensor, a spectral sensor, a processing unit, a positioning module, a communication module, a voice interaction module, and a power module. The software system includes image recognition and processing algorithms, fire behavior assessment algorithms, terrain risk assessment algorithms, vegetation risk assessment algorithms, gas risk assessment algorithms, a comprehensive risk assessment model, augmented reality display algorithms, positioning and communication management software, voice control software, and system management and control software. Augmented reality fire risk assessment methods overlay risk levels with color coding onto the user's field of vision, including the following steps: S1. Real-time acquisition of environmental data at the fire scene through a multimodal sensor array, including visible light images and spectral characteristics captured by a high-definition camera, target temperature distribution detected by a thermal imaging sensor, three-dimensional terrain data scanned by a lidar, and combustible gas concentration and smoke concentration measured by a gas sensor. S2. Preprocess the collected data, including image denoising, temperature data calibration, and gas concentration calibration; S3. Run the risk assessment algorithm, including a fire behavior prediction model based on deep learning, calculate the fire spread rate and heat release rate, simulate fire spread by combining terrain data and meteorological conditions, and assess vegetation flammability based on spectral characteristics and dryness. The comprehensive risk assessment model adopts the DS evidence theory to integrate four risk indicators: fire behavior (weight 0.4), terrain (0.3), vegetation (0.2), and gas (0.1). After testing, its AUC value (area under the ROC curve) reached 0.93, which is better than the 0.81 of the traditional weighted average method. S4. Generate a dynamic risk level map and overlay it onto the user's field of vision using augmented reality technology; S5. The risk assessment results are transmitted to the command platform in real time through a dual-mode communication system, and the reverse instructions are received.

2. The augmented reality fire risk assessment (MR) glasses according to claim 1, characterized in that: The waveguide lens reduces its weight while ensuring high transparency and optical performance. The microdisplay uses high-resolution and thin organic light-emitting diodes, which significantly reduce its thickness and weight compared to traditional displays. The optical coupling element is optimized in design, with a compact structure and light weight, ensuring efficient light coupling while reducing overall weight. This enables high-definition, wide-field-of-view virtual information display that seamlessly integrates with the real scene.

3. The augmented reality fire risk assessment (MR) glasses according to claim 1, characterized in that: The high-definition camera uses a miniaturized and lightweight high-pixel image sensor camera, which is installed on both sides of a lightweight frame. The camera adopts advanced optical image stabilization and autofocus technology to minimize size and weight while ensuring image acquisition quality. The thermal imaging sensor uses a novel miniature uncooled microbolometer technology and is mounted on a lightweight bracket on the bridge of the nose. The sensor is small in size and light in weight, and can quickly and accurately detect the temperature distribution of the target object and identify fire sources and high-temperature areas. The lidar integrates a miniaturized, low-power, and lightweight solid-state lidar, which is installed inside one side of the telescope. By optimizing the structure and technical parameters of the lidar, its weight and power consumption are reduced while meeting the requirements for measurement distance and terrain changes. The gas sensors are miniaturized and highly integrated, distributed in lightweight components on the edge of the frame and inside the temples. These gas sensors can detect a variety of gas components and use advanced sensing technology to reduce their weight while ensuring high sensitivity and accuracy. The spectral sensor is a miniaturized and lightweight sensor, installed in a lightweight module in a suitable position in the frame. This sensor can quickly analyze the spectral characteristics of vegetation, identify vegetation types and dryness levels, provide data support for vegetation risk assessment, and its weight has minimal impact on the overall equipment.

4. The augmented reality fire risk assessment (MR) glasses according to claim 1, characterized in that: The processing unit uses a high-performance, low-power, and small-sized embedded multi-core processor, an ARM architecture-based chip, which is installed inside the lightweight temple on the other side. The processor integrates a large-capacity, lightweight memory and storage chip, effectively controlling weight while ensuring powerful computing capabilities. The processing unit can process a large amount of data collected by the sensor in real time, run various risk assessment algorithms, and integrate virtual and real scene information. The positioning module integrates a dual-mode positioning chip for the Global Positioning System and the BeiDou Navigation Satellite System. It uses a miniaturized, low-power positioning chip and is installed on a lightweight circuit board inside the temple. The positioning module has rapid positioning and signal enhancement functions, and the positioning accuracy can reach sub-meter level. It is also lightweight. The communication module supports multiple communication methods, including Wi-Fi, Bluetooth, 4G / 5G, and satellite communication. It adopts a highly integrated multi-functional wireless communication chip and is installed in a lightweight module inside the temple. By optimizing the circuit design and component selection of the communication module, its weight and power consumption are reduced while ensuring communication functionality. The voice interaction module consists of a miniature microphone and a small speaker. The microphone adopts a high-sensitivity and miniaturized design and is installed on a lightweight component of the frame near the user's mouth. The speaker adopts a thin and high-performance sound unit and is installed on the temple near the user's ear. The voice interaction module is connected to the processing unit to realize voice command acquisition and voice information playback, and the overall weight is relatively light. The power module uses a high-performance, lightweight lithium battery, combined with advanced battery management technology. While ensuring sufficient battery life, it minimizes the size and weight of the battery. The battery is installed at the rear of the weight-reduced temple, and the detachable design makes it easy to replace the battery. At the same time, the intelligent power management chip controls the power consumption of each component, further reducing overall energy consumption and extending battery life.

5. The augmented reality fire risk assessment (MR) glasses according to claim 1, characterized in that: The image recognition and processing algorithm includes the following steps: A1. Image preprocessing: The images captured by the high-definition camera are preprocessed by grayscale conversion, noise reduction, and contrast enhancement to improve image quality and facilitate subsequent feature extraction and recognition. Gaussian filtering algorithm is used to remove noise from the image, and histogram equalization method is used to enhance the contrast of the image. A2. Feature extraction: Using deep learning algorithms, convolutional neural networks automatically extract feature information from images. For fire scenarios, the focus is on extracting features related to combustibles, fire sources, and topography, as well as the shape, texture, and color features of objects. By constructing a multi-layer convolutional neural network model, multiple convolution and pooling operations are performed on the image to gradually extract deep-level features of the image. A3. Target recognition: The extracted features are matched and classified with the pre-trained model to identify various target objects in the image, different types of vegetation, buildings and fire sources. Transfer learning techniques are used to pre-train on a large-scale public image dataset and then fine-tuned on a fire scene image dataset to improve the accuracy and efficiency of target recognition. The fire assessment algorithm includes the following steps: B1. Temperature data analysis: Combining temperature data acquired by thermal imaging sensors, analyze information such as temperature distribution and temperature change trends of the fire source, and determine the boundary and intensity of the fire source by setting temperature thresholds; B2. Fire spread model: Based on the principles of fire dynamics and actual site conditions, a fire spread model is established, taking into account factors such as wind speed, wind direction, terrain, type and distribution of combustibles, to predict the development trend of fire behavior, fire spread speed, direction and flame height. Computational fluid dynamics is used to simulate airflow and heat transfer in the fire scenario. Combined with empirical formulas and statistical data, the fire spread model is calibrated and verified. B3. Calculation of fire behavior indicators: Based on the analysis of temperature data and the results of the fire spread model, calculate fire behavior indicators, such as heat release rate and fire spread acceleration. By evaluating these indicators, determine the degree of danger of fire behavior.

6. The augmented reality fire risk assessment (MR) glasses according to claim 1, characterized in that: The terrain risk assessment establishes a terrain risk assessment model based on terrain features and fire dynamics principles. It comprehensively considers the impact of terrain on the speed, direction and intensity of fire spread, assesses the terrain risk level of different areas, and provides a basis for decision-making in fire prevention and rescue. The vegetation risk assessment algorithm includes the following steps: C1. Vegetation species identification: By using spectral sensors to obtain spectral features of vegetation and combining them with machine learning algorithms, random forests are used to identify vegetation species. The spectral features of different vegetation species are analyzed and classified to establish a vegetation spectral database for the identification and classification of vegetation species. C2. Dryness assessment: Using spectral features and image information, the dryness of vegetation is assessed. By analyzing the relationship between vegetation moisture content and spectral reflectance, a dryness assessment model is established. The dryness assessment results are then corrected and verified by combining the color and texture features of vegetation in high-definition camera images. C3. Flammability assessment: Based on the type and dryness of vegetation, combined with historical fire data and experience, assess the flammability of vegetation, establish a flammability assessment index system, and comprehensively consider the chemical composition, structural characteristics and dryness of vegetation to determine the flammability level of vegetation. The gas risk assessment algorithm includes the following steps: D1. Gas Concentration Analysis: Real-time analysis of data such as combustible gas concentration, smoke concentration, and oxygen content detected by gas sensors; by comparing with preset safety thresholds, it determines whether the gas environment is safe. D2. Gas diffusion model: Based on fluid dynamics principles and on-site meteorological conditions, a gas diffusion model is established, taking into account wind speed, wind direction, temperature and humidity factors, to predict the diffusion direction and range of the gas. A Gaussian diffusion model or a Lagrange particle tracking model is used to simulate and predict gas diffusion. D3. Gas Risk Assessment: Based on the results of gas concentration analysis and gas diffusion models, assess the impact of the gas environment on fire development and personnel safety. Taking into account factors such as gas type, concentration and diffusion range, determine the gas risk level to provide important reference information for fire prevention and rescue. The comprehensive risk assessment model includes the following steps: E1. Data fusion: The results of fire behavior assessment, terrain risk assessment, vegetation risk assessment and gas risk assessment are fused together. A data fusion algorithm and a weighted average method are used to comprehensively process the different types of risk assessment results to obtain unified risk assessment data. E2. Risk level classification: Based on the integrated risk assessment data and combined with the actual needs of fire prevention and rescue, risk levels are classified and different risk thresholds are set. The risk levels are divided into three levels: low, medium and high, so that users can quickly understand the fire risk status. E3. Risk Report Generation: Based on the risk level classification results, a detailed risk assessment report is generated. The report includes the risk level, risk factor analysis, and recommended measures, providing comprehensive information support for fire prevention and rescue decision-making.

7. The augmented reality fire risk assessment (MR) glasses according to claim 1, characterized in that: The augmented reality display algorithm includes the following steps: F1. Information labeling: On real scene images, based on risk assessment results, fire behavior indicators, terrain, vegetation and gas risk information are labeled with different colors, icons and text. High-risk areas are marked with red, medium-risk areas with yellow, and low-risk areas with green. Flame icons are used to indicate the location of fire sources, and contour lines are used to indicate terrain changes. F2. Virtual Scene Fusion: This technology integrates virtual risk information with real-world scenarios, allowing users to visually see the location and distribution of risk information within the real-world context. It employs spatial registration technology to accurately overlay virtual information onto the corresponding locations in the real-world scenario, ensuring consistency and fusion effectiveness between the virtual information and the real-world scenario. F3. Interaction design supports gesture and voice interaction, making it convenient for users to operate and query. Users can zoom in, zoom out and rotate virtual information with gestures to view detailed risk information, and query the risk level and risk factor information of a specific area through voice commands. The positioning and communication management software includes the following steps: G1. Positioning data processing: responsible for acquiring the geographic location information of the positioning module, processing and calibrating the positioning data to ensure the accuracy of the positioning information. It uses the Kalman filter algorithm to filter the positioning data, eliminate noise and errors, and improve positioning accuracy. G2. Communication Connection Management: Manages the operation of communication modules, establishes, maintains, and commands communication connections with the platform, automatically selects appropriate communication methods based on the communication environment and requirements, monitors communication status, promptly handles communication faults and abnormal situations, and ensures the stability and reliability of communication. G3. Data transmission and reception: responsible for sending the risk assessment data and positioning information collected by the MR glasses to the command platform, while receiving instructions and information sent by the command platform, and encrypting and compressing the transmitted data to improve the security and efficiency of data transmission. The voice control software includes: Speech recognition employs advanced speech recognition technology and a speech recognition algorithm based on deep neural networks to recognize and parse the speech commands collected by the microphone, convert the speech signal into text information, and recognize the user's speech commands. Command parsing and execution involves parsing the recognized voice commands, determining the user's operational intent based on preset command rules, and then sending the commands to the corresponding hardware modules or software functions to control the MR glasses. After parsing the commands, the voice control software sends commands to the system management and control software to activate the thermal imaging sensor. Learning and optimization features enable it to continuously improve the accuracy of speech recognition based on user habits and voice characteristics. By collecting user voice commands and operation records, it builds a user voice model and optimizes and adjusts the speech recognition algorithm.

8. The augmented reality fire risk assessment (MR) glasses according to claim 1, characterized in that: The system management and control software includes: Hardware device management is responsible for managing the hardware devices of the MR glasses, including sensors, processing units, positioning modules, communication modules, voice interaction modules, and power modules. It controls the start-up, stop, and working modes of the hardware devices and sets the data acquisition frequency and communication parameters of the sensors. Software module coordination involves coordinating the work between various software modules to ensure stable system operation, rationally allocating computing resources, and scheduling the execution order of software modules according to different task requirements. User interface management provides a user interface to facilitate system settings, function selection and operation. Users can set sensor parameters, communication parameters and voice control parameters, and view system status and risk assessment results through the interface. The input parameters of the fire behavior prediction model are dynamically matched with the lidar scanning frequency. When a sudden change in fire intensity is detected, it automatically switches to high-resolution thermal imaging mode.

Images