Multifunctional display system and method of self-made AR optical structure

Abstract

The invention discloses a multifunctional display system and method of a self-made AR optical structure. The multifunctional display system comprises an AR fire fighting helmet main control module, the self-made AR optical structure, a sensor module, a gyroscope data processing module, a GPS processing module, a voice module, a wireless module, a communication module and a cloud platform. All the modules are initialized, and the AR fire-fighting helmet main control module receives data of various sensors; the AR fire fighting helmet main control module based on the FreeRTOS embedded operating system synchronously reads and processes data transmitted by the peripheral module in real time; carrying out optimization processing on the data by utilizing a filtering algorithm; storing the current data so as to compare the current data with the previous data; and finally, a result processed by the AR fire-fighting helmet main control module is output to the self-made AR optical structure. According to the invention, the problems of possible image flipping and insufficient focal length when the light directly irradiates the semitransparent mirror are solved, it is ensured that a person sees an upright and clear image, the cost is low, and the application field is wide.

Description

Technical Field

[0001] The present invention relates to a display system and method, and in particular to a multifunctional display system and method of a self-made AR optical structure. Background Art

[0002] In the field of firefighting and rescue, traditional helmets have relatively simple functions. Due to their simple structure, they are sometimes unable to complete efficient search and rescue in complex environments, resulting in low rescue efficiency.

[0003] In the field of smart irrigation, when faced with large areas of farmland, especially when a large number of sensor devices are deployed, the maintenance of various irrigation facilities requires a lot of manpower and material resources, resulting in low inspection efficiency of irrigation equipment.

[0004] However, with the continuous development and maturity of AR, it has been widely used in various fields. However, there are also problems in the application such as high price and high equipment maintenance cost, which prevents it from being more widely used in many fields.

[0005] Existing helmet displays have low brightness and slow response speed, and are still not suitable for occasions that require fast response. In addition, early helmet displays used small cathode ray tubes (CRTs), which, although high resolution and high brightness, were heavy and bulky, and required high-voltage power supply. These factors limited their application in helmet displays, and the display screen may cause image flipping and insufficient focus when directly irradiating the semi-transparent mirror. Summary of the invention

[0006] The purpose of the present invention is to overcome the defects of the prior art and provide a multifunctional display system and method with a homemade AR optical structure, which solves the problems of image flipping and insufficient focal length that may occur when the semi-transparent mirror is directly irradiated, ensuring that people see an upright and clear image; it is easy to maintain, has strong anti-interference ability, good AR effect, low cost, and a wide range of applications.

[0007] The purpose of the present invention is achieved in one aspect as follows: a multifunctional display system of a self-made AR optical structure, comprising an AR fire helmet main control module, a self-made AR optical structure, a sensor module, a gyroscope data processing module, a GPS processing module, a voice module, a wireless module, a communication module and a cloud platform;

[0008] The AR fire helmet main control module is used to receive data from the sensor module, process it, exclude invalid data, output valid data, and transmit the data to the cloud platform and the self-made AR optical structure through the communication module;

[0009] The gyroscope data processing module is used to analyze the gyroscope data and transmit the calculated yaw angle to the AR fire helmet main control module in real time;

[0010] The GPS module is used to respectively calculate the read data, and finally obtain the longitude and latitude of the current location and further calculate the time;

[0011] The voice module is used to broadcast corresponding words according to different places;

[0012] The wireless module is used to transmit the sensor data to the cloud platform via the MQTT communication protocol;

[0013] The communication module is used to parse and process the data packets transmitted from the main control module of the AR fire helmet, and optimize the data transmission by using the frame header and frame tail;

[0014] The sensor module is used to detect on-site environmental data in real time and transmit the data to the AR fire helmet main control module;

[0015] The self-made AR optical structure includes a liquid crystal display, a full reflector and a half mirror; the liquid crystal display is used to display the effective data transmitted from the main control module of the AR fire helmet, and then the liquid crystal display is used as a light source, the light emitted by the liquid crystal display is captured by the full reflector and converted to a 90-degree angle, and then projected onto the half mirror, and finally enters the human eye together with the real scene;

[0016] The self-made AR optical structure, sensor module, gyroscope data processing module, GPS processing module, voice module, wireless module, communication module are electrically connected to the AR fire helmet main control module.

[0017] As a further limitation of the present invention, it also includes a power module, which is electrically connected to the sensor, the communication module and the AR fire helmet main control module.

[0018] As a further limitation of the present invention, the AR fire helmet main control module has a built-in FreeRTOS embedded operating system and LVGL library.

[0019] As a further limitation of the present invention, the AR fire helmet main control module adopts an STM32H7 series single-chip microcomputer.

[0020] Another aspect of the present invention is to achieve the purpose of: a multifunctional display method of a self-made AR optical structure, comprising the following steps:

[0021] 1) Initialize each module, and the main control module of the AR fire helmet receives various sensor data;

[0022] 2) The main control module of the AR fire helmet based on the FreeRTOS embedded operating system synchronously reads and processes the data transmitted by the peripheral modules in real time;

[0023] 3) Use filtering algorithm to optimize the data;

[0024] 4) Save the current data for comparison with the previous data, and store the map data of the rescue site in advance to ensure that the data is not lost when offline;

[0025] 5) Finally, the results processed by the main control module of the AR fire helmet are output on the homemade AR optical structure.

[0026] As a further limitation of the present invention, the step 2) comprises:

[0027] Step 2.1) creating multiple modules to perform different tasks respectively; the multiple modules include a GPS processing module, a gyroscope data processing unit, a voice module, a wireless module, a communication module and a sensor module;

[0028] Step 2.2) The GPS processing module will solve the read data separately, and finally get the longitude and latitude of the current location and can further calculate the time;

[0029] Step 2.3) The gyroscope data processing module is used to parse the gyroscope data. First, the code identifies the beginning of the data packet by detecting a specific frame header and frame ID in the received data stream. After confirming the frame header and frame ID, the code continues to receive data until a complete 11-byte data packet is collected. In this data packet, the low-byte and high-byte data used to calculate the gyroscope yaw angle are included. When calculating the yaw angle, the high byte is shifted left by 8 bits and ORed with the low byte to obtain a 16-bit integer value. This integer value is then divided by 32768.0, which is 2 to the 15th power, and multiplied by 180 to convert the raw gyroscope data into an angle value. If the calculated angle is negative, the range is adjusted by adding 360 to make it a positive value between 0 and 360 degrees. Finally, the final yaw angle value is obtained by subtracting the angle from 360 degrees.

[0030] Step 2.4) The voice module broadcasts the corresponding words according to different places; the wireless module transmits the sensor data to the cloud platform through the MQTT communication protocol; the communication module parses and processes the transmitted data packets and optimizes the data transmission by using the frame header and frame tail; the sensor module reads the data of the smoke on site in real time through the ADC port;

[0031] Step 2.5) The FreeRTOS embedded operating system creates and schedules tasks, creates multiple tasks, and schedules them according to their priorities to ensure that high-priority tasks can be executed in a timely manner.

[0032] As a further limitation of the present invention, the step 3) comprises:

[0033] Step 3.1) Before applying Kalman filtering, it is necessary to establish a state space model of the system, including the state equation and observation equation:

[0034] The state equation describes the evolution of the system state over time, and the formula is as follows:

[0035] x k =F K x k-1 +B k u k +w k

[0036] Among them, x k is the system state at time k, F k is the state transfer matrix, B k is the control matrix, u k is the control vector, w k is the process noise;

[0037] The observation equation describes the relationship between the observed value and the system state, and the formula is as follows:

[0038] z k =H k x k +v k

[0039] Among them, z k is the observed value at time k, H k is the observation matrix, v k is the observation noise;

[0040] Step 3.2) Initialization: Before filtering begins, the initial state estimate and the corresponding covariance matrix are given; the initial state estimate x0|0, the initial estimated covariance P0|0

[0041] Step 3.3) Prediction: Based on the state estimate at the previous moment, predict the state and covariance at the current moment;

[0042] Predicted state: x k |k-1=F K x K-1|K-1 +B k u k ;

[0043] Prediction covariance: Pk |k-1=F k P k-1 F K T +Q k ;

[0044] Among them, Q k is the covariance matrix of the process noise;

[0045] Step 3.4) Update: When new observation data arrives, update the state estimate and covariance; calculate the Kalman gain: Among them, R k is the covariance matrix of the observation noise;

[0046] Update state estimate: x k |k=x k |k+K k (z k -H k x k |k-1);

[0047] Update covariance: P K |k=(IK k H k ) k |k-1;

[0048] Where I is the identity matrix;

[0049] Step 3.5) Repeat the process: Use the updated state estimate and covariance as the initial value for the next iteration and repeat the prediction and update steps.

[0050] As a further limitation of the present invention, step 4) includes: using an SD card for data storage to store previously collected sensor data for data comparison and analysis of emergencies in the on-site environment; storing map data of the rescue site in the SD card in advance; and in the event of a network connection interruption, the smart rescue helmet can still instantly access and display a map of the surrounding environment.

[0051] As a further limitation of the present invention, the step 5) includes: transplanting the LVGL library in the main control module of the AR fire helmet, and then drawing a liquid crystal screen display screen through the SquareLine Studio software, and displaying the data collected by various sensors in real time on the interface designed based on LVGL;

[0052] The homemade AR optical structure includes a liquid crystal display, a full reflector and a half mirror; the liquid crystal display is used to display the valid data transmitted from the main control module of the AR fire helmet, and then the liquid crystal display is used as a light source, the light it emits is captured by the full reflector and converted to a 90-degree angle, and then projected onto the half mirror, and finally enters the human eye together with the real scene.

[0053] The present invention adopts the above technical solution, and has the following beneficial effects compared with the prior art: the present invention uses a self-made AR optical structure, and uses the liquid crystal display screen as the light source. The light emitted by the liquid crystal display screen is first captured by the total reflector and converted to a 90-degree angle, and then projected onto the semi-transparent mirror, and finally enters the human eye together with the real scene; this design solves the problems of image flipping and insufficient focal length that may occur when the semi-transparent mirror is directly illuminated, ensuring that the person sees an upright and clear image. The self-made AR optical structure is easy to maintain, and the cost of the self-made AR optical structure is low.

[0054] The present invention cleverly embeds the FreeRTOS operating system in the core processing unit of the single-chip microcomputer. The lightweight characteristics of FreeRTOS, like an exquisite design blueprint, manage system resources accurately and efficiently, ensuring that the smart helmet can operate stably like a rock in complex, changeable and harsh rescue scenarios. This optimized resource allocation mechanism not only maximizes the operating efficiency of the equipment, but also ensures that the smart helmet can quickly and accurately execute various instructions in an emergency, thereby playing a key role in the rescue operation. The introduction of FreeRTOS is undoubtedly a comprehensive improvement in the performance of rescue equipment, providing more reliable technical support for rescue personnel, ensuring that they can rely on the powerful functions of the smart helmet to make the most correct decisions when facing life-and-death moments.

[0055] The present invention realizes efficient and smooth user interface display by using the LVGL (Light and Versatile Graphics Library). The cross-platform feature of LVGL enables the AR helmet to maintain a consistent user experience on different hardware platforms, while providing rich graphical interface elements and enhancing the readability and interactivity of rescue information. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 System block diagram of the present invention.

[0057] Figure 2 A circuit schematic diagram of the system of the present invention.

[0058] Figure 3 Schematic diagram of the homemade AR optical structure of the present invention.

[0059] Figure 4Schematic diagram of the light reflection path of the homemade AR optical structure of the present invention.

[0060] Figure 5 System operation flow chart of the present invention.

[0061] Figure 6 It is the upper computer software display diagram of the present invention.

[0062] Among them, 1 is a liquid crystal display, 2 is a full reflection mirror, and 3 is a semi-transparent mirror. DETAILED DESCRIPTION

[0063] like Figure 1 A multifunctional display system with a homemade AR optical structure is shown, comprising an AR fire helmet main control module, a homemade AR optical structure, a sensor module, a gyroscope data processing module, a GPS processing module, a voice module, a wireless module, a communication module, a power module and a cloud platform;

[0064] The main control module of the AR fire helmet is used to receive data from the sensor module, process it, exclude invalid data, output valid data, and transmit the data to the cloud platform and the homemade AR optical structure through the communication module; the main control module of the AR fire helmet has a built-in FreeRTOS embedded operating system and LVGL library; the main control module of the AR fire helmet uses the STM32H750 XBH6 chip.

[0065] The gyroscope data processing module is used to analyze the gyroscope data and transmit the calculated yaw angle to the main control module of the AR fire helmet in real time; the gyroscope data processing module adopts the ATK-MS901M module;

[0066] The GPS module uses the Beidou GPS module, which is used to solve the read data separately, and finally obtain the longitude and latitude of the current location and further calculate the time;

[0067] The voice module is used to broadcast corresponding words according to different places;

[0068] The wireless module is used to transmit the sensor data to the cloud platform through the MQTT communication protocol; the wireless module uses the ESP32 networking module;

[0069] The communication module is used to parse and process the data packets transmitted from the main control module of the AR fire helmet, and optimize the data transmission by using the frame header and frame tail; the communication module adopts the LoRa communication processing module;

[0070] The sensor module is used to detect on-site environmental data in real time and transmit the data to the main control module of the AR fire helmet; the sensor module uses a smoke sensor module, which can monitor the concentration of harmful gases and smoke in the rescue environment in real time, and provide environmental safety warnings for rescue personnel.

[0071] like Figure 3-4 As shown, the self-made AR optical structure includes a liquid crystal display screen 1, a total reflection mirror 2, a semi-mirror 3 and a main body bottom plate, a space is set on the main body bottom plate to fix the liquid crystal display screen 1, a small hole is embedded on the top of the main body bottom plate for installing the total reflection mirror 2, a clamp structure is set at the lower end of the main body bottom plate, the clamp structure is fixed to the lower end of the main body bottom plate through a transfer rod, and the semi-mirror 3 is fixed by the clamp structure.

[0072] The LCD screen 1 is used to display the valid data transmitted by the main control module of the AR fire helmet. Then the LCD screen 1 is used as a light source. The light emitted by the LCD screen 1 is captured by the total reflector 2 and converted to a 90-degree angle, and then projected onto the semi-transparent mirror 3, and finally enters the human eye together with the real scene;

[0073] The homemade AR optical structure, sensor module, gyroscope data processing module, GPS processing module, voice module, wireless module, communication module are electrically connected to the AR fire helmet main control module; the power module is electrically connected to the sensor, communication module and the AR fire helmet main control module.

[0074] like Figure 2 As shown, the main control module of the AR fire helmet is the chip STM32H750, and the RX6 pin, RX22 pin, TX22 pin, RX1 pin, TX1 pin, RX3 pin, TX3 pin, TX6 pin, TX2 pin, RX2 pin, ADC13 pin, and ADC7 pin of the main control module are respectively connected to the RX6 pin of the gyroscope data processing module, the RX22 pin of the LoRa communication module, the TX22 pin of the LoRa communication module, the RX1 pin of the voice module, the TX1 pin of the voice module, the RX3 pin of the ESP32 networking module, the TX3 pin of the ESP32 networking module, the TX6 pin of the gyroscope module, the TX2 pin of the Beidou GPS module, the RX2 pin of the Beidou GPS module, and the ADC13 pin and ADC7 pin of the smoke sensor module.

[0075] A multifunctional display method of a self-made AR optical structure comprises the following steps:

[0076] 1) Initialize each module, and the main control module of the AR fire helmet receives various sensor data;

[0077] 2) The main control module of the AR fire helmet based on the FreeRTOS embedded operating system synchronously reads and processes the data transmitted by the peripheral modules in real time;

[0078] Step 2.1) creating multiple modules to perform different tasks respectively; the multiple modules include a GPS processing module, a gyroscope data processing unit, a voice module, a wireless module, a communication module and a sensor module;

[0079] Step 2.2) The GPS processing module will solve the read data separately, and finally get the longitude and latitude of the current location and can further calculate the time;

[0080] Step 2.3) The gyroscope data processing module is used to parse the gyroscope data. First, the code identifies the start of the data packet by detecting a specific frame header (0x55) and frame ID (0x01) in the received data stream. After confirming the frame header and frame ID, the code continues to receive data until a complete 11-byte data packet is collected. In this data packet, the low-byte and high-byte data used to calculate the gyroscope yaw angle are included. When calculating the yaw angle, the high byte is shifted left by 8 bits and ORed with the low byte to obtain a 16-bit integer value. This integer value is then divided by 32768.0, which is 2 to the 15th power, and multiplied by 180 to convert the raw gyroscope data into an angle value. If the calculated angle is negative, the range is adjusted by adding 360 to make it a positive value between 0 and 360 degrees. Finally, the final yaw angle value is obtained by subtracting the angle from 360 degrees.

[0081] Step 2.4) The voice module broadcasts the corresponding words according to different places; the wireless module transmits the sensor data to the cloud platform through the MQTT communication protocol; the communication module parses and processes the transmitted data packets and optimizes the data transmission by using the frame header and frame tail; the sensor module reads the data of the smoke on site in real time through the ADC port;

[0082] Step 2.5) Figure 5 As shown in the figure, the FreeRTOS real-time operating system is used as the task scheduling system, and the entire system is divided into multiple sub-units. Each sensor corresponds to an execution unit. Using the multi-threading of FreeRTOS, data from multiple sensors can be read at the same time, and there will be no asynchronous data reading. The sensor data can be processed synchronously, achieving the effect of simultaneous processing and decision-making of multiple data. The smart helmet has built-in environmental sensors such as smoke and CO2, which can monitor the concentration of harmful gases and smoke in the rescue environment in real time, and provide environmental safety warnings for rescuers.

[0083] 3) Use filtering algorithm to optimize the data;

[0084] Step 3.1) Before applying Kalman filtering, it is necessary to establish a state space model of the system, including the state equation and observation equation:

[0085] The state equation describes the evolution of the system state over time, and the formula is as follows:

[0086] x k =F K xk-1 +B k u k +w k

[0087] Among them, x k is the system state at time k, F k is the state transfer matrix, B k is the control matrix, u k is the control vector, w k is the process noise;

[0088] The observation equation describes the relationship between the observed value and the system state, and the formula is as follows:

[0089] z k =H k x k +v k

[0090] Among them, z k is the observed value at time k, H k is the observation matrix, v k is the observation noise;

[0091] Step 3.2) Initialization: Before filtering begins, the initial state estimate and the corresponding covariance matrix are given; the initial state estimate x0|0, the initial estimated covariance P0|0

[0092] Step 3.3) Prediction: Based on the state estimate at the previous moment, predict the state and covariance at the current moment;

[0093] Predicted state: x k |k-1=F K x K-1|K-1 +B k u k ;

[0094] Prediction covariance: P k |k-1=F k P k-1 F K T +Q k ;

[0095] Among them, Q k is the covariance matrix of the process noise;

[0096] Step 3.4) Update: When new observation data arrives, update the state estimate and covariance; calculate the Kalman gain: Among them, R k is the covariance matrix of the observation noise;

[0097] Update state estimate: x k|k=x k |k+K k (z k -H k x k |k-1);

[0098] Update covariance: P K |k=(IK k H k ) k |k-1;

[0099] Where I is the identity matrix;

[0100] Step 3.5) Repeat the process: Use the updated state estimate and covariance as the initial value for the next iteration and repeat the prediction and update steps.

[0101] 4) Save the current data for comparison with the previous data, and store the map data of the rescue site in advance to ensure that the data is not lost when offline;

[0102] SD cards are used for data storage, and previously collected sensor data is stored for comparison between data to analyze emergencies in the on-site environment; map data of the rescue site is stored in the SD card in advance; in the event of a network connection interruption, the smart rescue helmet can still instantly access and display the surrounding environment map.

[0103] 5) Finally, the results processed by the main control module of the AR fire helmet are output on the homemade AR optical structure.

[0104] Step 5) includes: transplanting the LVGL library in the main control module of the AR fire helmet, and then using SquareLineStudio software to draw the LCD screen display screen, and displaying the data collected by various sensors in real time on the interface designed based on LVGL;

[0105] The homemade AR optical structure includes a liquid crystal display 1, a total reflection mirror 2 and a half mirror 3; the liquid crystal display 1 is used to display the valid data transmitted from the main control module of the AR fire helmet, and then the liquid crystal display is used as a light source, the light emitted by the liquid crystal display is captured by the total reflection mirror 2 and converted to a 90-degree angle, and then projected onto the half mirror 3, and finally enters the human eye together with the real scene.

[0106] like Figure 6As shown, in terms of application, the present invention has developed an Android APP named "Monitoring Terminal" as an interface for users to interact with the system. The main function of this APP is to obtain various sensor data uploaded by the ESP32 terminal in real time, and provide multiple buttons for sending specific commands to the helmet. During the development of the APP, the present invention fully utilized the API provided by the Huawei Cloud IoT platform to achieve seamless integration with the cloud platform.

[0107] When the present invention is working, team members wearing AR fire helmets conduct air-to-ground coordinated patrols with drones. When the drone equipped with thermal imaging finds abnormal high temperature, it will issue an abnormal prompt. The drone operator confirms the disaster site through the image transmission perspective and sends the location data to the helmet. The AR helmet has built-in offline maps, environmental perception, real-time communication and other functions, which can help team members plan routes, detect the environment and receive real-time instructions from the APP command end.

[0108] The present invention provides a multifunctional display system and method of a self-made AR optical structure. Due to the low power consumption performance of the hardware structure and strong anti-interference performance, the detection threshold is set by the average background threshold of the front and rear window data to achieve accurate measurement of the data, and the surrounding environment information can be accurately obtained to ensure the efficiency of the rescue operation. It can work stably under various environmental conditions, provide stable and reliable data, and can adapt well to the changeable weather conditions often encountered in rescue operations and intelligent inspections. The installation of the self-made AR optical structure is relatively simple and does not require complex engineering. It can be quickly deployed in the use of smart fire helmets in smart rescue and smart inspection equipment in smart irrigation, which can reduce its operating and production costs.

[0109] The present invention is not limited to the above-mentioned embodiments. On the basis of the technical solution disclosed in the present invention, technicians in this field can make some substitutions and deformations to some technical features therein according to the disclosed technical content without creative labor, and these substitutions and deformations are all within the protection scope of the present invention.

Claims

1. A multifunctional display system with a self-made AR optical structure, characterized in that: It includes the main control module of the AR fire helmet, a self-made AR optical structure, a sensor module, a gyroscope data processing module, a GPS processing module, a voice module, a wireless module, a communication module and a cloud platform; The AR fire helmet main control module is used to receive data from the sensor module, process it, exclude invalid data, output valid data, and transmit the data to the cloud platform and the self-made AR optical structure through the communication module; The gyroscope data processing module is used to analyze the gyroscope data and transmit the calculated yaw angle to the AR fire helmet main control module in real time; The GPS module is used to respectively calculate the read data, and finally obtain the longitude and latitude of the current location and further calculate the time; The voice module is used to broadcast corresponding words according to different places; The wireless module is used to transmit the sensor data to the cloud platform via the MQTT communication protocol; The communication module is used to parse and process the data packets transmitted from the main control module of the AR fire helmet, and optimize the data transmission by using the frame header and frame tail; The sensor module is used to detect on-site environmental data in real time and transmit the data to the AR fire helmet main control module; The self-made AR optical structure includes a liquid crystal display, a full reflector and a half mirror; the liquid crystal display is used to display the effective data transmitted from the main control module of the AR fire helmet, and then the liquid crystal display is used as a light source, the light emitted by the liquid crystal display is captured by the full reflector and converted to a 90-degree angle, and then projected onto the half mirror, and finally enters the human eye together with the real scene; The self-made AR optical structure, sensor module, gyroscope data processing module, GPS processing module, voice module, wireless module, communication module are electrically connected to the AR fire helmet main control module.

2. A multifunctional display system of a self-made AR optical structure according to claim 1, characterized in that: It also includes a power module, which is electrically connected to the sensor, the communication module and the AR fire helmet main control module.

3. The multifunctional display system of a self-made AR optical structure according to claim 1, characterized in that: The AR fire helmet main control module has a built-in FreeRTOS embedded operating system and LVGL library.

4. The multifunctional display system of a self-made AR optical structure according to claim 1, characterized in that: The main control module of the AR fire helmet adopts the STM32H7 series single-chip microcomputer.

5. A multifunctional display method of a self-made AR optical structure, characterized in that: The following steps are involved: 1) Initialize each module, and the main control module of the AR fire helmet receives various sensor data; 2) The main control module of the AR fire helmet based on the FreeRTOS embedded operating system synchronously reads and processes the data transmitted by the peripheral modules in real time; 3) Use filtering algorithm to optimize the data; 4) Save the current data for comparison with the previous data, and store the map data of the rescue site in advance to ensure that the data is not lost when offline; 5) Finally, the results processed by the main control module of the AR fire helmet are output on the homemade AR optical structure.

6. The multifunctional display method of a self-made AR optical structure according to claim 5, characterized in that: The step 2) comprises: Step 2.1) creating multiple modules to perform different tasks respectively; the multiple modules include a GPS processing module, a gyroscope data processing unit, a voice module, a wireless module, a communication module and a sensor module; Step 2.2) The GPS processing module will solve the read data separately, and finally get the longitude and latitude of the current location and can further calculate the time; Step 2.3) The gyroscope data processing module is used to parse the gyroscope data. First, the code identifies the beginning of the data packet by detecting a specific frame header and frame ID in the received data stream. After confirming the frame header and frame ID, the code continues to receive data until a complete 11-byte data packet is collected. In this data packet, the low-byte and high-byte data used to calculate the gyroscope yaw angle are included. When calculating the yaw angle, the high byte is shifted left by 8 bits and ORed with the low byte to obtain a 16-bit integer value. This integer value is then divided by 32768.0, which is 2 to the 15th power, and multiplied by 180 to convert the raw gyroscope data into an angle value. If the calculated angle is negative, the range is adjusted by adding 360 to make it a positive value between 0 and 360 degrees. Finally, the final yaw angle value is obtained by subtracting the angle from 360 degrees. Step 2.4) The voice module broadcasts the corresponding words according to different places; the wireless module transmits the sensor data to the cloud platform through the MQTT communication protocol; the communication module parses and processes the transmitted data packets and optimizes the data transmission by using the frame header and frame tail; the sensor module reads the data of the smoke on site in real time through the ADC port; Step 2.5) The FreeRTOS embedded operating system creates and schedules tasks, creates multiple tasks, and schedules them according to their priorities to ensure that high-priority tasks can be executed in a timely manner.

7. The multifunctional display method of a self-made AR optical structure according to claim 5, characterized in that: The step 3) comprises: Step 3.1) Before applying Kalman filtering, it is necessary to establish a state space model of the system, including the state equation and observation equation: The state equation describes the evolution of the system state over time, and the formula is as follows: x k =F K x k-1 +B k u k +w k Among them, x k is the system state at time k, F k is the state transfer matrix, B k is the control matrix, u k is the control vector, w k is the process noise; The observation equation describes the relationship between the observed value and the system state, and the formula is as follows: z k =H k x k +v k Among them, z k is the observed value at time k, H k is the observation matrix, v k is the observation noise; Step 3.2) Initialization: Before filtering begins, the initial state estimate and the corresponding covariance matrix are given; the initial state estimate x0|0, the initial estimated covariance P0|0 Step 3.3) Prediction: Based on the state estimate at the previous moment, predict the state and covariance at the current moment; Predicted state: x k |k-1=F K x K-1|K-1 +B k u k ; Prediction covariance: Among them, Q k is the covariance matrix of the process noise; Step 3.4) Update: When new observation data arrives, update the state estimate and covariance; calculate the Kalman gain: Among them, R k is the covariance matrix of the observation noise; Update state estimate: x k |k=x k |k+K k (z k -H k x k |k-1); Update covariance: P K |k=(IK k H k ) k |k-1; Where I is the identity matrix; Step 3.5) Repeat the process: Use the updated state estimate and covariance as the initial value for the next iteration and repeat the prediction and update steps.

8. The multifunctional display method of a self-made AR optical structure according to claim 5, characterized in that: The step 4) includes: using an SD card for data storage to store previously collected sensor data for data comparison and analysis of emergencies in the on-site environment; storing map data of the rescue site in the SD card in advance; and in the event of a network connection interruption, the smart rescue helmet can still instantly access and display the surrounding environment map.

9. The multifunctional display method of a self-made AR optical structure according to claim 5, characterized in that: The step 5) includes: transplanting the LVGL library in the main control module of the AR fire helmet, and then drawing the LCD screen display screen through the SquareLine Studio software, and displaying the data collected by various sensors in real time on the interface designed based on LVGL; The homemade AR optical structure includes a liquid crystal display, a total reflection mirror and a half-mirror; the liquid crystal display is used to display the valid data transmitted from the main control module of the AR fire helmet, and then the liquid crystal display is used as a light source, the light it emits is captured by the total reflection mirror and converted to a 90-degree angle, and then projected onto the half-mirror, and finally enters the human eye together with the real scene.

Images