A fire scene cooperative auxiliary monitoring system and method based on fire fighting AR glasses

Abstract

The application discloses a fire scene cooperative auxiliary monitoring system and method based on fire-fighting AR glasses, and relates to the technical field of fire-fighting emergency rescue and intelligentized command. The monitoring system comprises an AR glass terminal device, a command terminal device and a cloud communication server; the AR glass terminal device collects fire scene video, voice, text and firefighter height, heart rate and blood oxygen information and uploads the information to the cloud communication server; the command terminal device receives and displays the information and issues command control information; the AR glass terminal device is internally provided with a network state detection module, maintains a TCP long connection between the AR glass terminal device and the cloud communication server, automatically reconnects when network interruption is detected and recovered or IP address is changed, and the cloud communication server actively terminates an old session of the same device identifier and establishes a new session, so that the command control information is always routed to a current effective session. The application realizes fire scene situation awareness, individual physiological monitoring and front and rear command cooperation, and improves the safety of fire extinguishing and rescue and the timeliness of command decision.

Description

Technical Field

[0001] This invention relates to the field of fire emergency rescue and intelligent command technology, and more specifically, to a fire scene collaborative auxiliary monitoring system and method based on fire AR glasses. Background Technology

[0002] As fire scenarios in urban complexes, high-rise buildings, and underground spaces become increasingly complex, firefighters generally face multiple risks when carrying out search and rescue and firefighting missions inside fire scenes, including high-temperature smoke, low visibility, noise interference, spatial disorientation, and physical exhaustion.

[0003] Commanders at the rear need to keep abreast of the fire's development, attack routes, and the distribution and physical condition of personnel attacking from within the fire zone in order to scientifically deploy forces and issue evacuation or reinforcement orders.

[0004] However, traditional individual soldier equipment systems often configure image transmission terminals, intercom terminals, heart rate monitoring terminals, and altitude indicators independently, resulting in prominent information silos and making it difficult to form a unified, continuous, and traceable collaborative data link.

[0005] In terms of fire situation awareness, some existing individual image transmission devices only have visible light imaging capabilities. Under conditions of dense smoke, darkness, or strong backlight, the quality of visible light images deteriorates rapidly, making it difficult to identify fire boundaries, concealed burning areas, and the location of openings in building structures. While standalone thermal imagers can help identify areas with abnormal temperatures, they often lack unified access to the command and dispatch system, making it impossible to synchronize and link voice, text, and physiological data to the same firefighter's identity. In terms of command and communication, on-site noise, breathing apparatus airflow noise, and building echoes significantly reduce the intelligibility of voice communication, posing a risk of missing or mishearing critical instructions. Pure voice channels are also not convenient for retaining structured records, which is not conducive to post-war review.

[0006] In terms of individual soldier safety, parameters such as heart rate, blood oxygen, and relative altitude are important indicators for assessing firefighters' physiological load and risk of getting lost. However, existing systems often transmit these parameters separately from image transmission and command channels, making it impossible for the command end to see "who is at what altitude, what their physical condition is, and what fire scene they are currently seeing" on a single interface in real time.

[0007] In addition, the configuration of equipment networks before deployment to a fire site usually relies on manual operation on site, which is inefficient and has a high error rate. After wireless link switching or network reconnection during a fire, the session status between the device and the cloud is inconsistent, which can easily lead to messages not being delivered and affect the continuity of command.

[0008] The above shortcomings need to be improved. Summary of the Invention

[0009] To overcome the problems of information silos, limited fire scene perception methods, susceptibility to environmental interference in command and communication, separation of personnel status monitoring and command and dispatch in existing fire protection systems, reliance on manual equipment deployment, and easy loss of connection after network switching leading to unreliable message delivery, this invention provides a collaborative auxiliary monitoring system and method for fire scenes based on fire protection AR glasses.

[0010] The technical solution of this invention is as follows: A fire scene collaborative auxiliary monitoring system based on fire-fighting AR glasses includes: AR glasses terminal devices are used to collect and upload fire monitoring information; The command terminal device is used to receive the fire monitoring information and send command and control information to the AR glasses terminal device for remote dispatch. A cloud communication server is used to establish a communication connection between the AR glasses terminal device and the command terminal device and to perform message routing. The AR glasses terminal device has a built-in network status detection module, which is used to detect the connectivity status and network address of the current network interface. The AR glasses terminal device maintains a long connection session based on the transmission control protocol with the cloud communication server. When the network status detection module detects that the network connection has been interrupted and then restored, or detects that the local network address has changed, the AR glasses terminal device disconnects the current long connection session and re-initiates a connection request and identity authentication to the cloud communication server. When the cloud communication server receives a reconnection request for the same device identifier, if the old long connection session corresponding to the device identifier is still active, the old long connection session is terminated and a new long connection session is established. At the same time, the mapping relationship between the device identifier and the session is updated.

[0011] Specifically, the fire monitoring information includes fire video information, and the AR glasses terminal device includes a visible light image acquisition module and a thermal imaging image acquisition module. The visible light image acquisition module acquires visible light images of the fire based on the video acquisition interface, and the thermal imaging image acquisition module acquires images of the fire temperature distribution based on an infrared thermal imaging sensor component. The AR glasses terminal device switches between visible light mode and thermal imaging mode based on response commands, and encodes the real-time images in the current acquisition mode and pushes them to the cloud communication server.

[0012] Specifically, the fire monitoring information and the command and control information include text interaction information. The AR glasses terminal device encapsulates the text information input by the firefighter into a first text frame and sends it to the command terminal device via the cloud communication server; the command terminal device encapsulates the input text information into a second text frame and sends it to the AR glasses terminal device via the cloud communication server.

[0013] Specifically, the fire monitoring information and the command and control information include voice interaction information. The AR glasses terminal device uploads the first voice message input by the firefighter to the object storage service module of the cloud communication server. Based on the object storage service module, a first voice resource address is generated and encapsulated into a first audio frame and sent to the command terminal device. The command terminal device receives the first audio frame, parses it to obtain the first voice resource address, and plays the first voice message. The command terminal device uploads the second voice message input by the command personnel to the object storage service module of the cloud communication server. Based on the object storage service module, it generates a second voice resource address and encapsulates it into a second audio frame, which is then sent to the AR glasses terminal device. The AR glasses terminal device receives the second audio frame, parses it to obtain the second voice resource address, and plays the second voice message.

[0014] Specifically, the collaborative auxiliary monitoring system also includes a wearable vital sign acquisition device, which is used to acquire heart rate and blood oxygen saturation values; the AR glasses terminal device includes a barometric altitude sensing unit, which is used to acquire the relative altitude value of the firefighter; the AR glasses terminal device is connected to the wearable vital sign acquisition device via a Bluetooth communication module, and the AR glasses terminal device encapsulates the altitude value, heart rate value, and blood oxygen saturation value into sensor data frames according to a preset period and uploads them to the cloud communication server.

[0015] Specifically, the command terminal device displays the online status, altitude, heart rate, and blood oxygen saturation of each firefighter on the monitoring interface; the command terminal device maps the firefighter's location to a floor zoning view of the fire scene based on the altitude value; when the altitude value is within a preset floor range, the command terminal device displays the firefighter's information on an independent interface.

[0016] Specifically, the command terminal device sends an emergency evacuation command frame to the designated AR glasses terminal device through the cloud communication server; after receiving the emergency evacuation command frame, the AR glasses terminal device displays evacuation prompt information on its display module and controls the audio playback module to emit an alarm sound.

[0017] Specifically, the AR glasses terminal device also includes a Bluetooth broadcast module, which is used to broadcast the device's unique identifier. The command terminal device reads the device's unique identifier via Bluetooth, binds the device's unique identifier with the firefighter's identity information, and uploads it to the cloud communication server. When the AR glasses terminal device initiates a connection, the cloud communication server verifies the binding relationship and only allows the AR glasses terminal device to establish the long connection session after the binding verification is successful.

[0018] Specifically, the command terminal device sends a wireless network configuration command to the AR glasses terminal device via Bluetooth. The wireless network configuration command includes a wireless network scanning command, a wireless network connection command, or a status refresh command. After executing the wireless network connection command, the AR glasses terminal device queries the currently active wireless network access point identifier, compares the currently active wireless network access point identifier with the target network name, and returns the configuration status to the command terminal device based on the comparison result.

[0019] A second aspect of the present invention also provides a fire scene collaborative auxiliary monitoring method based on fire-fighting AR glasses, applied to the fire scene collaborative auxiliary monitoring system for fire-fighting AR glasses as described above, comprising: The AR glasses terminal device establishes a long connection session with the cloud communication server based on the transmission control protocol, and uploads the collected fire monitoring information to the cloud communication server. The command terminal device receives the fire monitoring information through the cloud communication server and sends command and control information to the AR glasses terminal device based on the fire monitoring information; The AR glasses terminal device uses a built-in network status detection module to detect the connectivity status and network address of the current network interface. When the network status detection module detects that the network connection has been interrupted and then restored, or when it detects that the local network address has changed, the AR glasses terminal device disconnects the current long connection session and re-initiates a connection request and identity authentication to the cloud communication server. When the cloud communication server receives a reconnection request for the same device identifier, if the old long connection session corresponding to the device identifier is still active, the old long connection session is terminated and a new long connection session is established. At the same time, the mapping relationship between the device identifier and the session is updated.

[0020] According to the above-described solution, the beneficial effects of this invention are as follows: 1. This invention integrates visible light / thermal imaging dual-mode image acquisition, text information, voice messages, altitude sensing, heart rate and blood oxygen monitoring into the same cloud communication link, and centrally displays them on the command terminal by associating them with the same firefighter identity. This breaks the information silos of traditional individual equipment and realizes the integrated collaborative perception of multi-dimensional fire scene monitoring information, including video, voice, text, physiological and location data. 2. This invention enables firefighters and rear commanders to clearly identify fire boundaries, concealed burning areas, and building structure openings in low-visibility environments such as dense smoke, darkness, or strong backlighting through dual-mode image acquisition and real-time streaming of visible light and thermal imaging. This overcomes the perception limitations of a single visible light mode in complex fire environments. At the same time, through dual-channel interaction of text and voice messages, key instructions are presented in text form in noisy environments to avoid missing or mishearing them, and structured records are retained for post-battle review, making up for the shortcomings of pure voice communication, which is easily affected by environmental interference and cannot be traced back. 3. This invention obtains relative height values ​​through a pneumatic height sensing unit and maps them in conjunction with a floor zoning view. Combined with the periodic uploading and real-time display of physiological indicators such as heart rate and blood oxygen saturation, it enables commanders to simultaneously grasp the vertical position, physical load, and online status of firefighters on the same interface. This facilitates the timely detection of firefighters who have entered dangerous height zones or whose physiological indicators are abnormal, and timely rescue or evacuation measures can be taken. 4. This invention achieves secure access management for equipment by setting a unique identifier for the Bluetooth broadcasting device, and then having the command terminal complete the identity binding and the cloud verify the binding relationship before allowing access; it also enables remote and rapid network configuration before equipment deployment by sending wireless network scanning and connection commands via Bluetooth and verifying the connection results with the active access point identifier, thus avoiding the inefficiency and high error rate caused by manual operation on site. 5. This invention utilizes a network status detection module built into the AR glasses terminal to automatically terminate and rebuild the old connection when the network is restored or the IP address changes. At the same time, the cloud actively closes the old session with the same device identifier and binds the new session, ensuring the reliable delivery of command and control information and emergency evacuation instructions in the complex network environment of the fire scene, and solving the problem of "showing online but messages cannot be delivered" after network disconnection and reconnection. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a schematic diagram of the fire scene collaborative auxiliary monitoring system provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the internal modules of the AR glasses terminal device provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the fire scene collaborative auxiliary monitoring method provided in an embodiment of the present invention.

[0023] The following are the labeling elements in the figure: 1. AR glasses terminal equipment; 2. Command terminal equipment; 3. Cloud communication server; 4. Wearable vital sign collection equipment. Detailed Implementation

[0024] To make the technical problems to be solved, the technical solutions, and the beneficial effects of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0025] It should be noted that when a component is referred to as "fixed," "set," or "connected" to another component, it may be located directly or indirectly on that other component. The terms "upper," "lower," "left," "right," "front," "rear," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicate the orientation or position based on the accompanying drawings, and are for ease of description only, and should not be construed as limiting the technical solution. The terms "first," "second," etc., are used for ease of description only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features. "Many" means two or more, unless otherwise explicitly specified. "Several" means one or more, unless otherwise explicitly specified.

[0026] like Figures 1 to 2 As shown in the figure, this embodiment of the invention provides a fire scene collaborative auxiliary monitoring system based on fire-fighting AR glasses, including an AR glasses terminal device 1, a command terminal device 2, and a cloud communication server 3. The AR glasses terminal device 1 is communicatively connected to the cloud communication server 3, and the command terminal device 2 is communicatively connected to the cloud communication server 3.

[0027] Specifically, the AR glasses terminal device 1 and the cloud communication server 3 maintain a long-term connection session based on the Transmission Control Protocol (TCP). The AR glasses terminal device 1 has a built-in network status detection module, which detects the connectivity status and network address of the current network interface. When the network status detection module detects a network connection interruption followed by recovery or a change in the local network address, the AR glasses terminal device 1 disconnects the current long-term connection session and re-initiates a connection request and authentication to the cloud communication server 3. Upon receiving a reconnection request from the same device identifier, if the old long-term connection session corresponding to that device identifier is still active, the cloud communication server 3 terminates the old long-term connection session and establishes a new long-term connection session, while simultaneously updating the mapping relationship between the device identifier and the session.

[0028] In some embodiments, the AR glasses terminal device 1 sends heartbeat keep-alive messages to the cloud communication server 3 at fixed intervals (e.g., every 30 seconds). Upon receiving the messages, the cloud communication server 3 returns a heartbeat confirmation message. If the cloud communication server 3 does not receive a heartbeat keep-alive message within a predetermined time (e.g., 90 seconds), it determines that the long-connection session has expired and actively closes the corresponding channel. If the AR glasses terminal device 1 does not receive a heartbeat confirmation message within a predetermined time (e.g., 45 seconds), it determines that the network is abnormal and enters a reconnection preparation state.

[0029] Furthermore, the reconnection mechanism supports an exponential backoff strategy, meaning that after each failed reconnection, the reconnection interval increases exponentially (e.g., 1 second, 2 seconds, 4 seconds, 8 seconds... with a maximum interval of 60 seconds), avoiding signaling storms caused by frequent reconnections when the network is unstable.

[0030] Specifically, the AR glasses terminal device 1 includes a visible light image acquisition module and a thermal imaging image acquisition module. The visible light image acquisition module acquires visible light images of the fire scene based on a video acquisition interface, while the thermal imaging image acquisition module acquires images of the fire scene's temperature distribution based on an infrared thermal imaging sensor component. The AR glasses terminal device 1 switches between visible light mode and thermal imaging mode based on response commands, and encodes the real-time image in the current acquisition mode before streaming it to the cloud communication server 3.

[0031] In some embodiments, firefighters input a local mode switching command via buttons or physical function keys on the device. The embedded main control unit of the AR glasses terminal device 1 detects the button events and triggers the switching operation of the image acquisition module. In another example, a commander inputs a remote mode switching command via a command terminal device 2, which is then transmitted to the AR glasses terminal device 1 via a cloud communication server 3.

[0032] Furthermore, AR glasses terminal device 1 encodes the real-time video using the H.264 or H.265 video encoding module and then streams it to the streaming media service module of cloud communication server 3 via a real-time message transmission protocol. Terminal device 2 then retrieves and plays the real-time video from the streaming media service module of cloud communication server 3 as needed.

[0033] In fact, AR glasses terminal device 1 also supports picture-in-picture display mode, which means that thermal imaging images are displayed simultaneously in the corner of the visible light screen.

[0034] Specifically, the AR glasses terminal device 1 and the command terminal device 2 communicate bidirectionally via text messages through the cloud communication server 3. The AR glasses terminal device 1 encapsulates the text information input by the firefighter into a first text frame and sends it to the command terminal device 2 via the cloud communication server 3. The command terminal device 2 encapsulates the input text information into a second text frame and sends it to the AR glasses terminal device 1 via the cloud communication server 3. The AR glasses terminal device 1 displays the received text message on its local display module, and the command terminal device 2 displays the received text message on its dialogue interface.

[0035] In fact, the first text frame includes a header and data content. The header contains the message number, sending time, task identifier, sender identifier, receiver identifier, and message type identifier. The message routing subsystem of the cloud communication server 3 queries the mapping relationship between the device identifier and the session based on the receiver identifier in the header, determines that the current long-connection session is valid, and then forwards the text frame to the corresponding receiver.

[0036] Furthermore, the AR glasses terminal device 1 and the command terminal device 2 engage in bidirectional voice message interaction via a cloud communication server 3. The AR glasses terminal device 1 uploads the first voice signal input by the firefighter to the object storage service module of the cloud communication server 3, generates a first voice resource address, encapsulates it into a first audio frame, and sends it to the command terminal device 2. The command terminal device 2 receives the first audio frame, decodes it to obtain the first voice resource address, and plays the first voice signal. Similarly, the command terminal device 2 uploads the second voice signal input by the command personnel to the object storage service module of the cloud communication server 3, generates a second voice resource address, encapsulates it into a second audio frame, and sends it to the AR glasses terminal device 1. The AR glasses terminal device 1 receives the second audio frame, decodes it to obtain the second voice resource address, and plays the second voice signal.

[0037] In fact, the object storage service module is a component of the cloud communication server 3. The audio acquisition module of the AR glasses terminal device 1 acquires voice signals through its built-in microphone, uploads them to the object storage service module after audio encoding and compression, and the object storage service module assigns a globally unique resource identifier to the voice signal, generates a voice resource address, and returns it to the sender.

[0038] In some embodiments, the AR glasses terminal device 1 supports the voice broadcast function of text messages. That is, when a text message is received, the text content is converted into a voice signal through a local text-to-speech engine and played by an audio playback module.

[0039] Specifically, the AR glasses terminal device 1 includes a pneumatic height sensing unit, which is used to acquire the relative height value of the firefighter. The collaborative auxiliary monitoring system also includes a wearable vital sign acquisition device 4, which is used to acquire heart rate and blood oxygen saturation values. The AR glasses terminal device 1 and the wearable vital sign acquisition device 4 are connected via a Bluetooth communication module. The AR glasses terminal device 1 encapsulates the height, heart rate, and blood oxygen saturation values ​​into sensor data frames according to a preset period and uploads them to the cloud communication server 3.

[0040] In some embodiments, the barometric altitude sensing unit uses a Bmp390 barometric pressure sensor and communicates with the embedded main control unit of the AR glasses terminal device 1 via an I2C or SPI bus. The embedded main control unit calibrates based on the difference between the barometric pressure value and the standard atmospheric pressure, combined with the initial barometric pressure value at startup, to calculate the firefighter's relative altitude. The wearable vital signs acquisition device 4 is a Bluetooth bracelet or smartwatch worn on the firefighter's wrist, with a built-in photoplethysmography sensor that collects heart rate and blood oxygen saturation at a preset interval (e.g., every 5 seconds) and sends the data to the AR glasses terminal device 1 via Bluetooth Low Energy standard communication service.

[0041] Furthermore, the command terminal device 2 displays the online status, altitude, heart rate, and blood oxygen saturation of each firefighter in card format on the monitoring interface. The command terminal device 2 maps the firefighter's location to a floor-level zoning view of the fire scene based on the altitude value. When the altitude value falls within a preset floor range, the command terminal device 2 displays the firefighter's information in a separate card interface.

[0042] In some embodiments, the online status includes three states: "online," "offline," and "reconnecting," identified by green, gray, and yellow dots, respectively. The floor zoning view is generated based on building information model data or manually input floor height parameters, with each floor's height range preset (e.g., a standard floor height of 3.5 meters). When a firefighter's relative height falls within the height range of a certain floor, the command terminal device 2 moves the firefighter's card to the corresponding floor's zoning position and highlights the firefighter's card.

[0043] Specifically, when the command terminal device 2 sends an emergency evacuation command frame to the designated AR glasses terminal device 1 through the cloud communication server 3, the AR glasses terminal device 1 receives the emergency evacuation command frame, displays evacuation prompt information on its display module, and controls the audio playback module to emit an alarm sound.

[0044] In one example, the message type identifier in the header of the emergency evacuation command frame is set to "Emergency Evacuation," and the priority field is set to the highest priority. When the message routing subsystem of the cloud communication server 3 receives the emergency evacuation command frame, it processes it with priority over other message types. After receiving the emergency evacuation command frame, the AR glasses terminal device 1 displays the "Emergency Evacuation" prompt in full-screen or high-contrast color mode and controls the audio playback module to emit a high-frequency intermittent alarm sound.

[0045] In another example, the emergency evacuation command frame also includes evacuation route information. While displaying the evacuation prompt, the AR glasses terminal device 1 also overlays evacuation direction guidance in the form of arrows or path lines in the AR field of view.

[0046] Specifically, the AR glasses terminal device 1 also includes a Bluetooth broadcast module, which is used to broadcast the device's unique identifier. The command terminal device 2 reads the device's unique identifier via Bluetooth, binds the device's unique identifier with the firefighter's identity information, and uploads it to the cloud communication server 3. The cloud communication server 3 verifies the binding relationship when the AR glasses terminal device 1 initiates a connection, and only allows the AR glasses terminal device 1 to establish a long-term connection session after the binding verification is successful.

[0047] Furthermore, the Bluetooth broadcast module broadcasts the equipment service identifier and unique device identifier at preset intervals (e.g., 100ms) according to the Bluetooth broadcast protocol, and the broadcast data packet format follows the manufacturer-specific data format in the Bluetooth specification. The command terminal device 2 runs the equipment binding application, scans for surrounding equipment via Bluetooth, establishes a Bluetooth Low Energy connection with the target AR glasses terminal device 1, and then reads the device's unique identifier information via the Bluetooth GATT protocol.

[0048] In some embodiments, the binding relationship supports unbinding operations. The command terminal device 2 can send an unbinding request to the cloud communication server 3 via Bluetooth or a network interface, and the cloud communication server 3 will set the binding status in the corresponding device-person mapping table to invalid.

[0049] In practice, command terminal device 2 sends wireless network configuration commands to AR glasses terminal device 1 via Bluetooth. These commands include wireless network scanning commands, wireless network connection commands, or status refresh commands. When AR glasses terminal device 1 executes the wireless network connection command, it queries the currently active wireless network access point identifier and compares it with the target network name. Based on the comparison result, it returns the configuration status to command terminal device 2.

[0050] In one example, the wireless network connection command includes the target network name and connection password. After receiving the wireless network connection command, AR glasses terminal device 1 first calls the operating system's wireless network management interface to disconnect the current wireless connection, and then performs the target network connection operation. After the connection is completed, it queries the currently active wireless network access point identifier. If it matches the target network name, the connection is considered successful; otherwise, the connection is considered to have failed.

[0051] Specifically, the AR glasses terminal device 1 and the cloud communication server 3 use a length prefix frame packet method for message interaction. The message types of the interaction include at least heartbeat keep-alive frames, data frames, and command frames.

[0052] In a second aspect of the invention, a fire scene collaborative auxiliary monitoring method based on fire-fighting AR glasses is also provided, applied to the fire scene collaborative auxiliary monitoring system for fire-fighting AR glasses as described above, comprising: The AR glasses terminal device establishes a long connection session with the cloud communication server based on the transmission control protocol, and uploads the collected fire monitoring information to the cloud communication server. The command terminal device receives the fire monitoring information through the cloud communication server and sends command and control information to the AR glasses terminal device based on the fire monitoring information; The AR glasses terminal device uses a built-in network status detection module to detect the connectivity status and network address of the current network interface. When the network status detection module detects that the network connection has been interrupted and then restored, or when it detects that the local network address has changed, the AR glasses terminal device disconnects the current long connection session and re-initiates a connection request and identity authentication to the cloud communication server. When the cloud communication server receives a reconnection request for the same device identifier, if the old long connection session corresponding to the device identifier is still active, the old long connection session is terminated and a new long connection session is established. At the same time, the mapping relationship between the device identifier and the session is updated.

[0053] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A fire scene collaborative auxiliary monitoring system based on fire-fighting AR glasses, characterized in that, include: AR glasses terminal devices are used to collect and upload fire monitoring information; The command terminal device is used to receive the fire monitoring information and send command and control information to the AR glasses terminal device for remote dispatch. A cloud communication server is used to establish a communication connection between the AR glasses terminal device and the command terminal device and to perform message routing. The AR glasses terminal device has a built-in network status detection module, which is used to detect the connectivity status and network address of the current network interface. The AR glasses terminal device maintains a long connection session based on the transmission control protocol with the cloud communication server. When the network status detection module detects that the network connection has been interrupted and then restored, or detects that the local network address has changed, the AR glasses terminal device disconnects the current long connection session and re-initiates a connection request and identity authentication to the cloud communication server. When the cloud communication server receives a reconnection request for the same device identifier, if the old long connection session corresponding to the device identifier is still active, the old long connection session is terminated and a new long connection session is established. At the same time, the mapping relationship between the device identifier and the session is updated.

2. The fire scene collaborative auxiliary monitoring system based on fire-fighting AR glasses according to claim 1, characterized in that, The fire monitoring information includes fire video information. The AR glasses terminal device includes a visible light image acquisition module and a thermal imaging image acquisition module. The visible light image acquisition module acquires visible light images of the fire based on the video acquisition interface. The thermal imaging image acquisition module acquires images of the fire temperature distribution based on an infrared thermal imaging sensor component. The AR glasses terminal device switches between visible light mode and thermal imaging mode based on response commands, and encodes the real-time images in the current acquisition mode and pushes them to the cloud communication server.

3. A fire scene collaborative auxiliary monitoring system based on fire-fighting AR glasses according to claim 1, characterized in that, The fire monitoring information and the command and control information include text interaction information. The AR glasses terminal device encapsulates the text information input by the firefighter into a first text frame and sends it to the command terminal device via the cloud communication server. The command terminal device encapsulates the input text information into a second text frame, which is then sent to the AR glasses terminal device via the cloud communication server.

4. A fire scene collaborative auxiliary monitoring system based on fire-fighting AR glasses according to claim 1, characterized in that, The fire monitoring information and the command and control information include voice interaction information. The AR glasses terminal device uploads the first voice message input by the firefighter to the object storage service module of the cloud communication server. Based on the object storage service module, a first voice resource address is generated and encapsulated into a first audio frame and sent to the command terminal device. The command terminal device receives the first audio frame, parses it to obtain the first voice resource address, and plays the first voice message. The command terminal device uploads the second voice message input by the command personnel to the object storage service module of the cloud communication server. Based on the object storage service module, it generates a second voice resource address and encapsulates it into a second audio frame, which is then sent to the AR glasses terminal device. The AR glasses terminal device receives the second audio frame, parses it to obtain the second voice resource address, and plays the second voice message.

5. A fire scene collaborative auxiliary monitoring system based on fire-fighting AR glasses according to claim 1, characterized in that, The collaborative auxiliary monitoring system also includes a wearable vital sign acquisition device, which is used to acquire heart rate and blood oxygen saturation values; the AR glasses terminal device includes a barometric altitude sensing unit, which is used to acquire the relative altitude value of the firefighter; the AR glasses terminal device is connected to the wearable vital sign acquisition device via a Bluetooth communication module, and the AR glasses terminal device encapsulates the altitude value, heart rate value, and blood oxygen saturation value into sensor data frames according to a preset period and uploads them to the cloud communication server.

6. A fire scene collaborative auxiliary monitoring system based on fire-fighting AR glasses according to claim 5, characterized in that, The command terminal device displays the online status, altitude, heart rate, and blood oxygen saturation of each firefighter on the monitoring interface; the command terminal device maps the firefighter's position to a floor zoning view of the fire scene based on the altitude value; When the height value is within a preset floor range, the command terminal device displays the firefighter's information on an independent interface.

7. A fire scene collaborative auxiliary monitoring system based on fire-fighting AR glasses according to claim 1, characterized in that, The command terminal device sends an emergency evacuation command frame to the designated AR glasses terminal device through the cloud communication server; after receiving the emergency evacuation command frame, the AR glasses terminal device displays evacuation prompt information on its display module and controls the audio playback module to emit an alarm sound.

8. A fire scene collaborative auxiliary monitoring system based on fire-fighting AR glasses according to claim 1, characterized in that, The AR glasses terminal device also includes a Bluetooth broadcast module, which is used to broadcast the device's unique identifier. The command terminal device reads the device's unique identifier via Bluetooth, binds the device's unique identifier with the firefighter's identity information, and then uploads it to the cloud communication server. The cloud communication server verifies the binding relationship when the AR glasses terminal device initiates a connection, and only allows the AR glasses terminal device to establish the long connection session after the binding verification is successful.

9. A fire scene collaborative auxiliary monitoring system based on fire-fighting AR glasses according to claim 8, characterized in that, The command terminal device sends a wireless network configuration command to the AR glasses terminal device via Bluetooth. The wireless network configuration command includes a wireless network scanning command, a wireless network connection command, or a status refresh command. After executing the wireless network connection command, the AR glasses terminal device queries the currently active wireless network access point identifier, compares the currently active wireless network access point identifier with the target network name, and returns the configuration status to the command terminal device based on the comparison result.

10. A collaborative auxiliary monitoring method for fire scenes based on fire-fighting AR glasses, characterized in that, The fire-fighting AR glasses fire scene collaborative auxiliary monitoring system, as described in any one of claims 1 to 9, comprises: The AR glasses terminal device establishes a long connection session with the cloud communication server based on the transmission control protocol, and uploads the collected fire monitoring information to the cloud communication server. The command terminal device receives the fire monitoring information through the cloud communication server and sends command and control information to the AR glasses terminal device based on the fire monitoring information; The AR glasses terminal device uses a built-in network status detection module to detect the connectivity status and network address of the current network interface. When the network status detection module detects that the network connection has been interrupted and then restored, or when it detects that the local network address has changed, the AR glasses terminal device disconnects the current long connection session and re-initiates a connection request and identity authentication to the cloud communication server. When the cloud communication server receives a reconnection request for the same device identifier, if the old long connection session corresponding to the device identifier is still active, the old long connection session is terminated and a new long connection session is established. At the same time, the mapping relationship between the device identifier and the session is updated.

Images