Intelligent emergency rescue multifunctional persuasion device, system and method based on multi-modal perception

Abstract

This application relates to the technical field of intelligent management of scenic areas, and discloses a multi-functional intelligent emergency rescue and persuasion device, system, and method based on multi-modal perception. It includes a communication module, a management backend, and: a camera module for acquiring image information of the managed area; an infrared thermal imaging module for acquiring thermal imaging information of the managed area; a meteorological sensor for acquiring meteorological information of the managed area; a risk identification module for identifying risks based on the image information, thermal imaging information, and real-time meteorological information of the managed area; a persuasion and early warning module for issuing persuasion and distress calls based on the risk identification results; and a drone for conducting risk patrols based on distress calls. This invention dynamically adjusts the judgment process based on real-time meteorological information during the assessment of personnel safety in scenic areas, improving the rationality of the judgment process, enhancing the user experience for tourists, and ensuring the personal and property safety of tourists.

Description

Technical Field

[0001] This application relates to the technical field of intelligent management of scenic areas, and in particular to a multi-functional intelligent emergency rescue persuasion device, system and method based on multi-modal perception. Background Technology

[0002] In the process of scenic area management, ensuring the safety of people's lives and property is the top priority. Therefore, scenic areas will set up facilities such as guardrails and warning signs to remind tourists to avoid entering dangerous areas and reduce the occurrence of safety accidents. With the development trend of digitalization and informatization in scenic areas, many intelligent devices are applied to the safety management of scenic areas.

[0003] In existing technologies, intelligent safety management devices for scenic areas mainly rely on surveillance cameras installed at various locations within the scenic area. By identifying the location and behavior of personnel, risk warnings are issued, and informational devices such as speakers and displays within the scenic area are used to advise and guide visitors. However, existing intelligent scenic area safety management systems have the following shortcomings: First, the scope of dangerous areas varies depending on the scenario when assessing safety. Overly strict standards can negatively impact the visitor experience, while overly lenient standards cannot fully guarantee visitor safety. Second, surveillance cameras are affected by environmental factors (such as foggy weather), leading to inaccurate visitor identification and rendering the intelligent scenic area safety management system inoperable. Therefore, the fundamental problem this invention aims to solve is how to dynamically and reasonably adjust the standards for risk assessment of personnel in scenic areas and ensure the continuous operation of the scenic area safety management system. Summary of the Invention

[0004] In order to dynamically and reasonably adjust the standards for risk assessment of personnel in scenic areas and ensure the continuous operation of the scenic area safety management system, this application provides a multi-functional intelligent emergency rescue persuasion device, system and method based on multi-modal perception.

[0005] Firstly, this application provides a multi-functional intelligent emergency rescue persuasion system based on multi-modal perception, employing the following technical solution: A multi-modal sensing-based intelligent emergency rescue multi-functional persuasion system, including a communication module, a management backend, and: The camera module is used to acquire image information of the managed area; Infrared thermal imaging module, used to acquire thermal imaging information of the managed area; Meteorological sensors are used to acquire meteorological information for the managed area; The risk identification module is used to identify risks based on image information, thermal imaging information and real-time meteorological information of the managed area. The advisory and early warning module is used to issue advisory and distress commands based on the results of risk identification. Drones are used for risk patrols in response to distress calls.

[0006] By adopting the above technical solutions, the judgment process for assessing the safety of personnel in scenic areas can be dynamically adjusted using real-time meteorological information. This improves the rationality of the judgment process, enhances the user experience for tourists, and ensures the safety of tourists' persons and property.

[0007] Optionally, the risk identification process includes: Based on meteorological information, the management area is classified into risk areas and safe areas are identified. Personnel in the managed area are identified based on video and thermal imaging information. When personnel are identified as being in a risk area, their location is determined. The location of the identified personnel is then compared with the risk area and the safe area to determine the risk identification result.

[0008] By adopting the above technical solution, the management area is classified according to the risk level assessment method, thereby enabling dynamic adjustment of the assessment process based on real-time meteorological information. This improves the rationality of the assessment process, enhances the user experience for tourists, and ensures the personal and property safety of tourists.

[0009] Optionally, the system further includes an audio acquisition module for acquiring audio information of the management area; The audio information is identified to determine whether a person is sending a distress signal. If so, then issue a distress signal; If not, obtain the boundary lines of the risk area and the safe area, and determine the relative distance between the personnel's location and the boundary lines: If a person is in a safe area and the minimum relative distance to the critical line is greater than the preset distance, no advice or distress call will be issued. If the minimum relative distance between a person and the critical line is less than or equal to the preset distance, then both a warning instruction and a distress instruction will be issued simultaneously. If a person is in a risk area and the minimum relative distance to the critical line is greater than the preset distance, a distress signal will be issued.

[0010] By adopting the above technical solutions, identifying and judging audio information, and taking different countermeasures based on the minimum relative distance between the person and the critical line, the timeliness of rescue can be improved.

[0011] Optionally, the process of risk delineation for the management area includes: Pre-determine the correlation coefficient between the management area and each piece of meteorological information based on the type of management area; Each meteorological information is compared with its corresponding safe range. If the meteorological information is within the corresponding safe range, the risk value of the meteorological information is determined to be 0. If the meteorological information is not within the corresponding safe range, the absolute value of the difference between the corresponding value of the meteorological information and the median value of the corresponding safe range is normalized, and the absolute value of the normalized difference is taken as the risk value of the meteorological information. Calculate the sum of the products of all meteorological information correlation coefficients and risk values, compare the sum of the products with a preset threshold range, determine the risk level based on the threshold range in which the sum of the products falls, and classify the management area according to the classification method corresponding to the determined risk level.

[0012] By adopting the above technical solution, the management area is classified according to the risk level assessment method, thereby enabling dynamic adjustment of the assessment process based on real-time meteorological information. This improves the rationality of the assessment process, enhances the user experience for tourists, and ensures the personal and property safety of tourists.

[0013] Optionally, the process of identifying personnel within the managed area includes: Extract images from the image information, recognize the images based on a convolutional neural network, and obtain the first number of people in the managed area; Human body thermal maps are captured based on thermal imaging information, and the second number of people in the managed area is obtained by statistical analysis based on the human body thermal maps. If the first number and the second number are the same, the personnel are located based on the images in the image information; If the first number and the second number are different, the personnel are located according to the human body heat map.

[0014] By adopting the above technical solution, the accuracy of personnel identification in the management area can be guaranteed; when personnel are identified as being in a risk area, their location is determined, and the location of the personnel, along with the risk area and the safe area, is used as the result of risk identification.

[0015] Optionally, the process of a drone conducting a risk patrol based on a distress call includes: Control the drone to fly to the designated personnel location, acquire real-time images of the personnel's location through the drone, and transmit the real-time images to the management backend; Managers can issue rescue instructions by viewing real-time images in the management backend.

[0016] By adopting the above technical solutions, drones can be used to carry out preliminary steps in emergency rescue, thereby improving the efficiency of emergency rescue.

[0017] Optionally, the system further includes a solar power supply module and a satellite communication module; the solar power supply module provides power to the system, and the satellite communication module sends a distress signal to the drone and the management backend when the communication module loses signal.

[0018] By adopting the above technical solution, the system of this embodiment can still operate in dangerous scenarios such as no mains power, weak network, or no network.

[0019] Secondly, this application provides a smart emergency rescue multi-functional persuasion device based on multi-modal perception, which adopts the following technical solution: The device is a smart emergency rescue multi-functional persuasion device based on multi-modal perception, wherein the device operates any one of the above-mentioned smart emergency rescue multi-functional persuasion systems based on multi-modal perception.

[0020] Thirdly, this application provides a multi-modal sensing-based intelligent emergency rescue multi-functional persuasion method, which adopts the following technical solution: A multi-modal perception-based intelligent emergency rescue multi-functional persuasion method, wherein the method is executed by any one of the multi-modal perception-based intelligent emergency rescue multi-functional persuasion systems described above; including: Step 1: Acquire image and thermal imaging information of the managed area; Step 2: Obtain meteorological information for the managed area; Step 3: Conduct risk identification based on the image information, thermal imaging information, and real-time meteorological information of the managed area; issue warnings and distress calls based on the results of the risk identification. Step 4: Control the drone to conduct risk patrol according to the distress call instructions.

[0021] In summary, this application includes at least one of the following beneficial technical effects: This invention dynamically adjusts the assessment process for personnel safety in scenic areas using real-time weather information, improving the rationality of the assessment and enhancing the visitor experience while ensuring the safety of visitors' lives and property. The dual setup of a camera module and an infrared thermal imaging module avoids the problem of inaccurate visitor identification due to environmental factors. Attached Figure Description

[0022] Figure 1 This is a logic diagram of the intelligent emergency rescue multi-functional persuasion system based on multi-modal perception, which is based on the present invention.

[0023] Figure 2 This is a flowchart of the multi-modal perception-based intelligent emergency rescue multi-functional persuasion method of the present invention. Detailed Implementation

[0024] The embodiments of this application are described in detail below, and examples of the embodiments are shown in the accompanying drawings.

[0025] In the description of this specification, the references to "certain embodiments," "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples" refer to specific features, structures, materials, or characteristics described in connection with the described embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0026] This application discloses a multi-modal sensing-based intelligent emergency rescue multi-functional persuasion system, referring to... Figure 1 The system includes a communication module, a management backend, a camera module, an infrared thermal imaging module, a meteorological sensor, a risk identification module, a warning and guidance module, and a drone. When the system is running, the various modules work together to achieve a series of steps for the safety of people in the scenic area, from early warning to persuasion and then to emergency rescue.

[0027] It should be noted that the system in this embodiment is applied to high-risk public places such as scenic spots, so the modules and judgment strategies involved in the system are different from those of conventional monitoring systems in scenic spots.

[0028] The intelligent emergency rescue multi-functional guidance system in this embodiment is equipped with a camera module and an infrared thermal imaging module. The camera module is used to acquire image information of the managed area, and the infrared thermal imaging module is used to acquire thermal imaging information of the managed area. By using both camera and infrared thermal imaging modules, the problem of inaccurate identification of tourists due to environmental factors is avoided. At the same time, this embodiment is equipped with a meteorological sensor to acquire meteorological information of the managed area, including real-time rainfall, wind speed, high temperature, and probability of severe convective weather. Therefore, in the process of assessing the safety of people in the scenic area, the judgment process can be dynamically adjusted by using real-time meteorological information, which can improve the rationality of the judgment process, enhance the user experience of tourists, and ensure the personal and property safety of tourists.

[0029] The intelligent emergency rescue multi-functional persuasion system in this embodiment identifies risks through a risk identification module. This module identifies risks based on image information, thermal imaging information, and real-time meteorological information of the management area. This process includes: firstly, classifying the management area based on meteorological information to obtain risk areas and safe areas. This process can adjust the classification of the scenic area management area based on real-time meteorological conditions. In this embodiment, the risk classification process includes pre-setting the correlation coefficient between the management area and each piece of meteorological information according to the type of management area. Since the safety risks of different locations in the scenic area are related to different meteorological data, the correlation coefficient between each management area and different meteorological information is pre-set according to its type. For example, for low-lying areas such as canyons, excessive rainfall can cause flooding safety risks, so the correlation coefficient between the management area and rainfall is relatively large. For cliff areas, excessive wind speed poses a high safety risk, so the correlation coefficient between the management area and wind speed is relatively large. Then, each piece of meteorological information is judged individually, and each piece of meteorological information is compared with its corresponding safe zone. If the meteorological information falls within a certain safe range, it indicates a low safety risk to the managed area. Therefore, if the meteorological information is within the corresponding safe range, its risk value is determined to be 0. If the meteorological information is not within the corresponding safe range, a comprehensive analysis of various meteorological information is performed. The absolute value of the difference between the corresponding value of the meteorological information and the median value of the corresponding safe range is normalized, and the normalized absolute value of the difference is taken as the risk value of the meteorological information. The sum of the products of the correlation coefficients of all meteorological information and the risk value is calculated. The sum of the products is compared with a preset threshold range. The risk level is determined based on the threshold range in which the sum of the products falls. The threshold ranges are established based on big data fitting, and each threshold range corresponds to a risk level. However, the scenic area management personnel pre-determine a classification method according to the specific state of the managed area based on the risk level. Therefore, after determining the risk level, the managed area can be classified according to the classification method corresponding to the determined risk level. This allows for dynamic adjustment of the judgment process based on real-time meteorological information, improving the rationality of the judgment process, enhancing the user experience for tourists, and ensuring the personal and property safety of tourists.

[0030] Subsequently, the risk identification module in this embodiment identifies personnel in the managed area based on image and thermal imaging information. This process integrates image and thermal imaging information to obtain the most accurate judgment structure. First, the image is extracted from the image information, and the image is identified based on a convolutional neural network to obtain the first number of personnel in the managed area. This process utilizes existing face recognition technology and has high accuracy when the camera module's field of view is not interfered with. Simultaneously, the risk identification module captures a human body heat map based on the thermal imaging information and obtains a second number of personnel in the managed area based on the human body heat map. This process utilizes existing recognition algorithms and can be implemented even when the camera module's field of view is interfered with. The system can still identify personnel (though the overall accuracy is slightly lower than image recognition). If the first and second numbers are the same, the personnel are located based on the image in the video information. This ensures the accuracy of the identification results. If the first and second numbers are different, the personnel are located based on the human body heat map, indicating that the camera module's field of view is being interfered with. In this case, the personnel are located based on the human body heat map. Through the above identification process, the accuracy of identifying personnel in the managed area can be guaranteed. When personnel are identified as being in a risk area, their location is determined, and the location of the personnel, along with the risk area and the safe area, serves as the result of risk identification.

[0031] The system in this embodiment also includes an audio acquisition module, which can acquire audio information of the managed area. Based on existing speech recognition and AI technologies, it can identify and judge the audio information to determine whether a person has issued a distress signal. When a distress signal is detected, a distress command is issued, controlling the drone to conduct on-site patrols and simultaneously uploading the data to the management backend to notify management personnel to collect corresponding emergency rescue measures based on the real-time status. If it is determined that no distress signal has been issued, the system obtains the boundary lines of the risk area and the safe area, and judges the relative distance between the person's position and the boundary lines: if the person is in the safe area and the minimum relative distance from the boundary line is greater than a preset distance, it indicates that the person's status is relatively safe, and the warning module does not issue warning or distress commands. If the minimum relative distance between the person and the boundary line is less than or equal to the preset distance, it indicates that there is a high probability that the person has entered the risk area, and warning and distress commands are issued to reduce the probability of tourists entering the risk area and to promptly judge the tourists' real-time status. If the person is in the risk area and the minimum relative distance from the boundary line is greater than the preset distance, a distress command is issued to improve the timeliness of rescue.

[0032] Furthermore, the drone in this embodiment can conduct risk patrols based on distress commands. This process includes: first, controlling the drone to fly to the located personnel position, acquiring real-time images of the personnel position through the drone, and transmitting the real-time images to the management backend; the management personnel can issue rescue instructions by viewing the real-time images in the management backend. Through this process, the drone can realize the preliminary steps of emergency rescue, improving the efficiency of emergency rescue.

[0033] In one embodiment, the intelligent emergency rescue multi-functional persuasion system also includes a solar power supply module and a satellite communication module; the solar power supply module provides power to the system, and the satellite communication module sends a distress signal to the drone and the management backend when the communication module loses signal. Through the above modules, the system of this embodiment can still be guaranteed to operate in dangerous scenarios with no mains power, weak network / no network.

[0034] In one embodiment, a multi-modal sensing-based smart emergency rescue multi-functional persuasion device is also provided. This device integrates various modules of the system and is installed in the management area of ​​the scenic spot, enabling it to persuade tourists and provide emergency rescue.

[0035] In another embodiment, a multi-modal perception-based intelligent emergency rescue multi-functional persuasion method is also provided, referring to... Figure 2 The process includes: Step 1, acquiring image and thermal imaging information of the managed area; Step 2, acquiring meteorological information of the managed area; Step 3, identifying risks based on the image, thermal imaging, and real-time meteorological information of the managed area; issuing warnings and distress calls based on the risk identification results; Step 4, controlling drones to conduct risk patrols based on distress calls. Through these steps, the judgment process can be dynamically adjusted using real-time meteorological information, improving the rationality of the judgment process, enhancing the user experience for tourists, and ensuring the personal and property safety of tourists.

[0036] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A multi-modal sensing-based intelligent emergency rescue multi-functional persuasion system, characterized in that: Includes communication module, management backend and: The camera module is used to acquire image information of the managed area; Infrared thermal imaging module, used to acquire thermal imaging information of the managed area; Meteorological sensors are used to acquire meteorological information for the managed area; The risk identification module is used to identify risks based on image information, thermal imaging information and real-time meteorological information of the managed area. The advisory and early warning module is used to issue advisory and distress commands based on the results of risk identification. Drones are used for risk patrols in response to distress calls.

2. The intelligent emergency rescue multi-functional persuasion system based on multi-modal perception according to claim 1, characterized in that, The process of risk identification includes: Based on meteorological information, the management area is classified into risk areas and safe areas are identified. Personnel in the managed area are identified based on video and thermal imaging information. When personnel are identified as being in a risk area, their location is determined. The location of the identified personnel is then compared with the risk area and the safe area to determine the risk identification result.

3. The intelligent emergency rescue multi-functional persuasion system based on multi-modal perception according to claim 2, characterized in that, The system also includes an audio acquisition module for acquiring audio information of the management area; The audio information is identified to determine whether a person is sending a distress signal. If so, issue a distress signal; If not, obtain the boundary lines of the risk area and the safe area, and determine the relative distance between the personnel's location and the boundary lines: If a person is in a safe area and the minimum relative distance to the critical line is greater than the preset distance, no advice or distress call will be issued. If the minimum relative distance between a person and the critical line is less than or equal to the preset distance, then both a warning instruction and a distress instruction will be issued simultaneously. If a person is in a risk area and the minimum relative distance to the critical line is greater than the preset distance, a distress signal will be issued.

4. The intelligent emergency rescue multi-functional persuasion system based on multi-modal perception according to claim 2, characterized in that, The process of risk classification for management areas includes: Pre-determine the correlation coefficient between the management area and each piece of meteorological information based on the type of management area; Each meteorological information is compared with its corresponding safe range. If the meteorological information is within the corresponding safe range, the risk value of the meteorological information is determined to be 0. If the meteorological information is not within the corresponding safe range, the absolute value of the difference between the corresponding value of the meteorological information and the median value of the corresponding safe range is normalized, and the absolute value of the normalized difference is taken as the risk value of the meteorological information. Calculate the sum of the products of all meteorological information correlation coefficients and risk values, compare the sum of the products with a preset threshold range, determine the risk level based on the threshold range in which the sum of the products falls, and classify the management area according to the classification method corresponding to the determined risk level.

5. The intelligent emergency rescue multi-functional persuasion system based on multi-modal perception according to claim 2, characterized in that, The process of identifying personnel within the managed area includes: Extract images from the image information, recognize the images based on a convolutional neural network, and obtain the first number of people in the managed area; Human body thermal maps are captured based on thermal imaging information, and the second number of people in the managed area is obtained by statistical analysis based on the human body thermal maps. If the first number and the second number are the same, the personnel are located based on the images in the image information; If the first number and the second number are different, the personnel are located based on the human body heat map.

6. The intelligent emergency rescue multi-functional persuasion system based on multi-modal perception according to claim 1, characterized in that, The process of a drone conducting a risk patrol based on a distress call includes: Control the drone to fly to the designated personnel location, acquire real-time images of the personnel's location through the drone, and transmit the real-time images to the management backend; Managers can issue rescue instructions by viewing real-time images in the management backend.

7. The intelligent emergency rescue multi-functional persuasion system based on multi-modal perception according to claim 1, characterized in that, The system also includes a solar power module and a satellite communication module; the solar power module provides power to the system, and the satellite communication module sends a distress signal to the drone and the management backend when the communication module loses signal.

8. A multi-modal sensing-based intelligent emergency rescue multi-functional persuasion device, characterized in that: The device operates a smart emergency rescue multi-functional persuasion system based on multi-modal perception as described in any one of claims 1-7.

9. A multi-modal sensing-based intelligent emergency rescue multi-functional persuasion method, characterized in that: The method is executed by the intelligent emergency rescue multi-functional persuasion system based on multi-modal perception as described in any one of claims 1-7; include: Step 1: Acquire image and thermal imaging information of the managed area; Step 2: Obtain meteorological information for the managed area; Step 3: Conduct risk identification based on the image information, thermal imaging information, and real-time meteorological information of the managed area; issue warnings and distress calls based on the results of the risk identification. Step 4: Control the drone to conduct risk patrol according to the distress call instructions.

Images