Rescue system based on multi-mode positioning

Abstract

The invention discloses a rescue system based on multi-mode positioning, and belongs to the technical field of positioning rescue, and the system comprises an alarm receiving module which is configured to receive an alarm request sent by an alarm person through an alarm terminal, and carry out the positioning of the position of the alarm person; the data acquisition module is configured to acquire state information and surrounding environment videos of alarm personnel; the analysis module comprises a state evaluation module and an overall evaluation module, and the state evaluation module is configured to evaluate the health condition of the alarm personnel based on the state information of the alarm personnel to generate a first sub-result; the overall evaluation module is configured to perform analysis based on the surrounding environment video to generate a second sub-result; and the fusion decision module is configured to formulate a rescue scheme based on the first sub-result and the second sub-result.

Description

Technical Field

[0001] The present invention relates to the technical field of positioning and rescue, and in particular to a rescue system based on multi-modal positioning. Background Art

[0002] When an alarm is received, positioning is performed to quickly and accurately determine the location of the trapped person so that the rescue team can arrive at the scene as quickly as possible and carry out rescue operations in a timely and effective manner, thereby minimizing casualties and losses. Positioning information can help rescuers avoid wasting time and resources in complex or unknown environments, and can also ensure that they take the most appropriate rescue path and method, thereby improving the success rate and efficiency of rescue.

[0003] However, most of the existing alarm technologies use telephone alarms, and the police will go out to rescue after the telephone alarm. This method cannot effectively obtain the surrounding conditions and information of the alarm personnel, and cannot formulate a rescue plan that is more in line with the actual situation, which limits the targetedness in emergency rescue situations. Summary of the invention

[0004] In view of the above-mentioned prior art, the present invention provides a rescue system based on multi-modal positioning, which mainly solves the technical problems existing in the above-mentioned background technology.

[0005] To achieve the above object, the technical solution of the embodiment of the present invention is implemented as follows:

[0006] A rescue system based on multi-modal positioning, the system comprising:

[0007] An alarm receiving module is configured to receive an alarm request sent by an alarming person through an alarming terminal and locate the position of the alarming person;

[0008] The data acquisition module is configured to collect status information of the alarm personnel and surrounding environment video;

[0009] The analysis module includes a status evaluation module and an overall evaluation module, wherein the status evaluation module is configured to evaluate the health status of the alarming person based on the status information of the alarming person to generate a first sub-result; the overall evaluation module is configured to analyze based on the surrounding environment video to generate a second sub-result;

[0010] The fusion decision module is configured to formulate a rescue plan based on the first sub-result and the second sub-result.

[0011] Optionally, locating the position of the alarm person specifically includes: locating the position of the alarm person by combining the satellite signal and the network signal when there is a satellite signal and a network signal;

[0012] When there is a satellite signal but no network signal, basic positioning is performed through satellite signals, and 2.4GHz communication is used to assist communication and positioning;

[0013] When there is no satellite signal or network signal, communication and positioning are carried out through 2.4GHz communication.

[0014] Optionally, the status information of the alarming person includes health data information, and the health data information includes heart rate, body temperature, blood pressure, respiratory rate, and blood oxygen saturation.

[0015] Optionally, the step of evaluating the health status of the alarming person based on the status information of the alarming person to generate a first sub-result specifically includes:

[0016] Acquire a historical physiological data set of a normal human body, input the historical physiological data set into XGBoost, sort each physiological data in the historical physiological data feature set, and screen features;

[0017] The screening features obtained by the XGBoost are used to calculate the weights of the screening features based on the support vector machine, and then the comprehensive index features are obtained by weighted summation of the screening features;

[0018] Constructing a health prediction model, inputting the comprehensive indicator characteristics and the historical physiological data set into the health prediction model for training, using the cross entropy loss function and the stochastic gradient descent optimization algorithm to forward propagate the health prediction model, calculate the loss, and obtain a trained health prediction model after iterative updating;

[0019] The status information of the alarming person is input into the health prediction model, the health status category of the alarming person is predicted, and a first sub-result is generated.

[0020] Optionally, the generating a second sub-result by analyzing based on the surrounding environment information specifically includes:

[0021] Obtain rescue image data from historical cases in different scenarios;

[0022] Construct a scene recognition model, input the rescue image data into the scene recognition model based on YOLOV8 for training, and obtain a trained scene recognition model;

[0023] Based on the surrounding environment video, several frames of surrounding environment image data are obtained; the first frame of surrounding environment image data is input into the scene recognition model to perform scene recognition, and when a disaster scene is identified, the area ratio of the target scene to the surrounding environment image is calculated to generate a second sub-result, and if the area ratio exceeds a preset threshold, an early warning instruction is generated; if no disaster scene is identified, the next frame of surrounding environment image data is read.

[0024] Optionally, the step of formulating a rescue plan based on the first sub-result and the second sub-result specifically includes:

[0025] fusing the first sub-result and the second sub-result to obtain a fusion result, classifying the disaster level according to the fusion result, and formulating a rescue plan based on the disaster level;

[0026] The disaster levels include: Level I high risk, Level II medium risk, and Level III low risk.

[0027] Optionally, formulating a rescue plan based on the disaster level specifically includes:

[0028] When the disaster level is Level I high risk, the highest level of emergency response mechanism will be activated and rescue resources will be dispatched immediately;

[0029] When the disaster level is medium risk at level II, the intermediate-level emergency response mechanism will be activated to quickly dispatch rescue resources;

[0030] When the disaster level is level III low risk, the basic level emergency response mechanism will be activated and rescue resources will be appropriately dispatched;

[0031] The rescue resources include rescue personnel and medical staff, medical equipment and basic necessities of life.

[0032] The beneficial effects of the present invention are:

[0033] A rescue system based on multimodal positioning provided by an embodiment of the present invention locates the position of the alarm person after receiving an alarm request sent by the alarm person, and collects the status information and surrounding environment information of the alarm person, analyzes and evaluates the collected status information and surrounding environment information of the alarm person through an analysis module, generates a first sub-result and a second sub-result, and formulates a rescue plan based on the first sub-result and the second sub-result through a fusion decision module, so that corresponding strategies can be better made according to the actual situation. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A schematic diagram of a process of a rescue system based on multi-modal positioning provided in an embodiment of the present invention;

[0035] Figure 2The present invention provides a schematic diagram of a positioning process of a rescue system based on multi-modal positioning in an embodiment of the present invention. DETAILED DESCRIPTION

[0036] The technical solution of the present invention is further elaborated in detail below in conjunction with the drawings and specific embodiments of the specification. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention. In the following description, the expression "some embodiments" is related to a subset of all possible embodiments, but it should be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0037] In the following description, a large number of specific details are provided to provide a more thorough understanding of the present invention. However, it is apparent to those skilled in the art that the present invention can be implemented without one or more of these details. In other examples, in order to avoid confusion with the present invention, some technical features well known in the art are not described.

[0038] It should be understood that the present invention can be implemented in different forms and should not be interpreted as being limited to the embodiments proposed herein. On the contrary, providing these embodiments will make the disclosure thorough and complete, and the scope of the present invention will be fully conveyed to those skilled in the art. And the purpose of the terms used herein is only to describe specific embodiments and is not intended to be a limitation of the present invention. When used herein, the singular forms of "one", "one" and "said / the" are also intended to include plural forms, unless the context clearly indicates another way. It should also be understood that the terms "compose" and / or "include" when used in this specification determine the presence of the features, integers, steps, operations, elements and / or parts, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, parts and / or groups. When used herein, the term "and / or" includes any and all combinations of the relevant listed items.

[0039] It should also be noted that when an element is referred to as being "fixed to" another element, it may be directly on the other element or there may be a central element. When an element is considered to be "connected to" another element, it may be directly connected to the other element or there may be a central element at the same time. The terms "vertical", "horizontal", "inside", "outside", "left", "right" and similar expressions used herein are for illustrative purposes only and are not intended to be the only implementation method.

[0040] In order to fully understand the present invention, a detailed structure will be proposed in the following description to illustrate the technical solution proposed by the present invention. The optional embodiments of the present invention are described in detail as follows, but in addition to these detailed descriptions, the present invention may also have other implementations.

[0041] Example

[0042] Please refer to the attached Figure 1 and attached Figure 2 The present application provides a rescue system based on multi-modal positioning, the system comprising:

[0043] An alarm receiving module is configured to receive an alarm request sent by an alarming person through an alarming terminal and locate the position of the alarming person;

[0044] The data acquisition module is configured to collect status information of the alarm personnel and surrounding environment video;

[0045] The analysis module includes a status evaluation module and an overall evaluation module, wherein the status evaluation module is configured to evaluate the health status of the alarming person based on the status information of the alarming person to generate a first sub-result; the overall evaluation module is configured to analyze based on the surrounding environment video to generate a second sub-result;

[0046] a fusion decision module, configured to formulate a rescue plan based on the first sub-result and the second sub-result;

[0047] A rescue system based on multimodal positioning provided by the present invention is applied to emergency rescue scenarios, such as when disasters such as fire occur, or in emergency situations such as when people are injured. Specifically, when an alarm request sent by an alarm person is received, the position of the alarm person is located. For example, when an emergency occurs, the alarm person makes an emergency call or sends an alarm signal through a mobile phone or a bracelet and the like. After receiving the alarm signal, the alarm receiving module immediately locates the position of the alarm person through a positioning technology, such as using the GPS satellite positioning, WiFi, base station positioning, etc. of the device to determine the precise position of the alarm person. After the position of the alarm person is successfully located, the data acquisition module starts to work, collects the state information of the alarm person through the built-in sensor of the alarm device (such as a smart bracelet), and collects the environmental information around the alarm person through the sensor or external data source (such as a weather station, traffic monitoring, etc.), and sends the acquired state information of the alarm person and the environmental information around the alarm person to the analysis module. The analysis module evaluates the health status of the alarm person to generate a first sub-result and analyzes the surrounding environment video to generate a second sub-result. The fusion decision module fuses the first sub-result and the second sub-result, and formulates a corresponding rescue plan for the alarm person according to the fusion result.

[0048] As an optional implementation, locating the position of the alarm person specifically includes: locating the position of the alarm person by combining the satellite signal and the network signal when there is a satellite signal and a network signal;

[0049] When there is a satellite signal but no network signal, basic positioning is performed through satellite signals, and 2.4GHz communication is used to assist communication and positioning;

[0050] When there is no satellite signal or network signal, communication and positioning are carried out through 2.4GHz communication.

[0051] Emergency rescue may occur in different scenarios. Different communication technologies have different limitations when locating the alarm personnel in different scenarios. For example, satellite positioning signals are easily affected by the environment. In shielded environments such as cities with tall buildings, underground facilities or forests, satellite signals are weak or even cannot be received, resulting in reduced positioning accuracy or inability to locate. Satellite positioning consumes a lot of power, which affects the battery life of the device. The positioning response time is long, especially when the device is just started. Base station positioning requires the device to connect to multiple base stations. The positioning accuracy is limited by the distribution density of the base stations and the signal transmission environment. It is usually suitable for urban areas or places with poor satellite signals. The coverage of WiFi hotspots is limited, and the positioning accuracy depends on the distribution density of the hotspots. The positioning error is large, and in public areas or crowded environments, WiFi signal interference is large, which may lead to increased positioning errors; short-range wireless communication positioning technology (such as Bluetooth, UWB) has a limited communication distance, usually within a few dozen meters, which is difficult to use for large-scale positioning, and requires continuous communication between devices, which consumes a lot of power and is not suitable for long-term, low-power application scenarios; Therefore, the current positioning technologies have their own shortcomings and cannot meet the needs of all application scenarios, especially in complex environments, such as buildings, underground facilities or areas with weak signal coverage. It is difficult for a single positioning technology to achieve accurate and stable positioning services. This problem is mainly caused by the limitations of different technologies in terms of signal coverage, transmission distance, environmental adaptability and power consumption;

[0052] In the present invention, by combining satellite positioning, base station positioning, and short-range wireless communication, in the case of network signals and satellite signals, the base station signals in the mobile communication network are used to determine the location of the device. Specifically, by measuring the signal strength, arrival time difference (TDOA) and other information between the alarm device and multiple base stations, the distance and relative position of the alarm device are calculated through a triangulation algorithm. If the alarm device is connected to multiple base stations at the same time (usually at least three base stations are required), the two-dimensional or three-dimensional position of the device can be accurately determined through three-dimensional spatial positioning technology; in addition, combined with local map data and traffic IP address positioning information, the base station positioning results can be further calibrated and optimized; and combined with satellite positioning technology, the alarm device is used to receive multiple satellite signals, and the position of the alarm device is calculated by measuring the signal arrival time difference, so as to obtain a more accurate position of the alarm personnel;

[0053] When the network signal is poor or there is no network signal, you can rely on satellite positioning technology. Although the positioning accuracy may be reduced, it can still provide basic location information. Under extreme conditions, satellite positioning technology can still be used as a reliable backup method to ensure the continuity of rescue missions (such as text sending function).

[0054] In the absence of satellite signals or network signals, the use of 2.4GHz Wi-Fi technology as a portable communication means is particularly suitable for direct communication and positioning between devices in environments without network and satellite signals. On this basis, by enhancing the 2.4GHz signal power and networking technology, the communication coverage can be expanded. For example, rescuers can use high-power 2.4GHz communication equipment to achieve stable connection between devices in complex environments (such as buildings, underground facilities or forests) to ensure that the communication link is always maintained during the search and rescue process. Especially in the absence of other signal sources, 2.4GHz communication can be used to shorten the search and rescue distance and quickly locate trapped people or rescue equipment.

[0055] It should be noted that base station signal positioning and satellite signal positioning are preferred. In wild environments such as mountainous areas and forests, it can be basically determined that when an emergency occurs in wild environments such as mountainous areas and forests, the base station signal and satellite signal may be poor. At this time, positioning can be carried out by enhancing the 2.4GHz signal, transmitting signals between nodes, measuring the arrival time difference of signals between different nodes, and then calculating the target position coordinates of the alarm personnel.

[0056] As an optional implementation, the status information of the alarming person includes health data information, and the health data information includes heart rate, body temperature, blood pressure, respiratory rate, and blood oxygen saturation.

[0057] As an optional implementation manner, the health status of the alarming person is evaluated based on the status information of the alarming person to generate the first sub-result, specifically including:

[0058] Acquire a historical physiological data set of a normal human body, input the historical physiological data set into XGBoost, sort each physiological data in the historical physiological data feature set, and screen features;

[0059] The screening features obtained by the XGBoost are used to calculate the weights of the screening features based on the support vector machine, and then the comprehensive index features are obtained by weighted summation of the screening features;

[0060] Constructing a health prediction model, inputting the comprehensive indicator characteristics and the historical physiological data set into the health prediction model for training, using the cross entropy loss function and the stochastic gradient descent optimization algorithm to forward propagate the health prediction model, calculate the loss, and obtain a trained health prediction model after iterative updating;

[0061] Inputting the status information of the alarm person into the health prediction model, predicting the health status category of the alarm person, and generating a first sub-result;

[0062] Specifically, the historical physiological data set is input into the XGBoost model, and the XGBoost model is used to evaluate the importance of each physiological data in the physiological data feature set, and then the features are sorted and screened according to their importance. XGBoost can process large-scale data and can identify the features that have the greatest impact on model prediction; based on the screened features, the support vector machine will assign different weights according to the importance of the screened features, and then the weighted summation of the weighted screened features is performed to obtain comprehensive indicator features, and then the comprehensive indicator features and the historical physiological data set are input into the health prediction model for training, and optimized through the cross entropy loss function and stochastic gradient descent to obtain a trained health prediction model; then the real-time status information of the alarm personnel is input into the constructed health prediction model, and the health prediction model is used to analyze the status information of the alarm personnel to predict their health status category. For example, the health category can be healthy (all vital signs are within the normal range), injured (abnormal vital signs), severely injured (abnormal vital signs, possibly in an emergency), and the first sub-result is generated.

[0063] As an optional implementation manner, the analyzing based on the surrounding environment video to generate the second sub-result specifically includes:

[0064] Obtain rescue image data from historical cases in different scenarios;

[0065] Construct a scene recognition model, input the rescue image data into the scene recognition model based on YOLOV8 for training, and obtain a trained scene recognition model;

[0066] Based on the surrounding environment video, a plurality of frames of surrounding environment image data are obtained; the first frame of surrounding environment image data is input into the scene recognition model to perform scene recognition, and when a disaster scene is recognized, an area ratio of the target scene to the surrounding environment image is calculated to generate a second sub-result, and if the area ratio exceeds a preset threshold, an early warning instruction is generated; if no disaster scene is recognized, the next frame of surrounding environment image data is read;

[0067] Specifically, historical rescue image data in different scenarios are collected. These image data will be used to train a scene recognition model to identify different disaster scenes. YOLOV8 is used to build a scene recognition model. YOLOV8 can identify different objects and scenes from pictures. The collected rescue image data are input into a scene recognition model based on YOLOV8, and the scene recognition model is trained to determine the disaster scene in the picture to obtain a trained scene recognition model; the surrounding environment video collected in real time, such as the video stream monitoring data of a continuous period of time before and after the alarm personnel alarm, is input into the trained scene recognition model. The scene can be a fire scene, a forest, a seaside, etc., so as to identify the specific scene where the disaster occurs. For example, if a fire occurs, the scene recognition model identifies the area with fire and judges it as a fire scene, and then calculates the area value of the area with fire in the entire picture. If it exceeds the preset threshold, it means that the fire area is large and the fire is serious, then an early warning instruction is generated for reminder, and a second sub-result is generated based on the identified disaster scene.

[0068] As an optional implementation manner, the configuration of formulating a rescue plan based on the first sub-result and the second sub-result specifically includes:

[0069] fusing the first sub-result and the second sub-result to obtain a fusion result, classifying the disaster level according to the fusion result, and formulating a rescue plan based on the disaster level;

[0070] The disaster levels include: Level I high risk, Level II medium risk, and Level III low risk.

[0071] Specifically, when the person who called the police is identified as seriously injured, it will be defined as Level I high risk; when the person who called the police is identified as injured, it will be defined as Level II medium risk; when the person who called the police is identified as healthy, it will be defined as Level III low risk; when the scene is identified as a fire, earthquake, or other scene that may cause major casualties, it will be defined as Level I high risk; when a traffic accident or other scene that may cause minor injuries is identified, it will be defined as Level II medium risk; when a road blockage, equipment failure, or other scene that does not cause casualties is identified, it will be defined as Level III low risk; based on the above identification results, a corresponding rescue plan is formulated.

[0072] As an optional implementation, formulating a rescue plan based on the disaster level specifically includes:

[0073] When the disaster level is Level I high risk, the highest level of emergency response mechanism will be activated and rescue resources will be dispatched immediately;

[0074] When the disaster level is medium risk at level II, the intermediate-level emergency response mechanism will be activated to quickly dispatch rescue resources;

[0075] When the disaster level is level III low risk, the basic level emergency response mechanism will be activated and rescue resources will be appropriately dispatched;

[0076] The rescue resources include rescue personnel and medical staff, medical equipment and basic necessities of life.

[0077] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. The protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A rescue system based on multimodal positioning, characterized in that: The system comprises: An alarm receiving module is configured to receive an alarm request sent by an alarming person through an alarming terminal and locate the position of the alarming person; The data acquisition module is configured to collect status information of the alarm personnel and surrounding environment video; The analysis module includes a status evaluation module and an overall evaluation module, wherein the status evaluation module is configured to evaluate the health status of the alarming person based on the status information of the alarming person to generate a first sub-result; the overall evaluation module is configured to analyze based on the surrounding environment video to generate a second sub-result; The fusion decision module is configured to formulate a rescue plan based on the first sub-result and the second sub-result.

2. A rescue system based on multimodal positioning according to claim 1, characterized in that: Locating the position of the alarm person, specifically comprising: locating the position of the alarm person by combining the satellite signal and the network signal when there is a satellite signal and a network signal; When there is a satellite signal but no network signal, basic positioning is performed through satellite signals, and 2.4GHz communication is used to assist communication and positioning; When there is no satellite signal or network signal, communication and positioning are carried out through 2.4GHz communication.

3. A rescue system based on multimodal positioning according to claim 1, characterized in that: The status information of the alarming person includes health data information, and the health data information includes heart rate, body temperature, blood pressure, respiratory rate, and blood oxygen saturation.

4. A rescue system based on multimodal positioning according to claim 1, characterized in that: The step of evaluating the health status of the alarming person based on the status information of the alarming person to generate a first sub-result specifically includes: Obtain a historical physiological data set of a normal human body, input the historical physiological data set into XGBoost, sort each physiological data in the historical physiological data feature set, and screen features; The screening features obtained by the XGBoost are used to calculate the weights of the screening features based on the support vector machine, and then the comprehensive index features are obtained by weighted summation of the screening features; Constructing a health prediction model, inputting the comprehensive indicator characteristics and the historical physiological data set into the health prediction model for training, using the cross entropy loss function and the stochastic gradient descent optimization algorithm to forward propagate the health prediction model, calculate the loss, and obtain a trained health prediction model after iterative updating; The status information of the alarming person is input into the health prediction model, the health status category of the alarming person is predicted, and a first sub-result is generated.

5. A rescue system based on multi-modal positioning according to claim 1, characterized in that: The generating a second sub-result by analyzing the surrounding environment information specifically includes: Obtain rescue image data from historical cases in different scenarios; Constructing a scene recognition model, inputting the rescue image data into a scene recognition model based on YOLOV8 for training, and obtaining a trained scene recognition model; Based on the surrounding environment video, several frames of surrounding environment image data are obtained; the first frame of surrounding environment image data is input into the scene recognition model to perform scene recognition, and when a disaster scene is identified, the area ratio of the target scene to the surrounding environment image is calculated to generate a second sub-result, and if the area ratio exceeds a preset threshold, an early warning instruction is generated; if no disaster scene is identified, the next frame of surrounding environment image data is read.

6. The rescue system based on multi-modal positioning according to claim 1, characterized in that: The being configured to formulate a rescue plan based on the first sub-result and the second sub-result specifically comprises: fusing the first sub-result and the second sub-result to obtain a fusion result, classifying the disaster level according to the fusion result, and formulating a rescue plan based on the disaster level; The disaster levels include: Level I high risk, Level II medium risk, and Level III low risk.

7. A rescue system based on multi-modal positioning according to claim 6, characterized in that: The rescue plan formulated based on the disaster level specifically includes: When the disaster level is Level I high risk, the highest level of emergency response mechanism will be activated and rescue resources will be dispatched immediately; When the disaster level is medium risk at level II, the intermediate-level emergency response mechanism will be activated to quickly dispatch rescue resources; When the disaster level is level III low risk, the basic level emergency response mechanism will be activated and rescue resources will be appropriately dispatched; The rescue resources include rescue personnel and medical staff, medical equipment and basic necessities of life.

Images