Emergency rescue path planning method and system based on fall detection of old people

Abstract

The invention provides an emergency rescue path planning method and system based on old people falling detection, and relates to the technical field of path planning, and the method comprises the steps: collecting real-time video stream data in different regions of a pension community; inputting the real-time video stream data into a tumble detection model, and outputting an old man tumble detection result; when the old people fall down, determining accurate three-dimensional coordinates of the falling old people in a pre-constructed binocular SLAM map through a virtual binocular-scale consistency constraint algorithm; according to the accurate three-dimensional coordinates, semantic information in the binocular SLAM map and contextual information of rescuers, a Pareto optimal rescue path set is planned through a multi-objective optimization search algorithm; and introducing a dynamic weight vector, performing dynamic weighting operation on each Pareto optimal rescue path in the Pareto optimal rescue path set, and selecting the Pareto optimal rescue path with a minimum dynamic weighting result as a final rescue path.

Description

Technical Field

[0001] This invention relates to the field of path planning technology, and in particular to an emergency rescue path planning method and system based on fall detection in the elderly. Background Technology

[0002] With the deepening of population aging, falls have become one of the main risk factors threatening the lives, health, and personal safety of the elderly. In concentrated residential areas such as retirement communities, the elderly are frequently active and their physical functions are significantly declining. Once a fall occurs, if it is not detected and effectively rescued in time, it can easily lead to serious consequences such as fractures and internal bleeding, and even endanger their lives. Therefore, how to achieve rapid and accurate detection of falls in the elderly in retirement community environments, and on this basis, promptly carry out emergency rescue, has become an important technical problem that urgently needs to be solved in the field of smart elderly care.

[0003] In existing technologies, traditional fall detection methods for the elderly are mainly divided into wearable device-based and vision analysis-based methods. Wearable device-based methods typically use sensors such as accelerometers and gyroscopes built into smart bracelets, belts, etc., to detect sudden weightlessness or violent movement to determine whether a fall has occurred. However, this method relies on the elderly wearing the device continuously and correctly, and in practical applications, it is prone to failure due to forgetting to wear it, refusing to wear it, or insufficient device battery. It is also prone to false alarms during daily activities such as sitting down quickly or bending over. Vision analysis-based methods utilize video data collected by surveillance cameras to determine fall behavior through methods such as human posture recognition and motion trajectory analysis. It has the advantage of not requiring the wearable device. However, the monocular surveillance cameras deployed in this method have inherent scale uncertainty issues, which prevent them from being effectively fused with high-precision binocular SLAM maps, resulting in only providing fuzzy location information within the field of view of one camera.

[0004] In addition, traditional emergency rescue route planning methods mainly aim at the shortest path, without considering the complex environmental risk factors in elderly care communities, such as slippery ground, dense obstacles, insufficient lighting, and narrow corridors. The planned paths may not be suitable for rescuers to pass through quickly, and may even pose a risk of secondary injury. Summary of the Invention

[0005] In view of the shortcomings of the prior art, the purpose of this invention is to provide an emergency rescue route planning method based on fall detection for the elderly. This method can solve the problems of traditional fall detection methods for the elderly, which rely on the elderly to wear the device continuously and correctly. In practical applications, the device is prone to failure due to forgetting to wear it, refusing to wear it, or insufficient battery power. False alarms are also likely to occur during daily activities such as sitting down or bending over quickly. Due to the inherent scale uncertainty of the deployed monocular monitoring camera, it cannot be effectively fused with high-precision binocular SLAM maps, resulting in only providing fuzzy location information within the field of view of a single camera. Furthermore, traditional emergency rescue route planning methods mainly aim for the shortest path and do not consider the complex environmental risks in elderly care communities, such as slippery ground, dense obstacles, insufficient lighting, and narrow corridors. The planned path may not be suitable for the rapid passage of rescuers and may even pose a risk of secondary injury.

[0006] A first aspect of this invention proposes an emergency rescue path planning method based on fall detection in the elderly, comprising:

[0007] S1: Collect real-time video stream data from different areas of the senior living community;

[0008] S2: Input real-time video stream data into the fall detection model and output the fall detection results for the elderly;

[0009] S3: Based on the fall detection results, when there is a fall, the precise 3D coordinates of the fallen elderly person in the pre-built binocular SLAM map are determined through the virtual binocular-scale consistency constraint algorithm; otherwise, return to S1.

[0010] S4: Based on precise 3D coordinates, semantic information in the binocular SLAM map, and rescuer context information, a set of Pareto optimal rescue paths is planned using a multi-objective optimization search algorithm.

[0011] S5: Introduce a dynamic weight vector to dynamically weight each Pareto optimal rescue path in the Pareto optimal rescue path set, and select the Pareto optimal rescue path with the smallest dynamic weight result as the final rescue path.

[0012] S6: Perform rescue missions for elderly people who have fallen, based on the final rescue route.

[0013] A second aspect of this invention provides an emergency rescue path planning system based on fall detection in the elderly, comprising: a processor and a memory;

[0014] The memory stores programs or instructions that can run on the processor, which, when executed by the processor, implement the steps of the emergency rescue path planning method based on elderly fall detection as described in the first aspect.

[0015] A third aspect of the present invention provides a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the emergency rescue path planning method based on elderly fall detection as described in the first aspect.

[0016] The beneficial effects of the technical solutions provided by the embodiments of the present invention include at least the following:

[0017] In this embodiment of the invention, real-time video stream data from different areas of the elderly care community is collected and input into the fall detection model. The model outputs fall detection results for the elderly, and based on these results, it no longer relies on the elderly continuously and correctly wearing the device. This avoids the problem of devices failing due to forgetting to wear them, refusing to wear them, or insufficient battery power in practical applications. False alarms are less likely to occur during daily activities such as sitting down quickly or bending over. When a fall is detected, a virtual binocular-scale consistency constraint algorithm is used to determine the precise 3D coordinates of the fallen elderly person in a pre-constructed binocular SLAM map. This avoids the inherent scale uncertainty problem of deployed monocular monitoring cameras and allows for effective fusion with a high-precision binocular SLAM map. It can provide precise location information within the field of view of a certain camera. Based on precise 3D coordinates, semantic information in the binocular SLAM map, and contextual information of the rescuer, a set of Pareto optimal rescue paths is planned through a multi-objective optimization search algorithm. Dynamic weight vectors are introduced to dynamically weight each Pareto optimal rescue path in the set, and the Pareto optimal rescue path with the smallest dynamic weight result is selected as the final rescue path. It not only aims at the shortest path, but also considers the complex environmental risk factors in the elderly care community, such as slippery ground, dense obstacles, insufficient lighting, and narrow corridors. The planned path is more suitable for the rapid passage of rescuers and avoids the risk of secondary injury. Attached Figure Description

[0018] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts. Obviously, the drawings described below are merely some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without any creative effort.

[0019] Figure 1This is a flowchart illustrating an emergency rescue route planning method based on fall detection in the elderly, provided by an embodiment of the present invention.

[0020] Figure 2 This is a schematic diagram of the structure of an emergency rescue path planning system based on elderly fall detection provided in an embodiment of the present invention. Detailed Implementation

[0021] To enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. It should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0022] The emergency rescue path planning method based on elderly fall detection provided by the present invention will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0023] Reference manual attached Figure 1 The diagram illustrates a flowchart of an emergency rescue path planning method based on fall detection in the elderly, provided by an embodiment of the present invention.

[0024] This invention provides an emergency rescue route planning method based on fall detection in the elderly, which may include the following steps:

[0025] S1: Collect real-time video stream data from different areas of the senior living community.

[0026] Specifically, real-time video stream data consists of a sequence of consecutive frame images.

[0027] In one possible implementation, S1 specifically refers to:

[0028] By deploying multiple monocular surveillance cameras in different areas of the senior living community, real-time video stream data is collected from different areas of the senior living community.

[0029] In this embodiment of the invention, by deploying multiple monocular monitoring cameras in different areas of the elderly care community to collect real-time video stream data in the form of continuous frames, not only is the system hardware cost and deployment complexity reduced, but a stable and reliable data foundation is also provided for the temporal feature modeling of fall behavior, event spatial localization, and subsequent rescue path planning. This improves the accuracy of fall detection, the real-time performance of system response, and the engineering feasibility of the overall emergency rescue plan.

[0030] S2: Input real-time video stream data into the fall detection model and output the fall detection results for the elderly.

[0031] Specifically, the fall detection model is a fall detection model based on a spatiotemporal graph convolutional network.

[0032] It should be noted that spatiotemporal graph convolutional networks (SPCRNs) are deep learning models that simultaneously characterize spatial structural relationships and temporal dynamics. They model human joints as nodes in a graph structure, the topological connections between joints as edges, and introduce sequential relationships in the temporal dimension. This allows for the extraction of human posture structural features in the spatial dimension using graph convolution, and the modeling of continuous frame motion changes in the temporal dimension, achieving joint learning of temporal features of human behavior. This makes them suitable for the recognition and analysis of complex action patterns. SPCRNs are existing technology, and will not be elaborated upon further in this invention.

[0033] The fall detection results for the elderly include: whether the elderly person has fallen, whether the elderly person has not fallen, the pixel coordinates of the elderly person who fell in the current frame image, and the risk assessment score of the elderly person who fell.

[0034] In this embodiment of the invention, by using a fall detection model based on a spatiotemporal graph convolutional network to analyze real-time video stream data, not only is the human body structural features and action timing information fully utilized, improving the accuracy and stability of elderly fall recognition, but also the pixel coordinates of the fall location and risk assessment score are output simultaneously during the detection stage, providing direct and reliable input for subsequent three-dimensional spatial positioning and rescue path planning, thereby improving the intelligence level and response efficiency of the entire emergency rescue system.

[0035] S3: Based on the fall detection results, when there is a fall, the precise 3D coordinates of the fallen elderly person in the pre-built binocular SLAM map are determined through the virtual binocular-scale consistency constraint algorithm; otherwise, return to S1.

[0036] It should be noted that the Virtual Binocular-Scale Consistency Constraint Algorithm is a localization method that introduces a virtual baseline to construct a pseudo-binocular geometric relationship under monocular imaging conditions, and combines the scale information of the existing binocular SLAM map to constrain and optimize the spatial position of the target. It calculates disparity and estimates depth by generating a virtual right view, and uses the consistency constraint between the depth ratio and the scale of the 3D map points to construct an optimized model of joint pose and position, thereby achieving high-precision determination of the target's 3D coordinates without real binocular input.

[0037] In one possible implementation, determining the precise 3D coordinates of the fallen elderly person in the pre-built binocular SLAM map using the virtual binocular-scale consistency constraint algorithm in step S3 specifically includes sub-steps S301 to S307:

[0038] S301: Using a monocular surveillance camera as the left camera, a virtual right view is generated by introducing a predefined virtual baseline length.

[0039] In one possible implementation, S301 specifically includes sub-steps S3011 to S3014:

[0040] S3011: Calculate the coarse pose of the left camera.

[0041] In one possible implementation, S3011 specifically includes sub-steps S3011A to S3011D:

[0042] S3011A: In the current monocular image Extract ORB feature points to obtain the feature point set and corresponding feature descriptors of the current frame image.

[0043] S3011B: Global 3D map point cloud stored in a pre-built binocular SLAM map The keyframe information and its associated information are used to establish a correspondence between two-dimensional pixels and three-dimensional map points in the current monocular image and the three-dimensional map point cloud through a visual bag-of-words model or feature descriptor matching, forming several sets of keyframe information. Feature matching pairs ,in, This represents the pixel coordinates in the current monocular image. This represents the corresponding 3D map point.

[0044] S3011C: Based on Feature matching pairs are used to estimate camera pose using a pose estimation method based on random sampling consensus, specifically including:

[0045] First, a predetermined number of sample points are randomly selected from the feature matching pairs, and the EPnP algorithm is used to solve for the candidate camera pose. Second, the reprojection error of all feature matching pairs is calculated based on the candidate camera pose, and matching points with a reprojection error less than a predetermined threshold are identified as inliers. Finally, the above random sampling and pose estimation process is repeated, and the candidate camera pose with the most inliers is selected as the optimal hypothetical pose.

[0046] S3011D: Based on the optimal assumed pose and its corresponding set of inliers, the camera pose is refined using a nonlinear least squares optimization method to obtain a coarse pose for the left camera. .

[0047] S3012: Based on the coarse pose of the left camera, a coarse pose of the virtual right camera is generated by introducing a virtual baseline length.

[0048]

[0049]

[0050] Among them, T virtual This indicates the approximate pose of the virtual right camera. I represents the coarse pose of the left camera (i.e., the coarse pose of the monocular surveillance camera), I represents the 3×3 unit rotation matrix, and t represents the approximate pose of the left camera. v The subscript represents the translation vector of the virtual right camera relative to the left camera (i.e., the translation vector of the virtual right camera relative to the monocular surveillance camera). T This indicates the transpose operation.

[0051] S3013: Retrieve the 3D map point cloud within the field of view of the left camera in a coarse pose in a binocular SLAM map.

[0052] S3014: Based on the coarse pose of the virtual right camera, project the 3D map point cloud onto the virtual right camera plane to generate a virtual right view:

[0053]

[0054]

[0055] Among them, I virtual Let f represent the virtual right view, π represent the camera projection function, K represent the camera intrinsic parameter matrix, (X,Y,Z) represent the 3D coordinates of the 3D map point cloud in the virtual right camera plane, and f y The focal length in the vertical direction is represented by (c x ,c y () represents the coordinates of the principal point.

[0056] In this embodiment of the invention, S3011 extracts feature points from the current monocular image and establishes a correspondence between two-dimensional pixels and three-dimensional map points by combining them with a pre-constructed binocular SLAM map. A coarse pose of the monocular surveillance camera is obtained using a pose solution based on random sampling consistency and a nonlinear optimization method, thus providing a reliable initial pose reference for subsequent virtual binocular geometry construction. S3012 introduces a predefined virtual baseline length based on the coarse pose to generate a coarse pose for the virtual right camera, constructing a camera structure conforming to binocular geometry without adding any hardware. S3013 retrieves the three-dimensional map point cloud within the current left camera's field of view in the binocular SLAM map based on the coarse pose, reducing interference from irrelevant data in subsequent processing and improving computational efficiency. S3014 projects the three-dimensional map point cloud onto the virtual right camera plane according to the virtual right camera's pose and camera imaging model, generating a virtual right view, thus providing a unified and reliable image representation basis for subsequent feature matching, disparity calculation, and depth estimation with the current frame image.

[0057] S302: Perform ORB feature matching between the current frame image and the virtual right view to obtain a set of matching pairs, which includes multiple pairs of feature matching points:

[0058]

[0059] in, This represents the i-th pair of feature matching points. This represents the pixel coordinates of the i-th feature matching point in the current frame image. This represents the pixel coordinates of the i-th feature matching point in the virtual right view.

[0060] S303: Calculate the disparity of each pair of feature-matching points:

[0061]

[0062] Where, d i Represents the disparity of the i-th pair of feature matching points. This indicates Euclidean distance.

[0063] S304: Calculate the estimated depth of each pair of feature matching points based on disparity:

[0064]

[0065] in, f represents the estimated depth of the i-th pair of feature matching points. x 'b' represents the focal length in the horizontal direction, and 'b' represents the length of the virtual baseline.

[0066] It should be noted that those skilled in the art can set the size of the virtual baseline length according to actual needs, and this invention does not limit it. Optionally, the virtual baseline length is set to 0.5 meters.

[0067] S305: Based on the ratio of estimated depths between each pair of feature matching points, construct the scale consistency constraint equation:

[0068]

[0069] in, This represents the estimated depth of the j-th pair of feature matching points. This represents the 3D coordinates of the i-th feature matching point in a stereo SLAM map. τ represents the 3D point coordinates corresponding to the j-th feature matching point in the stereo SLAM map. scale This represents the tolerance threshold for scale consistency.

[0070] S306: Construct a graph optimization model describing the pose of the left camera and the 3D coordinates of the fallen elderly man:

[0071]

[0072]

[0073] in, This indicates the precise pose of the left camera (i.e., the precise pose of a monocular surveillance camera). This represents the precise 3D coordinates of the elderly person who fell in a pre-built binocular SLAM map, arg min represents minimization, ρ represents the robust kernel function (e.g., Huber loss function), and T wc P represents the camera pose of the left camera to be optimized. w This represents a 3D map point cloud, where p represents the pixel coordinates of the feature matching point in the image, λ represents the weighting coefficient of the scale consistency error term, and E... scale (i,j) represents the scale consistency error between each pair of feature matching points (i,j), μ represents the weighting coefficient of the pose prior error term, and T prior This represents the installation pose prior of the left camera (i.e., the installation pose prior of the monocular surveillance camera).

[0074] S307: Under the constraints of the scale consistency constraint equation, solve the graph optimization model to determine the precise pose of the left camera and the precise 3D coordinates of the fallen elderly man in the pre-constructed binocular SLAM map.

[0075] In this embodiment of the invention, S301 generates a virtual right view by introducing a predefined virtual baseline length under the imaging conditions of a monocular monitoring camera, constructing a pseudo-binocular geometric relationship without the need for a real binocular camera, providing a foundation for subsequent disparity and depth calculations. S302 performs feature matching between the current frame image and the virtual right view, establishing a stable and reliable pixel correspondence, improving the accuracy of subsequent spatial calculations. S303 converts two-dimensional image information into geometric quantities usable for spatial inference by calculating the disparity between feature matching points. S304 estimates the initial depth information of feature points based on disparity and camera intrinsic parameters, providing necessary depth priors for subsequent scale constraints and optimization. S305 effectively suppresses the scale drift problem in monocular depth estimation by constructing consistency constraints between the feature point depth ratio and the scale relationship of three-dimensional points in the binocular SLAM map. S306 integrates reprojection error, scale consistency error, and camera pose priors into a graph optimization model for joint optimization, improving the overall accuracy and stability of camera pose and the estimation of the three-dimensional coordinates of the fallen elderly person. S307 solves the graph optimization model under the above multiple constraints, and finally obtains the accurate three-dimensional coordinates of the elderly man who fell in the pre-constructed binocular SLAM map, thus providing a reliable spatial location basis for subsequent rescue route planning and emergency rescue decision-making.

[0076] S4: Based on precise 3D coordinates, semantic information in the binocular SLAM map, and rescuer context information, a set of Pareto optimal rescue paths is planned using a multi-objective optimization search algorithm.

[0077] The multi-objective optimization search algorithm is specifically the multi-objective A* search algorithm.

[0078] It should be noted that the multi-objective A-search algorithm is a path planning method that introduces multiple evaluation objectives for parallel optimization based on the traditional A-path search algorithm. It constructs a vector-based evaluation function for path nodes, incorporating multi-dimensional costs such as distance, risk, and reachability. During the search process, it simultaneously considers the trade-offs between objectives and uses dominance relationships to filter non-dominated paths, thereby obtaining a set of Pareto optimal paths that satisfy multi-objective constraints. This method is suitable for comprehensive path planning problems in complex environments. The multi-objective A-search algorithm is existing technology and will not be elaborated upon here.

[0079] In one possible implementation, S4 specifically includes sub-steps S401 to S403:

[0080] S401: Based on precise 3D coordinates, set the cost vector for each map location point in the stereo SLAM map. The cost vector includes distance cost, risk cost, reachability cost, and visibility cost.

[0081]

[0082] Where C(x,y) represents the cost vector of the map location point (x,y), c length c represents the distance cost. risk c represents the risk cost. access c represents the reachability cost. visibility This represents the cost of visibility.

[0083] Specifically, the distance cost is the Euclidean distance between the map location point and its precise three-dimensional coordinates.

[0084] Specifically, the risk costs are as follows:

[0085]

[0086] Among them, w k Let d represent the preset weight coefficient of the k-th class of risk semantic objects, exp() represent the natural exponential function, and d k σ represents the distance between a map location and a semantic object of the k-th risk class. k The Gaussian scale parameter represents the range of influence of the control risk of the object of the k-th risk semantic class.

[0087] Specifically, the accessibility cost is as follows:

[0088]

[0089] Where max represents taking the maximum value, w passable w represents the actual passable width traversed by the rescue mission. required Indicates the minimum passage width required for a rescue mission.

[0090] The visibility cost is adaptively set based on the image brightness, with higher costs for dark areas and lower costs for bright areas.

[0091] S402: Construct a first objective function with the goal of minimizing the sum of distance costs for all map locations. Construct a second objective function with the goal of minimizing the sum of risk costs for all map locations. Construct a third objective function with the goal of maximizing the sum of reachability costs for all map locations. Construct a fourth objective function with the goal of minimizing the sum of visibility costs for all map locations.

[0092] S403: Based on the first objective function, the second objective function, the third objective function, and the fourth objective function, plan the Pareto optimal rescue path set using the multi-objective A* search algorithm.

[0093] In this embodiment of the invention, S401 combines the precise three-dimensional coordinates of the fallen elderly person to construct a multi-dimensional cost vector in the binocular SLAM map for each map location point, including distance cost, risk cost, accessibility cost, and visibility cost. This allows path nodes to comprehensively reflect their spatial efficiency, safety risks, accessibility, and environmental visibility in a rescue scenario, thus providing a complete quantitative basis for multi-objective path planning. S402 constructs multiple optimization objectives such as distance, risk, accessibility, and visibility based on the multi-dimensional cost vector, enabling the path search process to move beyond a single shortest path and comprehensively balance efficiency, safety, and feasibility. S403 uses a multi-objective A* search algorithm to perform parallel search and dominance relationship filtering of candidate paths under the constraints of the above multiple objectives, obtaining a Pareto optimal rescue path set that satisfies the multi-objective constraints. This provides diverse and high-quality candidate solutions for subsequent selection of the optimal rescue path based on specific rescue needs and contextual information, thereby improving the overall safety, rationality, and adaptability of emergency rescue path planning.

[0094] S5: Introduce a dynamic weight vector to dynamically weight each Pareto optimal rescue path in the Pareto optimal rescue path set, and select the Pareto optimal rescue path with the smallest dynamic weight result as the final rescue path.

[0095] The dynamic weight vector is calculated as follows:

[0096] Calculate the dynamic weight vector based on the risk assessment score of the elderly person who fell:

[0097]

[0098] Where W represents the dynamic weight vector, w l w represents the weighting coefficient of the cumulative distance cost of the path. r w represents the weighting coefficient of the cumulative risk cost of the path. a The weights represent the cumulative reachability cost of the path, Softmax() represents the Softmax function, and α, β, and γ are all adjustment coefficients. score Rescuer indicates the fall risk assessment score. agility This parameter represents the mobility of rescuers. A preference vector representing the type of rescuer.

[0099] In one possible implementation, the dynamic weighting calculation performed on each Pareto optimal rescue path in the Pareto optimal rescue path set in S5, and the Pareto optimal rescue path with the smallest dynamic weighting result is selected as the final rescue path, specifically as follows:

[0100]

[0101] Where, Path final Indicates the final rescue route, f length f represents the cumulative distance cost of the path (i.e., the sum of the distance costs of all map locations). risk f represents the cumulative risk cost of the path (the sum of risk costs for all map locations). access This represents the cumulative reachability cost of the path (the sum of the reachability costs of all map locations).

[0102] In this embodiment of the invention, by introducing a dynamic weight vector based on fall risk assessment and rescuer context information on the basis of obtaining the Pareto optimal rescue path set, and by using a normalized weighting mechanism to comprehensively evaluate each candidate path, not only is the decision-making rigidity problem caused by the fixed weight strategy avoided, but also the adaptive adjustment of rescue path selection to the severity of the accident and the execution capability is realized, thereby improving the safety, feasibility and intelligence level of emergency rescue path planning results.

[0103] S6: Perform rescue missions for elderly people who have fallen, based on the final rescue route.

[0104] Specifically, alarm information containing precise three-dimensional coordinates and the final rescue path is sent to the rescue terminal, or the rescue robot is controlled to perform the rescue mission along the final rescue path.

[0105] In this embodiment of the invention, by directly executing the rescue task according to the final rescue path and sending alarm information containing precise three-dimensional coordinates and the rescue path to the rescue terminal or for controlling the rescue robot, not only is a closed-loop connection between fall detection, positioning, path planning and rescue execution realized, but also the timeliness of emergency rescue response, the accuracy of the execution process and the overall practicality and intelligence level of the system are improved.

[0106] Reference manual attached Figure 2 The diagram shows a structural schematic of an emergency rescue path planning system based on elderly fall detection provided by an embodiment of the present invention.

[0107] This invention provides an emergency rescue route planning system 20 based on elderly fall detection, comprising: a processor 201 and a memory 202;

[0108] The memory 202 stores programs or instructions that can run on the processor 201. When the program or instructions are executed by the processor 201, they implement the steps of the above-mentioned emergency rescue path planning method based on elderly fall detection and achieve the same technical effect. To avoid repetition, the present invention will not elaborate further.

[0109] It should be understood that the processor 201 in this embodiment of the invention may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0110] It should also be understood that the memory 202 in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DR RAM).

[0111] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0112] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0113] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0114] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0115] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0116] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0117] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0118] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0119] This invention provides a readable storage medium comprising: storing a program or instructions on the readable storage medium, wherein when the program or instructions are executed by a processor, the program or instructions implement the steps of the above-described emergency rescue path planning method based on elderly fall detection, and can achieve the same technical effect. To avoid repetition, this invention will not elaborate further.

[0120] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the embodiments of the present invention, and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the protection scope of the present invention.

Claims

1. An emergency rescue route planning method based on fall detection in the elderly, characterized in that, include: S1: Collect real-time video stream data from different areas of the elderly care community; S2: Input the real-time video stream data into the fall detection model and output the elderly fall detection results; S3: Based on the elderly fall detection results, when an elderly person falls, the precise three-dimensional coordinates of the fallen elderly person in the pre-built binocular SLAM map are determined through the virtual binocular-scale consistency constraint algorithm; otherwise, return to S1. S4: Based on the precise three-dimensional coordinates, the semantic information in the binocular SLAM map, and the rescuer's context information, a Pareto optimal rescue path set is planned using a multi-objective optimization search algorithm; S5: Introduce a dynamic weight vector to perform dynamic weighting calculations on each Pareto optimal rescue path in the Pareto optimal rescue path set, and select the Pareto optimal rescue path with the smallest dynamic weighting result as the final rescue path. S6: Perform a rescue mission on the elderly person who has fallen, according to the final rescue route.

2. The emergency rescue route planning method based on elderly fall detection according to claim 1, characterized in that, The real-time video stream data is specifically a sequence of consecutive frame images; Specifically, S1 is: Real-time video stream data is collected from different areas of the senior living community by deploying multiple monocular surveillance cameras in different areas of the community.

3. The emergency rescue route planning method based on elderly fall detection according to claim 1, characterized in that, The fall detection model is specifically a fall detection model based on a spatiotemporal graph convolutional network; The fall detection results for the elderly specifically include: whether an elderly person has fallen, whether an elderly person has not fallen, the pixel coordinates of the elderly person who fell in the current frame image, and the risk assessment score of the elderly person who fell.

4. The emergency rescue route planning method based on elderly fall detection according to claim 2, characterized in that, The precise 3D coordinates of the fallen elderly person in the pre-built binocular SLAM map, determined by the virtual binocular-scale consistency constraint algorithm in S3, specifically include: S301: Using the monocular surveillance camera as the left camera, a virtual right view is generated by introducing a predefined virtual baseline length; S302: Perform ORB feature matching between the current frame image and the virtual right view to obtain a set of matching pairs, the set of matching pairs including multiple pairs of feature matching points: ； in, This represents the i-th pair of feature matching points. This represents the pixel coordinates of the i-th feature matching point in the current frame image. This represents the pixel coordinates of the i-th feature matching point in the virtual right view; S303: Calculate the disparity of each pair of feature matching points: ； Where, d i Represents the disparity of the i-th pair of feature matching points. Indicates Euclidean distance; S304: Calculate the estimated depth for each pair of feature matching points based on the disparity: ； in, f represents the estimated depth of the i-th pair of feature matching points. x 'b' represents the focal length in the horizontal direction, and 'b' represents the length of the virtual baseline. S305: Based on the ratio of the estimated depths between each pair of feature matching points, construct the scale consistency constraint equation: ； in, This represents the estimated depth of the j-th pair of feature matching points. This represents the 3D coordinates of the i-th feature matching point in a stereo SLAM map. τ represents the 3D point coordinates corresponding to the j-th feature matching point in the stereo SLAM map. scale This represents the tolerance threshold for scale consistency. S306: Construct a graph optimization model describing the pose of the left camera and the 3D coordinates of the fallen elderly man: ； ； in, This indicates the precise pose of the left camera. This represents the precise 3D coordinates of the elderly person who fell in a pre-built stereo SLAM map, where arg min represents minimization, ρ represents the robust kernel function, and T... wc P represents the camera pose of the left camera to be optimized. w This represents a 3D map point cloud, where p represents the pixel coordinates of the feature matching point in the image, λ represents the weighting coefficient of the scale consistency error term, and E... scale (i,j) represents the scale consistency error between each pair of feature matching points (i,j), μ represents the weighting coefficient of the pose prior error term, and T prior This indicates the prior installation pose of the left camera; S307: Under the constraints of the scale consistency constraint equation, solve the graph optimization model to determine the precise pose of the left camera and the precise three-dimensional coordinates of the fallen elderly person in the pre-constructed binocular SLAM map.

5. The emergency rescue route planning method based on elderly fall detection according to claim 4, characterized in that, S301 specifically includes: S3011: Calculate the coarse pose of the left camera; S3012: Based on the coarse pose of the left camera, a coarse pose of the virtual right camera is generated by introducing the virtual baseline length: ； ； Among them, T virtual This indicates the approximate pose of the virtual right camera. The left camera's approximate pose is represented by I, which represents a 3×3 unit rotation matrix, and t... v This represents the translation vector of the virtual right camera relative to the left camera, with the subscript... T Indicates the transpose operation; S3013: Retrieve the 3D map point cloud within the field of view of the left camera under the coarse pose in the binocular SLAM map; S3014: Based on the coarse pose of the virtual right camera, project the 3D map point cloud onto the virtual right camera plane to generate the virtual right view: ； ； Among them, I virtual Let f represent the virtual right view, π represent the camera projection function, K represent the camera intrinsic parameter matrix, (X,Y,Z) represent the 3D coordinates of the 3D map point cloud in the virtual right camera plane, and f y The focal length in the vertical direction is represented by (c x ,c y () represents the coordinates of the principal point.

6. The emergency rescue route planning method based on elderly fall detection according to claim 1, characterized in that, The multi-objective optimization search algorithm is specifically: the multi-objective A* search algorithm; S4 specifically includes: S401: Based on the precise three-dimensional coordinates, set the cost vector for each map location point in the stereo SLAM map. The cost vector includes distance cost, risk cost, reachability cost, and visibility cost. ； Where C(x,y) represents the cost vector of the map location point (x,y), c length c represents the distance cost. risk c represents the risk cost. access c represents the reachability cost. visibility Indicates the cost of visibility; S402: Construct a first objective function with the goal of minimizing the sum of the distance costs of all map locations; construct a second objective function with the goal of minimizing the sum of the risk costs of all map locations; construct a third objective function with the goal of maximizing the sum of the reachability costs of all map locations; and construct a fourth objective function with the goal of minimizing the sum of the visibility costs of all map locations. S403: Based on the first objective function, the second objective function, the third objective function, and the fourth objective function, the Pareto optimal rescue path set is planned using the multi-objective A* search algorithm.

7. The emergency rescue route planning method based on elderly fall detection according to claim 6, characterized in that, The distance cost is specifically the Euclidean distance between the map location point and the precise three-dimensional coordinates; The specific risks and costs mentioned are as follows: ； Among them, w k Let d represent the preset weight coefficient of the k-th class of risk semantic objects, exp() represent the natural exponential function, and d k σ represents the distance between a map location and a semantic object of the k-th risk class. k The Gaussian scale parameter representing the scope of the control risk impact of the object of the k-th risk semantics; The reachability cost is specifically as follows: ； Where max represents taking the maximum value, w passable w represents the actual passable width traversed by the rescue mission. required Indicates the minimum passage width required for a rescue mission.

8. The emergency rescue route planning method based on elderly fall detection according to claim 3, characterized in that, The dynamic weight vector is calculated as follows: Calculate the dynamic weight vector based on the risk assessment score of the elderly person who fell: ； Where W represents the dynamic weight vector, w l w represents the weighting coefficient of the cumulative distance cost of the path. r w represents the weighting coefficient of the cumulative risk cost of the path. a The weights represent the cumulative reachability cost of the path, Softmax() represents the Softmax function, and α, β, and γ are all adjustment coefficients. score Rescuer indicates the fall risk assessment score. agility This parameter represents the mobility of rescuers. A preference vector representing the type of rescuer.

9. The emergency rescue route planning method based on elderly fall detection according to claim 8, characterized in that, The step S5 involves dynamically weighting each Pareto optimal rescue path in the Pareto optimal rescue path set and selecting the Pareto optimal rescue path with the smallest dynamic weighting result as the final rescue path. ； ； Where, Path final Indicates the final rescue route, f length f represents the cumulative distance cost of the path. risk f represents the cumulative risk cost of the path. access This represents the cumulative reachability cost of the path.

10. An emergency rescue route planning system based on fall detection in the elderly, characterized in that, include: Processor and memory; The memory stores programs or instructions that can run on the processor, which, when executed by the processor, implement the steps of the emergency rescue route planning method based on elderly fall detection as described in any one of claims 1 to 9.

Images