Hospital real-time navigation path planning method and system based on deep learning

Abstract

The invention provides a hospital real-time navigation path planning method and system based on deep learning, and relates to the technical field of wisdom medices.The walking posture of a patient is analyzed in real time through a deep learning network model, firstly, the patient with the normal walking posture and the patient with the abnormal walking posture are recognized and distinguished, and according to the actual environment condition of a hospital, the real-time navigation path of the hospital is planned; according to the method, a walking path line suitable for patient demands is planned, an optimal navigation walking path suitable for special crowds such as elderly patients, pregnant women and handicapped people can be provided for the special crowds such as the elderly patients, the pregnant women and the handicapped people, the problems of walking difficulty and cross infection of the special crowds are solved, and the doctor-seeing experience satisfaction degree of the patients and the hospital service efficiency are improved. Meanwhile, the walking posture of the patient is tracked, a secondary abnormity alarm mechanism is set, the path is re-planned in case of an abnormal posture, and an alarm is given, so that the patients of special crowds are rescued at the first time, and the service quality is further improved.

Description

Technical Field

[0001] This invention relates to the field of smart healthcare technology, and in particular to a method and system for real-time hospital navigation path planning based on deep learning. Background Technology

[0002] Current hospital navigation systems plan the shortest path based on static maps, but they do not consider patients' real-time posture and sudden environmental risks (uneven road surfaces, obstacles, high-density pedestrian traffic). This leads to risks of falls and collisions for vulnerable groups such as pregnant women and elderly patients. Therefore, current hospital navigation systems cannot meet the needs of all patient groups, especially those with special needs. Furthermore, current hospital navigation systems lack abnormal walking tracking and alarm functions. Even if abnormal information occurs, alarm messages cannot be sent to the patient; even if alarm messages are sent, it is impossible to determine the patient's condition and provide appropriate medical assistance. Summary of the Invention

[0003] The purpose of this invention is to provide a deep learning-based real-time hospital navigation path planning method and system. This system can provide optimal walking routes within the hospital for special groups such as elderly patients with mobility difficulties, pregnant women, and people with disabilities, solving problems related to walking difficulties and cross-infection, thereby improving patient satisfaction and hospital service efficiency. Simultaneously, a secondary anomaly alarm mechanism is implemented to replan the route and issue an alarm in the event of abnormal behavior, ensuring that special groups receive immediate assistance and further enhancing service quality.

[0004] To achieve the above objectives, the present invention provides the following technical solution:

[0005] The present invention provides a deep learning-based real-time navigation path planning method for hospitals, comprising: image data acquisition, acquiring image data of patients walking from visual sensors deployed in various locations within the hospital; walking analysis, performing real-time analysis of the patient's walking posture in the image data based on a deep learning network model, distinguishing between normal walking posture and multiple abnormal walking postures (such as gait instability, unilateral weakness, etc.); path planning, dynamically planning paths according to the results of the walking analysis, wherein, for patients in a normal walking state, a first-type path is planned; for patients in an abnormal state, multiple second-type paths are planned and one of the corresponding second-type paths is matched according to the abnormality category; and abnormality tracking and alarm, continuously monitoring the patient's walking posture, and triggering path replanning and alarm when an abnormality is detected.

[0006] In a second aspect, this invention provides a deep learning-based real-time hospital navigation path planning system for performing the aforementioned method, comprising: an image data acquisition module for acquiring image data of patient walking from visual sensors deployed throughout the hospital; a walking analysis module for real-time analysis of the patient's walking posture in the image data based on a deep learning network model, distinguishing between normal walking posture and multiple types of abnormal walking posture; a path planning module for dynamically planning paths according to the results of the walking analysis, wherein, for patients in a normal walking state, a first-type path is planned; for patients in an abnormal state, multiple second-type paths are planned and one of the corresponding second-type paths is matched according to the abnormality category; and an anomaly tracking and alarm module for continuously monitoring the patient's walking posture, triggering path replanning and an alarm when an anomaly is detected.

[0007] Compared with the prior art, the present invention has the following advantages:

[0008] This invention proposes a deep learning-based real-time hospital navigation path planning method and system. It analyzes patients' walking postures in real time using a deep learning network model, first identifying and distinguishing patients with normal or abnormal walking postures. Based on the actual hospital environment, it plans suitable walking routes to meet the needs of each patient. This provides optimal navigation routes within the hospital for special groups such as elderly patients with mobility impairments, pregnant women, and people with disabilities, solving problems related to walking difficulties and cross-infection, improving patient satisfaction and hospital service efficiency. Simultaneously, it tracks patient walking postures and sets up a secondary abnormality alarm mechanism. If abnormal postures recur, the system replans the route and issues an alarm, ensuring that special groups receive immediate assistance and further improving service quality. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0010] Figure 1 A flowchart illustrating a deep learning-based real-time hospital navigation path planning method according to a specific embodiment of the present invention;

[0011] Figure 2 A schematic diagram of key points of human walking in a hospital real-time navigation path planning method and system based on deep learning, provided as a specific embodiment of the present invention;

[0012] Figure 3The flowchart illustrates the execution of the residual module in the graph attention mechanism layer of a hospital real-time navigation path planning method and system based on deep learning, as provided in a specific embodiment of the present invention.

[0013] Figure 4 This invention provides a deep learning-based hospital real-time navigation path planning method and a spatiotemporal graph attention network learning model (GAT+LSTM+CNN spatiotemporal hybrid model) in the system, which is a specific embodiment of the present invention.

[0014] Figure 5 This is a block diagram of a hospital real-time navigation path planning system based on deep learning, provided as a specific embodiment of the present invention.

[0015] Reference numerals: Image data acquisition module 100, data preprocessing module 200, walking analysis module 300, path planning module 400, path sending module 500, and anomaly tracking alarm module 600. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] The following embodiments of the present invention provide a deep learning-based real-time hospital navigation path planning method and system, which can be used by all patients in a hospital, and is particularly suitable for special populations (elderly patients with mobility impairments, pregnant women, and people with disabilities). In summary, the deep learning-based real-time hospital navigation path planning method and system provided by the present invention, based on graph neural networks, analyzes the medical records and historical walking data of special population patients within the hospital to obtain their walking characteristics. These walking characteristics include pathological features, disease severity levels, and walking patterns. In-hospital location data refers to the environmental characteristics of special population patients while walking within the hospital, including ground flatness and road obstacle features. Based on the in-hospital navigation needs of special populations, and combining their walking characteristics and location data, a path planning model is established. Based on the path planning model, a path planning scheme is provided for special populations within the hospital. The method for obtaining the walking characteristics of special population patients includes: preprocessing the disease information and historical in-hospital walking posture image data of special population patients to obtain noise-filtered data; constructing a spatiotemporal graph attention network model and training the model to extract behavioral features from the preprocessed data to obtain the characteristics of special population patients. The spatiotemporal graph attention network includes a spatial attention network layer for extracting walking features of patients from special populations; a walking time feature for extracting walking features of patients; and finally outputs the features of patients from special populations.

[0018] One implementation provides a deep learning-based real-time hospital navigation path planning method, which mainly involves image data acquisition, walking analysis, path planning, and anomaly tracking and alarm steps. Correspondingly, a deep learning-based real-time hospital navigation path planning system for executing this method mainly consists of an image data acquisition module, a walking analysis module, a path planning module, and an anomaly tracking and alarm module.

[0019] Please refer to Figure 1 In a more preferred embodiment, a deep learning-based real-time hospital navigation path planning method further includes data preprocessing and path transmission steps, the specific contents of each step are as follows:

[0020] S1. Image Data Acquisition: Acquire image data of patient walking from visual sensors deployed throughout the hospital. In a preferred embodiment, the acquired image data of patient walking includes historical image data and real-time image data. Specifically, image data of patient walking is acquired from cameras deployed throughout the hospital; this image data is also called video data because video is composed of multiple frames. This step includes historical image (video) data acquisition, current day image (video) data acquisition, and obtaining real-time image (video) data from the current day image (video) data. Historical image (video) data acquisition refers to image (video) data of patient walking in the hospital stored for the previous 30 or 60 days from the current day; current day image (video) data acquisition refers to image (video) data of patient walking in the hospital on the current day. Before acquiring the image data, video cameras are pre-positioned at distances along the hospital's navigation paths, such as in the emergency building, outpatient building, inpatient building corridors, department floor corridors, elevators, and disabled access corridors; the locations are coded according to the hospital's 3D layout map, for example, the emergency building is JZ, the outpatient building is MZ, and the inpatient building is ZY, etc.

[0021] S2. Data preprocessing involves decompressing, denoising, and splitting the image data into static images of individual frames. In the walking analysis step, the patient's walking posture in the preprocessed image data is analyzed. Specifically, video image preprocessing refers to compressing, denoising, and framing the compressed image to obtain static images of individual frames. Preferably, filtering methods and CNN deep learning are used to remove image noise, and image processing segmentation tools are used to segment the image (video) data into frames to obtain static images of individual frames.

[0022] S3. Walking analysis: Based on a deep learning network model, the patient's walking posture in the image data is analyzed in real time to distinguish between normal walking posture and various abnormal walking postures. In a preferred embodiment, the walking analysis step is specifically implemented through the following sub-steps:

[0023] S31. Model Construction: Construct a spatiotemporal graph attention network learning model for judging abnormal patient walking posture. In a preferred embodiment, the deep learning network model adopts a spatiotemporal graph attention network learning model including a graph attention mechanism layer (GAT layer), an LSTM layer, and a CNN layer; the graph attention mechanism layer extracts the spatial features of key human body nodes representing walking posture to obtain the features of the patient's walking posture; the LSTM layer and the CNN layer extract the continuous temporal features of key human body nodes to distinguish between the patient's normal walking state and various abnormal walking states.

[0024] S32. Offline model training: Labeled historical image data is divided into training and testing sets, which are then input into the constructed spatiotemporal graph attention network learning model for classification training and testing, respectively. Labeling refers to assigning different labels to normal walking and various abnormal walking postures. Historical patient walking image (video) data is collected, labeling normal walking states and abnormal states (e.g., gait instability, unilateral weakness, balance disorders, sudden falls). The model is trained offline by labeling the historical image data. A normal / abnormal walking sample library is established. Graph attention mechanism layer: processes spatial feature data of human key points; LSTM layer: detects 10-30 second long-term abnormal walking postures; CNN layer: detects 0.5-2 second short-term sudden actions (e.g., sudden staggering). Output probability vector (normal label probability + abnormal state probability). More specifically, historical image data is manually labeled to distinguish between patients with normal walking (stable gait) and N types of abnormal walking (unsteady gait – such as significant lateral swaying, unilateral weakness – such as asymmetrical weight-bearing, balance disorders – such as abnormal center of gravity control, sudden falls – such as rapid falling movements, etc.). Normal walking is labeled 1, and abnormal walking patterns are labeled 0, -1, -2, -3, ..., -N-1. In a preferred embodiment, a patient skeletal keypoint model based on a hospital indoor scene is used to analyze each frame of data. The human body key nodes include at least 18 key points. The spatial coordinates of these 18 key points are obtained, and directed edges are established for each node. The features of these directed edges in spatial dimension are used as the spatial features of the human body key nodes. Specifically, the coordinate positions (x, y, z) of 18 key points representing human posture P are extracted, such as... Figure 2 As shown, the number of keypoints P is not limited to 18, but is considered for special populations. The coordinate parameters of 18 keypoints representing human posture P are: 1) Head keypoints include nose 1, right eye 2, left eye 3, right ear 4, left ear 5; 2) Upper body keypoints include left shoulder 7, right shoulder 8, left elbow 9, right elbow 10, left wrist 11, right wrist 12; 4) Lower body keypoints include left hip 13, right hip 14, left knee 15, right knee 16, left ankle 17, right ankle 18. Graph nodes are created for each frame. Nodes are 1-18, and node features are (x, y, z) coordinate values. Establish directed edges for the nodes: 1->2; 1->3; 2->4; 3->5; 1->6; 11->9; 9->7; 7->8; 8->10; 10->12; 7->13; 13->14; 13->15; 8->14; 15->17; 14->16; 16->18; 18->17.

[0025] The characteristics of a directed edge in spatial dimension are the distance between the two nodes connecting the edge and the edge itself. With all other edges The included angle θ between them is: The time-dimensional node edge features include hospital environmental parameters such as ground flatness, ground slope angle, obstacle density, and pedestrian density. These are obtained through real-time collection of the following environmental data: 1) Ground slope angle: A 3D point cloud is generated by LiDAR scanning the ground to calculate the slope angle; 2) Ground flatness: Point cloud standard deviation greater than or equal to 2cm is marked as "uneven"; 3) Dynamic obstacle density: Cameras detect obstacles (wheelchairs / medical equipment) in real-time; density formula = obstacle area / (corridor length * width); 4) Pedestrian density: Detected using a personnel tracking algorithm; pedestrian density = N people / (corridor length * width) / duration.

[0026] Establish a spatiotemporal graph attention network learning model. The spatiotemporal graph of human walking posture is defined as a set of four tuples π = {V}. t ,ε t ,χ t ,Γ}, where Γ={t1,t2,...,t T} represents the set of T timestamps representing video image frames of the patient's navigation movement within the hospital; V t V represents the set of key human body nodes of N patients in the image frame at time t. t ={v1,v2,..,v N};ε t This indicates that at time t, the key nodes of the patient's body in the image frame form a set of spatial and temporal dimensions, i.e. χ t χ represents the connection relationship between the same key points in consecutive frames. t ={v (t)i -v (t+1)j}, the spatial location of the human body key point at time t is the feature value (x) of node i. i ,y i ,z i ).

[0027] Suppose the video stream is decomposed into a sequence of T static frames, that is, the image sequence can be represented as G = {G1, G2, ..., G...} T}, where a single image frame is represented as G i =(V i E i V i The graph nodes formed by key points of the patient's body, E i This represents the connectivity between key points in the human body. GAT is used to update different weight values. i Adjacency matrix A i H is the feature matrix of the graph nodes.

[0028] In a preferred embodiment, when extracting spatial features, the graph attention mechanism layer calculates and constrains the attention weights based on predefined topological connections of human keypoints to shield interference between unrelated nodes. This constraint on the attention weights is achieved by introducing a binary mask matrix. This mask matrix is ​​generated based on the effective connections between human keypoints and is used to reset the weights of invalid connections to zero or approach negative infinity during attention calculation. Specifically, the spatiotemporal graph attention network learning model employs a directed adjacency matrix mask filter; the GAT graph attention network model sets a mask matrix, and the weight matrix is ​​used to obtain the masked attention weights. The mask matrix M... mask Nodes and edge data that are irrelevant to human walking posture and behavior are masked, where M mask (i,j) = -∞ indicates that the connection between nodes i and j is masked, M mask If (i,j) = 0, it means that a valid connection is retained. More specifically:

[0029] Directed edge feature mask filter MM = [1) 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0; 2) 00 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0; 3) 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0; 4) 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0; 5) 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0; 6) 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0; 7) 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0; 8) 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0; 9) 0 0 0 0 0 0 0 0 1 0 0 1 0 00 0 0; 10) 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0; 11) 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0; 12) 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0; 13) 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0; 14) 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0; 15) 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0; 16) 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0; 17) 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0; 18) 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0].

[0030] The attention weights of nodes to nodes (i,j) in the graph attention mechanism layer are:

[0031]

[0032] Where X i Input the feature vector X for node i j Input the feature vector for node j; Θ s Θ t Θ e This is a learnable weight matrix used to transform the linear dimension of node features; A learnable attention vector used for computer attention scores (i.e., node pair importance); α i,j The normalized attention coefficient represents the importance of node j to node i; e i,j Let e ​​be the eigenvector of edge L(i,j). i,k Let L(i,k) be the feature vector of edge; after weighted summation of the neighboring nodes of i, output the new feature of node i.

[0033] The graph attention mechanism layer sets a mask matrix, and the weight matrix is ​​passed through the mask matrix to obtain the masked attention weights. The mask is set based on the directed adjacency matrix, and the weight mask matrix M... mask=[0 0 0 -∞ -∞ 0 -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞; -∞ 0 -∞ 0 -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞; -∞ -∞ 0 -∞ 0 -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞; -∞ -∞ -∞ -∞ 0 -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ 0 -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ 0 -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ 0 0 -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞; -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ 0 -∞ 0-∞ -∞ -∞ 0 -∞ -∞ -∞ 0 -∞ -∞ -∞ -∞ -∞ ; -∞ -∞ -∞ -∞ -∞ -∞ -∞ 0 -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ 0 -∞ -∞ -∞ -∞ -∞ -∞ -∞ ; -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ 0-∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ 0 -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ 0 0 0 -∞ -∞ -∞ ; -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ 0 -∞ 0 -∞ -∞; -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ 0 -∞ 0 -∞; -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ 0 -∞ 0; -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ -∞ 0 0], where M mask (i,j) = -∞ indicates that the connection between nodes i and j is masked, M mask (i,j) = 0 indicates that a valid connection is retained.

[0034] After the masking, pay attention to the matrix for:

[0035] [a 1,1 a 1,2 a 1,3 0 0 a 1,6 0 0 0 0 0 0 0 0 0 0 0 0;0 a 2,2 0 a 2,4 0 0 0 0 0 00 0 0 0 0 0 0 0;0 0 a 3,3 0 a 3,5 0 0 0 0 0 0 0 0 0 0 0 0 0 0;0 0 0 a 4,4 0 0 0 0 00 0 0 0 0 0 0 0 0;0 0 0 0 a 5,5 0 0 0 0 0 0 0 0 0 0 0 0 0;0 0 0 0 0 a 6,6 0 0 0 00 0 0 0 0 0 0 0;0 0 0 0 0 0 0 a 7,7 a 7,8 0 0 0 0 0 0 0 0 0 0;0 0 0 0 0 0 0 0 a 8,8 0a 8,10 0 0 0 a 8,14 0 0 0 0;0 0 0 0 0 0 a 9,7 0 a 9,9 0 0 0 0 0 0 0 0 0;0 0 0 0 0 0 0 0 00 0 0 0 a 10,12 0 0 0 0 0 0;0 0 0 0 0 0 0 0 0 a 11,9 0 a 11,11 0 0 0 0 0 0 0;0 0 0 0 00 0 0 0 0 0 0 a 12,12 0 0 0 0 0 0;0 0 0 0 0 0 0 0 0 0 0 0 0 a 13,13 a 13,14 a 13,15 0 0 0;00 0 0 0 0 0 0 0 0 0 0 0 a 14,14 0 a 14,16 0 0;0 0 0 0 0 0 0 0 0 0 0 0 0 0 a 15,15 0a 15,170;0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 α 16,16 0 α 16,18 ;0 0 0 0 0 0 0 0 0 0 0 0 0 00 α 17,17 0;0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 α 18,17 α 18,18 ].

[0036] For the spatiotemporal graph attention network learning model in step S31, model construction, more specifically, the graph attention mechanism layer extracts the spatial features between key points in the human walking image, and then enters the LSTM layer to extract the continuous temporal features of the human key points under the patient's walking path; the CNN layer extracts the continuous temporal node features of the patient's instantaneous abnormal fall.

[0037] In a preferred embodiment, the spatiotemporal graph attention network learning model also has a fully connected layer. The graph attention mechanism layer extracts and outputs the feature vectors of human key points for each time frame; the feature vectors extracted by the graph attention mechanism layer for each frame are combined into a time series, and then simultaneously input into an LSTM layer and a CNN layer. The LSTM layer processes global temporal features and selectively memorizes patterns through a gating mechanism to process temporal dependencies of time frames, capturing the entire process of abnormal walking by the patient; the CNN layer is used to detect local temporal features of short-term sudden abnormal movement mutations; the final output of the LSTM layer and the globally pooled features of the CNN layer are concatenated, and then classified through a fully connected layer. In a preferred embodiment, the graph attention mechanism layer consists of 6 residual modules, and the input of each residual module is fused with the output calculated by at least two layers of graph attention mechanism layers to optimize the training process of the deep network. Please refer to [reference needed]. Figure 3 , Figure 3 The flowchart for each residual module is shown below. Its graph attention mechanism layer uses a 2-head attention mechanism (M=2), with a feature layer dropout rate of 0.03 and an attention score dropout rate of 0.2 to prevent overfitting. The LeakyReLU activation function used in the residual module (residual network) has a negative parameter β=0.01, and an improved LeakyReLU residual activation function is employed, the expression of which is:

[0038]

[0039] Where β is a preset small negative slope parameter.

[0040] According to the example diagram above, note that the input dimension of the mechanism layer is N1=7, the hidden layer dimension is N2=8, and the number of categories is N3=5.

[0041] The graph attention mechanism layer outputs feature vectors of graph structure data (human key points) for each time frame. The feature vectors extracted from each frame by the GAT are combined into a time series, which is then simultaneously input into LSTM and CNN. LSTM processes global temporal features and selectively remembers patterns through gating mechanisms (forget gate / input gate / output gate) to handle temporal dependencies between time frames, capturing the entire process of "body tilting - unsteady supporting leg - fall" during abnormal walking in patients.

[0042] In a preferred embodiment, to save on manual classification and labeling workload, a self-iterative semi-supervised learning labeling algorithm is employed. A portion of the data (e.g., 5%) is selected as the original dataset for partial manual labeling, while the remaining data (e.g., 95%) is labeled using a semi-supervised self-iterative learning algorithm. This algorithm first trains a basic model using a small amount (e.g., 5%) of manually labeled data as initial seeds. Then, this model predicts the labels for the remaining large amount (e.g., 95%) of unlabeled data, and selects the samples with the highest prediction confidence and their "pseudo-labels" to add to the training set. Next, a more powerful model is retrained using the expanded dataset, and this iterative process of "prediction-selection-expansion-training" is repeated continuously until all or most of the unlabeled data is automatically labeled, thus efficiently generating a large-scale, high-quality labeled dataset with extremely low manual cost. After labeling, the image data is shuffled to form a labeled dataset, which is then proportionally divided into training and test sets and input into a spatiotemporal graph attention network learning model (GAT+LSTM+CNN spatiotemporal hybrid model) for classification training and testing. During model training, the predicted data generated by connecting and fusing the spatiotemporal features of GAT and LSTM-CNN meets the accuracy requirements of the cross-entropy loss function. If so, the model training is completed, and the preprocessed test data is input. If the model accuracy reaches the prediction accuracy requirement, the model is obtained, and online data can be input for abnormal behavior classification. Otherwise, the parameters of each part of the model are adjusted, including the number of network layers, the number of iterations, the learning rate, etc., until the accuracy is met.

[0043] For a more detailed process of step S32, offline model training, please refer to [link / reference]. Figure 4 .

[0044] S33. Online Analysis: The acquired real-time image data is input into the spatiotemporal graph attention network learning model, which has undergone classification training and testing, for online real-time analysis to distinguish between normal walking postures and various abnormal walking postures of the patient. Specifically, during online analysis, a video stream (30fps) is received, and image segmentation tools are used to extract the coordinate parameters of key human body points. A 30-frame temporal buffer (1-second time window) is maintained, and the data is input into the spatiotemporal model for inference (completed within 100ms), outputting the current state.

[0045] S4. Path Planning: Based on the results of the walking analysis and combined with the hospital environment, dynamic path planning is performed. For patients with normal walking posture, a first-class path is planned; for patients with abnormal posture, multiple second-class paths are planned, and one of the corresponding second-class paths is matched according to the abnormality category. Specifically, in the path planning step, walking posture classification data (normal walking posture or multiple abnormal walking postures) is obtained from the walking analysis, and selective path planning is performed. The input includes the walking analysis results, patient case characteristics, and real-time environmental data. The path is dynamically planned according to the walking analysis results and combined with the hospital environment. The process is as follows: If the normal walking posture is "normal," a first-class path is planned for patients with normal walking posture, which is the normal walking path. Hospital environmental factors are not considered, and it is generally the shortest path. The shortest path leads to the destination department. The shortest path can be planned using the Dijkstra algorithm, or the hospital navigation system can be directly switched, or path replanning can be avoided. For patients with abnormal walking posture, the abnormality type is distinguished: multiple second-class paths are drawn, and one of the corresponding second-class paths is matched according to the abnormality category. In short, first, the abnormal walking patient category is determined, and then a specific walking route is selected based on the category. As a reference example: Gait instability: Avoid uneven ground and choose paths with handrails; Unilateral weakness: Avoid narrow passages and areas requiring bilateral exertion; Balance difficulties: Use assistive wheelchairs. Sudden fall: Take the shortest path to the rest area. More specifically, the path planning step ultimately outputs a set of nodes and segmented paths based on the planned path and walking analysis results.

[0046] S5. Path Transmission: Based on the planned path and walking instructions, a directed graph of the patient's walking posture is generated and sent to the hospital's in-hospital navigation system. In this step, specific paths are found based on each segmented path and the patient's abnormal behavior, and finally, the directed graph data based on the patient's walking posture is merged. Specifically, this includes: a) Segment Feature Mapping: The segment feature database is accessed to retrieve the environment type of the segmented path; b) Adaptive Policy Binding: Adaptive policies are generated for each segmented path based on the patient's abnormality type; c) Constructing the Directed Graph Data Structure: A directed graph containing a set of nodes, a set of edges, and edge attributes binding adaptive policies is generated; d) Data Transmission: The directed graph data is transmitted to the in-hospital navigation system. The directed graph data structure includes: a set of nodes (physical locations in path_nodes), a set of edges (segmented paths in path_segments), and edge attributes (adaptive policies bound to each edge). A predefined segment feature database is accessed, which stores the mapping relationship between different segment types (e.g., smooth corridors, uneven surfaces) and abnormal behaviors (e.g., gait instability, unilateral weakness, balance disorders). Each path segment is traversed, and based on the current patient's abnormality type, the required adaptive strategy for traversing that segment (e.g., "needs handrail support," "reduces walking speed") is retrieved from the path segment characteristic database. The directed graph data is then transmitted in real-time to the hospital navigation system, driving the navigation terminal to generate adaptive walking guidance for the patient. As a reference example: the structured data contains json{"patient": "Zhang San", "path_nodes": ["uneven ground", "rest area 1", "outpatient hall", "smooth corridor", "elevator A", "rest area 2", "neurology department"], "path_segments": [["uneven ground", "rest area 1"], ["rest area 1", "outpatient hall"], ["outpatient hall", "elevator A"], ["elevator A", "smooth corridor"], ["smooth corridor", "neurology department"]]. The graph data is then sent to the hospital navigation system to correct the patient's walking route.

[0047] S6. Anomaly Tracking Alarm: Continuously monitors the patient's walking posture. When an anomaly is detected, path replanning and an alarm are triggered. This step is implemented by inputting data into the spatiotemporal graph attention network learning model trained and tested in the walking analysis step for analysis. It determines whether the patient will exhibit abnormal walking again under the navigation path and whether the patient can walk normally with the support of assistive tools. If abnormal walking occurs, an alarm message is issued, and the path is replanned. As a reference example: Monitoring patient Zhang San's walking process begins: Frame 0: Status = Normal, Instruction = Please maintain current position; ... Frame 10: Status = Normal, Instruction = Please go to the Reflexology Department; Frame 11: Patient Zhang San exhibits an abnormal state: Unstable gait. Patient Zhang San (Unstable gait): Planned path ["Reflexology Department" "Rest Area 1", "Discharge Area"]. Send Zhang San's path: ["Reflexology Department" "Rest Area 1", "Discharge Area"].

[0048] Frame 20: Status = Unsteady gait, Command = Unsteady gait, Command = Please proceed to rest area 1; Frame 30: Status = Unsteady gait, Command = Please proceed to discharge area; Alarm! Patient Zhang San has an abnormal status: sudden fall, Patient Zhang San (sudden fall): Planned path ["Discharge area", "Emergency exit"], Path sent to Zhang San: ["Discharge area", "Emergency exit"]. Patient Zhang San needs immediate assistance. Frame 40: Status = Sudden fall, Command = Please proceed to emergency exit.

[0049] Please refer to Figure 5 Correspondingly, a deep learning-based real-time hospital navigation path planning system for executing the preferred method described above also includes a data preprocessing module and a path transmission module. The specific execution contents of each module are as follows: an image data acquisition module 100, used to acquire image data of patient walking from visual sensors deployed throughout the hospital; a data preprocessing module 200, used to decompress, denoise, and split the image data into static images of unit frames; and a walking analysis module 300, used to perform real-time analysis of the patient's walking posture in the image data based on a deep learning network model, distinguishing between normal walking postures and various abnormal walking postures, and classifying the walking posture... The analysis module analyzes the patient's walking posture in the preprocessed image data; the path planning module 400 dynamically plans paths based on the walking analysis results. For patients in a normal walking state, a first-type path is planned; for patients in an abnormal state, multiple second-type paths are planned, and one of the corresponding second-type paths is matched according to the abnormality category; the path sending module 500 generates a directed graph of the patient's walking posture path based on the planned path and sends it to the hospital's in-hospital navigation system; the abnormality tracking and alarm module 600 continuously monitors the patient's walking posture and triggers path replanning and an alarm when an abnormality is detected.

[0050] In a preferred embodiment, the deep learning network model employs a spatiotemporal graph attention network learning model comprising a graph attention mechanism layer, an LSTM layer, and a CNN layer. The graph attention mechanism layer extracts the spatial features of key human body nodes representing walking posture to obtain the features of the patient's walking posture. The LSTM layer and the CNN layer extract the continuous temporal features of key human body nodes to distinguish between the patient's normal walking state and various abnormal walking states.

[0051] In a preferred embodiment, the image data acquisition module acquires image data of the patient's walking, including historical image data and real-time image data. The walking analysis module specifically includes: a model construction submodule, used to construct a spatiotemporal graph attention network learning model for judging abnormal patient walking postures; an offline model training submodule, used to divide the labeled historical image data into training and testing sets, and input them into the constructed spatiotemporal graph attention network learning model for classification training and testing, wherein labeling refers to assigning different labels to normal walking and various abnormal walking postures; and an online analysis submodule, used to input the acquired real-time image data into the spatiotemporal graph attention network learning model after classification training and testing for online real-time analysis, distinguishing whether the patient has a normal walking posture or multiple abnormal walking postures.

[0052] In a preferred embodiment, the graph attention mechanism layer is based on a residual network, which consists of two graph attention network layers and employs a modified residual activation function, Sleaky ReLU.

[0053] In a preferred embodiment, the labeling employs a self-iterative semi-supervised learning labeling algorithm, selecting a portion of the data as the original dataset for manual labeling, and using a pseudo-labeling algorithm for iterative labeling of the remaining data.

[0054] In a preferred embodiment, the human body key nodes include at least 18 key points, namely: nose, right eye, left eye, right ear, left ear, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, right knee, left ankle, and right ankle; the spatial coordinates of the 18 key points are obtained, and directed edges are established for the nodes, and the features of the directed edges in the spatial dimension are obtained as the spatial features of the human body key nodes; the hospital environment includes the following parameters: ground flatness, ground slope angle, obstacle density, and pedestrian flow density.

[0055] In a preferred embodiment, the spatiotemporal graph attention network learning model further includes a fully connected layer; the graph attention mechanism layer extracts and outputs feature vectors of human key points in each time frame; the feature vectors of each frame extracted by the graph attention mechanism layer are combined into a time series, and then simultaneously input into the LSTM layer and the CNN layer. The LSTM layer processes global temporal features and selectively memorizes patterns through a gating mechanism to process temporal dependencies of time frames and capture the entire process of abnormal walking by the patient; the CNN layer is used to detect local temporal features of short-term sudden abnormal movement mutations; the final output of the LSTM layer and the global pooled features of the CNN layer are concatenated and then classified through the fully connected layer.

[0056] For more specific and better execution content of each module, please refer to the corresponding better deep learning-based real-time hospital navigation path planning method mentioned above.

[0057] In addition, the deep learning-based real-time hospital navigation path planning system also has a data acquisition subsystem. In addition to acquiring image data of the patient's walking, it also collects the patient's personal information (walking location information or added medical record information) and hospital condition data (including environmental data such as ground flatness, ground slope angle, obstacle density and pedestrian density).

[0058] Based on the above specific implementation methods, the following beneficial technical effects can be achieved:

[0059] This invention utilizes an innovative spatiotemporal graph attention network learning model to analyze the walking posture of patients (elderly, pregnant women, and people with disabilities) in real time, accurately distinguishing between normal gait and various abnormal states (gait instability, unilateral weakness, balance disorders, and sudden falls). Based on this dynamic analysis, the system proactively plans personalized safety paths: avoiding uneven surfaces, narrow passages, and other risk areas related to the type of abnormality for patients with abnormal conditions, and selecting paths with handrails, flat surfaces, or direct access to rest areas; providing the shortest path navigation for patients with normal conditions. This effectively reduces the risk of falls and collisions for special populations walking within hospitals, solves the problem that existing static navigation cannot cope with real-time posture and sudden environmental risks, and significantly improves the safety and experience of special populations seeking medical care.

[0060] This invention not only plans pathways but also continuously tracks the patient's walking posture using a neural network model. Once a new abnormal state is detected, the system immediately triggers an alarm (information is directly sent to the hospital system) and replans the pathway in real time, forming a closed loop of "monitoring-analysis-planning-tracking-alarm-replanning." This proactive intervention mechanism can quickly respond to emergencies such as patient falls, gaining valuable rescue time and avoiding delays. Simultaneously, by providing optimal pathways (safe or shortest paths) for patients in different states, it reduces delays caused by ineffective walking and potential injuries, improving both patient access to medical care and hospital service efficiency.

[0061] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0062] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification. Furthermore, the above embodiments only illustrate several implementation methods of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the present invention. For those skilled in the art, several modifications and improvements can be made without departing from the concept of the present invention, and these all fall within the protection scope of the present invention.

Claims

1. A deep learning-based real-time navigation path planning method for hospitals, characterized in that, include: Image data acquisition: Acquire image data of patients walking from sensory sensors deployed throughout the hospital; Walking analysis is based on a deep learning network model to analyze the patient's walking posture in image data in real time, and to distinguish whether the patient has a normal walking posture or multiple abnormal walking postures. Path planning involves dynamically planning paths based on the results of walking analysis and the hospital environment. For patients in a normal walking state, a first-class path is planned; for patients in an abnormal state, multiple second-class paths are planned, and one of the second-class paths is matched according to the type of abnormality. Anomaly tracking alarms continuously monitor the patient's walking posture, and trigger path replanning and alarms when an anomaly is detected.

2. The hospital real-time navigation path planning method based on deep learning according to claim 1, characterized in that, It also includes data preprocessing, which involves compressing, denoising, and framing the image data to obtain a static image per unit frame; In the walking analysis step, the patient's walking posture is analyzed in the image data after data preprocessing.

3. The hospital real-time navigation path planning method based on deep learning according to claim 1, characterized in that, The deep learning network model adopts a spatiotemporal graph attention network learning model that includes a graph attention mechanism layer, an LSTM layer, and a CNN layer; the graph attention mechanism layer extracts the spatial features of key human body nodes that represent walking posture to obtain the features of the patient's walking posture. The LSTM and CNN layers extract continuous temporal features of key human body nodes to distinguish between the patient's normal walking state and various abnormal walking states.

4. The hospital real-time navigation path planning method based on deep learning according to claim 3, characterized in that, In the image data acquisition step, the acquired image data of the patient walking includes historical image data and real-time image data; The walking analysis steps specifically include: Model construction: Constructing a spatiotemporal graph attention network learning model for judging abnormal patient walking posture; The model is trained offline by dividing the labeled historical image data into training and testing sets, which are then input into the constructed spatiotemporal graph attention network learning model for classification training and testing. Labeling refers to assigning different labels to normal walking postures and various abnormal walking postures. Online analysis involves inputting the acquired real-time image data into a spatiotemporal graph attention network learning model that has undergone classification training and testing for online real-time analysis, distinguishing between patients with normal walking postures or multiple abnormal walking postures.

5. The deep learning-based real-time hospital navigation path planning method according to claim 3 or 4, characterized in that, When extracting spatial features, the graph attention mechanism layer calculates and constrains the attention weights based on the predefined topological connection relationships of human key points to shield interference between unrelated nodes. The calculation and constraint of the attention weights is achieved by introducing a binary mask matrix, which is generated based on the effective connection relationships between human key points and is used to reset the weights of invalid connections to zero or tend to negative infinity in the attention calculation.

6. The hospital real-time navigation path planning method based on deep learning according to claim 5, characterized in that, The graph attention mechanism layer consists of multiple stacked residual modules. The input of each residual module is fused with the output calculated by at least two graph attention mechanism layers. The activation function used in the residual modules is a modified LeakyReLU function, whose expression is: Where β is a preset small negative slope parameter.

7. The hospital real-time navigation path planning method based on deep learning according to claim 1, characterized in that, Also includes: Path transmission involves generating a directed graph of the patient's walking posture based on the planned path, combined with walking instructions, and sending it to the hospital's in-hospital navigation system. Specifically, this includes: a) Segment characteristic mapping: calling the segment characteristic database to retrieve the environment type of the segmented path. b) Adaptive strategy binding: Generate a passability adaptive strategy for each segmented path based on the patient's abnormality type; c) Construct a directed graph data structure: Generate directed graph data containing a set of nodes, a set of edges, and edge attributes that bind the adaptive strategy; d) Data transmission: Transmit the directed graph data to the hospital navigation system.

8. The hospital real-time navigation path planning method based on deep learning according to claim 3, characterized in that, The human body key nodes include at least 18 key points, namely: nose, right eye, left eye, right ear, left ear, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, right knee, left ankle, and right ankle; obtain the spatial coordinates of the 18 key points, establish directed edges for the nodes, and obtain the spatial features of the directed edges in the spatial dimension as the spatial features of the human body key nodes; the hospital environment includes the following parameters: ground flatness, ground slope angle, obstacle density, and pedestrian flow density.

9. The hospital real-time navigation path planning method based on deep learning according to claim 3, characterized in that, The spatiotemporal graph attention network learning model also includes a fully connected layer; the graph attention mechanism layer extracts and outputs the feature vectors of human key points in each time frame; The feature vectors of each frame extracted by the graph attention mechanism layer are combined into a time series, and then simultaneously input into the LSTM layer and the CNN layer. The LSTM layer processes global temporal features and selectively memorizes patterns through a gating mechanism to process temporal dependencies of time frames and capture the entire process of the patient's abnormal walking. The CNN layer is used to detect local temporal features of short-term sudden abnormal movement mutations. The final output of the LSTM layer and the globally pooled features of the CNN layer are concatenated, and then classified through a fully connected layer.

10. A deep learning-based real-time hospital navigation path planning system, used to perform the method described in any one of claims 1 to 9, characterized in that, include: The image data acquisition module is used to acquire image data of patients walking from visual sensors deployed throughout the hospital; The walking analysis module is used to perform real-time analysis of the patient's walking posture in image data based on a deep learning network model, and to distinguish whether the patient has a normal walking posture or multiple abnormal walking postures. The path planning module is used to dynamically plan paths based on the results of walking analysis. For patients in a normal walking state, a first-class path is planned; for patients in an abnormal state, multiple second-class paths are planned and one of the corresponding second-class paths is matched according to the abnormality category. The anomaly tracking and alarm module is used to continuously monitor the patient's walking posture and trigger path replanning and alarm when an anomaly is detected.

Images