A fall detection method and system based on transformer

Abstract

The application provides a fall detection method and system based on a transformer, and relates to the technical field of artificial intelligence. The method comprises the following steps: acquiring an image sequence comprising continuous multiple frames of images, and taking a label corresponding to the last frame of image in the multiple frames of images as a classification label of the image sequence; pre-processing the multiple frames of images in the image sequence to obtain a feature map of a preset dimension; performing a straightening operation on the feature map and inputting the feature map and the classification label into a pre-constructed inverted triangular network; performing feature fusion on spatial features output by the inverted triangular network to form a new classification label; extracting time features between the multiple frames of images according to the new classification label, and performing classification by using a fully connected network to complete binary classification of fall detection. In this way, by extracting spatial features and time features between the continuous multiple frames of images and performing classification by using the fully connected network, the recognition capability for human fall events can be enhanced, and the correctness of the fall detection result can be improved.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence technology, and in particular relates to a fall detection method and system based on Transformer. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art known to those skilled in the art.

[0003] Among the many issues affecting the daily safety of the elderly, falls have become one of the leading causes of injury and death among them. Addressing falls early can reduce serious consequences, and appropriate fall detection and rescue services can ensure the safety of the elderly population as much as possible. Therefore, the development of intelligent detection and protection systems has become a focus of attention.

[0004] In recent years, machine learning and deep learning algorithms have been widely applied in the field of fall detection. In deep learning, existing fall detection methods can be summarized into three categories: wearable sensor-based, environment-based, and computer vision-based fall detection systems.

[0005] Among them, fall detection methods based on wearable sensors typically use accelerometers and pressure sensors to acquire features such as human speed to detect falls. For example, the paper "A hardware framework for fall detection using inertial sensors and compressed sensing" published in *Microprocessors and Microsystems* in 2022 (pages 104514-104521) proposed a fall detection hardware framework based on accelerometers and gyroscopes to acquire data; the paper "Applying deep learning technology for automatic fall detection using mobile sensors" published in *Biomedical Signal Processing and Control* in 2022 (pages 103355-103363) proposed a fall detection method that acquires data through accelerometers, gyroscopes, and orientation sensors in smartphones and smartwatches; and the paper "Killer heuristic" published in *Measurement* in 2021 (pages 108258-108267) proposed a method that acquires data through accelerometers, gyroscopes, and orientation sensors in smartphones and smartwatches. The paper "Optimized Convolution Neural Network-Based Fall Detection with Wearable IoT Sensor Devices" uses wearable sensor devices composed of magnetometers, gyroscopes, and accelerometers to acquire data and applies the AlexNet convolutional network for fall detection. However, methods based on wearable sensors usually require long-term wear, which is uncomfortable and easy to forget. Therefore, the researchers proposed a more convenient environment-based fall detection method.

[0006] Environment-based fall detection methods use sensors (infrared, ultrasonic, and other non-visual sensors) deployed in the environment to acquire data for fall detection. For example, the 2022 paper "Fall detection system based on infrared array sensor and multi-dimensional feature fusion" published in *Measurement* (pages 110870-110879) effectively expands the detection area and improves the accuracy of fall detection methods by using infrared sensors for data acquisition. The 2021 paper "Elderly fall detection with vital signs monitoring using CW Doppler radar" published in *IEEE Sensors Journal* (pages 16969-16978) proposed a low-cost, high-precision fall detection system based on Doppler radar. However, environment-based fall detection methods are costly and susceptible to external interference. Therefore, researchers have proposed more efficient and lower-cost computer vision-based methods.

[0007] Computer vision-based methods typically detect falls by analyzing videos or images. For example, the 2022 paper "Fall detection using body geometry and human pose estimation in video sequences" published in the *Journal of Visual Communication and Image Representation* (pages 103407-103419) proposed a fall detection method that extracts human geometric features from different frames of a video sequence. Another 2022 paper, "Person Fall Detection System Based on Video Stream Analysis" published in *Procedia Computer Science* (pages 676-681), proposed a fall detection method based on pose estimation and LSTM, extracting key points and temporal features from the video stream.

[0008] However, existing computer vision-based fall detection methods suffer from high false positive rates and low accuracy. Therefore, providing a computer vision-based fall detection system with high accuracy is a pressing issue that needs to be addressed. Summary of the Invention

[0009] To address the aforementioned issues, this invention provides a fall detection method and system based on Transformer. By extracting spatial and temporal features between consecutive multi-frame images and using a fully connected network for classification, the method enhances the ability to recognize human fall events and improves the accuracy of fall detection results.

[0010] To achieve the above objectives, the present invention mainly includes the following aspects:

[0011] In a first aspect, embodiments of the present invention provide a fall detection method based on Transformer, comprising:

[0012] Obtain an image sequence consisting of multiple consecutive frames, and use the label corresponding to the last frame of the multiple frames as the classification label for the image sequence;

[0013] The image sequence is preprocessed to obtain a feature map of a preset dimension;

[0014] The feature map is straightened and input together with the classification label into a pre-constructed inverted triangular network. The spatial features output by the inverted triangular network are fused to form a new classification label. The inverted triangular network includes two Pooling-Transformer networks connected sequentially. A pooling operation is added between the multi-head attention mechanism and the multilayer perceptron in the Transformer encoding module to obtain the Pooling-Transformer network.

[0015] Based on the new classification labels, the temporal features between multiple frames are extracted, and a fully connected network is used for classification to complete the binary classification of fall detection.

[0016] In one possible implementation, the preprocessing of multiple frames in the image sequence to obtain a feature map of a preset dimension includes:

[0017] After performing layer normalization on multiple frames of images in the image sequence, the pre-built image reduction module is used to convert the multiple frames of images into feature maps of a preset dimension. The image reduction module includes two convolutional layers and two pooling layers. Each convolution and pooling operation reduces the width and height of the image to half of its original value, and doubles the number of channels.

[0018] In one possible implementation, the feature map is straightened in the following way:

[0019] The feature map is divided into blocks of a preset size, and then merged with the number of channels to obtain an image tensor of a specific dimension. The image tensor is encoded, and positional encoding is added to the image tensor. Then, the dimensions are merged to obtain a flattened image tensor.

[0020] In one possible implementation, the flattened image tensor and classification label are concatenated and input into an inverted triangular network. After passing through two interconnected Pooling-Transformer networks, the spatial features of multiple frames of images are output.

[0021] In one possible implementation, in the Pooling-Transformer network, the image tensor output by the multi-head attention mechanism is set as X. The dimensions of the image tensor X are reshaped, and then the width and height of the image tensor X are halved through average pooling and max pooling operations. The results of average pooling and max pooling are concatenated along the channel dimension, doubling the number of channels. Finally, the image tensor is reshaped again to complete the straightening operation.

[0022] In one possible implementation, the output space features of the inverted triangular network are separated into an image tensor and a classification label; the separated image tensor is reshaped into four dimensions, and a global average pooling operation is performed on its channel dimension to reduce the channel dimension to 1 dimension, thereby reducing the four-dimensional image tensor to a three-dimensional tensor.

[0023] After reducing the dimensionality to a 3D tensor, the image tensor and classification labels are separated and concatenated in the width and height dimensions. Then, a global pooling operation is performed on the concatenated dimension to obtain the new classification labels.

[0024] In one possible implementation, the new classification label is input into the Transformer encoding module, multi-head self-attention mechanism is performed between multiple frames of images, and then the image is processed through a multilayer perceptron module to obtain the temporal features between the multiple frames of images.

[0025] Secondly, embodiments of the present invention also provide a fall detection system based on Transformer, comprising:

[0026] The acquisition module is used to acquire an image sequence including multiple consecutive frames of images, and to use the label corresponding to the last frame of the multiple frames as the classification label of the image sequence;

[0027] The preprocessing module is used to preprocess multiple frames of images in the image sequence to obtain feature maps of a preset dimension.

[0028] The feature fusion module is used to straighten the feature map and input it together with the classification label into a pre-constructed inverted triangular network. The spatial features output by the inverted triangular network are fused to form a new classification label. The inverted triangular network includes two Pooling-Transformer networks connected sequentially. A pooling operation is added between the multi-head attention mechanism and the multilayer perceptron in the Transformer encoding module to obtain the Pooling-Transformer network.

[0029] The fall detection module is used to extract temporal features between multiple frames of images based on the new classification labels, and then use a fully connected network for classification to complete the binary classification of fall detection.

[0030] Thirdly, embodiments of the present invention provide a computer device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the computer device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, they perform the steps of the fall detection method based on Transformer as described in the first aspect and any possible implementation of the first aspect.

[0031] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the Transformer-based fall detection method as described in the first aspect and any possible implementation thereof.

[0032] Based on the above technical solution, the present invention has the following beneficial effects:

[0033] 1. Considering that falling is a process, it is difficult to identify a fall from a single frame. This invention obtains an image sequence consisting of multiple consecutive frames, extracts the spatial and temporal features between the multiple frames, and uses a fully connected network for classification. This can enhance the ability to identify human fall events and improve the accuracy of fall detection results.

[0034] 2. This invention reduces computational complexity by dividing the dimensions of the image sequence and using an image reduction module based on a convolutional neural network to reduce the width and height of the input image sequence. Then, the output feature map is straightened and input into an inverted triangular network along with the positional encoding and classification labels for spatial feature extraction. The tensor output by the inverted triangular network and the classification labels are fused to form new classification labels. Finally, the new classification labels are input into a Transformer encoding structure for temporal feature extraction, and a fully connected network is used for classification. This fully utilizes the features of the image sequence and classification labels to accurately detect fall behavior with fast detection speed and good real-time performance. Attached Figure Description

[0035] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0036] Figure 1 This is one of the flowcharts of the fall detection method based on Transformer provided in Embodiment 1 of the present invention;

[0037] Figure 2 This is the second flowchart of the fall detection method based on Transformer provided in Embodiment 1 of the present invention;

[0038] Figure 3 This is a schematic diagram of the image reduction module based on a convolutional neural network provided in Embodiment 1 of the present invention;

[0039] Figure 4 This is a schematic diagram of the inverted triangular network and Pooling-Transformer module provided in Embodiment 1 of the present invention;

[0040] Figure 5 This is a flowchart of the pooling operation in Transformer provided in Embodiment 1 of the present invention;

[0041] Figure 6 This is a flowchart of feature fusion provided in Embodiment 1 of the present invention. Detailed Implementation

[0042] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0043] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0044] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0045] Example 1

[0046] like Figure 1 As shown, this embodiment of the invention provides a fall detection method based on Transformer, specifically including the following steps:

[0047] S101: Obtain an image sequence consisting of multiple consecutive frames, and use the label corresponding to the last frame of the multiple frames as the classification label of the image sequence.

[0048] In practical implementation, since falling is a process, a single frame image is insufficient to identify a fall. Therefore, this embodiment uses an image sequence comprising multiple consecutive frames as a sample. The length of the image sequence can be set according to actual needs, such as using seven consecutive frames as a sample. Each sample has dimensions of 7×3×480×640, where 7 represents the number of frames, 3 represents the number of channels, and 480 and 640 represent the width and height of the image, respectively. This data partitioning method allows the network to improve the accuracy of fall recognition by extracting temporal features.

[0049] S102: Preprocess multiple frames of images in the image sequence to obtain feature maps of a preset dimension.

[0050] As an optional implementation, such as Figure 2 As shown, the image sequence is input into the image reduction module based on a convolutional neural network to reduce the image size and obtain a feature map of a preset dimension. Specifically, the following method is used to perform layer normalization on the samples:

[0051]

[0052] Where T is the sample, μ represents the mean of the layer normalized data, σ represents the variance of the layer normalized data, ε is a constant added to prevent the denominator from being zero, γ and β are the learning parameters of the model, taking values ​​of 1 and 0 respectively, and T′ is the sample after layer normalization.

[0053] To reduce the computational cost of the model, an image reduction module was designed. The image reduction module consists of two convolutional layers and two pooling layers, and its specific structure is as follows: Figure 3As shown. Each convolution and pooling operation reduces the image's width and height to half their original values, while doubling the number of channels. The image data undergoes convolution, ReLU activation, and average pooling operations, with the calculation process as follows:

[0054] z = avg(ReLU(C 3×3 T′));

[0055] Where avg represents average pooling, ReLU represents the ReLU activation function, and C 3×3 The expression represents a 3×3 convolution operation, where z represents the output of the image sequence after the first convolution pooling operation.

[0056] After passing through the image reduction module, sample T′ is transformed into a feature of size 7×12×120×160. This achieves the goal of reducing the size of the input image sequence in terms of width and height, thereby reducing the number of network parameters and computational complexity.

[0057] S103: The feature map is straightened and input together with the classification label into a pre-constructed inverted triangular network. The spatial features output by the inverted triangular network are fused to form a new classification label. The inverted triangular network includes two interconnected Pooling-Transformer networks. A pooling operation is added between the multi-head attention mechanism and the multilayer perceptron in the Transformer encoding module to obtain the Pooling-Transformer network.

[0058] In practical implementation, pooling layers in convolutional networks can reduce the width and height of feature maps, thereby reducing the computational cost of the model and making the feature maps translation-invariant, rotation-invariant, and scale-invariant. Inspired by convolutional networks, this embodiment adds a pooling structure to the original Transformer, allowing the Transformer to extract features on tensors of different dimensions, thereby reducing computational cost.

[0059] Specifically, the feature map z′={z′1,z′2,…,z′7} output by the image reduction module is divided into blocks of size p×p, resulting in (120 / p)×(160 / p)=H×W blocks. Then, the number of channels and p×p are merged to obtain an image tensor of dimensions F×N×C, where F is the number of frames, N=(H×W) is called the width and height dimension, and C=(p×p×c) is called the channel dimension. The image tensor is encoded, and positional encoding is added to preserve positional information. Then, the F and C dimensions are merged to flatten the tensor. The classification labels and the flattened image tensor are concatenated to form X. p Input into the inverted triangle network.

[0060] A pooling operation is added between the multi-head attention mechanism and the multilayer perceptron in the original Transformer encoding module, resulting in an improved Transformer encoding structure, named Pooling-Transformer. The specific structure is as follows: Figure 4 As shown. The image tensor output by the multi-head attention mechanism module is set to X, and then pooling operations are performed in the Transformer. First, the dimensions of the image tensor X are reshaped. Then, average pooling and max pooling operations are used to halve the width and height of the tensor. Next, the results of average pooling and max pooling are concatenated along the channel dimension, doubling the number of channels. Finally, a final reshaping operation is performed to complete the straightening process. The above operations are expressed as follows:

[0061] X′=RS2(concat(avg(RS1(X)),max(RS1(X))));

[0062] Here, RS1 and RS2 represent reshaping operations, and max represents max pooling. The classification labels are input into the fully connected network so that the width and height dimensions of the classification labels are consistent with those of X′. The specific operations described above are as follows: Figure 5 As shown.

[0063] By connecting two Pooling-Transformers one after the other, the width and height dimensions of the output tensor of each Pooling-Transformer become one-quarter of the input tensor, and the channel dimension becomes twice the original. Therefore, the tensor gradually becomes narrower and taller, and the whole looks like an "inverted triangle" structure. Thus, the two stacked Pooling-Transformer networks are called inverted triangle networks.

[0064] The aforementioned inverted triangular network based on Poling-Transformer reduces network computation while still extracting spatial features from the feature map.

[0065] After the classification labels are encoded using the Transformer structure, attention is calculated between them and various image patches, enabling feature interaction. Since the dimensionality is relatively low, Transformer-based classification networks typically use these classification labels for subsequent operations or for direct classification. However, using only classification labels is too limiting, and the features fused from the classification labels may not be comprehensive enough. Therefore, this embodiment proposes a feature fusion method to obtain new classification labels for subsequent feature extraction operations.

[0066] The output X″ of the inverted triangle network p Separate into image tensor X T and category tag XC Then the image tensor X T The image tensor is reshaped into four dimensions, and a global average pooling operation is performed on its channel dimension to reduce the channel dimension to 1 dimension, thereby reducing the four-dimensional image tensor to a three-dimensional tensor, as shown in the following equation:

[0067] X′ T =gavg(RS3(X) T ));

[0068] Here, RS3 represents the reshaping operation, and gavg represents global average pooling. After being reduced to a 3D tensor, the image tensor and the classification labels have the same dimensions. The image tensor and the classification labels are concatenated along the width and height dimensions, and then a global pooling operation is performed along the concatenated dimension. The entire process is as follows: Figure 6 As shown, the above operation is expressed in the following formula:

[0069] X′ C =gavg(concat(X) C ,X′ T ));

[0070] After the above operations, the feature fusion of the image tensor and the classification label is completed, and a new classification label is formed. The new classification label is then used for the subsequent feature extraction and classification operations.

[0071] S104: Extract the temporal features between multiple frames of images based on the new classification labels, and use a fully connected network for classification to complete the binary classification of fall detection.

[0072] In practice, the new classification label X′ will be used. C The input is processed by the Transformer encoding module, which performs multi-head self-attention mechanism operations between image frames, and then passes through the multilayer perceptron module to complete the extraction of temporal features from the image.

[0073] After the new classification labels pass through the Transformer encoding module, the classification labels with spatiotemporal features are input into the fully connected layer to complete the binary classification of fall detection, that is, distinguishing between a fall or normal behavior. To prevent the model from overfitting and improve its generalization ability, a random deactivation layer is added after the fully connected layer, with the random deactivation rate set to 0.3.

[0074] The image is reduced in width and height by a convolutional network, then its spatiotemporal features are extracted by an inverted triangular network and the original Transformer encoding module, and finally it is passed through a classification network to obtain the probability of the image sequence belonging to each class, thereby completing the classification of falls and normal conditions and realizing fall detection.

[0075] Example 2

[0076] This invention also provides a fall detection system based on Transformer, comprising:

[0077] The acquisition module is used to acquire an image sequence including multiple consecutive frames of images, and to use the label corresponding to the last frame of the multiple frames as the classification label of the image sequence;

[0078] The preprocessing module is used to preprocess multiple frames of images in the image sequence to obtain feature maps of a preset dimension.

[0079] The feature fusion module is used to straighten the feature map and input it together with the classification label into a pre-constructed inverted triangular network. The spatial features output by the inverted triangular network are fused to form a new classification label. The inverted triangular network includes two Pooling-Transformer networks connected sequentially. A pooling operation is added between the multi-head attention mechanism and the multilayer perceptron in the Transformer encoding module to obtain the Pooling-Transformer network.

[0080] The fall detection module is used to extract temporal features between multiple frames of images based on the new classification labels, and then use a fully connected network for classification to complete the binary classification of fall detection.

[0081] The Transformer-based fall detection system provided in this embodiment is used to implement the aforementioned Transformer-based fall detection method. Therefore, the specific implementation of the Transformer-based fall detection system can be found in the embodiment section of the Transformer-based fall detection method above, and will not be repeated here.

[0082] Example 3

[0083] This invention also provides a computer device, including a processor, a memory, and a bus.

[0084] The memory stores machine-readable instructions executable by the processor. When the computer device is running, the processor communicates with the memory via a bus. When the machine-readable instructions are executed by the processor, they can perform the operations described above. Figure 1 and 2 The steps of the fall detection method based on Transformer in the method embodiment shown are explained in detail in the method embodiment, and will not be repeated here.

[0085] Example 4

[0086] Based on the same inventive concept, embodiments of the present invention also provide a computer-readable storage medium storing a computer program, wherein the computer program, when run by a processor, executes the steps of the fall detection method based on Transformer described in the above method embodiments.

[0087] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0088] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A fall detection method based on Transformer, characterized in that, The application relates to a fall detection method based on image sequence. The application comprises the following steps: An image sequence comprising continuous multiple frames of images is acquired, and a label corresponding to the last frame of image in the multiple frames of images is taken as a classification label of the image sequence; The multiple frames of images in the image sequence are preprocessed to obtain a feature map with a preset dimension; The feature map is subjected to a straightening operation and is input into a pre-constructed inverted triangular network together with the classification label, spatial features output by the inverted triangular network are subjected to feature fusion to form a new classification label; wherein the inverted triangular network comprises two connected Pooling-Transformer networks, a pooling operation is added between a multi-head attention mechanism and a multi-layer perception in a Transformer coding module to obtain the Pooling-Transformer network; 2.The Transformer-based fall detection method of claim 1, wherein, According to the new classification label, time features between the multiple frames of images are extracted, a full connection network is used for classification, and a binary classification of fall detection is completed. The multiple frames of images in the image sequence are preprocessed to obtain a feature map with a preset dimension, which comprises the following steps: 3.The Transformer-based fall detection method of claim 2, wherein, After layer normalization processing is performed on the multiple frames of images in the image sequence, the multiple frames of images are respectively converted into feature maps with a preset dimension by using a pre-constructed image reduction module; the image reduction module comprises two convolution layers and two pooling layers, each convolution and pooling operation reduces the width and height of the image to one half of the original, and the channel number becomes twice that of the original. The feature map is subjected to a straightening operation in the following manner: 4.The Transformer-based fall detection method of claim 3, wherein, The feature map is divided into blocks according to a preset size, then is subjected to dimension merging with the channel number to obtain an image tensor with a specific dimension; the image tensor is coded, position coding is added on the image tensor, then dimension merging is performed to obtain a flattened image tensor. 5.The Transformer-based fall detection method of claim 4, wherein, The flattened image tensor and the classification label are spliced and input into the inverted triangular network, and spatial features of the multiple frames of images are output through the two connected Pooling-Transformer networks. 6.The Transformer-based fall detection method of claim 5, wherein, In the Pooling-Transformer network, an image tensor output by the multi-head attention mechanism is taken as X, the dimension of the image tensor X is reshaped, then the width and height of the image tensor X are reduced to one half of the original through average pooling operation and maximum pooling operation, the results of the average pooling and the maximum pooling are spliced in the channel dimension, the channel number is doubled, and finally reshaping is performed again to complete the straightening operation. The output spatial features of the inverted triangular network are separated into an image tensor and a classification label; the separated image tensor is reshaped into four dimensions, and global average pooling operation is performed on the channel dimension, so that the channel dimension is reduced to 1 dimension, thereby reducing the four-dimensional image tensor into a three-dimensional tensor; After being reduced into a three-dimensional tensor, the image tensor and the classification label are spliced in the width and height dimensions, then global pooling operation is performed on the spliced dimension to obtain a new classification label. 7.The Transformer-based fall detection method of claim 1, wherein, The new classification label is input into a Transformer encoding module, a multi-head self-attention mechanism is operated between multiple frames of images, and then a multi-layer perception module is passed through to obtain time features between the multiple frames of images. 8.A fall detection system based on Transformer, characterized in that, The method comprises the steps of: An acquisition module is configured to acquire an image sequence comprising continuous multiple frames of images, and take a label corresponding to a last frame of image in the multiple frames of images as a classification label of the image sequence. A preprocessing module is configured to perform preprocessing on the multiple frames of images in the image sequence to obtain a feature map of a preset dimension. A feature fusion module is configured to perform a straightening operation on the feature map and input the feature map and the classification label into a pre-constructed inverted triangular network, perform feature fusion on spatial features output by the inverted triangular network, and compose a new classification label. A fall detection module is configured to extract time features between the multiple frames of images according to the new classification label, perform classification using a full connection network, and complete binary classification of fall detection.

9. A computer device, comprising: The computer device comprises a processor, a memory and a bus. The computer readable storage medium stores a computer program.

10. A computer-readable storage medium, characterized in that, When the computer program is executed by the processor, the steps of the fall detection method based on the Transformer are executed.

Images