Fall detection method and system based on video semantics

Abstract

The invention provides a fall detection method and system based on video semantics, and the method comprises the steps: extracting a visual input vector corresponding to a behavior video clip of a to-be-detected individual, carrying out the extraction of multi-channel structure semantic information through employing a Transform deep neural network, obtaining the multi-channel structure semantic information of the to-be-detected individual, and carrying out the detection of the to-be-detected individual. And then multi-channel structure semantic information is processed by using a tense aggregation method based on an efficient channel attention mechanism, and then tumble prediction is performed by means of a support vector machine, so that the problem that left and right foot monitoring data are lack of relation and the problem that each piece of measurement data is lack of uniformity in a traditional method are effectively solved. And thus, more accurate fall risk prediction is provided.

Description

Technical Field

[0001] The present invention belongs to the technical field related to fall detection, and in particular relates to a fall detection method and system based on video semantics. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] The prediction and prevention of falls has become a key research topic in the field of public health. Although a variety of fall prediction systems have appeared on the market, these systems still have several limitations in practical applications, such as insufficient adaptability to specific populations such as the elderly and inaccurate feature selection.

[0004] The existing solutions for fall detection have the following problems: 1. Poor versatility. Existing fall detection is usually difficult to adapt to people of different ages and physical conditions. There are significant differences in physiological characteristics between the elderly and the young, but existing fall prediction often performs poorly in distinguishing these key differences; 2. Insufficient feature selection. Many detection methods rely on traditional feature selection methods, which may fail to fully identify and utilize those features that have a significant impact in fall detection. Therefore, the prediction performance may be limited, and features cannot be extracted globally and their actual utility cannot be evaluated; 3. Traditional methods have significant limitations. Traditional timed mobility tests are not accurate enough. There are certain limitations in using laboratory equipment (gold standard) to collect data related to falls, such as inconvenience in use, relatively expensive prices, and some indicators and other monitoring data need to be improved. Traditional IMU devices are usually unable to measure universal semantic information of fall behavior, which is an important indicator for predicting falls. The structural semantic information of the human body is a key indicator for assessing the risk of falls. However, the application of this important information is often overlooked in current fall prediction systems. Since the structural semantic information of the human body can reflect the stability and coordination of human form and movement, failure to fully utilize this information may lead to inaccurate assessment of individual fall risk, thus affecting the effective implementation of preventive measures. Therefore, it is particularly important to develop a system that can integrate structural semantic information for accurate fall risk assessment, which can not only improve the accuracy of fall prediction, but also help to adaptively formulate more suitable prevention strategies for high-risk individuals.

[0005] In summary, how to effectively extract and utilize structural semantic information that reflects human body shape and movement for fall prediction and achieve more accurate prediction is a problem that needs to be solved at present. Summary of the invention

[0006] In order to overcome the shortcomings of the above-mentioned prior art, the present invention provides a fall detection method and system based on video semantics, which utilizes the initial structure semantic vector and visual input vector, and performs feature extraction, aggregation, and prediction based on the Transformer network, efficient channel attention mechanism and support vector machine, thereby improving the accuracy of fall detection.

[0007] In order to achieve the above object, the present invention adopts the following technical solution:

[0008] In a first aspect, the present invention provides a fall detection method based on video semantics, comprising:

[0009] Obtaining behavioral video clips of the individual to be tested;

[0010] Dividing the frame image corresponding to the behavior video clip of the individual to be tested into grid areas, and performing linear transformation to obtain a visual input vector;

[0011] The initial structure semantic vector learned through training is input into the Transformer network together with the visual input vector to obtain multi-channel structure semantic information;

[0012] An efficient channel attention mechanism is used to assign corresponding weights to the structural semantic information of multiple channels for temporal aggregation, thus obtaining temporally aggregated structural semantic information.

[0013] Based on the temporally aggregated structural semantic information, a support vector machine is used to perform prediction to obtain a detection result of whether the individual to be tested has fallen.

[0014] In a second aspect, the present invention provides a fall detection system based on video semantics, comprising:

[0015] An acquisition unit is configured to: acquire a behavior video clip of the individual to be tested;

[0016] A first feature extraction unit is configured to: divide the frame image corresponding to the behavior video clip of the individual to be tested into grid areas, and perform linear transformation to obtain a visual input vector;

[0017] A second feature extraction unit is configured to: input the initial structure semantic vector learned through training and the visual input vector into the Transformer network to obtain multi-channel structure semantic information;

[0018] The temporal aggregation unit is configured to: use an efficient channel attention mechanism to assign corresponding weights to the multi-channel structural semantic information for temporal aggregation, thereby obtaining temporally aggregated structural semantic information;

[0019] A prediction unit is configured to: perform prediction using a support vector machine based on the temporally aggregated structural semantic information to obtain a detection result of whether the individual to be tested has fallen.

[0020] In a third aspect, the present invention provides an electronic device comprising a memory and a processor, and computer instructions stored in the memory and executed on the processor, wherein when the computer instructions are executed by the processor, the method described in the first aspect is performed.

[0021] In a fourth aspect, the present invention provides a computer-readable storage medium for storing computer instructions, wherein when the computer instructions are executed by a processor, the method described in the first aspect is performed.

[0022] In a fifth aspect, the present invention provides a computer program product, comprising a computer program, which, when executed by a processor, implements the method described in the first aspect.

[0023] One or more of the above technical solutions have the following beneficial effects:

[0024] In the present invention, the visual input vector corresponding to the behavior video clip of the individual to be tested is extracted, and the Transformer deep neural network is used to extract multi-channel structural semantic information to obtain the multi-channel structural semantic information of the individual to be tested, and then the multi-channel structural semantic information is processed by the temporal aggregation method based on the efficient channel attention mechanism, and then the fall prediction is performed with the help of a support vector machine; the Transformer deep neural network is used. Compared with the traditional method and the CNN-based deep learning method, this method has a smaller amount of calculation and a faster speed when predicting the risk of falling, and at the same time improves the efficiency, convenience, connectivity and globality of information acquisition, and effectively solves the problem of lack of connection between the left and right foot monitoring data and the problem of lack of uniformity of the measurement data of the traditional method; through the efficient channel attention mechanism, the structural semantic information at different times interacts, the correlation between the structural semantic information at different times is learned, the recognition ability of the predicted behavior is enhanced, and a more accurate fall risk prediction is provided.

[0025] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0027] Figure 1 This is a schematic diagram of the Transformer deep neural network structure in Example 1 of the present invention;

[0028] Figure 2 This is a flow chart of a fall detection method based on video semantics in Embodiment 1 of the present invention. DETAILED DESCRIPTION

[0029] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0030] It should be noted that the terms used herein are for describing specific embodiments only and are not intended to be limiting of exemplary embodiments according to the present invention.

[0031] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0032] Embodiment 1

[0033] This embodiment discloses a fall detection method based on video semantics, including:

[0034] Obtaining behavioral video clips of the individual to be tested;

[0035] Dividing the frame image corresponding to the behavior video clip of the individual to be tested into grid areas, and performing linear transformation to obtain a visual input vector;

[0036] During the training process, a set of initial structure semantic vectors that are independent of any specific image and beneficial to the model prediction effect are learned, and the initial structure semantic vectors and the corresponding visual input vectors are input into the Transformer network to obtain multi-channel structure semantic information;

[0037] An efficient channel attention mechanism is used to assign corresponding weights to the structural semantic information of multiple channels for temporal aggregation, thus obtaining temporally aggregated structural semantic information.

[0038] Based on the temporally aggregated structural semantic information, a support vector machine is used to perform prediction to obtain a detection result of whether the individual to be tested has fallen.

[0039] This embodiment uses the Transformer deep neural network to extract structural semantic information from the behavioral video clips of the individual to be tested, obtains the structural semantic information of the individual to be tested, and then uses the temporal aggregation method based on the efficient channel attention mechanism to process the structural semantic information, and then uses the prediction model to predict falls. The semantic information obtained through video analysis not only includes the parameter information in the traditional method, but also can extract hidden information that the traditional method cannot obtain. This embodiment uses the Transformer deep neural network. Compared with the traditional method and the CNN-based deep learning method, this method has a smaller amount of calculation and faster speed when predicting the risk of falling. At the same time, it improves the efficiency, convenience, connectivity, and globality of information acquisition, effectively solves the problem of lack of connection between the left and right foot monitoring data and the lack of uniformity of the measurement data of the traditional method, and thus provides a more accurate prediction of the risk of falling. This innovative method not only improves the reliability of fall prediction, but also provides strong technical support for the safety monitoring of the elderly, and further improves the accuracy, rapidity and overall performance of fall prediction.

[0040] Combine the following Figure 1-Figure 2 A fall detection method based on video semantics proposed in this embodiment is described in detail, specifically including:

[0041] Step 1: Obtain the behavioral video clips of the individual to be tested, divide the frame images corresponding to the behavioral video clips of the individual to be tested into grid areas, and perform linear transformation to obtain the visual input vector; input the initial structural semantic vector and the corresponding visual input vector learned during the training process into the Transformer network to obtain multi-channel structural semantic information.

[0042] In this embodiment, the behavior video of the individual to be tested is also preprocessed, and the preprocessing includes video cropping, normalization, denoising, contrast enhancement, etc., so as to improve the accuracy of semantic information detection for further analysis and application.

[0043] In this embodiment, a visual input vector is extracted from the preprocessed behavioral video of the individual to be tested, specifically: a two-dimensional image of the current frame is obtained at a certain interval (such as a period T = 0.5s), and its size is [H, W], and the two-dimensional image of the current frame is divided into multiple grid areas of the same size. After a linear transformation, the original video data is converted into an embedding vector in the [1, d] format and used as the input of the Transformer deep neural network. The structural semantic information of the two-dimensional image is extracted using the Transformer deep neural network based on the self-attention mechanism.

[0044] Specifically, in the process of obtaining the visual input vector, the image is first divided into a grid of size [P_h, P_w], and the image information of each grid is flattened into a one-dimensional vector of size P_h*P_w, the number of which is:

[0045] L = (H × W) / (P_h × P_w) (1)

[0046] The L one-dimensional vectors are mapped into an embedding vector of dimension d through a linear projection function. Since human structure semantic extraction is a position-sensitive task, positional encoding is used on the L embedding vectors to obtain the visual input vector.

[0047] The initial structure semantic vector is represented by N learnable d-dimensional embedding vectors. The initial structure semantic vector is independent of any specific image and is a set of initial data that is beneficial to the model prediction effect.

[0048] At the beginning of the first round of training, these N d-dimensional embedding vectors are randomly initialized according to Gaussian distribution. During training, these N d-dimensional embedding vectors are input into the Transformer deep neural network together with the visual input vector of the training sample; during the training process, these N d-dimensional embedding vectors will be updated. After the training is completed, the updated N d-dimensional embedding vectors are the initial structural semantic vectors learned by the model.

[0049] The input of the Transformer deep neural network consists of two parts: a visual input vector and an initial structural semantic vector.

[0050] The visual input vector and the initial structural semantic vector are spliced ​​together and input into the Transformer deep neural network. The Transformer deep neural network mentioned in this embodiment only uses the encoder part (encoders) of the standard Transformer deep neural network, and learns the structural semantic representation by superimposing M encoder modules. Each module includes two components: a multi-head self-attention component and a feedforward neural network component. The output of each component adopts a residual connection and a layer normalization (LayerNorm) structure to improve the stability of the model and enhance the generalization ability.

[0051] The output of the Transformer deep neural network is a structural semantic vector based on the original two-dimensional image of the current frame. The structural semantic information extracted from the original two-dimensional image is represented by a matrix composed of N structural semantic vectors.

[0052] For a single two-dimensional image, its structural semantic matrix is ​​obtained through the deep neural network in the above manner. For a piece of video data, the two-dimensional image of the current frame is processed and input into the Transformer deep neural network at a certain interval (such as period T = 0.5s), and K structural semantic matrices can be obtained. These structural semantic matrices are saved in the form of tensors to form multi-channel structural semantic information.

[0053] In this embodiment, a training sample data set is constructed, and the training sample data set includes video data of known falling or non-falling states, wherein the video data is collected by shooting a video with a camera. The training sample data set is used to train the Transformer deep neural network. During training, the structural semantic information output by the Transformer deep neural network is mapped through a linear projection layer to obtain a two-dimensional heat map of shape [H, W]. The Transformer deep neural network is optimized by predicting the difference between the two-dimensional heat map and the real two-dimensional heat map, and the loss function used is the mean square error (MSE).

[0054] In order to accurately assess and prevent the risk of falling, it is essential to collect multi-channel structural semantic information for comprehensive analysis. In addition, multi-channel structural semantic information can effectively reflect the stability of an individual's gait, thereby more accurately capturing the individual's biological structural information. By utilizing multi-channel structural semantic information, the sensitivity and predictive ability of potential fall risks can be enhanced, thus providing a scientific basis for preventing falls.

[0055] Step 2: Use an efficient channel attention mechanism to assign corresponding weights to the structural semantic information of multiple channels for temporal aggregation to obtain temporally aggregated structural semantic information; based on the temporally aggregated structural semantic information, use the support vector machine model to make predictions to obtain the detection result of whether the individual to be tested has fallen.

[0056] The machine learning model of this embodiment includes two parts: a temporal aggregation part based on the attention mechanism and a support vector machine classification part.

[0057] In this embodiment, the first part is temporal aggregation based on an efficient channel attention mechanism, which uses the channel attention mechanism to integrate multi-channel structural semantic information extracted from frames of video data and assign corresponding weights to the structural semantic information of different channels.

[0058] Specifically, the multi-channel structural semantic information is temporally aggregated based on the attention mechanism. The efficient channel attention mechanism module (ECA) is used to learn the correlation between the channels of the input feature map and assign different weights to each channel. The specific method is:

[0059] Step 201: First, perform global average pooling on the multi-channel structural semantic information with a channel number of K to obtain channel features with a scale of 1x1xK.

[0060] Step 202: Then, a convolution kernel with a channel number of 1 and an adaptive size is used to perform feature extraction to achieve cross-channel interaction.

[0061] During the convolution process, appropriate padding is used to ensure that the input and output sizes are consistent.

[0062] Step 203: The output of the convolutional layer participates in the operation of the Sigmoid activation function to assign different weights to each channel and obtain weight features.

[0063] Step 204: Multiply the weighted features with the multi-channel structural semantic information predicted by the Transformer deep neural network to obtain the structural semantic information that has undergone temporal aggregation based on the attention mechanism.

[0064] The efficient channel attention mechanism module (ECA) is particularly suitable for processing the initially extracted multi-channel structural semantic information because it can effectively take into account the correlation between the channels of the input multi-channel structural semantic information. The channels of the multi-channel structural semantic information represent the time dimension, and the structural semantic matrices of different channels represent the structural semantic information at different discrete moments in a continuous time.

[0065] This embodiment uses an efficient channel attention mechanism module (ECA) to perform temporal aggregation on multi-channel structural semantic data based on the attention mechanism, so that the structural semantic information at different times interacts, and learns the correlation between the structural semantic information at different times, which especially enhances the model's ability to recognize difficult-to-predict behaviors, thereby significantly improving the model's generalization ability on diversified data and overall fall prediction performance. This method enables the model to not only learn the broad characteristics of the data, but also accurately adjust to cope with various complex situations, ensuring the accuracy and reliability of the prediction.

[0066] The second part is to use support vector machine (SVM). The temporal aggregated structural semantic information is first flattened into a vector and then input into the trained support vector machine to obtain the prediction result of whether to fall.

[0067] The structural semantic information after temporal aggregation is flattened into a vector and input into the trained support vector machine as an input sample. The output of the support vector machine is the final fall prediction result. The support vector machine (SVM) maps the input sample from the input space to the new feature space through the Gaussian kernel function (RBF), and uses the penalty parameter C and the kernel function parameter g to find a hyperplane that separates the semantic information, thereby completing the classification of the semantic information.

[0068] Mathematical expression model of SVM model:

[0069]

[0070] st y i (ω T x i +b)≥1-ξ i ,ξ i ≥0 (2)

[0071] Where ω is the normal vector of the hyperplane, b is the offset of the hyperplane, and x i is the i-th data in the training data set, y i is x i The label, ξ i is a slack variable used to deal with noise or outliers in the data, and C is a penalty parameter used to balance the complexity of the model and the degree of fit of the training data.

[0072] The mathematical expression of the Gaussian kernel function is as follows:

[0073] k(‖x1-x2‖)=exp(-g‖x1-x2‖ 2 )(3)

[0074] Among them, g is the kernel function parameter, g>0.

[0075] In this embodiment, various gait parameters are input into a machine learning model for predicting fall risk for training, thereby evaluating the prediction performance index of each combination and finally obtaining a trained machine learning model.

[0076] Specifically, the multi-channel structural semantic data of all training samples are input into the machine learning model for predicting fall risk to train the model. When the number of training iterations exceeds the set number, or the total loss function value is less than the set threshold, the training is stopped to obtain a trained machine learning model.

[0077] By analyzing the video, the visual input vector is extracted after preprocessing the video data, and the visual input vector and the initial structural semantic vector are input into the Transformer deep neural network together to obtain preliminary structural semantic information, which can provide important information about the individual's action state. The preliminary structural semantic information is temporally aggregated based on the attention mechanism to analyze the fluctuations and changes in the individual's behavior in the continuous time process, which can reflect the individual's movement characteristics and help assess the risk of falling.

[0078] The present invention extracts the structural semantic information of an individual and performs temporal aggregation of the structural semantic information based on an attention mechanism. The result after temporal aggregation reflects the stability of the individual's gait. By comprehensively utilizing this information, the system can capture the individual's behavior more comprehensively and improve the sensitivity to the risk of falling. This method of comprehensively utilizing structural semantic information and temporal aggregation operations based on an attention mechanism makes the system more advantageous in fall prediction.

[0079] In order to effectively learn and adapt to the differences between different individuals and accurately predict the risk of falling, the present invention adopts a deep learning neural network model based on Transformer for extracting structural semantic information and a machine learning model based on support vector machine for predicting the risk of falling. The deep learning neural network model can adaptively adjust the weights and biases of the model and enhance the learning ability for individuals that are difficult to predict. By introducing this deep learning model, the system of the present invention can better adapt to diversity and improve the overall fall prediction performance.

[0080] This embodiment provides a real-time, effective and highly adaptable fall prediction method, which is applicable to multiple fields such as sports biomechanics and rehabilitation medicine. This method introduces a deep learning model, which significantly improves the intelligence and accuracy of fall prediction and provides new technical means for research and practical application in these fields.

[0081] The solution of this embodiment is applicable to individuals of different age groups and physiological states, which significantly improves the versatility of the system. It has broad application prospects, especially in the fields of elderly health management and medical auxiliary equipment.

[0082] Embodiment 2

[0083] The purpose of this embodiment is to provide a fall detection system based on video semantics, including:

[0084] An acquisition unit is configured to: acquire a behavior video clip of the individual to be tested;

[0085] A first feature extraction unit is configured to: divide the frame image corresponding to the behavior video clip of the individual to be tested into grid areas, and perform linear transformation to obtain a visual input vector;

[0086] A second feature extraction unit is configured to: during the training process, learn a set of initial structure semantic vectors that are independent of any specific image and are beneficial to the model prediction effect, input the initial structure semantic vectors and the corresponding visual input vectors into the Transformer network, and obtain multi-channel structure semantic information;

[0087] The temporal aggregation unit is configured to: use an efficient channel attention mechanism to assign corresponding weights to the multi-channel structural semantic information for temporal aggregation, thereby obtaining temporally aggregated structural semantic information;

[0088] A prediction unit is configured to: perform prediction using a support vector machine based on the temporally aggregated structural semantic information to obtain a detection result of whether the individual to be tested has fallen.

[0089] In further embodiments, there is also provided:

[0090] An electronic device includes a memory and a processor, and computer instructions stored in the memory and executed on the processor, wherein when the computer instructions are executed by the processor, the method described in Embodiment 1 is performed. For the sake of brevity, no further description is given here.

[0091] It should be understood that in this embodiment, the processor may be a central processing unit CPU, and the processor may also be other general-purpose processors, digital signal processors DSP, application-specific integrated circuits ASIC, off-the-shelf programmable gate arrays FPGA or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0092] The memory may include a read-only memory and a random access memory, and provide instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type.

[0093] A computer-readable storage medium is used to store computer instructions. When the computer instructions are executed by a processor, the method described in embodiment 1 is completed.

[0094] The method in the first embodiment can be directly embodied as a hardware processor, or a combination of hardware and software modules in the processor. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware. To avoid repetition, it will not be described in detail here.

[0095] A computer program product includes a computer program, and when the computer program is executed by a processor, the method described in the first embodiment is implemented.

[0096] The present invention also provides at least one computer program product tangibly stored on a non-transitory computer-readable storage medium. The computer program product includes computer executable instructions, such as instructions included in a program module, which are executed in a device on a real or virtual processor of the target to perform the process / method as described above. Typically, program modules include routines, programs, libraries, objects, classes, components, data structures, etc. that perform specific tasks or implement specific abstract data types. In various embodiments, the functions of program modules can be combined or divided between program modules as needed. Machine executable instructions for program modules can be executed in local or distributed devices. In distributed devices, program modules can be located in local and remote storage media.

[0097] The computer program code for implementing the method of the present invention can be written in one or more programming languages. These computer program codes can be provided to the processor of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that the program code, when executed by the computer or other programmable data processing device, causes the function / operation specified in the flow chart and / or block diagram to be implemented. The program code can be executed completely on a computer, partially on a computer, as an independent software package, partially on a computer and partially on a remote computer or completely on a remote computer or server.

[0098] In the context of the present invention, computer program codes or related data may be carried by any appropriate carrier to enable a device, apparatus or processor to perform the various processes and operations described above. Examples of carriers include signals, computer readable media, etc. Examples of signals may include electrical, optical, radio, acoustic or other forms of propagation signals, such as carrier waves, infrared signals, etc.

[0099] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with this embodiment can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0100] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A fall detection method based on video semantics, characterized in that: include: Obtaining behavioral video clips of the individual to be tested; Dividing the frame image corresponding to the behavior video clip of the individual to be tested into grid areas, and performing linear transformation to obtain a visual input vector; The initial structure semantic vector learned through training is input into the Transformer network together with the visual input vector to obtain multi-channel structure semantic information; An efficient channel attention mechanism is used to assign corresponding weights to the structural semantic information of multiple channels for temporal aggregation, thus obtaining temporally aggregated structural semantic information. Based on the temporally aggregated structural semantic information, a support vector machine is used to perform prediction to obtain a detection result of whether the individual to be tested has fallen.

2. A fall detection method based on video semantics as claimed in claim 1, characterized in that: The frame image corresponding to the behavior video clip of the individual to be tested is divided into grid areas, and a linear transformation is performed to obtain a visual input vector, specifically: Dividing the frame image corresponding to the behavior video clip of the individual to be tested into grid areas, and flattening the image information of each grid area into a one-dimensional vector; The one-dimensional vector corresponding to each grid area is mapped to an embedded vector through a linear projection function; The embedding vector is encoded to obtain a visual input vector.

3. A fall detection method based on video semantics as claimed in claim 1, characterized in that: The initial structure semantic vector learned is obtained, specifically: Initialize N learnable d-dimensional embedding vectors randomly according to Gaussian distribution to represent the initial structure semantic vector; During the training process of the fall detection model, the training samples and the initial structural semantic vector are input into the fall detection model, and the initial structural semantic vector learned by the fall detection model is obtained after the training.

4. A fall detection method based on video semantics as claimed in claim 1, characterized in that: The initial structure semantic vector and the corresponding visual input vector are input into the Transformer network to obtain multi-channel structure semantic information, specifically: Using a Transformer encoder to extract features from the visual input vector corresponding to the frame image and the learned initial structural semantic vector to obtain a structural semantic matrix; The structural semantic matrix of each frame image is saved in the form of a tensor to form multi-channel structural semantic information.

5. A fall detection method based on video semantics as claimed in claim 1, characterized in that: The efficient channel attention mechanism is used to assign corresponding weights to the structural semantic information of multiple channels for temporal aggregation, and the temporally aggregated structural semantic information is obtained, specifically: Perform global average pooling on the multi-channel structural semantic information to obtain channel features; Using convolution to extract the channel features, and obtain cross-channel interactive features; The cross-channel interaction features are calculated using an activation function to obtain weight features; The weight feature is multiplied by the multi-channel structural semantic information to obtain temporally aggregated structural semantic information.

6. A fall detection method based on video semantics as claimed in claim 1, characterized in that: Based on the temporally aggregated structural semantic information, a support vector machine is used to perform prediction to obtain a detection result of whether the individual to be tested has fallen, specifically: Mapping the temporally aggregated structural semantic information to a feature space through a Gaussian kernel function; By using the penalty parameter and the kernel function parameter, an optimal hyperplane for separating data information is constructed, and the temporally aggregated structural semantic information is classified to obtain a detection result of whether the individual to be tested has fallen.

7. A fall detection system based on video semantics, characterized in that: include: An acquisition unit is configured to: acquire a behavior video clip of the individual to be tested; A first feature extraction unit is configured to: divide the frame image corresponding to the behavior video clip of the individual to be tested into grid areas, and perform linear transformation to obtain a visual input vector; A second feature extraction unit is configured to: input the initial structure semantic vector learned through training and the visual input vector into the Transformer network to obtain multi-channel structure semantic information; The temporal aggregation unit is configured to: use an efficient channel attention mechanism to assign corresponding weights to the multi-channel structural semantic information for temporal aggregation, thereby obtaining temporally aggregated structural semantic information; A prediction unit is configured to: perform prediction using a support vector machine based on the temporally aggregated structural semantic information to obtain a detection result of whether the individual to be tested has fallen.

8. An electronic device, characterized in that: The method comprises a memory and a processor and computer instructions stored in the memory and executed on the processor, wherein when the computer instructions are executed by the processor, the method according to any one of claims 1 to 6 is completed.

9. A computer-readable storage medium, characterized in that: Used to store computer instructions, which, when executed by a processor, complete the method described in any one of claims 1 to 6.

10. A computer program product, characterized in that The invention comprises a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 6.

Images