Fall detection method and device based on video, medium and equipment

Abstract

The invention discloses a falling detection method and device based on a video, a medium and equipment, and the method comprises the steps: obtaining video stream data, and obtaining a human body target Box box; a DeepSORT target tracking algorithm is adopted to dynamically track the track of each human body target Box box; according to the track of the human body target Box box, behavior recognition is carried out on each human body target object, and the behavior category of the human body target object is determined; generating TSM model input data based on the human body target frame queue data; and inputting the TSM model input data into a fall detection TSM model to obtain a detection result. According to the method, behavior trajectory tracking is realized based on DeepSORT, the behaviors are primarily classified, then tumble or non-tumble fine classification identification is carried out on the single individual behavior in the toppling state through the tumble detection TSM model, the extracted behavior time sequence characteristics are fully utilized, and the detection accuracy is improved.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to a video-based fall detection method, device, medium and equipment. Background Art

[0002] With the advent of an aging society, the scale of home-based elderly care continues to expand. After an elderly person falls, it is particularly important to be able to accurately alarm and notify relatives or caregivers for timely nursing intervention to achieve the goals of safe and smart elderly care. In addition, in public places such as kindergartens and shopping malls, it is also necessary to promptly notify teachers or staff to help deal with the situation after someone falls.

[0003] Compared with the sensor-based fall detection scheme, the video-based fall detection method makes full use of general surveillance cameras, is simple and convenient to deploy, has low cost, and is suitable for large-scale promotion. However, there is currently no video-based fall detection method that can detect falls according to the temporal behavioral characteristics of the actor. Summary of the invention

[0004] In view of the above problems, the present invention is proposed to provide a video-based fall detection method, device, medium and equipment that overcome the above problems or at least partially solve the above problems.

[0005] Other features and advantages of the present invention will become apparent from the following detailed description, or may be learned in part by practice of the present invention.

[0006] According to a first aspect of an embodiment of the present invention, a video-based fall detection method is provided, and the video-based fall detection method includes:

[0007] Acquire video stream data, detect and identify the video stream data based on a target detection tool, confirm the human target object in the video stream data, and obtain a human target Box frame;

[0008] Using the DeepSORT target tracking algorithm, dynamically tracking the trajectory of each human target Box, intercepting a number of video frame data evenly spaced in the video stream data, saving the number of video frame data to a first queue according to a time sequence, and saving the trajectory of each human target Box frame in the number of video frame data to a second queue;

[0009] According to the trajectory of the human target Box, behavior recognition is performed on each of the human target objects to determine the behavior category of the human target object, and according to the behavior category of the human target object, the data of the second queue is determined to be discarded or retained;

[0010] Calculate the target behavior state data according to the retained data of the second queue, and cut out the human target frame queue data from the data of the first queue based on the target behavior state data;

[0011] Based on the human target frame queue data, generating TSM model input data;

[0012] The TSM model input data is input into the fall detection TSM model to obtain a detection result, and when the detection result meets the alarm condition, an alarm message is generated.

[0013] In some embodiments of the present invention, performing behavior recognition on each of the human target objects according to the trajectory of the human target Box, determining the behavior category of the human target object, and determining whether the data in the second queue is discarded or retained according to the behavior category of the human target object includes:

[0014] According to the trajectory of the human target Box, behavior recognition is performed on each of the human target objects to determine whether the behavior category of the human target object is the first state or the second state;

[0015] When the behavior category of the human target object is in the first state, discarding the corresponding data of the second queue;

[0016] When the behavior category of the human target object is in the second state, retaining the corresponding data of the second queue;

[0017] When all the data in the second queue are discarded, the human target Box is reacquired.

[0018] In some embodiments of the present invention, generating TSM model input data based on the human target frame queue data includes:

[0019] Determine the size requirement of the input image corresponding to the fall detection TSM model;

[0020] The human target frame queue data is scaled according to the size requirement of the input image to generate the TSM model input data.

[0021] In some embodiments of the present invention, the method includes training to obtain the fall detection TSM model, including:

[0022] Collect sample video data;

[0023] intercepting the sample video data to obtain a number of sample video data at even intervals;

[0024] Classifying the plurality of sample video data respectively to obtain first sample data in a first state and second sample data in a second state;

[0025] Performing frame extraction processing on the first sample data and the second sample data respectively to obtain a number of sample video frame data at even intervals;

[0026] A data set is generated according to the sample video frame data, and model training is performed based on the data set to obtain the fall detection TSM model.

[0027] According to a second aspect of an embodiment of the present invention, a video-based fall detection device is provided, and the video-based fall detection device includes:

[0028] A data acquisition module is used to acquire video stream data, detect and identify the video stream data based on a target detection tool, confirm the human target object in the video stream data, and obtain a human target Box frame;

[0029] A frame extraction module is used to adopt the DeepSORT target tracking algorithm to dynamically track the trajectory of each human target Box frame, intercept a number of video frame data evenly spaced in the video stream data, save the number of video frame data to a first queue according to a time sequence, and save the trajectory of each human target Box frame in the number of video frame data to a second queue;

[0030] a target recognition module, configured to perform behavior recognition on each of the human target objects according to the trajectory of the human target Box, determine the behavior category of the human target object, and determine whether the data in the second queue is to be discarded or retained according to the behavior category of the human target object;

[0031] A first data processing module is used to calculate target behavior state data according to the retained data of the second queue, and cut out human target frame queue data from the data of the first queue based on the target behavior state data;

[0032] A second data processing module is used to generate TSM model input data based on the human target frame queue data;

[0033] The data detection module is used to input the TSM model input data into the fall detection TSM model to obtain a detection result, and generate an alarm message when the detection result meets the alarm condition.

[0034] In some embodiments of the present invention, the target recognition module is used to:

[0035] According to the trajectory of the human target Box, behavior recognition is performed on each of the human target objects to determine whether the behavior category of the human target object is the first state or the second state;

[0036] When the behavior category of the human target object is in the first state, discarding the corresponding data of the second queue;

[0037] When the behavior category of the human target object is in the second state, retaining the corresponding data of the second queue;

[0038] When all the data in the second queue are discarded, the human target Box is reacquired.

[0039] In some embodiments of the present invention, the second data processing module is used to:

[0040] Determine the size requirement of the input image corresponding to the fall detection TSM model;

[0041] The human target frame queue data is scaled according to the size requirement of the input image to generate the TSM model input data.

[0042] In some embodiments of the present invention, the data detection module is further used to train and obtain the fall detection TSM model, including:

[0043] Collect sample video data;

[0044] intercepting the sample video data to obtain a number of sample video data at even intervals;

[0045] Classifying the plurality of sample video data respectively to obtain first sample data in a first state and second sample data in a second state;

[0046] Performing frame extraction processing on the first sample data and the second sample data respectively to obtain a number of sample video frame data at even intervals;

[0047] A data set is generated according to the sample video frame data, and model training is performed based on the data set to obtain the fall detection TSM model.

[0048] According to a third aspect of an embodiment of the present invention, a computer-readable storage medium is provided, in which computer program instructions are stored. The computer program instructions are loaded and executed by a processor to implement the operations performed by any of the methods described above.

[0049] According to a fourth aspect of an embodiment of the present invention, there is provided an electronic device, comprising a processor and a memory, wherein the memory stores computer program instructions that can be executed by the processor, and when the processor executes the computer program instructions, it implements instructions of any of the methods described above.

[0050] The technical solution provided in the embodiments of the present invention has at least the following technical effects or advantages:

[0051] The embodiments of the present invention provide a video-based fall detection method, device, medium and equipment. The method described in the embodiments of the present invention integrates the target tracking tool DeepSORT (or SORT) and the time drift model fall detection TSM model to realize fall detection, realizes behavior trajectory tracking based on DeepSORT, performs initial classification of behavior, and then uses the fall detection TSM model to classify and identify the fall or non-fall of a single individual behavior in a tilting state, making full use of the extracted behavior time series features, and improving the detection accuracy compared with the model that uses a single static picture to classify falls and non-falls.

[0052] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0054] Figure 1 A schematic diagram of a flow chart of a video-based fall detection method provided by an embodiment of the present invention;

[0055] Figure 2 A reference diagram of the behavior categories of human target objects;

[0056] Figure 3 This is a schematic diagram of the Shift operation;

[0057] Figure 4 A schematic diagram of the principle structure of a video-based fall detection method provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0058] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings.

[0059] Various structural schematic diagrams according to embodiments of the present disclosure are shown in the accompanying drawings. These figures are not drawn to scale, and some details are magnified and some details may be omitted for the purpose of clear expression. The shapes of various regions and layers shown in the figures and the relative sizes and positional relationships therebetween are only exemplary, and may deviate in practice due to manufacturing tolerances or technical limitations, and those skilled in the art may further design regions / layers with different shapes, sizes, and relative positions according to actual needs.

[0060] In the context of the present disclosure, when a layer / element is referred to as being "on" another layer / element, the layer / element may be directly on the other layer / element, or there may be an intervening layer / element therebetween. Additionally, if a layer / element is "on" another layer / element in one orientation, the layer / element may be "below" the other layer / element when the orientation is reversed. In the context of the present disclosure, similar or identical components may be denoted by the same or similar reference numerals.

[0061] In order to better understand the above-mentioned technical scheme, the above-mentioned technical scheme will be described in detail below in combination with specific implementation methods. It should be understood that the embodiments of the present disclosure and the specific features in the embodiments are detailed descriptions of the technical scheme of the present invention, rather than limitations on the technical scheme of the present invention. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments can be combined with each other.

[0062] Figure 1 is a flow chart of a video-based fall detection method provided by an embodiment of the present invention, such as Figure 1 As shown, the video-based fall detection method includes the following steps:

[0063] S1. Obtain video stream data, detect and identify the video stream data based on a target detection tool, confirm the human target object in the video stream data, and obtain a human target Box frame;

[0064] In an embodiment of the present invention, the target detection tool may be a tool such as a YOLO detection model, which is used to confirm the human target object in the video stream data and extract the human target Box frame.

[0065] S2, using the DeepSORT target tracking algorithm to dynamically track the trajectory of each human target Box frame, intercepting a number of video frame data evenly spaced in the video stream data, saving the number of video frame data to a first queue according to a time sequence, and saving the trajectory of each human target Box frame in the number of video frame data to a second queue;

[0066] Specifically, for example, 2 seconds of video data in the video stream data is intercepted and frame extraction is performed, 8 frames of video frame data are extracted at even intervals and saved in the first queue QueueV; and the trajectory of each human target Box in the 8 frames of video frame data is saved in the second queue QueueB.

[0067] S3, according to the trajectory of the human target Box, perform behavior recognition on each of the human target objects, determine the behavior category of the human target object, and determine whether the data in the second queue is discarded or retained according to the behavior category of the human target object;

[0068] The embodiment of the present invention performs behavior recognition on each of the human target objects according to the trajectory of the human target Box, determines the behavior category of the human target object, and determines whether the data in the second queue is discarded or retained according to the behavior category of the human target object. Specifically includes: performing behavior recognition on each of the human target objects according to the trajectory of the human target Box, and determining whether the behavior category of the human target object is the first state or the second state; when the behavior category of the human target object is the first state, discarding the corresponding data in the second queue; when the behavior category of the human target object is the second state, retaining the corresponding data in the second queue; when all the data in the second queues are discarded, reacquiring the human target Box.

[0069] Specifically, combined Figure 2 As shown, for example, the first state may be that the human target object is in a stable state such as "moving" or "staying", and the second state may be that the human target object is in an unstable state such as "falling over", that is, the human target object is in a state where it may fall over. When the embodiment of the present invention confirms that the behavior category of the human target object is the first state, the corresponding data of the second queue QueueB is discarded, and when all the data of the second queues are discarded, step S1 is executed to reacquire the human target Box, and only when it is confirmed that the behavior category of the human target object is the second state, subsequent steps are executed to further detect and confirm whether there is a falling state, so as to save computing resources.

[0070] Among them, the judgment criteria for the behavior category of the human target object in the embodiment of the present invention are shown as follows (the area of ​​the intersection of the 8 human target Box frames after frame alignment exceeds 80% of the average area of ​​the 8 human target Box frames):

[0071]

[0072] in,

[0073] The effects of InterBox in the first state and OuterBox in the second state are as follows Figure 2 See reference.

[0074] S4, according to the retained data of the second queue, calculate the target behavior state data, and cut out the human target frame queue data from the data of the first queue based on the target behavior state data;

[0075] For each human target object in the retained "fallen over" state, the target behavior state data OuterBox is calculated according to the second queue QueueB corresponding to the human target Box frame, and a new human target frame queue QueueObj is cut out from the first queue QueueV based on the target behavior state data OuterBox.

[0076] The calculation method of OuterBox in the embodiment of the present invention is shown as follows:

[0077]

[0078] S5, generating TSM model input data based on the human target frame queue data;

[0079] In order to make the input data meet the input requirements of the fall detection TSM model, an embodiment of the present invention generates TSM model input data based on the human target frame queue data, including: determining the size requirement of the input image corresponding to the fall detection TSM model; scaling the human target frame queue data according to the size requirement of the input image to generate the TSM model input data.

[0080] The embodiment of the present invention calculates the size requirements of the input image corresponding to the TSM model, including the width W and the height H, scales the image frame sizes in the N human target frame queues QueueObj to WxH, and generates TSM model input data DataIns.

[0081] In the embodiment of the present invention, the format of the TSM model input data DataIns is as shown in the following reference:

[0082] N×C×H×W

[0083] Among them, C=8×3=24, which means that each target frame queue has 8 images, each image has three channels of R, G, and B, and each target frame queue has a total of 24 channels.

[0084] The calculation method of W and H is shown in the following reference:

[0085] W=max(W1,…,W N ),H=max(H1,…,H N )

[0086] W1,…,WN and H1,…,HN are the width and height of the images in the queue of N human target frames respectively.

[0087] S6. Input the TSM model input data into the fall detection TSM model to obtain a detection result, and generate an alarm message when the detection result meets the alarm condition.

[0088] Before inputting the TSM model input data into the fall detection TSM model, an embodiment of the present invention further includes training to obtain the fall detection TSM model, including the following steps: collecting sample video data; intercepting the sample video data to obtain a number of sample video data at uniform intervals; classifying the number of sample video data respectively to obtain first sample data of a first state and second sample data of a second state; performing frame extraction processing on the first sample data and the second sample data respectively to obtain a number of sample video frame data at uniform intervals; generating a data set according to the sample video frame data, performing model training based on the data set, and obtaining the fall detection TSM model.

[0089] Specifically, in the embodiment of the present invention, a network TSMNet for generating a fall detection TSM model is constructed. From top to bottom, it includes two parts: backbone and head. All convolutional layers Conv, pooling layers Pool, and regularization layers BatchNorm are 2D;

[0090] The network module generated by combining the convolution layer Conv, the pooling layer Pool, the regularization layer BatchNorm and other basic network units is as follows:

[0091] The ConvBN block includes 1 Conv2D unit, 1 BatchNorm2D unit, and 1 LeakyReLU unit from top to bottom;

[0092] The NetBlock block includes 1 Shift block, 2 ConvBN blocks, 1 ResAdd block, and 1 LeakyReLU unit from top to bottom. The ResAdd block takes the output of the previous ConvBN block as the residual Res, superimposes it on the input data of the NetBlock block, and inputs it into the LeakyReLU unit of the subsequent block as the return result of the ResAdd block.

[0093] The Shift block is implemented as follows:

[0094] Input: X(T,C1,H1,W1), T represents the time dimension;

[0095] Return: Y(T,C1,H1,W1);

[0096] Ns=int(C1 / 8)

[0097] Y(0:T,0:C1,0:H1,0:W1)=X(0:T,0:C1,0:H1,0:W1)

[0098] Y(0,0:Ns,0:H1,0:W1)=0

[0099] Y(1:T,0:Ns,0:H1,0:W1)=X(0:T-1,0:Ns,0:H1,0:W1)

[0100] Shift operation diagram Figure 3 As shown in the reference, Figure 3 In the example, Ns=2, C1=6, and T=4.

[0101] The backbone part includes 2 ConvBN blocks, 1 MaxPool2D unit, and 4 NetBlock blocks from top to bottom.

[0102] The head part includes 1 AdaptiveAvgPool2D unit, 1 Dropout unit, 1 fully connected unit, and 1 cross entropy loss function CrossEntropyLoss from top to bottom.

[0103] The fall detection TSM model based on the TSMNet network needs to be generated through sample training first. The fall detection TSM model is a classification model for time series data, which can realize joint modeling of feature fusion between different frames without additional computing cost. The number of its classifications is 2, including: fall (for example, the second state is falling), not fall (for example, the first state is moving or staying).

[0104] An embodiment of the present invention collects sample video data, wherein the human target object in the sample video data needs to be the same person, and the sample video data is various behavior trajectories of the human target object, including but not limited to walking, running, squatting, sitting, lying down, falling down and other states.

[0105] The sample video data is cut into 2 to 4 seconds of video clips, each of which contains a single behavior: walking, running, squatting, sitting, lying down, falling down, etc. The video clips are divided into two categories, the first sample data of the first state and the second sample data of the second state, i.e., falling down, not falling down (including walking, running, squatting, sitting down, lying down, etc. except falling down), and the number of video clips in the two categories is roughly equal, and the video clips of the falling category contain as many falling trajectories as possible in different scenes, different individuals, and different postures.

[0106] Each video clip is sampled at equal intervals to obtain 8 frames of sample video frame data. The two types of video clips generate two types of image samples, each type of image sample contains a roughly equal number of image sequence units, and each image sequence unit contains 8 sample video frame data.

[0107] For example, the embodiment of the present invention generates 3400 samples of two types of images, namely, falling and not falling, and each type of sample contains 1700 image sequence units. The number of samples is 3400, of which 3000 are training sets and 400 are validation sets.

[0108] The fall detection TSM model training parameter settings are shown in the following reference:

[0109] img-size: 640x640

[0110] epochs: 500

[0111] batch-size: 16

[0112] learning-rate: 0.01

[0113] After the model training was completed, the accuracy on the validation set was 92.53%.

[0114] The embodiment of the present invention can thus generate a data set according to the sample video frame data, perform model training based on the data set, and obtain the fall detection TSM model.

[0115] Compared with the prior art, the video-based fall detection method described in the embodiment of the present invention integrates the target tracking tool DeepSORT (or SORT) and the time drift model fall detection TSM model to realize fall detection, realizes behavior trajectory tracking based on DeepSORT, performs initial classification of behavior, and then uses the fall detection TSM model to classify and identify the fall or non-fall of a single individual behavior in a tilting state, making full use of the extracted behavior time series features, and improving the detection accuracy compared with the model that uses a single static picture to classify falls and non-falls.

[0116] Based on the above embodiment, as the above Figure 1 The present invention provides an embodiment of a video-based fall detection device, which is similar to Figure 1 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices, see Figure 4 As shown, the video-based fall detection device includes:

[0117] The data acquisition module 100 is used to acquire video stream data, detect and identify the video stream data based on a target detection tool, confirm the human target object in the video stream data, and obtain a human target Box frame;

[0118] The frame extraction module 200 is used to adopt the DeepSORT target tracking algorithm to dynamically track the trajectory of each human target Box frame, intercept a number of video frame data evenly spaced in the video stream data, save the number of video frame data to a first queue according to a time sequence, and save the trajectory of each human target Box frame in the number of video frame data to a second queue;

[0119] The target recognition module 300 is used to perform behavior recognition on each of the human target objects according to the trajectory of the human target Box, determine the behavior category of the human target object, and determine whether the data in the second queue is discarded or retained according to the behavior category of the human target object;

[0120] The first data processing module 400 is used to calculate the target behavior state data according to the retained data of the second queue, and cut out the human target frame queue data from the data of the first queue based on the target behavior state data;

[0121] A second data processing module 500 is used to generate TSM model input data based on the human target frame queue data;

[0122] The data detection module 600 is used to input the TSM model input data into the fall detection TSM model to obtain a detection result, and generate an alarm message when the detection result meets the alarm condition.

[0123] In the embodiment of the present invention, the target recognition module 300 is used to:

[0124] According to the trajectory of the human target Box, behavior recognition is performed on each of the human target objects to determine whether the behavior category of the human target object is the first state or the second state;

[0125] When the behavior category of the human target object is in the first state, discarding the corresponding data of the second queue;

[0126] When the behavior category of the human target object is in the second state, retaining the corresponding data of the second queue;

[0127] When all the data in the second queue are discarded, the human target Box is reacquired.

[0128] In the embodiment of the present invention, the second data processing module 500 is used to:

[0129] Determine the size requirement of the input image corresponding to the fall detection TSM model;

[0130] The human target frame queue data is scaled according to the size requirement of the input image to generate the TSM model input data.

[0131] In the embodiment of the present invention, the data detection module 600 is further used to train and obtain the fall detection TSM model, including:

[0132] Collect sample video data;

[0133] intercepting the sample video data to obtain a number of sample video data at even intervals;

[0134] Classifying the plurality of sample video data respectively to obtain first sample data in a first state and second sample data in a second state;

[0135] Performing frame extraction processing on the first sample data and the second sample data respectively to obtain a number of sample video frame data at even intervals;

[0136] A data set is generated according to the sample video frame data, and model training is performed based on the data set to obtain the fall detection TSM model.

[0137] The video-based fall detection device described in the embodiment of the present invention can execute the video-based fall detection method provided in the above embodiment. The video-based fall detection device has the corresponding functional steps and beneficial effects of the video-based fall detection method described in the above embodiment. Please refer to the embodiment of the above-mentioned video-based fall detection method for details. The embodiment of the present invention will not be repeated here.

[0138] The embodiment of the present invention also provides an electronic device, which may include a processor and a memory, wherein the processor and the memory may be connected via a bus or other means. The processor may be a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components and other chips, or a combination of the above-mentioned various types of chips. The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs, non-transient computer executable programs and modules, such as program instructions / modules corresponding to the video-based fall detection method in the embodiment of the present invention. The processor executes various functional applications and data processing of the processor by running the non-transient software programs, instructions and modules stored in the memory, that is, realizing the video-based fall detection method in the above method embodiment.

[0139] The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required by at least one function; the data storage area may store data created by the processor, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. The one or more modules are stored in the memory, and when executed by the processor, a video-based fall detection method as in the above-mentioned method embodiment is executed. The specific details of the above-mentioned electronic device can be understood by corresponding to the corresponding descriptions and effects in the above-mentioned method embodiment, and will not be repeated here. Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment method can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and the program, when executed, may include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium can be a read-only memory (ROM), a random access memory (RAM), a flash memory (Flash Memory), a hard disk (HDD) or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above types of memory.

[0140] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0141] Similarly, it should be understood that in order to streamline the present disclosure and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this disclosed method should not be interpreted as reflecting the intention that the claimed invention requires more features than those expressly recited in each claim. Rather, as reflected in the claims, inventive aspects lie in less than all of the features of the individual embodiments previously disclosed. Therefore, the claims that follow the detailed description are hereby expressly incorporated into the detailed description, with each claim itself serving as a separate embodiment of the present invention.

[0142] It should be noted that the above-mentioned embodiments illustrate rather than limit the invention and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims.

Claims

1. A video-based fall detection method, characterized in that: The video-based fall detection method includes: Acquire video stream data, detect and identify the video stream data based on a target detection tool, confirm the human target object in the video stream data, and obtain a human target Box frame; Using the DeepSORT target tracking algorithm, dynamically tracking the trajectory of each human target Box, intercepting a number of video frame data evenly spaced in the video stream data, saving the number of video frame data to a first queue according to a time sequence, and saving the trajectory of each human target Box frame in the number of video frame data to a second queue; According to the trajectory of the human target Box, behavior recognition is performed on each of the human target objects to determine the behavior category of the human target object, and according to the behavior category of the human target object, the data of the second queue is determined to be discarded or retained; Calculate the target behavior state data according to the retained data of the second queue, and cut out the human target frame queue data from the data of the first queue based on the target behavior state data; Based on the human target frame queue data, generating TSM model input data; The TSM model input data is input into the fall detection TSM model to obtain a detection result, and when the detection result meets the alarm condition, an alarm message is generated.

2. The video-based fall detection method according to claim 1, characterized in that: The step of performing behavior recognition on each of the human target objects according to the trajectory of the human target Box, determining the behavior category of the human target object, and determining whether the data in the second queue is discarded or retained according to the behavior category of the human target object comprises: According to the trajectory of the human target Box, behavior recognition is performed on each of the human target objects to determine whether the behavior category of the human target object is the first state or the second state; When the behavior category of the human target object is in the first state, discarding the corresponding data of the second queue; When the behavior category of the human target object is in the second state, retaining the corresponding data of the second queue; When all the data in the second queue are discarded, the human target Box is reacquired.

3. The video-based fall detection method according to claim 1, characterized in that: The generating TSM model input data based on the human target frame queue data comprises: Determine the size requirement of the input image corresponding to the fall detection TSM model; The human target frame queue data is scaled according to the size requirement of the input image to generate the TSM model input data.

4. The video-based fall detection method according to claim 2, characterized in that: The method includes training and acquiring the fall detection TSM model, including: Collect sample video data; intercepting the sample video data to obtain a number of sample video data at even intervals; Classifying the plurality of sample video data respectively to obtain first sample data in a first state and second sample data in a second state; Performing frame extraction processing on the first sample data and the second sample data respectively to obtain a number of sample video frame data at even intervals; A data set is generated according to the sample video frame data, and model training is performed based on the data set to obtain the fall detection TSM model.

5. A video-based fall detection device, characterized in that: The video-based fall detection device comprises: A data acquisition module is used to acquire video stream data, detect and identify the video stream data based on a target detection tool, confirm the human target object in the video stream data, and obtain a human target Box frame; A frame extraction module is used to adopt the DeepSORT target tracking algorithm to dynamically track the trajectory of each human target Box frame, intercept a number of video frame data evenly spaced in the video stream data, save the number of video frame data to a first queue according to a time sequence, and save the trajectory of each human target Box frame in the number of video frame data to a second queue; a target recognition module, configured to perform behavior recognition on each of the human target objects according to the trajectory of the human target Box, determine the behavior category of the human target object, and determine whether the data in the second queue is to be discarded or retained according to the behavior category of the human target object; A first data processing module is used to calculate target behavior state data according to the retained data of the second queue, and cut out human target frame queue data from the data of the first queue based on the target behavior state data; A second data processing module is used to generate TSM model input data based on the human target frame queue data; The data detection module is used to input the TSM model input data into the fall detection TSM model to obtain a detection result, and generate an alarm message when the detection result meets the alarm condition.

6. The video-based fall detection device according to claim 5, characterized in that: The target recognition module is used for: According to the trajectory of the human target Box, behavior recognition is performed on each of the human target objects to determine whether the behavior category of the human target object is the first state or the second state; When the behavior category of the human target object is in the first state, discarding the corresponding data of the second queue; When the behavior category of the human target object is in the second state, retaining the corresponding data of the second queue; When all the data in the second queue are discarded, the human target Box is reacquired.

7. The video-based fall detection device according to claim 5, characterized in that: The second data processing module is used for: Determine the size requirement of the input image corresponding to the fall detection TSM model; The human target frame queue data is scaled according to the size requirement of the input image to generate the TSM model input data.

8. The video-based fall detection device according to claim 6, characterized in that: The data detection module is also used to train and obtain the fall detection TSM model, including: Collect sample video data; intercepting the sample video data to obtain a number of sample video data at even intervals; Classifying the plurality of sample video data respectively to obtain first sample data in a first state and second sample data in a second state; Performing frame extraction processing on the first sample data and the second sample data respectively to obtain a number of sample video frame data at even intervals; A data set is generated according to the sample video frame data, and model training is performed based on the data set to obtain the fall detection TSM model.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, which are loaded and executed by a processor to implement the operations performed by the method according to any one of claims 1 to 4.

10. An electronic device comprising a processor and a memory, characterized in that: The memory stores computer program instructions that can be executed by the processor, and when the processor executes the computer program instructions, the processor implements instructions of any one of the methods described in claims 1-4.

Images