Fall detection method, apparatus, electronic device, and computer-readable storage medium

Abstract

Embodiments of the present application relate to a fall detection method and device, electronic equipment and computer readable storage medium. The method comprises: performing human body detection on a current frame image to obtain a first detection box corresponding to the current frame image; performing human body key point detection on each first detection box to obtain a first key point corresponding to each first detection box; obtaining a second detection box and a second key point corresponding to a previous frame image; clustering pixel points contained in the first detection box and the second detection box to obtain a first clustering center and a second clustering center, respectively; matching the first detection box and the second detection box to determine a second detection box matched with each first detection box; and performing fall detection on the key points of each first detection box and the matched second detection box through a fall detection model to obtain a fall detection result. The fall detection method, device, electronic equipment and computer readable storage medium can improve the accuracy of detecting whether a person has fallen.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, specifically to a fall detection method, apparatus, electronic device, and computer-readable storage medium. Background Technology

[0002] Falls in the elderly, if not detected and treated promptly, can lead to disability. Therefore, more accurate assessments of falls and minimizing their impact have become a hot research topic. Summary of the Invention

[0003] This application discloses a fall detection method, apparatus, electronic device, and computer-readable storage medium, which can improve the accuracy of detecting whether a person has fallen.

[0004] In a first aspect, embodiments of this application disclose a fall detection method, including:

[0005] Human detection is performed on the current frame image to obtain at least one first detection box corresponding to the current frame image;

[0006] Human key point detection is performed on each of the first detection boxes to obtain the first key point corresponding to each of the first detection boxes;

[0007] Obtain at least one second detection box corresponding to the previous frame image, and obtain the second key point corresponding to each second detection box;

[0008] The pixels contained in each of the first detection boxes and each of the second detection boxes are clustered to obtain the first cluster center corresponding to each of the first detection boxes and the second cluster center corresponding to each of the second detection boxes.

[0009] Based on the first cluster center corresponding to each of the first detection boxes and the second cluster center corresponding to each of the second detection boxes, the at least one first detection box is matched with the at least one second detection box to determine the second detection box that matches each of the first detection boxes;

[0010] The fall detection model is used to perform fall detection based on the first key point corresponding to each first detection box and the second key point corresponding to the matched second detection box, and the fall detection result is obtained.

[0011] As an optional implementation, in a first aspect of this application, the step of matching at least one first detection box with at least one second detection box based on the first cluster center corresponding to each of the first detection boxes and the second cluster center corresponding to each of the second detection boxes to determine the second detection box matching each of the first detection boxes includes:

[0012] Based on the first cluster center corresponding to the target first detection box and the second cluster center corresponding to each of the second detection boxes, the distance between the target first detection box and each of the second detection boxes is calculated; the target first detection box is any one of the first detection boxes.

[0013] Based on the distances between the target first detection box and each of the second detection boxes, a second detection box matching the target first detection box is determined.

[0014] As an optional implementation, in a first aspect of this application, calculating the distance between the target first detection box and each of the second detection boxes based on the first cluster center corresponding to the target first detection box and the second cluster center corresponding to each of the second detection boxes includes:

[0015] The M first cluster centers corresponding to the first target detection box are matched with the M second cluster centers corresponding to the second target detection box to obtain M sets of center pairs. Each set of center pairs includes the first cluster center of the first target detection box and the second cluster center matched by the second target detection box; the second target detection box is any second detection box; and M is a positive integer.

[0016] The distance between the first target detection box and the second target detection box is calculated based on the distance between the first cluster center and the matching second cluster center contained in each pair of centers.

[0017] As an optional implementation, in a first aspect of the embodiments of this application, the step of matching the M first cluster centers corresponding to the target first detection box with the M second cluster centers corresponding to the target second detection box to obtain M sets of center pairs includes:

[0018] The proportion of the number of pixels contained in the first cluster center of the target to the total number of pixels contained in the first detection box of the target is calculated to obtain the first proportion corresponding to the first cluster center of the target; the first cluster center of the target is any first cluster center corresponding to the first detection box of the target.

[0019] Calculate the ratio of the number of pixels contained in the target second cluster center to the total number of pixels contained in the target second detection box, and obtain the second ratio corresponding to the target second cluster center; the target second cluster center is any second cluster center corresponding to the target second detection box;

[0020] The distance between the first target cluster center and the second target cluster center is obtained based on the first ratio corresponding to the first target cluster center and the second ratio corresponding to the second target cluster center.

[0021] Based on the distance between the target first cluster center and each of the second cluster centers corresponding to the target second detection box, the second cluster center that matches the target first cluster center is determined, thus obtaining the center pair corresponding to the target first cluster center.

[0022] As an optional implementation, in a first aspect of the embodiments of this application, obtaining the distance between the target first cluster center and the target second cluster center based on the first ratio corresponding to the target first cluster center and the second ratio corresponding to the target second cluster center includes:

[0023] Determine the absolute value of the difference between the first proportion corresponding to the target first cluster center and the second proportion corresponding to the target second cluster center;

[0024] Determine the sum of the first proportion corresponding to the target first cluster center and the second proportion corresponding to the target second cluster center;

[0025] Divide the absolute value of the difference by the sum to obtain the first function value;

[0026] Calculate the Euclidean distance between the target first cluster center and the target second cluster center to obtain the second function value;

[0027] The distance between the target first cluster center and the target second cluster center is obtained by weighted summation of the first function value and the second function value.

[0028] As an optional implementation, in a first aspect of the embodiments of this application, before performing fall detection using a fall detection model based on the first key points corresponding to each of the first detection boxes and the second key points corresponding to the matched second detection boxes to obtain the fall detection result, the method further includes:

[0029] Key point preprocessing is performed on each first key point corresponding to the first target detection box and each second key point corresponding to the second detection box matching the first target detection box to obtain the relative offset of each first key point corresponding to the first target detection box; the first target detection box is any of the first detection boxes.

[0030] The fall detection model performs fall detection based on the first key point corresponding to each first detection box and the second key point corresponding to the matched second detection box, and obtains the fall detection result, including:

[0031] An offset sequence is generated based on the relative offsets between each first key point corresponding to the first detection box of the target and the corresponding historical images of multiple frames; the historical images are images acquired before the current frame image.

[0032] The offset sequence is input into the fall detection model, and the fall detection model performs fall detection based on the offset sequence to obtain the fall detection result corresponding to the first detection box of the target.

[0033] As an optional implementation, in a first aspect of this application, the step of performing key point preprocessing on each first key point corresponding to the target first detection box and each second key point corresponding to the second detection box matching the target first detection box to obtain the relative offset of each first key point corresponding to the target first detection box includes:

[0034] Subtract the x-coordinate of the first target key point from the x-coordinate of the second target key point, and divide the resulting x-coordinate difference by the width of the first target detection box to obtain the first offset value of the first target key point; the first target key point is any first key point in the first target detection box; the second target key point is the second key point in the second detection box that matches the first target detection box and corresponds to the first target key point.

[0035] Subtract the ordinate of the first key point of the target from the ordinate of the second key point of the target, and divide the difference in ordinates by the length of the first detection frame of the target to obtain the second offset value of the first key point of the target.

[0036] The relative offset of the first key point of the target is determined based on the first offset value and the second offset value of the first key point of the target.

[0037] Secondly, embodiments of this application disclose a fall detection device, comprising:

[0038] The detection box acquisition module is used to perform human detection on the current frame image and the previous frame image respectively, and obtain at least one first detection box corresponding to the current frame image and at least one second detection box corresponding to the previous frame image.

[0039] The key point acquisition module is used to perform human key point detection on each of the first detection boxes to obtain the first key point corresponding to each of the first detection boxes, and to perform human key point detection on each of the second detection boxes to obtain the second key point corresponding to each of the second detection boxes.

[0040] The detection box matching module is used to cluster the pixels contained in each of the first detection boxes and each of the second detection boxes to obtain a first cluster center corresponding to each of the first detection boxes and a second cluster center corresponding to each of the second detection boxes; and to match the at least one first detection box with the at least one second detection box according to the first cluster center corresponding to each of the first detection boxes and the second cluster center corresponding to each of the second detection boxes to determine the second detection box that matches each of the first detection boxes.

[0041] The fall detection module is used to perform fall detection based on the first key point corresponding to each first detection box and the second key point corresponding to the matched second detection box using a fall detection model, and obtain the fall detection result.

[0042] Thirdly, embodiments of this application disclose an electronic device, including a memory and a processor. The memory stores a computer program, and when the computer program is executed by the processor, the processor causes the processor to implement the method described in any of the above embodiments.

[0043] Fourthly, embodiments of this application disclose a computer-readable storage medium that stores a computer program, which, when executed by a processor, implements the methods described in any of the above embodiments.

[0044] The fall detection method, apparatus, electronic device, and computer-readable storage medium disclosed in this application perform human body detection on the current frame image and the previous frame image respectively to obtain a first detection box corresponding to the current frame image and a second detection box corresponding to the previous frame image; perform human body key point detection on each first detection box and each second detection box respectively to obtain a first key point corresponding to each first detection box and a second key point corresponding to each second detection box; cluster the pixels contained in each first detection box and each second detection box respectively to obtain a first cluster center corresponding to each first detection box and a second cluster center corresponding to each second detection box; match at least one first detection box with at least one second detection box according to the first cluster center corresponding to each first detection box and the second cluster center corresponding to each second detection box to determine the second detection box that matches each first detection box; and perform fall detection using a fall detection model according to the first key point corresponding to each first detection box and the second key point corresponding to the matched second detection box to obtain a fall detection result. In this embodiment, by clustering the pixels contained in each detection box in the image, and matching the first detection box corresponding to the current frame image with the second detection box corresponding to the previous frame image according to the cluster center, the accuracy of tracking detection boxes is improved, and the previous and next frames can be associated. Fall detection is performed using key points in detection boxes with temporal relationships, thereby improving the accuracy of fall detection. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 Here is a flowchart of a fall detection method in one embodiment;

[0047] Figure 2 This is a YOLOv7 network architecture in one embodiment;

[0048] Figure 3 Here is an example of the architecture of ViTPose;

[0049] Figure 4 This is a flowchart of the detection box matching in one embodiment;

[0050] Figure 5 This is a flowchart illustrating the calculation of the detection box distance in one embodiment;

[0051] Figure 6 This is a flowchart of cluster center matching in one embodiment;

[0052] Figure 7 Here is a flowchart of a fall detection method in another embodiment;

[0053] Figure 8 Here is an overall flowchart of the fall detection method in another embodiment;

[0054] Figure 9 This is a block diagram of a fall detection device in one embodiment;

[0055] Figure 10 This is a structural block diagram of an electronic device in one embodiment. Detailed Implementation

[0056] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0057] It should be noted that the terms "comprising" and "having," and any variations thereof, in the embodiments and accompanying drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the steps or units listed, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0058] It is understood that the terms "first," "second," etc., used in this application may be used herein to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish one element from another. For example, without departing from the scope of this application, a first detection box may be referred to as a second detection box, and similarly, a second detection box may be referred to as a first detection box.

[0059] The first and second detection boxes are both detection boxes, but they are not the same detection boxes.

[0060] In related technologies, most commonly used vision-based fall detection methods rely on single-frame images, meaning that a fall is determined as soon as a person is detected close to the ground in the image. While this method is relatively simple, it cannot distinguish between a person intentionally touching the ground (such as an electrician lying down to check equipment) and an accidental fall, resulting in an inherent deficiency in false detection rate. Furthermore, current frame-by-frame tracking and association schemes based on the spatial features of the detection box are easily affected by factors such as shooting angle and target occlusion, leading to inaccurate fall detection.

[0061] This application discloses a fall detection method, apparatus, electronic device, and computer-readable storage medium, which improves the accuracy of detecting whether a person has fallen.

[0062] like Figure 1 As shown, in one embodiment, a fall detection method is provided, which can be applied to electronic devices, including but not limited to mobile phones, smart wearable devices, tablets, PCs (Personal Computers), computers, etc., and can be used in places where people frequently appear, such as parks, corridors, senior citizen activity communities, nursing homes, squares, and parks. The method may include the following steps:

[0063] Step 110: Perform human detection on the current frame image to obtain at least one first detection box corresponding to the current frame image.

[0064] Electronic devices can perform human detection on each frame of images captured by a camera, and obtain the detection boxes contained in each frame of images. One human body in the image can correspond to one detection box.

[0065] Human detection refers to detecting all human bodies within an image and marking the position of each body within the image to obtain human outlines or bounding boxes containing human bodies. The first detection box refers to the bounding box containing human bodies obtained after human detection of the current frame image.

[0066] In some embodiments, human detection can be performed on each frame of image captured by the camera using various human detection methods. When performing human detection on the current frame, the result of human detection on the previous frame is combined to obtain at least one first detection box corresponding to the current frame. Human detection methods may include, but are not limited to, motion detection-based algorithms (including Gaussian mixture models, PBAS (Pixel-Based Adaptive Segmenter) algorithm, frame differencing algorithm, etc.), machine learning-based methods (artificial feature + classifier method), and deep learning-based methods (including Faster-RCNN (Faster Regions with CNN features) framework, SSD (Single Shot MultiBox Detector) framework, FPN (Feature Pyramid Network) framework, YOLO (you only look once) framework, etc.).

[0067] In some specific embodiments, the YOLOv7 network architecture can be used to perform human detection on each frame of the image captured by the camera, obtaining at least one first detection box corresponding to each current frame. The YOLOv7 network architecture is a typical deep learning-based human detection method, belonging to the seventh generation of the YOLO series network architecture, such as... Figure 2 As shown, the basic components of the YOLOv7 network architecture include an input module, a backbone module, and a head module. The input module is used to acquire the input image, primarily by obtaining feature maps through the backbone module, and then the head module is used for training and inference to obtain bounding boxes containing the human body.

[0068] Step 120: Perform human body key point detection on each first detection box to obtain the first key point corresponding to each first detection box.

[0069] After obtaining the detection boxes contained in each frame of the image, human key point detection can be performed on each detection box to obtain the key points corresponding to each detection box.

[0070] Human keypoint detection, also known as human pose estimation, refers to identifying human pose in an image by labeling key parts of the human body with nodes as keypoints. Typically, keypoint tracking is performed after keypoint detection to capture and analyze human pose and movement. Therefore, the first keypoint refers to the keypoints corresponding to the human body within the first detection box in the current frame image.

[0071] In some embodiments, human keypoint detection algorithms can be used to detect human keypoints in each first detection box to obtain the first keypoints corresponding to each first detection box. Human keypoint detection algorithms are mainly divided into 2D keypoint detection and 3D keypoint detection, which can obtain human keypoints from both planar and three-dimensional perspectives. Among them, 2D keypoint detection includes, but is not limited to, CPM (Convolutional Pose Machines) and MSPN (Rethinking on Multi-Stage Networks for Human Pose Estimation).

[0072] In some specific embodiments, the ViTPose architecture can be used to detect human keypoints in bounding boxes in an image, obtaining the keypoints corresponding to the bounding boxes. The ViTPose architecture uses a Transformer structure for 2D human keypoint estimation, such as... Figure 3 As shown, the basic architecture of ViTPose includes an Encoder module and a Decoder module. Generally, a regular ViT (Vision Transformer) structure is used as the Encoder, which is responsible for extracting features from the detection box. Combined with a lightweight Decoder, which mainly consists of upsampling convolutional layers, it is used to regress all key points of the human body based on the features provided by the Encoder.

[0073] Step 130: Obtain at least one second detection box corresponding to the previous frame image, and obtain the second key points corresponding to each second detection box.

[0074] The second detection box refers to all the bounding boxes containing the human body obtained from the previous frame image after human detection. The second keypoint refers to the keypoints corresponding to the human body within the second detection box obtained by performing human keypoint detection on the second detection box in the previous frame image.

[0075] In some embodiments, the current frame image is one frame of the image captured by the camera. When analyzing the next frame image, the next frame image is taken as the new current image, and the original current frame image becomes the previous frame image corresponding to the new current image; the process is performed frame by frame.

[0076] In a specific embodiment, if the camera captures n frames of images, human detection is performed on each of the captured n frames. Assuming the current frame is the m-th frame, the detection box corresponding to the m-th frame is the first detection box, and the detection box corresponding to the (m-1)-th frame is the second detection box, where m is less than n, and both m and n are positive integers; if the current frame is the (m+1)-th frame, the detection box corresponding to the (m+1)-th frame is the first detection box, and the detection box corresponding to the m-th frame is the second detection box; and so on.

[0077] Step 140: Cluster the pixels contained in each first detection box and each second detection box respectively to obtain the first cluster center corresponding to each first detection box and the second cluster center corresponding to each second detection box.

[0078] A pixel is an image element, the smallest basic unit in an image. Each pixel in an image has a specific location and numerical value, which can be used to represent information such as the pixel's color or brightness. Generally, the higher the image resolution, the more pixels the image contains, and the clearer and more detailed the image. Therefore, pixels can also be used as features to describe an image.

[0079] Clustering is a typical form of unsupervised learning that divides a dataset into different classes or clusters according to a specific criterion (such as distance), maximizing the similarity of data objects within the same cluster while maximizing the dissimilarity of data objects in different clusters. Commonly used clustering algorithms can be mainly divided into partition-based methods, density-based methods, and hierarchical methods.

[0080] In some embodiments, pixels within each first detection box and each second detection box can be partitioned and clustered separately, and the first cluster center corresponding to each first detection box and the second cluster center corresponding to each second detection box can be calculated. The coordinates of each cluster center are represented by the RGB (red, green, and blue) values ​​of the pixel. The first cluster center refers to the coordinates of the center point of each cluster obtained by clustering pixels within the first detection box; the second cluster center refers to the coordinates of the center point of each cluster obtained by clustering pixels within the second detection box; and the number of first cluster centers and the number of second cluster centers are not necessarily equal.

[0081] In some specific embodiments, the kmeans clustering method can be used to cluster the pixels contained in each first detection box and each second detection box, respectively, to obtain M first cluster centers corresponding to each first detection box and M second cluster centers corresponding to each second detection box, where M is a positive integer. The kmeans clustering method is a partitioning clustering method, which requires specifying the number of clusters or cluster centers in advance, and iterating repeatedly until the goal of "points within a cluster are close enough and points between clusters are far enough apart" is finally achieved.

[0082] Specifically, M pixels are randomly set as the original centroids in each detection box. When the cluster assignment result of any pixel changes, the process is iterated: for each pixel in each detection box, the distance between each centroid and each pixel is calculated, each pixel is assigned to the cluster with the smallest distance, and the mean value of the pixels contained in each cluster is calculated. The mean value is used as the centroid for the next iteration until the cluster assignment result of all pixels no longer changes. At this point, the centroid is the cluster center, and M clusters of candidate boxes are obtained.

[0083] Step 150: Based on the first cluster center corresponding to each first detection box and the second cluster center corresponding to each second detection box, match at least one first detection box with at least one second detection box to determine the second detection box that matches each first detection box.

[0084] In some embodiments, the first cluster center corresponding to each first detection box can be used as the feature point of each first detection box, and the second cluster center corresponding to each second detection box can be used as the feature point of each second detection box. Based on feature point matching, at least one first detection box is matched with at least one second detection box to determine the second detection box that matches each first detection box.

[0085] In some embodiments, a brute-force matching method can be used to match at least one first detection box with at least one second detection box. The brute-force matching method calculates the distance between a given feature point and all other feature points, sorts the obtained distances, and selects the closest one as the matching point.

[0086] Step 160: Using the fall detection model, fall detection is performed based on the first key points corresponding to each first detection box and the second key points corresponding to the matched second detection boxes to obtain the fall detection results.

[0087] Fall detection models can be used to predict whether a fall will occur. Fall detection models can employ YOLO network architectures, ALBERT model architectures, BERT (Bidirectional Encoder Representations from Transformers) model architectures, etc.

[0088] In some embodiments, the first keypoints corresponding to each first detection box and the second keypoints corresponding to the matched second detection boxes can be input into the ALBERT (A Lite Bidirectional Encoder Representations from Transformers) model for fall detection to obtain the fall detection result. The ALBERT model is generally used in natural language processing. In some embodiments, each first keypoint corresponding to each first detection box in the current frame image can be understood as a "word" in a sentence, and each first keypoint corresponding to each first detection box in the current frame image and each second keypoint corresponding to the matched second detection box in the previous frame image form a temporal relationship. Connecting each first keypoint and each second keypoint with a temporal relationship forms a "sentence," and the "sentence" is then processed by the ALBERT model for fall detection prediction to obtain the fall detection result.

[0089] In some embodiments, the fall detection model needs to be trained with keypoint sequences before predicting falls. The sample fall detection dataset may contain multiple frames of images, corresponding bounding boxes and keypoints, and actual fall detection results for each frame. It may also include the relative offsets of each keypoint for each frame. The sample fall detection dataset is preprocessed and then input into the pre-trained fall detection model to obtain predicted fall detection results. The accuracy is calculated based on the predicted and actual fall detection results. If the accuracy is less than a set threshold, the loss function of the pre-trained fall detection model is calculated. The model is then fine-tuned based on the gradient descent direction of the loss function, and the process is repeated until the accuracy reaches the set threshold. The sample fall detection dataset can be obtained through FDD (the fall detection dataset), URFD (the UR fall detection dataset), etc.

[0090] In this embodiment, by clustering the pixels contained in each detection box in the image, and matching the first detection box corresponding to the current frame image with the second detection box corresponding to the previous frame image according to the cluster center, the accuracy of tracking detection boxes is improved, and the previous and next frames can be associated. Fall detection is performed using key points in detection boxes with temporal relationships, thereby improving the accuracy of fall detection.

[0091] like Figure 4 As shown, in one embodiment, the step of matching at least one first detection box with at least one second detection box based on the first cluster center corresponding to each first detection box and the second cluster center corresponding to each second detection box may include the following steps:

[0092] Step 410: Calculate the distance between the target first detection box and each of the second detection boxes based on the first cluster center corresponding to the target first detection box and the second cluster center corresponding to each of the second detection boxes.

[0093] The target first detection box is any first detection box in the current frame image.

[0094] In some embodiments, calculating the distance between the target first detection box and each of the second detection boxes is essentially calculating the distance between two clusters that each contain M cluster centers. Therefore, the distances between the M first cluster centers corresponding to the target first detection box and the M second cluster centers corresponding to each of the second detection boxes can be calculated to obtain the distances between the target first detection box and each of the second detection boxes.

[0095] like Figure 5 As shown, in one embodiment, step 410 calculates the distance between the target first detection box and each of the second detection boxes, including steps 502 to 504.

[0096] Step 502: Match the M first cluster centers corresponding to the first target detection box with the M second cluster centers corresponding to the second target detection box to obtain M sets of center pairs.

[0097] Each set of centers includes the first cluster center of the first target detection box and the second cluster center that matches the second target detection box; the second target detection box is any second detection box; M is a positive integer.

[0098] Calculating the distance between the first target detection box and the second target detection box essentially involves calculating the distance between two clusters, each containing M cluster centers. However, the distance between two clusters is not easily calculated directly. In some embodiments, the M first cluster centers corresponding to the first target detection box and the M second cluster centers corresponding to the second target detection box can be paired to form center pairs. These paired center pairs assist in determining the distance between the two clusters, thus achieving the matching of the first and second target detection boxes.

[0099] In some embodiments, such as Figure 6 As shown, the step of matching the M first cluster centers corresponding to the first target detection box with the M second cluster centers corresponding to the second target detection box to obtain M sets of center pairs may include steps 610 to 620.

[0100] Step 610: Calculate the distance between the first target cluster center and the second target cluster center based on the first target cluster center and the second target cluster center.

[0101] Wherein, the first cluster center of the target is any first cluster center corresponding to the first detection box of the target; the second cluster center of the target is any second cluster center corresponding to the first detection box of the target.

[0102] In order to match the M first cluster centers corresponding to the first target detection box with the M second cluster centers corresponding to the second target detection box to form a center pair, it is necessary to first calculate the distance from each first cluster center to each second cluster center, and then match the target first cluster center with each target second cluster center according to the distance to form a center pair.

[0103] In some embodiments, the proportion of the number of pixels contained in the first target cluster center to the total number of pixels contained in the first target detection box can be calculated to obtain a first proportion corresponding to the first target cluster center; the first target cluster center can be any first cluster center corresponding to the first target detection box; the proportion of the number of pixels contained in the second target cluster center to the total number of pixels contained in the second target detection box can be calculated to obtain a second proportion corresponding to the second target cluster center; the second target cluster center can be any second cluster center corresponding to the second target detection box; the distance between the first target cluster center and the second target cluster center can be obtained based on the first proportion corresponding to the first target cluster center and the second proportion corresponding to the second target cluster center.

[0104] In some embodiments, the distance between the target first cluster center and the target second cluster center can be calculated using a distance calculation formula based on a first ratio corresponding to the target first cluster center and a second ratio corresponding to the target second cluster center. The distance calculation formula indicates that if two cluster centers are sufficiently similar, then the proportion of pixels they each contain and their coordinates in the RGB coordinate system should be relatively similar. In other words, if two cluster centers each contain a similar number of pixels and their coordinates in the RGB coordinate system are similar, then these two cluster centers can be matched.

[0105] In some specific embodiments, the absolute value of the difference between the first proportion corresponding to the target first cluster center and the second proportion corresponding to the target second cluster center can be determined; the sum of the first proportion corresponding to the target first cluster center and the second proportion corresponding to the target second cluster center can be determined; the absolute value of the difference can be divided by the sum to obtain the first function value; the Euclidean distance from the target first cluster center to the target second cluster center can be calculated to obtain the second function value; and the first function value and the second function value can be weighted and summed to obtain the distance between the target first cluster center and the target second cluster center.

[0106] Specifically, the distance between the target first cluster center and the target second cluster center can be calculated using the distance calculation formula (1):

[0107]

[0108] Among them, Dis c (c pi ,c bj ) represents the distance between the first cluster center of the i-th target corresponding to the first detection box of target p in the current frame and the second cluster center of the j-th target corresponding to the second detection box of target b in the previous frame; c pi r represents the first cluster center of the i-th target in the cluster obtained by k-means clustering of the first detection box of target p in the current frame. piThis represents the first proportion corresponding to the first cluster center of the i-th target; similarly, c bj r represents the j-th second cluster center of the target's second detection bounding box in the previous frame, obtained by k-means clustering. bj Represents the second proportion corresponding to the second cluster center of the j-th target; ||c pi -c bj ||2 represents the Euclidean distance between the coordinates of the first cluster center of the i-th target and the second cluster center of the j-th target.

[0109] Step 620: Based on the distance between the target first cluster center and each second cluster center corresponding to the target second detection box, determine the second cluster center that matches the target first cluster center, and obtain the center pair corresponding to the target first cluster center.

[0110] In some embodiments, the target first cluster center and the target second cluster center can be matched using a brute-force matching method based on the distance between the target first cluster center and each of the second cluster centers corresponding to the target second detection box. Alternatively, a distance threshold can be set to retain the connections between the target first cluster center and the target second cluster center whose distance to each of the second cluster centers corresponding to the target second detection box is less than the distance threshold. Then, the Hungarian algorithm is used to match the target first cluster center with each of the second cluster centers corresponding to the target second detection box to determine the second cluster centers that match each of the first cluster centers, forming center pairs.

[0111] Step 504: Calculate the distance between the first target detection box and the second target detection box based on the distance between the first cluster center and the matching second cluster center contained in each group of center pairs.

[0112] In some embodiments, the distance between the first cluster center and the matching second cluster center included in the M group center pairs corresponding to the first target detection box can be summed to obtain the distance between the first target detection box and the second target detection box.

[0113] Specifically, the distance between the first detection box and the second detection box of the target can be calculated using formula (2):

[0114] Dis b (box1,box2)=∑Dis c (c 1i ,c 2j (2)

[0115] Among them, Dis b(box1, box2) represents the distance from the first target detection box in the current frame to the second target detection box in the previous frame; box1 represents the first target detection box in the current frame; box2 represents the second target detection box in the previous frame; Dis c (c 1i ,c 2j ) represents the distance from the i-th cluster center of the first target detection box to the j-th cluster center of the second target detection box; c 1i c represents the first cluster center of the i-th target in the first detection box; 2j This represents the j-th target second cluster center of the second detection box; and c 2j For the first cluster center c 1i The second matching cluster center.

[0116] Step 420: Determine the second detection box that matches the first detection box based on the distances between the first detection box and each second detection box.

[0117] After determining the distances between the target first detection box and each corresponding second detection box, the second detection boxes whose distances are less than a distance threshold can be selected, and the connection relationships between the corresponding second detection boxes whose distances are less than the distance threshold and the target first detection box can be retained. For each first detection box in the current frame image, the corresponding retained connection relationship can be determined according to the distance threshold.

[0118] In some embodiments, a second detection box matching each first detection box can be determined based on the Hungarian algorithm and the preserved connection relationships corresponding to each first detection box.

[0119] In some specific embodiments, based on the reserved connection relationships corresponding to each first detection box, each first detection box selects a matching second detection box in ascending order of distance values; if the current first detection box matches the first second detection box with a connection relationship, and this second detection box has no matching relationship with any previous first detection boxes, then this second detection box is the second detection box that matches the current first detection box; if the second detection box already has a matching relationship with a previous first detection box, then the current first detection box forcibly matches the second detection box, and the first detection box that previously matched the second detection box abandons its matching relationship with the second detection box and instead matches the next second detection box with a connection relationship, until all the reserved connection relationships between the previous first detection box and the first detection box cannot be matched, then the matching operation of the previous first detection box is abandoned.

[0120] Specifically, assume there are four first detection boxes a, b, c, and d, and four second detection boxes 1, 2, 3, and 4. The connection relationships retained for the first detection boxes are: (a, 1) with a connection distance of 5, (a, 2) with a connection distance of 8, (a, 4) with a connection distance of 10, (b, 2) with a connection distance of 6, (b, 3) with a connection distance of 7, (c, 2) with a connection distance of 7, and (d, 1) with a connection distance of 6. The matching process is as follows: a matches with 1 first; b matches with 2; c matches with 2. At this point, c conflicts with b, and c completes the matching after b, so c is retained for matching with 2, and b chooses to match with 3; d matches with 1. At this point, d conflicts with a, and d completes the matching after a, so d is retained for matching with 1, and a chooses to match with 2. C still conflicts with a, and a completes the matching after c, so a is retained for matching with 2, and c has no other connection relationships, so matching is abandoned. Ultimately, the second detection box 2 was found to be a successful match for the first detection box a, the second detection box 3 was a successful match for the first detection box b, and the second detection box 1 was a successful match for the first detection box d.

[0121] The Hungarian algorithm is a combinatorial optimization algorithm that solves the task assignment problem in polynomial time and has driven the development of subsequent primal-dual methods. The Hungarian algorithm requires high accuracy in redline connections, meaning that components such as motion and appearance models must make relatively accurate predictions, or a high threshold must be preset to only send edges with high confidence to the Hungarian algorithm for matching, thus achieving good results. Therefore, before performing Hungarian algorithm matching, a distance threshold is set to retain only the connections between the first and second bounding boxes of targets with distances less than the threshold, thereby improving matching accuracy.

[0122] In this embodiment, the target first cluster center and each second cluster center are matched by the distance between the target first cluster center and each second cluster center to generate center pairs. Then, based on the distance between the target first cluster center and the matched second cluster center in each center pair, the distance from the target first detection box to each second detection box is obtained. The matching relationship between the first detection box corresponding to the current frame image and the second detection box corresponding to the previous frame image is determined, making the tracking matching relationship between the first detection box corresponding to the current frame image and the second detection box corresponding to the previous frame image more accurate, improving the accuracy of tracking detection boxes. Furthermore, it can associate previous and subsequent frames and use key points within detection boxes with temporal relationships for fall detection, thereby improving the accuracy of fall detection.

[0123] like Figure 7 As shown, in another embodiment, a fall detection method is provided, which can be applied to the above-mentioned electronic device. The method may include the following steps:

[0124] Step 702: Perform human detection on the current frame image to obtain at least one first detection box corresponding to the current frame image;

[0125] Step 704: Perform human key point detection on each of the first detection boxes to obtain the first key point corresponding to each of the first detection boxes;

[0126] Step 706: Obtain at least one second detection box corresponding to the previous frame image, and obtain the second key point corresponding to each second detection box.

[0127] Step 708: Cluster the pixels contained in each first detection box and each second detection box respectively to obtain the first cluster center corresponding to each first detection box and the second cluster center corresponding to each second detection box.

[0128] Step 710: Based on the first cluster center corresponding to each first detection box and the second cluster center corresponding to each second detection box, match at least one first detection box with at least one second detection box to determine the second detection box that matches each first detection box.

[0129] The descriptions of steps 702 to 710 can be found in the descriptions of steps 110 to 150 in the above embodiments, and will not be repeated here.

[0130] Step 712: Perform key point preprocessing on each first key point corresponding to the first detection box of the target and each second key point corresponding to the second detection box that matches the first detection box of the target to obtain the relative offset of each first key point corresponding to the first detection box of the target; the first detection box of the target can be any first detection box.

[0131] The target first detection box and the second detection box that matches the target first detection box are matched. Therefore, there is a one-to-one correspondence between each first key point corresponding to the target first detection box and each second key point corresponding to the second detection box that matches the target first detection box. This can represent the actions of the same human body in the current frame and the previous frame. By calculating the relative offset of each first key point corresponding to the target first detection box, the changes in the human body's actions in the current frame and the previous frame can be analyzed, which assists the fall detection model in performing fall detection analysis.

[0132] In some specific embodiments, the abscissa of the first target key point is subtracted from the abscissa of the second target key point, and the difference in abscissas is divided by the width of the first target detection box to obtain the first offset value of the first target key point; the first target key point is any first key point in the first target detection box; the second target key point is the second key point in the second detection box that matches the first target detection box and corresponds to the first target key point; the ordinate of the first target key point is subtracted from the ordinate of the second target key point, and the difference in abscissas is divided by the length of the first target detection box to obtain the second offset value of the first target key point; the relative offset of the first target key point is determined based on the first offset value and the second offset value of the first target key point.

[0133] Specifically, the relative offset of the first key point of the target can be calculated using formula (3):

[0134]

[0135] Among them, new_x i This represents the first offset value of the first keypoint of the i-th target in the first detection box; new_y i This represents the second offset value of the first keypoint of the i-th target in the first detection frame; The x-coordinate of the first key point of the target; w represents the x-coordinate of the second key point of the target. p This indicates the width of the first detection bounding box of the target; The vertical coordinate represents the first key point of the target; h represents the ordinate of the second key point of the target. p This indicates the length of the first detection box of the target.

[0136] Step 714: Generate an offset sequence based on the relative offsets of each first key point corresponding to the first detection box of the target and the corresponding historical images of multiple frames.

[0137] Historical images are those captured before the current frame image.

[0138] In some embodiments, the relative offsets corresponding to the current frame image and each frame image preceding the current frame image are calculated respectively, and an offset sequence is generated based on the relative offsets corresponding to the current frame image and each frame image preceding the current frame image.

[0139] In some specific embodiments, assuming the current frame image is the nth frame image, the relative offset of the current frame image can be calculated by combining it with the previous frame image; the relative offset of the nth frame image is calculated based on the nth frame image and the (n-1)th frame image; the relative offset of the (n-1)th frame image is calculated based on the (n-1)th frame image and the (n-2)th frame image; and so on until the relative offset of the 1st frame image is obtained; where n is a positive integer, and the previous frame image of the 1st frame image is the origin coordinate; the relative offsets corresponding to the 1st to nth frames image are used to generate an offset sequence.

[0140] In order to be processed by the fall detection model, the relative offsets of each first keypoint corresponding to the first detection box of the target need to be generated into an offset sequence. The offset sequence refers to the sequence formed by arranging the relative offsets of each first keypoint corresponding to the first detection box of the target in multiple frames as sequence elements in the temporal order of the multiple frames. This sequence can be recognized by the fall detection model and used for fall detection.

[0141] Step 716: Input the offset sequence into the fall detection model, and use the fall detection model to perform fall detection based on the offset sequence to obtain the fall detection result corresponding to the first detection box of the target.

[0142] The description of step 716 can be referred to the relevant description of step 420 in the above embodiments, and will not be repeated here.

[0143] In a specific embodiment, such as Figure 8 As shown, the fall detection process is as follows:

[0144] First, human detection is performed on both the current frame and the previous frame, resulting in a first detection box for the current frame and a second detection box for the previous frame. Then, the pixels of each first and second detection box are clustered, and detection box matching and tracking are performed based on the cluster centers to obtain the second detection box from the previous frame that matches each first detection box in the current frame. Simultaneously, human keypoint detection is performed on each first and second detection box, resulting in a first keypoint for each first detection box and a second keypoint for each second detection box. Next, relative offsets are generated for each first and second keypoint based on the matching and tracking results. Finally, the relative offsets of all keypoints across all frames are fed into a fall detection model to obtain the fall detection results.

[0145] In this embodiment, by clustering the pixels contained in each detection box in the image, and matching the first detection box corresponding to the current frame image with the second detection box corresponding to the previous frame image according to the cluster center, the accuracy of tracking detection boxes is improved, and the relative offset of key points in the detection boxes with temporal relationship can be associated with previous and subsequent frames. Fall detection is performed by using the relative offset of key points in the detection boxes with temporal relationship, thereby improving the accuracy of fall detection.

[0146] like Figure 9 As shown, in one embodiment, a fall detection device 900 is provided, which can be applied to the above-described electronic device. The fall detection device 900 may include a detection frame acquisition module 910, a key point acquisition module 920, a detection frame matching module 930, and a fall detection module 940.

[0147] The detection box acquisition module 910 is used to perform human detection on the current frame image, obtain at least one first detection box corresponding to the current frame image, and obtain at least one second detection box corresponding to the previous frame image.

[0148] The key point acquisition module 920 is used to perform human key point detection on each first detection box, obtain the first key point corresponding to each first detection box, and obtain the second key point corresponding to each second detection box.

[0149] The detection box matching module 930 is used to cluster the pixels contained in each first detection box and each second detection box respectively to obtain the first cluster center corresponding to each first detection box and the second cluster center corresponding to each second detection box; based on the first cluster center corresponding to each first detection box and the second cluster center corresponding to each second detection box, at least one first detection box is matched with at least one second detection box to determine the second detection box that matches each first detection box.

[0150] The fall detection module 940 is used to perform fall detection based on the first key point corresponding to each first detection box and the second key point corresponding to the matched second detection box using a fall detection model, and obtain the fall detection result.

[0151] In some embodiments, the detection box matching module 930 includes a detection box distance calculation unit and a matching determination unit.

[0152] The detection box distance calculation unit is used to calculate the distance between the target first detection box and each second detection box based on the first cluster center corresponding to the target first detection box and the second cluster center corresponding to each second detection box; the target first detection box is any first detection box.

[0153] The matching determination unit is used to determine the second detection box that matches the first detection box of the target based on the distances between the first detection box of the target and each of the second detection boxes.

[0154] The detection box distance calculation unit is also used to match the M first cluster centers corresponding to the target first detection box with the M second cluster centers corresponding to the target second detection box to obtain M sets of center pairs. Each set of center pairs includes the first cluster center of the target first detection box and the second cluster center matched by the target second detection box; the target second detection box is any second detection box; M is a positive integer; and the distance between the target first detection box and the target second detection box is calculated based on the distance between the first cluster center and the matched second cluster center contained in each set of center pairs.

[0155] In some embodiments, the detection box distance calculation unit is further configured to calculate the ratio of the number of pixels contained in the target first cluster center to the total number of pixels contained in the target first detection box, to obtain a first ratio corresponding to the target first cluster center; the target first cluster center is any first cluster center corresponding to the target first detection box; calculate the ratio of the number of pixels contained in the target second cluster center to the total number of pixels contained in the target second detection box, to obtain a second ratio corresponding to the target second cluster center; the target second cluster center is any second cluster center corresponding to the target second detection box; based on the first ratio corresponding to the target first cluster center and the second ratio corresponding to the target second cluster center, obtain the distance between the target first cluster center and the target second cluster center; based on the distance between the target first cluster center and each of the second cluster centers corresponding to the target second detection box, determine the second cluster center that matches the target first cluster center, to obtain the center pair corresponding to the target first cluster center.

[0156] As an optional implementation, the detection box distance calculation unit is further configured to: determine the absolute value of the difference between the first proportion corresponding to the first target cluster center and the second proportion corresponding to the second target cluster center; determine the sum of the first proportion corresponding to the first target cluster center and the second proportion corresponding to the second target cluster center; divide the absolute value of the difference by the sum to obtain a first function value; calculate the Euclidean distance from the first target cluster center to the second target cluster center to obtain a second function value; and perform a weighted summation of the first function value and the second function value to obtain the distance between the first target cluster center and the second target cluster center.

[0157] As an optional implementation, the fall detection device 900 also includes an offset calculation module and a sequence generation module.

[0158] The offset calculation module is used to perform key point preprocessing on each first key point corresponding to the first detection box of the target and each second key point corresponding to the second detection box that matches the first detection box of the target, to obtain the relative offset of each first key point corresponding to the first detection box of the target; the first detection box of the target can be any first detection box.

[0159] The sequence generation module is used to generate an offset sequence based on the relative offsets of each first key point corresponding to the first detection box of the target and the corresponding historical images of multiple frames; the historical images are images acquired before the current frame image.

[0160] The fall detection module 940 is also used to input the offset sequence into the fall detection model, and the fall detection model performs fall detection based on the offset sequence to obtain the fall detection result corresponding to the first detection box of the target.

[0161] The offset calculation module is also used to subtract the abscissa of the first target key point from the abscissa of the second target key point, and divide the difference in abscissas by the width of the first target detection box to obtain the first offset value of the first target key point; the first target key point is any first key point in the first target detection box; the second target key point is the second key point in the second detection box that matches the first target detection box and corresponds to the first target key point; subtract the ordinate of the first target key point from the ordinate of the second target key point, and divide the difference in ordinates by the length of the first target detection box to obtain the second offset value of the first target key point; and determine the relative offset of the first target key point based on the first offset value and the second offset value.

[0162] In this embodiment, by clustering the pixels contained in each detection box in the image, and matching the first detection box corresponding to the current frame image with the second detection box corresponding to the previous frame image according to the cluster center, the accuracy of tracking detection boxes is improved, and the relative offset of key points in the detection boxes with temporal relationship can be associated with previous and subsequent frames. Fall detection is performed by using the relative offset of key points in the detection boxes with temporal relationship, thereby improving the accuracy of fall detection.

[0163] Figure 10 This is a structural block diagram of an electronic device in one embodiment. The electronic device can be a mobile phone, tablet computer, smart wearable device, etc. Figure 10 As shown, the electronic device 1000 may include one or more of the following components: a processor 1010 and a memory 1020 coupled to the processor 1010, wherein the memory 1020 may store one or more computer programs, which may be configured to implement the methods described in the above embodiments when executed by one or more processors 1010.

[0164] The processor 1010 may include one or more processing cores. The processor 1010 connects to various parts within the electronic device 1000 using various interfaces and lines, and performs various functions and processes data of the electronic device 1000 by running or executing instructions, programs, code sets, or instruction sets stored in the memory 1020, and by calling data stored in the memory 1020. Optionally, the processor 1010 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 1010 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 1010 and may be implemented separately using a communication chip.

[0165] The memory 1020 may include random access memory (RAM) or read-only memory (ROM). The memory 1020 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 1020 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as touch functionality, sound playback functionality, image playback functionality, etc.), and instructions for implementing the various method embodiments described above. The data storage area may also store data created by the electronic device 1000 during use.

[0166] Understandably, the electronic device 1000 may include more or fewer structural elements than those shown in the above block diagram, such as power supply, input buttons, camera, speaker, screen, RF (Radio Frequency) circuit, Wi-Fi (Wireless Fidelity) module, Bluetooth module, sensor, etc., and may not be limited herein.

[0167] This application discloses a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the methods described in the above embodiments.

[0168] This application discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program can be executed by a processor to implement the methods described in the above embodiments.

[0169] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), etc.

[0170] Any references to memory, storage, databases, or other media used herein may include non-volatile and / or volatile memory. Suitable non-volatile memory may include ROM, Programmable ROM (PROM), Erasable PROM (EPROM), Electrically Erasable PROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which is used as an external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as Static RAM (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), Rambus DRAM (RDRAM), and Direct Rambus DRAM (DRDRAM).

[0171] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Those skilled in the art should also recognize that the embodiments described in the specification are optional embodiments, and the actions and modules involved are not necessarily essential to this application.

[0172] In the various embodiments of this application, it should be understood that the sequence number of each process does not necessarily imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0173] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0174] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they can be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0175] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0176] If the aforementioned integrated units are implemented as software functional units and sold or used as independent products, they can be stored in a computer-accessible memory. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several requests to cause a computer device (which can be a personal computer, server, or network device, specifically a processor in the computer device) to execute some or all of the steps of the methods described in the various embodiments of this application.

[0177] The foregoing has provided a detailed description of a fall detection method, apparatus, electronic device, and computer-readable storage medium disclosed in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A fall detection method, characterized in that, The method includes: Human detection is performed on the current frame image to obtain at least one first detection box corresponding to the current frame image; Human key point detection is performed on each of the first detection boxes to obtain the first key point corresponding to each of the first detection boxes; Obtain at least one second detection box corresponding to the previous frame image, and obtain the second key point corresponding to each second detection box; The pixels contained in each of the first detection boxes and each of the second detection boxes are clustered to obtain the first cluster center corresponding to each of the first detection boxes and the second cluster center corresponding to each of the second detection boxes. The proportion of the number of pixels contained in the first cluster center of the target to the total number of pixels contained in the first detection box of the target is calculated to obtain the first proportion corresponding to the first cluster center of the target; the first cluster center of the target is any first cluster center corresponding to the first detection box of the target; the first detection box of the target is any first detection box of the target. The proportion of pixels contained in the target second cluster center to the total number of pixels contained in the target second detection box is calculated to obtain the second proportion corresponding to the target second cluster center; the target second cluster center is any second cluster center corresponding to the target second detection box; the target second detection box is any second detection box; The distance between the first target cluster center and the second target cluster center is obtained based on the first ratio corresponding to the first target cluster center and the second ratio corresponding to the second target cluster center. Based on the distance between the target first cluster center and each of the second cluster centers corresponding to the target second detection box, a second cluster center matching the target first cluster center is determined, resulting in a center pair corresponding to the target first cluster center; each center pair includes the first cluster center of the target first detection box and the second cluster center matching the target second detection box; The distance between the first target detection box and the second target detection box is calculated based on the distance between the first cluster center and the matching second cluster center contained in each pair of centers; Based on the distances between the target first detection box and each of the second detection boxes, a second detection box matching the target first detection box is determined; The fall detection model is used to perform fall detection based on the first key point corresponding to each first detection box and the second key point corresponding to the matched second detection box, and the fall detection result is obtained.

2. The method according to claim 1, characterized in that, The step of obtaining the distance between the target first cluster center and the target second cluster center based on the first ratio corresponding to the target first cluster center and the second ratio corresponding to the target second cluster center includes: Determine the absolute value of the difference between the first proportion corresponding to the target first cluster center and the second proportion corresponding to the target second cluster center; Determine the sum of the first proportion corresponding to the target first cluster center and the second proportion corresponding to the target second cluster center; Divide the absolute value of the difference by the sum to obtain the first function value; Calculate the Euclidean distance between the target first cluster center and the target second cluster center to obtain the second function value; The distance between the target first cluster center and the target second cluster center is obtained by weighted summation of the first function value and the second function value.

3. The method according to claim 1, characterized in that, Before obtaining the fall detection result by performing fall detection using the fall detection model based on the first key points corresponding to each of the first detection boxes and the second key points corresponding to the matched second detection boxes, the method further includes: Key point preprocessing is performed on each first key point corresponding to the first detection box of the target, and on each second key point corresponding to the second detection box that matches the first detection box of the target, to obtain the relative offset of each first key point corresponding to the first detection box of the target; the first detection box of the target is any of the first detection boxes. The fall detection model performs fall detection based on the first key point corresponding to each first detection box and the second key point corresponding to the matched second detection box, and obtains the fall detection result, including: An offset sequence is generated based on the relative offsets between each first key point corresponding to the first detection box of the target and the corresponding historical images of multiple frames; the historical images are images acquired before the current frame image. The offset sequence is input into the fall detection model, and the fall detection model performs fall detection based on the offset sequence to obtain the fall detection result corresponding to the first detection box of the target.

4. The method according to claim 3, characterized in that, The step of performing keypoint preprocessing on each first keypoint corresponding to the first detection box of the target and each second keypoint corresponding to the second detection box matching the first detection box of the target to obtain the relative offset of each first keypoint corresponding to the first detection box of the target includes: Subtract the x-coordinate of the first target key point from the x-coordinate of the second target key point, and divide the resulting x-coordinate difference by the width of the first target detection box to obtain the first offset value of the first target key point; the first target key point is any first key point in the first target detection box; the second target key point is the second key point in the second detection box that matches the first target detection box and corresponds to the first target key point. Subtract the ordinate of the first key point of the target from the ordinate of the second key point of the target, and divide the difference in ordinates by the length of the first detection frame of the target to obtain the second offset value of the first key point of the target. The relative offset of the first key point of the target is determined based on the first offset value and the second offset value of the first key point of the target.

5. A fall detection device, characterized in that, The device includes: The detection box acquisition module is used to perform human detection on the current frame image, obtain at least one first detection box corresponding to the current frame image, and obtain at least one second detection box corresponding to the previous frame image. The key point acquisition module is used to perform human key point detection on each of the first detection boxes to obtain the first key point corresponding to each of the first detection boxes and to obtain the second key point corresponding to each of the second detection boxes. The detection box distance calculation unit is used to calculate the ratio of the number of pixels contained in the first cluster center of the target to the total number of pixels contained in the first detection box of the target, to obtain a first ratio corresponding to the first cluster center of the target; the first cluster center of the target is any first cluster center corresponding to the first detection box of the target; the first detection box of the target is any first detection box of the target; calculate the ratio of the number of pixels contained in the second cluster center of the target to the total number of pixels contained in the second detection box of the target, to obtain a second ratio corresponding to the second cluster center of the target; the second cluster center of the target is any second cluster center corresponding to the second detection box of the target; the second detection box of the target is any second detection box of the target; based on the first cluster center of the target... The distance between the target first cluster center and the target second cluster center is obtained by using the corresponding first ratio and the second ratio corresponding to the target second cluster center; based on the distance between the target first cluster center and each second cluster center corresponding to the target second detection box, the second cluster center matching the target first cluster center is determined, thus obtaining the center pair corresponding to the target first cluster center; each center pair includes the first cluster center of the target first detection box and the second cluster center matching the target second detection box; based on the distance between the first cluster center and the matching second cluster center included in each center pair, the distance between the target first detection box and the target second detection box is calculated; The matching determination unit is used to determine the second detection box that matches the first detection box of the target based on the distance between the first detection box of the target and each of the second detection boxes; The fall detection module is used to perform fall detection based on the first key point corresponding to each first detection box and the second key point corresponding to the matched second detection box using a fall detection model, and obtain the fall detection result.

6. An electronic device, characterized in that, The system includes a memory and a processor, wherein the memory stores a computer program that, when executed by the processor, causes the processor to perform the method as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method as described in any one of claims 1 to 4.

Images