Method, device, storage medium and system for judging personnel fall

Abstract

The embodiment of the application provides a personnel fall judgment method, device, storage medium and system, the method comprises the following steps: acquiring the latest personnel image of the target personnel, acquiring the key point information corresponding to each key point of the target personnel in each personnel image; judging whether the target personnel falls based on the acquired key point information, and generating first judgment information; inputting all the acquired key point information into a personnel key point sequence classifier, judging whether the target personnel falls through the personnel key point sequence classifier, and generating second judgment information; acquiring the personnel image information of each personnel image, inputting all the acquired personnel image information into a personnel image sequence classifier, judging whether the target personnel falls through the personnel image sequence classifier, and generating third judgment information; judging whether the target personnel really falls based on the first judgment information, the second judgment information and the third judgment information, so that the accuracy of judging whether the personnel falls can be improved.

Description

Technical Field

[0001] This application relates to the field of video surveillance technology, and in particular to a method, device, storage medium and system for determining a person's fall. Background Technology

[0002] In underground coal mining, worker safety is of paramount importance. Current fall detection algorithms often use a single algorithm to determine if a person has fallen, but the accuracy of a single algorithm is low, and the false alarm rate is high. Therefore, improving the accuracy of fall detection is a technical problem that needs to be solved. Summary of the Invention

[0003] This application provides a method, device, storage medium, and system for determining whether a person has fallen, which can improve the accuracy of determining whether a person has fallen and reduce the false alarm rate.

[0004] This application provides a method for determining a person's fall, the method comprising:

[0005] Acquire the latest multi-frame images of the target person, and acquire key point information corresponding to each key point among multiple key points of the target person in each frame of the image;

[0006] Based on the obtained key point information, it is determined whether the target person has fallen, and first judgment information is generated;

[0007] All the acquired key point information is input into a personnel key point sequence classifier trained by the GRU deep learning algorithm. The personnel key point sequence classifier determines whether the target person has fallen and generates second judgment information.

[0008] The personnel image information of each frame of the personnel image is obtained, and all the obtained personnel image information is input into a pre-trained personnel image sequence classifier. The personnel image sequence classifier is used to determine whether the target personnel has fallen and to generate third judgment information.

[0009] Based on the first judgment information, the second judgment information, and the third judgment information, determine whether the target person has actually fallen.

[0010] In the method for determining a person's fall as described in this application embodiment, the multiple key points include the target person's nose, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left waist, right waist, left knee, right knee, left ankle, and right ankle. The key point information includes the X coordinate, Y coordinate, and confidence level of the key point.

[0011] In the method for determining whether a person has fallen, as described in the embodiments of this application, determining whether the target person has fallen based on the acquired key point information includes:

[0012] Based on the obtained Y-coordinates of the nose, left shoulder, right shoulder, left waist, right waist, left knee, right knee, left ankle, and right ankle, it is determined whether the target person has fallen.

[0013] In the method for determining whether a person has fallen, as described in this application embodiment, determining whether the target person has fallen based on the acquired Y-coordinates of the nose, left shoulder, right shoulder, left waist, right waist, left knee, right knee, left ankle, and right ankle includes:

[0014] The Y coordinates of the left waist, right waist, left knee, right knee, left ankle, and right ankle are compared with the Y coordinates of the nose, left shoulder, and right shoulder.

[0015] If the Y-coordinate of any one of the key points among the left waist, right waist, left knee, right knee, left ankle, and right ankle is greater than the Y-coordinate of any one of the key points among the nose, left shoulder, and right shoulder, then it is determined that the target person has fallen.

[0016] If the Y-coordinate of any of the key points among the left waist, right waist, left knee, right knee, left ankle, and right ankle is greater than the Y-coordinate of any of the key points among the nose, left shoulder, and right shoulder, then it is determined that the target person has not fallen.

[0017] In the method for determining a person's fall as described in the embodiments of this application, the first determination information, the second determination information, and the third determination information all include falls and non-falls.

[0018] In the method for determining whether a person has actually fallen, as described in the embodiments of this application, determining whether the target person has actually fallen based on the first determination information, the second determination information, and the third determination information includes:

[0019] If at least two of the first, second, and third judgment information indicate a fall, then the target person is determined to have actually fallen.

[0020] If at least two of the first, second, and third judgment information indicate that the person has not actually fallen, then the person is determined not to have actually fallen.

[0021] In the method for determining whether a person has fallen as described in the embodiments of this application, the step of determining whether the target person has fallen using the person key point sequence classifier includes:

[0022] Obtain the output of the personnel key point sequence classifier;

[0023] If the output of the personnel key point sequence classifier is labeled as 1, then it is determined that the target person has fallen.

[0024] If the output label of the personnel key point sequence classifier is 0, it is determined that the target personnel did not fall.

[0025] This application embodiment also provides a person fall detection device, the device comprising:

[0026] The acquisition module is used to acquire the latest multi-frame images of the target person, and to acquire key point information corresponding to each key point among multiple key points of the target person in each frame of the image.

[0027] The first generation module is used to determine whether the target person has fallen based on the acquired key point information, and to generate first judgment information;

[0028] The second generation module is used to input all the acquired key point information into a personnel key point sequence classifier trained by the GRU deep learning algorithm, and to determine whether the target person has fallen through the personnel key point sequence classifier, and generate second judgment information.

[0029] The third generation module is used to acquire personnel image information of each frame of the personnel image, and input all the acquired personnel image information into a pre-trained personnel image sequence classifier. The personnel image sequence classifier determines whether the target personnel has fallen and generates third judgment information.

[0030] The judgment module is used to determine whether the target person has actually fallen based on the first judgment information, the second judgment information, and the third judgment information.

[0031] This application also provides a fall detection system, which includes the fall detection device described in the above embodiments.

[0032] This application also provides a computer-readable storage medium storing a computer program that, when run on a computer, causes the computer to execute the person fall judgment method described in any of the above embodiments.

[0033] The method for determining whether a person has fallen, as provided in this application, acquires the latest multi-frame images of the target person and obtains key point information corresponding to each key point among multiple key points of the target person in each frame of the image. Based on the acquired key point information, it determines whether the target person has fallen and generates first judgment information. Simultaneously, all acquired key point information is input into a person key point sequence classifier trained with the GRU deep learning algorithm. The person key point sequence classifier determines whether the target person has fallen and generates second judgment information. Furthermore, it acquires the person image information of each frame of the image and inputs all acquired person image information into a pre-trained person image sequence classifier. The person image sequence classifier determines whether the target person has fallen and generates third judgment information. Finally, based on the first, second, and third judgment information, it determines whether the target person has actually fallen. This method uses three algorithms to comprehensively determine whether the target person has fallen, focusing on both the feature information of the person's key points and the feature information of the person's image. Compared to existing methods for detecting falls, it only increases the computational load slightly but improves the accuracy and reduces the probability of false alarms. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 This is a flowchart illustrating the method for determining a person's fall, as provided in an embodiment of this application.

[0036] Figure 2 This is a schematic diagram of the structure of the personnel fall detection device provided in the embodiments of this application.

[0037] Figure 3 Another structural schematic diagram of the personnel fall detection device provided in the embodiments of this application.

[0038] Figure 4 This is a schematic diagram of the structure of the personnel fall detection system provided in an embodiment of this application. Detailed Implementation

[0039] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the protection scope of this application.

[0040] This application provides a method for determining whether a person has fallen, which can be applied to a system for determining whether a person has fallen.

[0041] Please see Figure 1 , Figure 1 This is a flowchart illustrating a method for determining a person's fall, provided in an embodiment of this application. The method, applied in a fall detection system, may include the following steps:

[0042] Step 101: Obtain the latest multi-frame images of the target person, and obtain key point information corresponding to each key point among multiple key points of the target person in each frame of the image.

[0043] Among them, the multi-frame personnel images can be 30 frames of personnel images.

[0044] There is no limit to the number of target personnel; it can be one or more. When there are multiple target personnel, a unique ID can be assigned to each target personnel using the Byte Track tracking algorithm. Then, the latest multi-frame images of each tracked target personnel are retrieved and stored in a personnel image information storage, named with the target personnel's ID number. The latest multi-frame images of the target personnel can then be retrieved from the personnel image information storage. All target personnel can be detected using the YOLOv8 target detection algorithm.

[0045] In some embodiments, the plurality of key points include the target person’s nose, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left waist, right waist, left knee, right knee, left ankle, and right ankle, and the key point information includes the X coordinate, Y coordinate, and confidence level of the key points.

[0046] Specifically, the key information of each target person can be stored separately in the Personnel Key Information Storage, named by the person's ID number.

[0047] Step 102: Based on the obtained key point information, determine whether the target person has fallen, and generate first judgment information.

[0048] In some embodiments, it is determined whether the target person has fallen based on the Y coordinates of the nose, left shoulder, right shoulder, left waist, right waist, left knee, right knee, left ankle, and right ankle.

[0049] In some embodiments, determining whether the target person has fallen based on the acquired Y-coordinates of the nose, left shoulder, right shoulder, left waist, right waist, left knee, right knee, left ankle, and right ankle includes:

[0050] The Y coordinates of the left waist, right waist, left knee, right knee, left ankle, and right ankle are compared with the Y coordinates of the nose, left shoulder, and right shoulder.

[0051] If the Y-coordinate of any one of the key points among the left waist, right waist, left knee, right knee, left ankle, and right ankle is greater than the Y-coordinate of any one of the key points among the nose, left shoulder, and right shoulder, then it is determined that the target person has fallen.

[0052] If the Y-coordinate of any of the key points among the left waist, right waist, left knee, right knee, left ankle, and right ankle is greater than the Y-coordinate of any of the key points among the nose, left shoulder, and right shoulder, then it is determined that the target person has not fallen.

[0053] Step 103: Input all the obtained key point information into the personnel key point sequence classifier trained by the GRU deep learning algorithm, and use the personnel key point sequence classifier to determine whether the target person has fallen and generate second judgment information.

[0054] For example, all keypoint information in the personnel keypoint information storage is retrieved. The keypoint information of each target person is one dimension, and the total data dimension is N*30*13*3. Here, N is the number of target people detected, 30 represents 30 frames of personnel images, 13 represents the 13 keypoints of each target person (nose, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left waist, right waist, left knee, right knee, left ankle, and right ankle), and 3 represents the X coordinate, Y coordinate, and confidence level of each keypoint.

[0055] You can input key information for one target person at a time, or you can input key information for multiple target persons at the same time.

[0056] When inputting keypoint information for only one target person at a time, the process for determining whether all target people have fallen is as follows: input the keypoint information for one target person at a time, and only after determining whether that target person has fallen is the keypoint information for the next target person inputted, and so on. For example, if there are two target people, A and B, and the goal is to determine whether these two target people have fallen using a personnel keypoint sequence classifier, first input all the keypoint information for A into the personnel keypoint sequence classifier, and then use the personnel keypoint sequence classifier to determine whether A has fallen. After determining whether A has fallen, then input all the keypoint information for B into the personnel keypoint sequence classifier, and then use the personnel keypoint sequence classifier to determine whether B has fallen.

[0057] When inputting keypoint information for multiple target individuals simultaneously, the number of target individuals can be preset, for example, 8. In this case, the maximum input per detection is the keypoint information for 8 target individuals. For instance, if 17 target individuals are detected simultaneously in one frame, the input will be divided into three batches: the first batch contains keypoint information for 8 target individuals, the second batch contains keypoint information for 8 target individuals, and the third batch contains keypoint information for 1 target individual. The second judgment information for the first batch consists of 8 pieces of information, corresponding to the 8 target individuals in that batch; the second judgment information for the second batch also consists of 8 pieces of information, corresponding to the 8 target individuals in that batch; and the second judgment information for the third batch consists of 1 piece of information, corresponding to the 1 target individual in that batch.

[0058] In some embodiments, determining whether the target person has fallen using the personnel key point sequence classifier includes:

[0059] Obtain the output of the personnel key point sequence classifier;

[0060] If the output of the personnel key point sequence classifier is labeled as 1, then it is determined that the target person has fallen.

[0061] If the output label of the personnel key point sequence classifier is 0, it is determined that the target personnel did not fall.

[0062] The output is 1, which means the person fell down; and 0, which means the person did not fall down.

[0063] Step 104: Obtain the personnel image information of each frame of the personnel image, and input all the obtained personnel image information into a pre-trained personnel image sequence classifier. Use the personnel image sequence classifier to determine whether the target personnel has fallen and generate third judgment information.

[0064] The personnel image information includes the width and height of the personnel image. For example, the personnel image information in the personnel image information storage is retrieved, with each target personnel's personnel image information as one dimension, and the total data dimensions are N*3*30*224*224. Here, N is the number of target personnel, 3 is the number of image channels, 30 is the latest 30 frames of personnel image information for each target personnel, and 224*224 is the width and height of the personnel image.

[0065] The system allows users to input the image information of one target person at a time, or to input the image information of multiple target persons simultaneously.

[0066] When inputting the image information of only one target person at a time, the steps for determining whether all target persons have fallen are as follows: input the image information of one target person at a time, and after determining whether that target person has fallen, input the image information of the next target person to determine whether the next target person has fallen. For example, if there are two target persons, A and B, and the goal is to determine whether these two target persons have fallen using a personnel image sequence classifier, first input the image information of A into the personnel image sequence classifier, and use the personnel keypoint sequence classifier to determine whether A has fallen. After determining whether A has fallen, then input the image information of B into the personnel image sequence classifier, and use the personnel image sequence classifier to determine whether B has fallen.

[0067] When inputting images of multiple target individuals simultaneously, the number of target individuals can be preset, for example, 8. In this case, the maximum input per detection cycle is images of 8 target individuals. For instance, if 17 target individuals are detected simultaneously in one frame, the input will be divided into three batches: the first batch contains images of 8 target individuals, the second batch contains images of 8 target individuals, and the third batch contains images of 1 target individual. The third judgment information for the first batch consists of 8 pieces of information, corresponding to the 8 target individuals in that batch; the third judgment information for the second batch also consists of 8 pieces of information, corresponding to the 8 target individuals in that batch; and the third judgment information for the third batch consists of 1 piece of information, corresponding to the single target individual in that batch.

[0068] The personnel image sequence classifier can be a classifier trained using the deep learning algorithm Videomae.

[0069] In some embodiments, determining whether the target person has fallen using the person image sequence classifier includes:

[0070] Obtain the output of the personnel image sequence classifier;

[0071] If the output of the personnel image sequence classifier is labeled as 1, then it is determined that the target person has fallen.

[0072] If the output of the personnel image sequence classifier is labeled 0, it is determined that the target personnel did not fall.

[0073] The output is 1, which means the person fell down; and 0, which means the person did not fall down.

[0074] Step 105: Based on the first judgment information, the second judgment information, and the third judgment information, determine whether the target person has actually fallen.

[0075] In some embodiments, the first judgment information, the second judgment information, and the third judgment information all include falling and not falling.

[0076] That is, the first judgment information is whether the person fell or not, the second judgment information is whether the person fell or not, and the third judgment information is whether the person fell or not.

[0077] In some embodiments, determining whether the target person has actually fallen based on the first judgment information, the second judgment information, and the third judgment information includes:

[0078] If at least two of the first, second, and third judgment information indicate a fall, then the target person is determined to have actually fallen.

[0079] If at least two of the first, second, and third judgment information indicate that the person has not actually fallen, then the person is determined not to have actually fallen.

[0080] For example, if the first and second judgment information are both "falling" and the third judgment information is "not falling", then the target person is judged to have actually fallen; or if the first, second, and third judgment information are all "falling", then the target person is judged to have actually fallen.

[0081] For example, if the first and second judgment information are both "not a fall" and the third judgment information is "fall", then it is determined that the target person did not fall; or if the first, second, and third judgment information are all "not a fall", then it is determined that the target person did not fall.

[0082] All of the above-mentioned optional technical solutions can be combined in any way to form the optional embodiments of this application, and will not be described in detail here.

[0083] In practice, this application is not limited by the execution order of the described steps. Without causing conflicts, some steps may be performed in other orders or simultaneously.

[0084] As can be seen from the above, the method for determining whether a person has fallen, provided in this application, acquires the latest multi-frame images of the target person and obtains key point information corresponding to each key point among multiple key points of the target person in each frame of the image. Then, it determines whether the target person has fallen based on the acquired key point information and generates first judgment information. Simultaneously, it inputs all acquired key point information into a person key point sequence classifier trained by the GRU deep learning algorithm, and uses the person key point sequence classifier to determine whether the target person has fallen and generates second judgment information. In addition, it acquires the person image information of each frame of the image and inputs all acquired person image information into a pre-trained person image sequence classifier, and uses the person image sequence classifier to determine whether the target person has fallen and generates third judgment information. Finally, it determines whether the target person has actually fallen based on the first, second, and third judgment information. That is, it uses three algorithms to comprehensively determine whether the target person has fallen, paying attention to both the feature information of the person's key points and the feature information of the person's image. Compared with existing methods for detecting falls, it only increases the computational load by a small amount, but improves the accuracy of judgment and reduces the probability of false alarms.

[0085] This application also provides a person fall detection device, which can be integrated into a person fall detection system.

[0086] Please see Figure 2 , Figure 2 This is a schematic diagram of the structure of the personnel fall detection device provided in an embodiment of this application. The personnel fall detection device 30 may include:

[0087] The acquisition module 31 is used to acquire the latest multi-frame images of the target person, and to acquire key point information corresponding to each key point among multiple key points of the target person in each frame of the image.

[0088] The first generation module 32 is used to determine whether the target person has fallen based on the acquired key point information, and to generate first judgment information;

[0089] The second generation module 33 is used to input all the acquired key point information into a personnel key point sequence classifier trained by the GRU deep learning algorithm, and to determine whether the target person has fallen through the personnel key point sequence classifier, and generate second judgment information.

[0090] The third generation module 34 is used to acquire personnel image information of each frame of the personnel image, and input all the acquired personnel image information into a pre-trained personnel image sequence classifier, and use the personnel image sequence classifier to determine whether the target personnel has fallen, and generate third judgment information.

[0091] The judgment module 35 is used to determine whether the target person has actually fallen based on the first judgment information, the second judgment information, and the third judgment information.

[0092] In some embodiments, the first generation module 32 is used to determine whether the target person has fallen based on the acquired Y coordinates of the nose, left shoulder, right shoulder, left waist, right waist, left knee, right knee, left ankle, and right ankle.

[0093] In some embodiments, the first generation module 32 is used to compare the Y coordinates of the left waist, right waist, left knee, right knee, left ankle, and right ankle with the Y coordinates of the nose, left shoulder, and right shoulder; if the Y coordinate of any key point among the left waist, right waist, left knee, right knee, left ankle, and right ankle is greater than the Y coordinate of any key point among the nose, left shoulder, and right shoulder, then it is determined that the target person has fallen; if the Y coordinate of no key point among the left waist, right waist, left knee, right knee, left ankle, and right ankle is greater than the Y coordinate of any key point among the nose, left shoulder, and right shoulder, then it is determined that the target person has not fallen.

[0094] In some embodiments, the second generation module 33 is used to obtain the output result of the personnel key point sequence classifier;

[0095] If the personnel key point sequence classifier outputs a label of 1, it is determined that the target person has fallen; if the personnel key point sequence classifier outputs a label of 0, it is determined that the target person has not fallen.

[0096] In some embodiments, the judgment module 35 is configured to determine that the target person has actually fallen if at least two of the first, second, and third judgment information indicate a fall; and to determine that the target person has not actually fallen if at least two of the first, second, and third judgment information indicate no fall.

[0097] In practice, the above modules can be implemented as independent entities or combined in any way to be implemented as the same or several entities.

[0098] As can be seen from the above, the personnel fall judgment device 30 provided in this application embodiment acquires the latest multi-frame images of the target personnel through the acquisition module 31, and acquires key point information corresponding to each key point among multiple key points of the target personnel in each frame of the personnel image; the first generation module 32 judges whether the target personnel has fallen based on the acquired key point information and generates first judgment information; the second generation module 33 inputs all the acquired key point information into a personnel key point sequence classifier trained by the GRU deep learning algorithm, and the personnel key point sequence classifier judges whether the target personnel has fallen and generates second judgment information; the third generation module 34 acquires personnel image information of each frame of the personnel image, and inputs all the acquired personnel image information into a pre-trained personnel image sequence classifier, and the personnel image sequence classifier judges whether the target personnel has fallen and generates third judgment information; the judgment module 35 judges whether the target personnel has actually fallen based on the first judgment information, the second judgment information, and the third judgment information. This application embodiment uses three algorithms to comprehensively determine whether a target person has fallen. It focuses on both the feature information of the person's key points and the feature information of the person's image. Compared with the existing methods for detecting falls, it only increases the amount of computation by a small amount, but improves the accuracy of judgment and reduces the probability of false alarms.

[0099] Please see Figure 3 , Figure 3 This is another schematic diagram of the personnel fall detection device provided in this application embodiment. The personnel fall detection device 30 includes a memory 120, one or more processors 180, and one or more application programs, wherein the one or more application programs are stored in the memory 120 and configured to be executed by the processors 180; the processors 180 may include an acquisition module 31, a first generation module 32, a second generation module 33, a third generation module 34, and a judgment module 35. For example, the structure and connection relationship of the above components can be as follows:

[0100] Memory 120 can be used to store applications and data. The applications stored in memory 120 contain executable code. Applications can be composed of various functional modules. Processor 180 executes various functional applications and data processing by running the applications stored in memory 120. Furthermore, memory 120 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, memory 120 may also include a memory controller to provide processor 180 with access to memory 120.

[0101] The processor 180 is the control center of the device, connecting various parts of the terminal through various interfaces and lines. It performs various functions and processes data by running or executing applications stored in the memory 120 and calling data stored in the memory 120, thereby providing overall monitoring of the device. Optionally, the processor 180 may include one or more processing cores; preferably, the processor 180 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications.

[0102] Specifically, in this embodiment, the processor 180 loads the executable code corresponding to the processes of one or more applications into the memory 120 according to the following instructions, and the processor 180 runs the applications stored in the memory 120 to achieve various functions:

[0103] The acquisition module 31 is used to acquire the latest multi-frame images of the target person, and to acquire key point information corresponding to each key point among multiple key points of the target person in each frame of the image.

[0104] The first generation module 32 is used to determine whether the target person has fallen based on the acquired key point information, and to generate first judgment information;

[0105] The second generation module 33 is used to input all the acquired key point information into a personnel key point sequence classifier trained by the GRU deep learning algorithm, and to determine whether the target person has fallen through the personnel key point sequence classifier, and generate second judgment information.

[0106] The third generation module 34 is used to acquire personnel image information of each frame of the personnel image, and input all the acquired personnel image information into a pre-trained personnel image sequence classifier, and use the personnel image sequence classifier to determine whether the target personnel has fallen, and generate third judgment information.

[0107] The judgment module 35 is used to determine whether the target person has actually fallen based on the first judgment information, the second judgment information, and the third judgment information.

[0108] In some embodiments, the first generation module 32 is used to determine whether the target person has fallen based on the acquired Y coordinates of the nose, left shoulder, right shoulder, left waist, right waist, left knee, right knee, left ankle, and right ankle.

[0109] In some embodiments, the first generation module 32 is used to compare the Y coordinates of the left waist, right waist, left knee, right knee, left ankle, and right ankle with the Y coordinates of the nose, left shoulder, and right shoulder; if the Y coordinate of any key point among the left waist, right waist, left knee, right knee, left ankle, and right ankle is greater than the Y coordinate of any key point among the nose, left shoulder, and right shoulder, then it is determined that the target person has fallen; if the Y coordinate of no key point among the left waist, right waist, left knee, right knee, left ankle, and right ankle is greater than the Y coordinate of any key point among the nose, left shoulder, and right shoulder, then it is determined that the target person has not fallen.

[0110] In some embodiments, the second generation module 33 is used to obtain the output result of the personnel key point sequence classifier;

[0111] If the personnel key point sequence classifier outputs a label of 1, it is determined that the target person has fallen; if the personnel key point sequence classifier outputs a label of 0, it is determined that the target person has not fallen.

[0112] In some embodiments, the judgment module 35 is configured to determine that the target person has actually fallen if at least two of the first, second, and third judgment information indicate a fall; and to determine that the target person has not actually fallen if at least two of the first, second, and third judgment information indicate no fall.

[0113] like Figure 4 As shown, the personnel fall detection system 1200 may include a personnel fall detection device 30, a personnel key point sequence classifier 36, and a personnel image sequence classifier 37.

[0114] The personnel key point sequence classifier 36 is communicatively connected to the personnel fall detection device 30 and is used to determine whether the target personnel have fallen.

[0115] The personnel image sequence classifier 37 is communicatively connected to the personnel fall detection device 30 and is used to determine whether the target personnel have fallen.

[0116] The processor 180 in the personnel fall detection system 1200 is used to perform the following instructions:

[0117] Acquire the latest multi-frame images of the target person, and acquire key point information corresponding to each key point among multiple key points of the target person in each frame of the image;

[0118] Based on the obtained key point information, it is determined whether the target person has fallen, and first judgment information is generated;

[0119] All the acquired key point information is input into a personnel key point sequence classifier trained by the GRU deep learning algorithm. The personnel key point sequence classifier determines whether the target person has fallen and generates second judgment information.

[0120] The personnel image information of each frame of the personnel image is obtained, and all the obtained personnel image information is input into a pre-trained personnel image sequence classifier. The personnel image sequence classifier is used to determine whether the target personnel has fallen and to generate third judgment information.

[0121] Based on the first judgment information, the second judgment information, and the third judgment information, determine whether the target person has actually fallen.

[0122] In some embodiments, the processor 180 is used to determine whether the target person has fallen based on the acquired Y coordinates of the nose, left shoulder, right shoulder, left waist, right waist, left knee, right knee, left ankle, and right ankle.

[0123] In some embodiments, the processor 180 is configured to compare the Y-coordinates of the left waist, right waist, left knee, right knee, left ankle, and right ankle with the Y-coordinates of the nose, left shoulder, and right shoulder; if the Y-coordinate of any one of the key points among the left waist, right waist, left knee, right knee, left ankle, and right ankle is greater than the Y-coordinate of any one of the key points among the nose, left shoulder, and right shoulder, then it is determined that the target person has fallen; if the Y-coordinate of no one of the key points among the left waist, right waist, left knee, right knee, left ankle, and right ankle is greater than the Y-coordinate of any one of the key points among the nose, left shoulder, and right shoulder, then it is determined that the target person has not fallen.

[0124] In some embodiments, the processor 180 is configured to determine that the target person has actually fallen if at least two of the first, second, and third judgment information indicate a fall; and to determine that the target person has not actually fallen if at least two of the first, second, and third judgment information indicate no fall.

[0125] In some embodiments, the processor 180 is used to obtain the output result of the personnel key point sequence classifier; if the output result of the personnel key point sequence classifier is labeled as 1, it is determined that the target person has fallen; if the output result of the personnel key point sequence classifier is labeled as 0, it is determined that the target person has not fallen.

[0126] As described above, this application provides a fall detection system 1200, which performs the following steps: acquiring the latest multi-frame images of a target person, and acquiring key point information corresponding to each key point among multiple key points of the target person in each frame of the image; determining whether the target person has fallen based on the acquired key point information, and generating first judgment information; inputting all the acquired key point information into a person key point sequence classifier trained by the GRU deep learning algorithm, and determining whether the target person has fallen through the person key point sequence classifier, and generating second judgment information; acquiring the person image information of each frame of the image, and inputting all the acquired person image information into a pre-trained person image sequence classifier, and determining whether the target person has fallen through the person image sequence classifier, and generating third judgment information; determining whether the target person has actually fallen based on the first judgment information, the second judgment information, and the third judgment information. Through the above steps, this application can improve the accuracy of determining whether a person has fallen and reduce the false alarm rate.

[0127] This application also provides a computer-readable storage medium storing a computer program. When the computer program is run on a computer, the computer executes the person fall judgment method described in any of the above embodiments.

[0128] It should be noted that, for the personnel fall detection method described in this application, those skilled in the art will understand that implementing all or part of the process of the personnel fall detection method described in the embodiments of this application can be accomplished by a computer program controlling related hardware. The computer program can be stored in a computer-readable storage medium, such as the memory of the personnel fall detection system, and executed by at least one processor within the personnel fall detection system. During execution, it can include the process of the embodiments of the personnel fall detection method described. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), random access memory (RAM), etc.

[0129] The foregoing has provided a detailed description of the method, apparatus, storage medium, and system for determining falls provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for determining whether a person has fallen, characterized in that, The method includes: The latest multi-frame images of the target person are obtained, and key point information corresponding to each key point among multiple key points of the target person in each frame of the image is obtained. The key point information includes the X coordinate, Y coordinate and confidence level of the key point. Based on the Y coordinates of the obtained key points, a comparison is made to determine whether the target person has fallen, and first judgment information is generated; All the acquired key point information is input into a personnel key point sequence classifier trained by the GRU deep learning algorithm. The personnel key point sequence classifier determines whether the target person has fallen and generates second judgment information. The personnel image information of each frame of the personnel image is obtained, and all the obtained personnel image information is input into a pre-trained personnel image sequence classifier. The personnel image sequence classifier is used to determine whether the target personnel has fallen and to generate third judgment information. The first, second, and third judgment information all include falls and non-falls; Determining whether the target person has actually fallen based on the first judgment information, the second judgment information, and the third judgment information includes: If at least two of the first, second, and third judgment information indicate a fall, then the target person is determined to have actually fallen. If at least two of the first, second, and third judgment information indicate that the person has not actually fallen, then the person is determined not to have actually fallen.

2. The method for determining a person's fall as described in claim 1, characterized in that, The key points include the target person's nose, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left waist, right waist, left knee, right knee, left ankle, and right ankle.

3. The method for determining a person's fall as described in claim 2, characterized in that, Determining whether the target person has fallen based on a comparison of the obtained Y-coordinates of the key points includes: Based on the obtained Y-coordinates of the nose, left shoulder, right shoulder, left waist, right waist, left knee, right knee, left ankle, and right ankle, it is determined whether the target person has fallen.

4. The method for determining a person's fall as described in claim 3, characterized in that, The step of determining whether the target person has fallen based on the acquired Y-coordinates of the nose, left shoulder, right shoulder, left waist, right waist, left knee, right knee, left ankle, and right ankle includes: The Y coordinates of the left waist, right waist, left knee, right knee, left ankle, and right ankle are compared with the Y coordinates of the nose, left shoulder, and right shoulder. If the Y-coordinate of any one of the key points—left waist, right waist, left knee, right knee, left ankle, and right ankle—is greater than the Y-coordinate of any one of the key points—nose, left shoulder, and right shoulder, then the target person is determined to have fallen. If the Y-coordinate of any of the key points among the left waist, right waist, left knee, right knee, left ankle, and right ankle is greater than the Y-coordinate of any of the key points among the nose, left shoulder, and right shoulder, then it is determined that the target person has not fallen.

5. The method for determining a person's fall as described in claim 1, characterized in that, The step of determining whether the target person has fallen using the personnel key point sequence classifier includes: Obtain the output of the personnel key point sequence classifier; If the output of the personnel key point sequence classifier is labeled as 1, then it is determined that the target person has fallen. If the output label of the personnel key point sequence classifier is 0, it is determined that the target personnel did not fall.

6. A device for detecting falls, characterized in that, The device includes: The acquisition module is used to acquire the latest multi-frame images of the target person, and to acquire key point information corresponding to each key point among multiple key points of the target person in each frame of the image, wherein the key point information includes the X coordinate, Y coordinate and confidence level of the key point; The first generation module is used to compare and determine whether the target person has fallen based on the Y coordinate of the obtained key point, and generate first judgment information; The second generation module is used to input all the acquired key point information into a personnel key point sequence classifier trained by the GRU deep learning algorithm, and to determine whether the target person has fallen through the personnel key point sequence classifier, and generate second judgment information. The third generation module is used to acquire personnel image information of each frame of the personnel image, and input all the acquired personnel image information into a pre-trained personnel image sequence classifier. The personnel image sequence classifier determines whether the target personnel has fallen and generates third judgment information. The judgment module is used to determine whether the target person has actually fallen based on the first judgment information, the second judgment information, and the third judgment information, including: The first, second, and third judgment information all include falls and non-falls; If at least two of the first, second, and third judgment information indicate a fall, then the target person is determined to have actually fallen. If at least two of the first, second, and third judgment information indicate that the person has not actually fallen, then the person is determined not to have actually fallen.

7. A fall detection system for individuals, characterized in that, The personnel fall detection system includes the personnel fall detection device as described in claim 6.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when run on a computer, causes the computer to perform the person fall judgment method according to any one of claims 1 to 5.

Images