Human body abnormal action recognition method and device, computer equipment and medium
By combining video surveillance and Kinect technology, real-time identification of changes in human skeleton points has been solved, and the existing monitoring system is difficult to identify abnormal movements of the human body is achieved, and accurate analysis and timely alarms of human behavior are achieved.
Patent Information
- Application Number
- CN202411004131.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-25
- Publication Date
- 2024-11-26
AI Technical Summary
The existing monitoring system is difficult to identify and warn abnormal human movements in real time, especially in places with dense crowds, which makes it difficult to form a mechanism of active early warning and timely alarm.
Video streams and Kinect bone data are collected in real time through video surveillance equipment, and target detection model, human weight recognition network and feature fusion model are used, combining the principles of time synchronization and spatial synchronization to identify changes in human bone points, match abnormal behavior types, and pass through nerves The network model triggers the alarm system.
It realizes effective identification and analysis of abnormal movements of human bodies, reduces the false alarm rate, improves the accuracy and practicality of human behavior analysis, and can promptly trigger alarms and record video clips of abnormal behaviors.
Smart Images

Figure CN119028011A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of human motion recognition, and in particular to a method, device, computer equipment and medium for recognizing abnormal human motion. Background Art
[0002] Places with large crowds, such as transportation stations, have large spaces and are prone to incidents such as falls, conflicts, and group panic. Preventing the occurrence of the above incidents through video surveillance is of great practical significance for maintaining people's livelihood security.
[0003] However, the existing monitoring system relies on long-term human supervision by staff, and it is difficult to display a large number of video scene information at the same time and complete identification and judgment. As a result, a large number of video monitoring systems can only use video storage to collect evidence after the event, and it is difficult to form a positive mechanism for active early warning and timely alarm.
[0004] Detect designated abnormal events and send out real-time alarms to remind staff to confirm the events and take corresponding measures. However, it is still difficult to dynamically monitor the behavior of multiple people or groups and effectively analyze and detect abnormal behaviors (such as fighting, falling, etc.) in a timely and effective manner through continuous video recording. Summary of the invention
[0005] In order to overcome the problems existing in the related art, the present disclosure provides a method, device, computer equipment and medium for identifying abnormal human movements to solve the technical problem that the related art cannot effectively analyze and detect abnormal behaviors.
[0006] One or more embodiments of this specification provide a method for identifying abnormal human movements, including the steps of:
[0007] The video stream is collected in real time by a video monitoring device, and the Kinect skeleton data is collected in real time by a skeleton point collection device, wherein the Kinect skeleton data includes a human skeleton sequence and three-dimensional coordinate features of each skeleton point;
[0008] Based on the video stream, the target detection model is used to realize the human body region frame and human body identification, and the human body weight recognition network is used to convert the detection results of the target detection model into human body weight recognition features;
[0009] Based on the principle of time synchronization and space synchronization, the three-dimensional coordinate features of each node in the human skeleton sequence are fused with the human weight recognition features through the feature fusion model to obtain a fused feature image sequence with human body identification.
[0010] as well as
[0011] The fused feature image sequence is input into the abnormal action recognition model, and whether the human behavior characteristics are abnormal behaviors is determined based on the changes in the human skeleton points and the preset change threshold, and the type of abnormal behavior is determined according to the matching of the abnormal behavior category library; if abnormal behavior is recognized, the alarm system is triggered, wherein the initial abnormal action recognition model is a neural network model obtained by training the abnormal behavior video stream, wherein the abnormal behavior video stream data is a video stream collected by the skeleton point acquisition device, and the neural network model recognizes and classifies abnormal behaviors according to the skeleton points of continuous actions in the video stream and the preset change thresholds of each skeleton point, and obtains an abnormal behavior category library composed of the skeleton point change situation and the abnormal behavior type, wherein the skeleton point change threshold includes a position change threshold and a motion speed change threshold.
[0012] One or more embodiments of this specification provide a device for identifying abnormal human movements, which is characterized by comprising:
[0013] The video acquisition module is used to acquire video streams and Kinect skeleton data in real time, wherein the Kinect skeleton data includes the human skeleton sequence and the three-dimensional coordinate features of each skeleton point;
[0014] The first video data processing module is used to realize the human body region frame and human body identification through the target detection model based on the video stream, and convert the detection result of the target detection model into human body weight recognition features through the human body weight recognition network;
[0015] The second video data processing module is used to fuse the three-dimensional coordinate features of each node in the acquired human skeleton sequence with the human body weight recognition features through a feature fusion model based on the principles of time synchronization and space synchronization, and obtain a fused feature image sequence with human body identification;
[0016] as well as
[0017] The abnormal action prediction module is used to input the fused feature image sequence into the abnormal action recognition model, determine whether the human behavior characteristics are abnormal behaviors based on the changes in the human skeleton points and the preset change threshold, and determine the abnormal behavior type according to the abnormal behavior category library matching; if abnormal behavior is recognized, the alarm system is triggered, wherein the initial abnormal action recognition model is a neural network model obtained by training the abnormal behavior video stream, wherein the abnormal behavior video stream data is the video stream collected by the video acquisition module, and the neural network model recognizes and classifies abnormal behaviors according to the skeleton points of continuous actions in the video stream and the preset change thresholds of each skeleton point, and obtains an abnormal behavior category library composed of the skeleton point changes and abnormal behavior types, wherein the skeleton point change threshold includes a position change threshold and a motion speed change threshold.
[0018] One or more embodiments of the present specification provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned abnormal human motion recognition method when executing the computer program.
[0019] One or more embodiments of the present specification provide a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the above-mentioned method for identifying abnormal human body movements.
[0020] The present disclosure provides a method, device, computer equipment and medium for identifying abnormal human movements. The advantages are that real-time video data of a human body is obtained through a video monitoring device; the real-time video data is processed and each human body is numbered; then, based on the Kinect skeleton data collected in real time by a Kinect device, based on the principles of time synchronization and space synchronization, the three-dimensional coordinates of each node in the acquired human skeleton sequence are fused with the human body weight recognition feature through a feature fusion model, and then the abnormal behavior type is determined by matching the changes of the skeleton points detected by the abnormal action recognition model with the abnormal behavior type set. When the abnormal human behavior is identified, the alarm system is triggered, and the video clip and related information of the behavior are recorded. The method of the embodiment makes full use of continuous video image information, and through human body weight recognition, posture estimation and behavior recognition technology, it can effectively identify and analyze the behavior of the human body, and can also avoid the problem of high false alarm rate of human behavior analysis based on a single image, thereby improving the accuracy and practicality of human behavior analysis. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] In order to more clearly illustrate one or more embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0022] Figure 1 A flowchart of a method for identifying abnormal human movements provided by one or more embodiments of this specification;
[0023] Figure 2 A block diagram of a device for identifying abnormal human movements provided in one or more embodiments of this specification; and
[0024] Figure 3 A schematic diagram of the structure of a computer device provided for one or more embodiments of this specification. DETAILED DESCRIPTION
[0025] In order to enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the technical solutions in one or more embodiments of this specification will be clearly and completely described below in conjunction with the drawings in one or more embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the protection scope of the present invention.
[0026] The present invention is described in detail below in conjunction with specific implementation methods and the accompanying drawings.
[0027] Method Embodiment
[0028] According to an embodiment of the present invention, a method for identifying abnormal human movements is provided. Figure 1 As shown in FIG. 1 , a flow chart of a method for identifying abnormal human movements provided in this embodiment is provided. The abnormal human movement identification according to an embodiment of the present invention includes:
[0029] Step S1, collects video stream in real time by video monitoring equipment, and Kinect skeleton data collected in real time by Kinect device, wherein, Kinect skeleton data comprises the three-dimensional coordinate feature of human skeleton sequence and each skeleton point. In the present embodiment, the continuous video data of human body is captured in real time by video monitoring equipment. These video monitoring equipment are usually installed in key positions such as entrances and exits, platforms, corridors of the station, guaranteeing to comprehensively cover each area in the station. Video monitoring equipment can be high-definition camera, infrared camera etc., to obtain high-quality video data.
[0030] Step S2, based on the video stream, realizes each human body region frame and human body identification through the target detection model, and converts the detection result of the target detection model into human body weight recognition features through the human body weight recognition network.
[0031] In this embodiment, after acquiring real-time video data, the video data is processed, each person in the video is numbered, and behavioral features are extracted. A pedestrian re-identification algorithm is used to identify the facial features, body features or clothing features of the human body, accurately obtain the coordinates of each person in the video image, and number each person.
[0032] Step S3, based on the principles of time synchronization and space synchronization, the three-dimensional coordinates of each node in the acquired human skeleton sequence are fused with the human body weight recognition feature through a feature fusion model to obtain a fused feature image sequence with human body identification.
[0033] Step S4, inputting the fused feature image sequence into the abnormal action recognition model, determining whether the human behavior feature is abnormal behavior based on the change of human skeleton points and the preset change threshold, and determining the abnormal behavior type according to the abnormal behavior matching the abnormal behavior type set; if abnormal behavior is recognized, triggering the alarm system, wherein the initial abnormal action recognition model is a neural network model trained by the abnormal behavior video stream including Kinect skeleton data information, the neural network model recognizes and classifies abnormal behavior according to the skeleton points of continuous actions in the video stream and the preset change threshold of each skeleton point, and obtains an abnormal behavior category library composed of the skeleton point change situation and the abnormal behavior type, wherein the skeleton point change threshold includes a position change threshold and a motion speed change threshold.
[0034] The abnormal human action recognition method provided in the present embodiment obtains real-time video data of the human body through a video monitoring device; processes the real-time video data and numbers each human body; then based on the Kinect skeleton data collected in real time by the Kinect device, based on the principles of time synchronization and space synchronization, the three-dimensional coordinates of each node in the acquired human skeleton sequence are fused with the human body weight recognition feature through a feature fusion model, and then the abnormal behavior type is determined by matching the changes of the skeleton points detected by the abnormal action recognition model with the abnormal behavior type set. When the abnormal human behavior is recognized, the alarm system is triggered, and the video clip and related information of the behavior are recorded. The method of the present embodiment makes full use of continuous video image information, and through human weight recognition, posture estimation and behavior recognition technology, it can effectively recognize and analyze the behavior of the human body, and can also avoid the problem of high false alarm rate of human behavior analysis based on a single image, thereby improving the accuracy and practicality of human behavior analysis.
[0035] The idea of abnormal actions in this embodiment is to identify the body posture of each person through the abnormal action recognition model, extract the trunk and limbs of each person, and use the image classification algorithm to classify the trunk into categories such as standing, lying flat, and tilted, and the limbs into categories such as natural drooping, waving, and horizontal raising. The coordinates and number of each person are matched with the body posture information obtained by posture estimation, so as to obtain the body posture of each person in the continuous video, which constitutes the human behavior characteristics. The specific implementation process is as follows.
[0036] In this embodiment, during the training process, the initial neural network model uses the initial frame coordinates of the skeleton point sequence in the video stream as the initial coordinates, and then identifies and determines the change information of the skeleton point sequence of each video frame, the change information includes whether the skeleton point position has changed, the position change distance, and the change speed. According to the preset skeleton point change distance threshold and change speed threshold, as well as the change distance and change speed, the corresponding abnormal behavior video is classified into the corresponding abnormal behavior data set according to the abnormal behavior type label, wherein the abnormal behavior characterization of the abnormal behavior data set is to determine which abnormal behavior the behavior in the corresponding video stream is through the change distance and change speed of the skeleton point, because different abnormal behaviors, such as fighting and falling, or normal walking, the change distance and change speed of the human skeleton point are definitely different, and these two parameters are used to more accurately represent the changes in the abnormal behavior of the human body, which will greatly improve the accuracy of abnormal behavior action recognition in the human body action recognition process and reduce the probability of false alarms.
[0037] This embodiment is preferred because the number of occurrences of different abnormal human behavior actions is also an important factor in determining abnormal behavior. For example, in a wrestling action, the change distance and change speed of the human skeleton points generally occur once, but in a fight, the distance change and speed change of the human skeleton points will change multiple times. Therefore, this embodiment also sets thresholds for the occurrence of various types of abnormal behaviors. According to the abnormal behavior type and the number of occurrences determined by the abnormal action recognition model, the number of occurrences is compared with the threshold of the corresponding abnormal behavior number of occurrences. If they are equal, the occurrence of the corresponding abnormal behavior is determined, and an alarm is issued.
[0038] In this embodiment, determining the change of coordinates, the change distance and the change speed through the change of coordinates of any point is an existing conventional technology and will not be described in detail here.
[0039] In this embodiment, the training set used for training the initial abnormal action recognition model can find out the patterns and trends of human behavior by clustering and classifying historical abnormal action video data, including the patterns of changes in bone points, which may include patterns of changes in distance and speed, thereby optimizing the behavior pattern library and improving the accuracy of human behavior analysis.
[0040] In this embodiment, the Kinect skeleton data collected in real time by the Kinect device is specifically implemented through the following steps:
[0041] Through the built Kinect abnormal action recognition data acquisition system, the real-time skeleton data of the human body in the scene is obtained and put into the cache; each frame of data is composed of a skeleton point sequence based on a collection of Joint types. The Joint type data includes: JonitType skeleton point type, that is, the bone name; Position bone position information, that is, X, Y, Z three-dimensional coordinates, to obtain the Kinect skeleton data of each human body.
[0042] In one embodiment, based on the principles of time synchronization and space synchronization, the three-dimensional coordinate features of each node in the acquired human skeleton sequence are fused with the human weight recognition features through a feature fusion model, specifically including the following steps:
[0043] Step 11: Unify and integrate the coordinate system of the Kinect device and the coordinate system of the video surveillance device to achieve spatial fusion of the video data collected by the Kinect device and the video surveillance device.
[0044] In this embodiment, in order to ensure that data processing is simpler and more accurate and the recognition error is smaller, the setting distance between the Kinect device and the video surveillance device should not be too large, and the preferred setting distance difference between the two devices is 1-10m.
[0045] Step 12, using multi-threaded processing to achieve time synchronization between the Kinect skeleton data collected by the Kinect device and the video stream data of each human body captured in real time by the video surveillance device, so as to achieve time fusion of the video data collected by the Kinect device and the video surveillance device; wherein, the video frames of the two video data can be aligned in time according to the time frame.
[0046] Step 13, associating the video data processed by step 11 and step 12 with the GIS data, establishing a tracking and positioning map of the human body to be identified, and performing video tracking of the human body through the target detection model by the video monitoring device.
[0047] In this embodiment, since the coordinate system used by the Kinect device is different from the coordinate system of the video surveillance device, the two need to be fused in the recognition method, so that the data of the Kinect device and the data collected by the video surveillance device are placed in the same coordinate system and displayed on the same graph, so that the user can understand the recognition situation more conveniently and intuitively, that is, to achieve spatial fusion of data.
[0048] In this embodiment, the three-dimensional coordinates of each node in the acquired human skeleton sequence are fused with the human weight recognition feature through the feature fusion model, which specifically includes the following steps:
[0049] The three-dimensional coordinates of each node in the acquired human skeleton sequence and the human body weight identification features are fused using an image fusion algorithm or an image fusion neural network. The feature fusion model is a pre-trained neural network, and the fusion result is set with a corresponding human body unique identification label, which can use the human body unique identification generated during the human body detection process of the target detection model.
[0050] Device Embodiment
[0051] According to an embodiment of the present invention, a device for identifying abnormal human movements is provided. Figure 2 As shown in FIG. 1 , a block diagram of a human body abnormal motion recognition device provided in this embodiment is provided. The human body abnormal motion recognition device according to an embodiment of the present invention comprises:
[0052] The video acquisition module 10 is used to acquire video streams and Kinect skeleton data in real time, wherein the Kinect skeleton data includes a human skeleton sequence and three-dimensional coordinate features of each skeleton point.
[0053] The first video data processing module 20 is used to realize each human body region frame and human body identification through the target detection model based on the video stream, and convert the detection result of the target detection model into a human body weight recognition feature through the human body weight recognition network.
[0054] The second video data processing module 30 is used to fuse the three-dimensional coordinate features of each node in the acquired human skeleton sequence with the human body weight recognition features through a feature fusion model based on the principles of time synchronization and space synchronization, so as to obtain a fused feature image sequence with human body identification.
[0055] The abnormal action prediction module 40 is used to input the fused feature image sequence into the abnormal action recognition model, determine whether the human behavior characteristics are abnormal behaviors based on the changes in the human skeleton points and the preset change threshold, and determine the abnormal behavior type according to the abnormal behavior category library matching; if abnormal behavior is recognized, the alarm system is triggered, wherein the initial abnormal action recognition model is a neural network model obtained by training the abnormal behavior video stream, wherein the abnormal behavior video stream data is a video stream collected by the Kinect device, and the neural network model recognizes and classifies abnormal behaviors according to the skeleton points of continuous actions in the video stream and the preset change thresholds of each skeleton point, and obtains an abnormal behavior category library composed of the skeleton point change situation and the abnormal behavior type, wherein the skeleton point change threshold includes a position change threshold and a motion speed change threshold.
[0056] In the apparatus for identifying abnormal human motion provided by the present embodiment, the video acquisition module 10 acquires real-time video data of the human body through a video monitoring device; the first video data processing module 20 processes the real-time video data and numbers each human body; based on the Kinect skeleton data acquired in real time by the Kinect device, the second video data processing module 30 fuses the three-dimensional coordinates of each node in the acquired human skeleton sequence with the human body weight recognition feature through a feature fusion model based on the principles of time synchronization and space synchronization; the abnormal motion prediction module 40 determines the abnormal behavior type by matching the changes of the skeleton points detected by the abnormal motion recognition model with the abnormal behavior type set; when the abnormal human behavior is identified, the alarm system is triggered, and the video clip and related information of the behavior are recorded; the method of the present embodiment makes full use of continuous video image information, and can effectively identify and analyze the human behavior through human body weight recognition, posture estimation and behavior recognition technology, and can also avoid the problem of high false alarm rate of human behavior analysis based on a single image, thereby improving the accuracy and practicality of human behavior analysis.
[0057] In this embodiment, during the training process of the initial neural network model, the coordinates of the skeleton point sequence in the video stream according to the initial frame are used as the initial coordinates, and then the change information of the skeleton point sequence is determined according to each video frame in the video stream, including determining whether the position has changed, the change distance, and the change speed. According to the preset change distance threshold and change speed threshold of each skeleton point, as well as the change distance and change speed, the corresponding abnormal behavior video is classified into the corresponding abnormal behavior data set according to the abnormal behavior type label, wherein the abnormal behavior qualitative characterization of the abnormal behavior data set is to determine which abnormal behavior the behavior in the corresponding video stream is through the change distance and change speed of the skeleton points, because different abnormal behaviors, such as fighting and falling, or normal walking, the change distance and change speed of the human skeleton points are definitely different, and the two parameters are used to more accurately represent the changes in the abnormal behavior of the human body, which will greatly improve the accuracy of abnormal behavior action recognition in the process of human action recognition and reduce the probability of false alarms.
[0058] This embodiment is also provided with a threshold setting module 40, which is used to set the threshold for the occurrence of various types of abnormal behaviors. According to the threshold for the occurrence of corresponding abnormal behaviors, the type of abnormal behavior and the number of occurrences determined by the abnormal action recognition model are compared with the threshold for the occurrence of corresponding abnormal behaviors. If they are equal, the occurrence of the corresponding abnormal behavior is determined and an alarm is issued.
[0059] In this embodiment, the second video data processing module 30 is configured to implement the following steps:
[0060] Step 21: Unify and integrate the coordinate system of the Kinect device and the coordinate system of the video surveillance device to achieve spatial fusion of the video data collected by the Kinect device and the video surveillance device.
[0061] In this embodiment, in order to ensure that data processing is simpler and more accurate and the recognition error is smaller, the setting distance between the Kinect device and the video surveillance device should not be too large, and the preferred setting distance difference between the two devices is 1-10m.
[0062] Step 22, using multi-threaded processing to achieve time synchronization between the Kinect skeleton data collected by the Kinect device and the video stream data of each human body captured in real time by the video surveillance device, so as to achieve time fusion of the video data collected by the Kinect device and the video surveillance device; wherein, the video frames of the two video data can be aligned in time according to the time frame.
[0063] Step 23, associating the video data processed by step 21 and step 22 with the GIS data, establishing a tracking and positioning map of the human body to be identified, and performing video tracking of the human body through the target detection model by the video monitoring device.
[0064] The embodiment of the present invention is an apparatus embodiment corresponding to the above-mentioned method embodiment. The specific operations of the processing steps of each module can be understood by referring to the description of the method embodiment, which will not be repeated here.
[0065] like Figure 3 As shown, the present invention further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the method for identifying abnormal human movements in the above embodiment is implemented; or when the computer program is executed by a processor, the method for identifying abnormal human movements in the above embodiment is implemented. When the computer program is executed by the processor, the following method steps are implemented:
[0066] Step S1, collecting video streams in real time through video surveillance equipment, and collecting Kinect skeleton data in real time through a Kinect device, wherein the Kinect skeleton data includes a human skeleton sequence and three-dimensional coordinate features of each skeleton point.
[0067] Step S2, based on the video stream, realizes each human body region frame and human body identification through the target detection model, and converts the detection result of the target detection model into human body weight recognition features through the human body weight recognition network.
[0068] Step S3, based on the principle of time synchronization and space synchronization, the three-dimensional coordinates of each node in the acquired human skeleton sequence are fused with the human body weight recognition feature through a feature fusion model to obtain a fused feature image sequence with human body identification;
[0069] Step S4, inputting the fused feature image sequence into the abnormal action recognition model, determining whether the human behavior feature is abnormal behavior based on the change of human skeleton points and the preset change threshold, and determining the abnormal behavior type according to the abnormal behavior matching the abnormal behavior type set; if abnormal behavior is recognized, triggering the alarm system, wherein the initial abnormal action recognition model is a neural network model trained by the abnormal behavior video stream including Kinect skeleton data information, the neural network model recognizes and classifies abnormal behavior according to the skeleton points of continuous actions in the video stream and the preset change threshold of each skeleton point, and obtains an abnormal behavior category library composed of the skeleton point change situation and the abnormal behavior type, wherein the skeleton point change threshold includes a position change threshold and a motion speed change threshold.
[0070] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
[0071] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The device and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.
[0072] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein by equivalents. These modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention, and the contents not described in detail in the specification of the present invention belong to the common knowledge of those skilled in the art.
Claims
1. A method for identifying abnormal human movements. Features Includes steps: The video stream is collected in real time by a video monitoring device, and the Kinect skeleton data is collected in real time by a skeleton point collection device, wherein the Kinect skeleton data includes a human skeleton sequence and three-dimensional coordinate features of each skeleton point; Based on the video stream, the target detection model is used to realize the human body region frame and human body identification, and the human body weight recognition network is used to convert the detection results of the target detection model into human body weight recognition features; Based on the principle of time synchronization and space synchronization, the three-dimensional coordinate features of each node in the human skeleton sequence are fused with the human weight recognition features through the feature fusion model to obtain a fused feature image sequence with human body identification. as well as The fused feature image sequence is input into the abnormal action recognition model, and whether the human behavior characteristics are abnormal behaviors is determined based on the changes in the human skeleton points and the preset change threshold, and the abnormal behavior type is determined according to the abnormal behavior category library; if abnormal behavior is recognized, the alarm system is triggered, wherein the initial abnormal action recognition model is a neural network model obtained by training the abnormal behavior video stream, wherein the abnormal behavior video stream data is a video stream collected by the skeleton point acquisition device, and the neural network model recognizes and classifies abnormal behaviors according to the skeleton points of continuous actions in the video stream and the preset skeleton point change threshold learning, and obtains the abnormal behavior category library composed of the skeleton point change situation and the abnormal behavior type, wherein the skeleton point change threshold includes the position change threshold and the movement speed change threshold.
2. The method for identifying abnormal human movements according to claim 1, It is characterized in that During the training process, the initial neural network model uses the initial frame coordinates of the skeleton point sequence in the video stream as the initial coordinates, and then identifies and determines the change information of the skeleton point sequence in each video frame. The change information includes whether the position of the skeleton point has changed, the position change distance, and the change speed. According to the preset change distance threshold and change speed threshold of each skeleton point, as well as the change distance and change speed, the corresponding abnormal behavior video is classified into the corresponding abnormal behavior data set according to the abnormal behavior type label.
3. The method for identifying abnormal human movements according to claim 1, It is characterized in that Also includes the steps: Set the threshold for the number of occurrences of various types of abnormal behaviors. According to the abnormal behavior type and number of occurrences determined by the abnormal action recognition model, compare the number of occurrences with the corresponding threshold for the number of occurrences of abnormal behaviors. If they are equal, an alarm will be issued.
4. The method for identifying abnormal human movements according to claim 1, It is characterized in that The acquisition of the Kinect skeleton data is specifically achieved through the following steps: Through the built Kinect abnormal motion recognition data acquisition system, the skeleton data of the human body in the scene is acquired and stored in real time; each frame of data is composed of a skeleton point sequence based on a Joint type set. The Joint type data contains the skeleton point type and the skeleton position information, so as to obtain the Kinect skeleton data of each human body.
5. The method for identifying abnormal human movements according to claim 1, It is characterized in that The feature fusion model fusion result sets a corresponding human unique identification label, and the unique identification label uses the human unique identification generated during the human body detection process of the target detection model.
6. A device for identifying abnormal human movements, Features include: The video acquisition module is used to acquire video streams and Kinect skeleton data in real time, wherein the Kinect skeleton data includes the human skeleton sequence and the three-dimensional coordinate features of each skeleton point; The first video data processing module is used to realize the human body region frame and human body identification through the target detection model based on the video stream, and convert the detection result of the target detection model into human body weight recognition features through the human body weight recognition network; The second video data processing module is used to fuse the three-dimensional coordinate features of each node in the acquired human skeleton sequence with the human body weight recognition features through a feature fusion model based on the principles of time synchronization and space synchronization, and obtain a fused feature image sequence with human body identification; as well as The abnormal action prediction module is used to input the fused feature image sequence into the abnormal action recognition model, determine whether the human behavior characteristics are abnormal behaviors based on the changes in the human skeleton points and the preset change thresholds, and determine the abnormal behavior types according to the abnormal behavior category library; if abnormal behavior is recognized, the alarm system is triggered, wherein the initial abnormal action recognition model is a neural network model obtained by training the abnormal behavior video stream, wherein the abnormal behavior video stream data is the video stream collected by the video acquisition module, and the neural network model recognizes and classifies abnormal behaviors according to the skeleton points of continuous actions in the video stream and the preset change thresholds of each skeleton point, and obtains an abnormal behavior category library composed of the skeleton point changes and abnormal behavior types, wherein the skeleton point change thresholds include position change thresholds and motion speed change thresholds.
7. The device for identifying abnormal human movements according to claim 6, It is characterized in that During the training process, the initial neural network model uses the initial frame coordinates of the skeleton point sequence in the video stream as the initial coordinates, and then identifies and determines the change information of the skeleton point sequence in each video frame. The change information includes whether the position of the skeleton point has changed, the position change distance, and the change speed. According to the preset change distance threshold and change speed threshold of each skeleton point, as well as the change distance and change speed, the corresponding abnormal behavior video is classified into the corresponding abnormal behavior data set according to the abnormal behavior type label.
8. The device for identifying abnormal human movements as claimed in claim 7, It is characterized in that It also includes a threshold setting module, which is used to set the threshold of the number of occurrences of various types of abnormal behaviors. According to the abnormal behavior type and number of occurrences determined by the abnormal action recognition model, the number of occurrences is compared with the corresponding abnormal behavior number of occurrence thresholds. If they are equal, an alarm is issued.
9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the computer program, the method for identifying abnormal human movements as described in any one of claims 1 to 5 is implemented.
10. A computer-readable storage medium storing a computer program. It is characterized in that When the computer program is executed by a processor, the method for identifying abnormal human movements as described in any one of claims 1 to 5 is implemented.
Citation Information
Patent Citations
Human body movement identification method based on Kinect
CN103529944A
Behavior analysis method, device, equipment and system based on video, and storage medium
CN108875708A
Machine vision human body abnormal behavior recognition method based on multi-feature fusion
CN110633624A
Abnormal behavior detection system and method for monitoring old people living alone
CN115116133A
Physical education teaching test method and system based on human body posture estimation
CN116503954A
Cited By
AI-based factory abnormal behavior identification monitoring method and system
CN120088737A
