Fighting behavior recognition method, device and electronic equipment
By using the target extraction and posture recognition network in the security monitoring system combined with the pedestrian tracking model, the fighting behavior in the crowded scenes is identified, and the problems of missed detection and false alarms in the existing technology are solved, and the high-accurate fighting behavior recognition is achieved and prompt warning.
Patent Information
- Application Number
- CN202211043986.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-08-30
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2042-08-30
AI Technical Summary
The existing security monitoring system is difficult to identify fighting behavior in a timely manner in crowded scenarios, and there are problems of missed detection and false alarms, especially when pedestrian postures change in various ways and there may be occlusion.
A method of fighting behavior recognition is adopted to extract pedestrian outlines by obtaining images and inputting them into the target extraction network, and obtaining key point coordinates and pose categories with the posture recognition network. The pedestrian tracking model is used to calculate the displacement velocity and direction of key point, and combining distance Measure whether a fight occurs and issue a warning.
It effectively improves the accuracy of identification of fighting behaviors, reduces false alarms and missed reports, and is suitable for public scenes with dense crowds, and can accurately identify fighting behaviors and issue warnings in a timely manner.
Smart Images

Figure CN115424341B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image recognition technology, and in particular to a fighting behavior recognition method, device and electronic equipment. Background Art
[0002] With the development of technology and the advancement of science and technology, my country has made qualitative breakthroughs in key monitoring technologies such as artificial intelligence perception, machine learning, 5G communications, the Internet of Things, cloud computing and edge computing, and video optoelectronic technology. The security industry has become the most important application field for the deep integration of artificial intelligence and the real economy. Leading companies have deployed high-end artificial intelligence algorithms, and AI cameras with artificial intelligence algorithm analysis have gradually occupied the market.
[0003] In existing security monitoring systems, abnormal situations are usually reported through instant feedback from monitoring personnel, but the manual monitoring method has the disadvantages of not being able to guarantee immediacy and wasting a lot of human resources. In crowded public scenes such as rail transit, the flow of people is large. If a fight occurs and the alarm cannot be given in time, it will cause traffic congestion and even stampede.
[0004] Although there are some existing methods that use deep learning and image processing technology to detect fighting behavior, these methods are all traditional pedestrian detection-based solutions. Pedestrians are non-rigid objects, especially during a fight, whose postures change in various ways, have high posture freedom, and may be obstructed. This detection method is prone to missed detections. Summary of the invention
[0005] The purpose of the present invention is to provide a fighting behavior recognition method, which is suitable for public scenes with dense crowds and can effectively improve the accuracy of recognition and reduce false positives and missed positives.
[0006] To achieve the above object, the present invention first provides a fighting behavior recognition method, comprising the following steps:
[0007] Step S1, obtaining an image to be detected, inputting the image to be detected into a preset target extraction network, and extracting the contour image of each target pedestrian;
[0008] Step S2, inputting the outline image of the target pedestrian into a preset posture recognition network to obtain the key point coordinates and posture category of each target pedestrian;
[0009] Step S3, tracking the target pedestrian through a preset pedestrian tracking model, and calculating the displacement speed and displacement direction of the key points of the target pedestrian according to the changes in the coordinates of the key points of the target pedestrian during the tracking process;
[0010] Step S4, determining whether the target pedestrian is in a fighting action according to a comparison result between the displacement speed of the key point of the target pedestrian and a preset speed threshold, a pointing relationship between the displacement direction of the key point of the target pedestrian and other target pedestrians, and an area overlap relationship between the key point of the target pedestrian and other target pedestrians;
[0011] Step S5, recording the number of occurrences of the target pedestrian's fighting actions during the tracking process and the number of times the target pedestrian's posture category switches during the tracking process, and calculating the distance between each target pedestrian;
[0012] Step S6: When the distance between a target pedestrian and another target pedestrian is less than a preset distance threshold, and the number of fighting actions of the target pedestrian during the tracking process is greater than a preset action threshold and the number of times the target pedestrian switches the posture category during the tracking process is greater than a preset switching threshold, a fighting warning is issued.
[0013] Optionally, the step S3 of calculating the displacement speed and displacement direction of the key points of the target pedestrian according to the change of the key point coordinates of the target pedestrian during the tracking process specifically includes: calculating the displacement speed and displacement direction of the key points of the target pedestrian according to the change of the key point coordinates of the target pedestrian in the current frame and the image n frames before the current frame, where n is a positive integer;
[0014] The step S4 specifically includes:
[0015] Determine whether the displacement speed of the key point of the target pedestrian exceeds a preset speed threshold, whether the displacement direction points to another target pedestrian, and whether the key point of the target pedestrian overlaps with another target pedestrian;
[0016] When the displacement speed of the key point of the target pedestrian exceeds a preset speed threshold and the displacement direction points to another target pedestrian and the key point of the target pedestrian overlaps with the appearance area of another target pedestrian, it is determined that the target pedestrian has performed a fighting action.
[0017] Optionally, the calculating of the displacement speed of the key points of the target pedestrian according to the change of the coordinates of the key points of the target pedestrian in the current frame and the image n frames before the current frame and the judging whether the displacement speed of the key points of the target pedestrian exceeds a preset speed threshold specifically include:
[0018] Calculate the displacement speed of multiple key points of the target pedestrian according to a preset first speed calculation formula;
[0019] Calculating and determining the maximum displacement speed and the average displacement speed among the displacement speeds of the plurality of key points;
[0020] When the maximum displacement speed among the displacement speeds of the multiple key points exceeds a preset maximum speed threshold and the average displacement speed exceeds a preset average speed threshold, it is determined that the displacement speed of the key points of the target pedestrian exceeds the preset speed threshold.
[0021] Optionally, the first speed calculation formula is:
[0022]
[0023] Among them, V i represents the displacement velocity of the i-th key point, and is the horizontal and vertical coordinates of the i-th key point in the current frame, and is the horizontal and vertical coordinates of the i-th key point in the image n frames before the current frame, t is the interval between the current frame and the image n frames before the current frame, and i is a positive integer.
[0024] Optionally, the multiple key points of the target pedestrian include: a right wrist key point, a left wrist key point, a right leg ankle key point, and a left leg ankle key point.
[0025] Optionally, the formula for calculating the distance between each target pedestrian in step S5 is:
[0026]
[0027] in, are the horizontal and vertical coordinates of the hip joint midpoint of the j-th target pedestrian, are the horizontal and vertical coordinates of the hip joint midpoint of the kth target pedestrian, is the distance between the jth target pedestrian and the kth target pedestrian, where j and k are both positive integers.
[0028] Optionally, the step S1 specifically includes:
[0029] The face frames of the target pedestrians and the alignment parameters of the target pedestrians are obtained by identifying the target extraction network;
[0030] Establishing a contour map of each target pedestrian according to the face frame of the target pedestrian and the alignment parameters of the target pedestrian; the alignment parameters of each target pedestrian specifically include: midpoint parameters of the face, midpoint parameters of the shoulders, midpoint parameters of the hip joints, and parameters of the circumscribed circle of the human body;
[0031] The step S2 specifically includes: identifying the key point coordinates of each target pedestrian according to the contour map of the target pedestrian;
[0032] The posture of the target pedestrian is classified according to the key point coordinates of the target pedestrian and the contour map of the target pedestrian to obtain the posture category of the target pedestrian; the posture category includes fighting posture and normal posture.
[0033] Optionally, in step S5, each time the target pedestrian switches from a normal posture to a fighting posture or from a fighting posture to a normal posture during the tracking process, it is considered that the target pedestrian has completed a switch in posture category during the tracking process.
[0034] The present invention also provides a fighting behavior recognition device, comprising the following steps:
[0035] An extraction unit, used to obtain an image to be detected, input the image to be detected into a preset target extraction network, and extract the contour image of each target pedestrian;
[0036] A recognition unit, used to input the outline of the target pedestrian into a preset posture recognition network to obtain the key point coordinates and posture category of each target pedestrian;
[0037] A first calculation unit is used to track the target pedestrian through a preset pedestrian tracking model, and calculate the displacement speed and displacement direction of the key points of the target pedestrian according to the changes in the key point coordinates of the target pedestrian during the tracking process;
[0038] A behavior judgment unit is used to determine whether the target pedestrian has a fighting action based on a comparison result between the displacement speed of the key point of the target pedestrian and a preset speed threshold, a pointing relationship between the displacement direction of the key point of the target pedestrian and other target pedestrians, and an area overlap relationship between the key point of the target pedestrian and other target pedestrians;
[0039] A behavior recording unit is used to record the number of fighting actions of the target pedestrian during the tracking process and the number of times the target pedestrian switches its posture category during the tracking process, and calculate the distance between each target pedestrian;
[0040] An alarm unit is used to issue a fighting warning when the distance between a target pedestrian and another target pedestrian is less than a preset distance threshold, and the number of fighting actions of the target pedestrian during the tracking process is greater than a preset action threshold and the number of times the posture category of the target pedestrian switches during the tracking process is greater than a preset switching threshold.
[0041] In addition, the present invention also provides an electronic device, including: a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the above method.
[0042] Beneficial effects of the invention: The invention provides a fighting behavior recognition method, device and electronic device, the method comprising the following steps: acquiring an image to be detected, inputting the image to be detected into a preset target extraction network, extracting a contour map of each target pedestrian; inputting the contour map of the target pedestrian into a preset posture recognition network, obtaining the key point coordinates and posture category of each target pedestrian; tracking the target pedestrian through a preset pedestrian tracking model, and calculating the key point displacement speed and displacement direction of the target pedestrian according to the change of the key point coordinates of the target pedestrian during the tracking process; comparing the key point displacement speed of the target pedestrian with a preset speed threshold, the directional relationship between the key point displacement direction of the target pedestrian and other target pedestrians, and the key point of the target pedestrian and its The overlapping relationship between the areas of the target pedestrians and other target pedestrians is used to determine whether the target pedestrian shows fighting behavior; the number of fighting behaviors of the target pedestrians during the tracking process and the number of times the target pedestrians switch their posture categories during the tracking process are recorded, and the distance between each target pedestrian is calculated; when the distance between a target pedestrian and another target pedestrian is less than a preset distance threshold and the number of fighting behaviors of the target pedestrian during the tracking process is greater than a preset action threshold and the number of times the target pedestrians switch their posture categories during the tracking process is greater than a preset switching threshold, a fighting warning is issued. By accurately identifying the key point coordinates and posture categories of the target pedestrians and calculating the key point displacement speed, the combination of key point coordinates, posture categories and key point displacement speed is used to complete the fighting behavior recognition, which can effectively improve the recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] In order to further understand the features and technical contents of the present invention, please refer to the following detailed description and drawings of the present invention. However, the drawings are only provided for reference and illustration and are not intended to limit the present invention.
[0044] In the accompanying drawings,
[0045] Figure 1 is a flow chart of the fighting behavior identification method of the present invention;
[0046] Figure 2 and Figure 3 A flowchart of fighting action judgment in the fighting behavior recognition method of the present invention;
[0047] Figure 4 A schematic diagram of key points in the fighting behavior identification method of the present invention;
[0048] Figure 5 to Figure 6 A schematic diagram of an embodiment of a fighting behavior recognition method of the present invention;
[0049] Figures 7 and 8 A schematic diagram of another embodiment of the fighting behavior recognition method of the present invention;
[0050] Fig. 9 is a schematic diagram of a fighting behavior recognition device of the present invention;
[0051] Fig.10 Schematic diagram of an electronic device of the present invention. DETAILED DESCRIPTION
[0052] To further illustrate the technical means and effects of the present invention, the following is a detailed description in conjunction with the preferred embodiments of the present invention and the accompanying drawings.
[0053] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.
[0054] In the description of the present application, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, which are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present application. In addition, the terms "first" and "second" are used only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more features. In the description of the present application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.
[0055] In this application, the word "exemplary" is used to mean "used as an example, illustration, or description." Any embodiment described in this application as "exemplary" is not necessarily to be construed as being preferred or advantageous over other embodiments. The following description is given to enable any technician in the field to implement and use the present application. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present application can be implemented without using these specific details. In other instances, well-known structures and processes will not be elaborated in detail to avoid obscuring the description of the present application with unnecessary details. Therefore, the present application is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in the present application.
[0056] See also Figure 1 The present invention provides a fighting behavior recognition method, comprising the following steps:
[0057] Step S1: Acquire an image to be detected, input the image to be detected into a preset target extraction network, and extract the contour of each target pedestrian.
[0058] Specifically, the step S1 includes: obtaining the face frame of each target pedestrian and the alignment parameters of each target pedestrian through the target extraction network recognition;
[0059] Establishing the contour map of each target pedestrian according to the face frame of the target pedestrian and the alignment parameters of the target pedestrian;
[0060] The alignment parameters of each target pedestrian specifically include: midpoint parameters of the face, midpoint parameters of both shoulders, midpoint parameters of the hip joints, and parameters of the circumscribed circle of the human body.
[0061] In detail, in some embodiments of the present invention, the step S1 specifically includes: reading a video stream, obtaining an image frame, wherein the image frame is the image to be detected, and inputting the image to be detected into a preset target extraction network. Preferably, the target extraction network is a BlazeFace deep neural network, and then the face frame of each target pedestrian in the image to be detected and the alignment parameters of the corresponding pedestrian are obtained through the target extraction network. In a typical embodiment of the present invention, the alignment parameters include: the midpoint parameters of the face, the midpoint parameters of the shoulders, the midpoint parameters of the hip joints, and the parameters of the circumscribed circle of the human body. Finally, the face frame of the target pedestrian detected by the target extraction network and the alignment parameters of the corresponding pedestrian are used to construct a contour map of the target pedestrian.
[0062] It should be noted that since the key point detection network (i.e., posture recognition network) used subsequently is a technology designed to realize real-time single-person key point detection, and the fighting scenes targeted by the present invention are all multi-person scenes, the present invention utilizes the detected face frame of the target pedestrian and the alignment parameters of the corresponding pedestrian to construct a contour map of the target pedestrian, and then extracts the human body and inputs it into the key point detection network to extract key point information, thereby realizing real-time key point detection technology for multiple people.
[0063] Step S2, inputting the outline image of the target pedestrian into a preset posture recognition network to obtain the key point coordinates and posture category of each target pedestrian;
[0064] Specifically, the step S2 specifically includes:
[0065] Identify the key point coordinates of each target pedestrian according to the contour map of the target pedestrian;
[0066] The posture of the target pedestrian is classified according to the key point coordinates of the target pedestrian and the contour map of the target pedestrian to obtain the posture category of the target pedestrian; the posture category includes fighting posture and normal posture.
[0067] It should be noted that if Figure 4 As described, in some embodiments of the present invention, the posture recognition network outputs 14 key point information. When the pedestrian is subsequently tracked, the 14 key point information of the previous frame can be used to construct a human body external frame and input into the next frame image to help obtain the key point information of the next frame image, thereby achieving effective human body tracking. When the target pedestrian is lost, the process returns to step S1 and restarts a cycle.
[0068] Furthermore, the posture recognition network of the present invention is an improved BlazePose network. The traditional BlazePose network has only one key point branch in the reasoning phase output, while the present invention adds a classification branch to the network structure of BlazePose to obtain the posture recognition network of the present invention. The classification branch is used to distinguish whether the human body is in a fighting posture. At the same time, the classification branch uses key point information to assist classification, which improves the classification accuracy without increasing the reasoning time consumption of the model too much.
[0069] Step S3: Combination Figures 5 to 8 The target pedestrian is tracked by a preset pedestrian tracking model, and the displacement speed and displacement direction of the key points of the target pedestrian are calculated according to the changes of the key point coordinates of the target pedestrian during the tracking process.
[0070] Specifically, Figure 2 As shown, in some embodiments of the present invention, the step S3 determines the target pedestrian according to the change of the key point coordinates of the target pedestrian during the tracking process and calculates the displacement speed and displacement direction of the key points of the target pedestrian, which specifically includes:
[0071] The displacement speed and displacement direction of the key points of the target pedestrian are calculated according to the change in the coordinates of the key points of the target pedestrian in the current frame and the n frames before the current frame, where n is a positive integer.
[0072] Furthermore, if Figure 4 As shown, the step S3 calculates the displacement speed of the key points of the target pedestrian according to the change of the key point coordinates of the target pedestrian in the current frame and the n frames before the current frame, and determines whether the displacement speed of the key points of the target pedestrian exceeds the preset speed threshold, specifically including:
[0073] Calculate the displacement speed of multiple key points of the target pedestrian according to a preset first speed calculation formula;
[0074] Calculating and determining the maximum displacement speed and the average displacement speed among the displacement speeds of the plurality of key points;
[0075] When the maximum displacement speed among the displacement speeds of the multiple key points exceeds a preset maximum speed threshold and the average displacement speed exceeds a preset average speed threshold, it is determined whether the displacement speed of the key points of the target pedestrian exceeds a preset speed threshold.
[0076] It should be noted that in general fighting, only punching and kicking are involved, which is reflected in the movement of the key points of the hands and legs. The position information of the key points of the hands and legs in the current frame is recorded. Using the tracking information, the key point information of the hands and legs recorded in the current frame is calculated for displacement with the corresponding key point information of the same target pedestrian in the detection area of the previous n frames of images, and the displacement speed of the key points can be obtained. The specific calculation formula is the following first speed calculation formula:
[0077] ;
[0078] Among them, V i represents the displacement velocity of the i-th key point, and is the horizontal and vertical coordinates of the i-th key point in the current frame, and is the horizontal and vertical coordinates of the i-th key point in the image n frames before the current frame, t is the interval between the current frame and the image n frames before the current frame, and i is a positive integer.
[0079] Since fighting behavior only needs to focus on the key points of the hands and feet, the present invention selects the right wrist key point 5, the left wrist key point 8, the right leg ankle key point 11 and the left leg ankle key point 14 among the above 14 key points to calculate the displacement speed of the key points, and calculates the maximum displacement speed of these key points and the average displacement velocity , calculated as follows:
[0080] ;
[0081] ;
[0082] in, They respectively represent the key point displacement speeds of the right hand wrist key point 5, the left hand wrist key point 8, the right leg ankle key point 11 and the left leg ankle key point 14.
[0083] When the maximum displacement speed exceeds a preset maximum speed threshold and the average displacement speed exceeds a preset average speed threshold, it is determined whether the key point displacement speed of the target pedestrian exceeds a preset speed threshold.
[0084] If the key point coordinates of the target pedestrian move towards another target pedestrian between the current frame and the image n frames before the current frame, it is considered that the displacement direction of the key point of the target pedestrian points to the other target pedestrian.
[0085] Step S4, determining whether the target pedestrian is in a fighting action based on the comparison result between the displacement speed of the key point of the target pedestrian and the preset speed threshold, the directional relationship between the displacement direction of the key point of the target pedestrian and other target pedestrians, and the regional overlap relationship between the key point of the target pedestrian and other target pedestrians.
[0086] Specifically, in step S4, first, it is determined whether the displacement speed of the key point of the target pedestrian exceeds a preset speed threshold, whether the displacement direction points to another target pedestrian, and whether the key point of the target pedestrian overlaps with another target pedestrian;
[0087] When the displacement speed of the key point of the target pedestrian exceeds a preset speed threshold and the displacement direction points to another target pedestrian and the key point of the target pedestrian overlaps with the appearance area of another target pedestrian, it is determined that the target pedestrian has performed a fighting action.
[0088] Step S5, recording the number of occurrences of the target pedestrian's fighting actions during the tracking process and the number of times the target pedestrian's posture category switches during the tracking process, and calculating the distance between each target pedestrian;
[0089] Specifically, the formula for calculating the distance between each target pedestrian in step S5 is:
[0090]
[0091] in, are the horizontal and vertical coordinates of the hip joint midpoint of the j-th target pedestrian, are the horizontal and vertical coordinates of the hip joint midpoint of the kth target pedestrian, is the distance between the jth target pedestrian and the kth target pedestrian, where j and k are both positive integers.
[0092] Furthermore, the posture category includes a normal posture and a fighting posture. Each time the target pedestrian switches from a normal posture to a fighting posture or from a fighting posture to a normal posture during the tracking process, it is considered that the posture category of the target pedestrian has completed a switch during the tracking process.
[0093] Step S6: When the distance between a target pedestrian and another target pedestrian is less than a preset distance threshold, and the number of fighting actions of the target pedestrian during the tracking process is greater than a preset action threshold and the number of times the target pedestrian switches the posture category during the tracking process is greater than a preset switching threshold, a fighting warning is issued.
[0094] It should be noted that, since fighting is a continuous behavior, it is very easy to misdetect daily behaviors by only counting one effective fighting action or one posture category switch. Therefore, in the present invention, an action threshold for the occurrence of fighting actions and a switching threshold for posture category switching are set. When the number of fighting actions of the target pedestrian during the tracking process is greater than the preset action threshold and the number of posture category switches of the target pedestrian during the tracking process is greater than the preset switching threshold, an alarm will be triggered, thereby effectively avoiding false alarms.
[0095] It is worth mentioning that the fighting behavior recognition method of the present invention is applicable to both fights between two people and group fights between multiple people.
[0096] Finally, during a fight, pedestrians have various posture changes, high posture freedom, and possible occlusion. The traditional key point detection method based on pedestrian detection is prone to missed detection. The present invention realizes key point detection of multiple people and solves the problem of ID tracking by combining the BlazeFace and BlazePose models. This method is conducive to improving missed detection and model reasoning speed, and further creatively uses the displacement speed of key points of continuous frames to distinguish fighting behavior. At the same time, it combines the switching of fighting posture categories and the displacement speed of key points to detect fighting behavior, which can effectively improve the accuracy of the algorithm and effectively improve the accuracy of recognition.
[0097] See also Fig. 9 , a fighting behavior recognition device, comprising the following steps:
[0098] The extraction unit 10 is used to obtain the image to be detected, input the image to be detected into a preset target extraction network, and extract the contour image of each target pedestrian;
[0099] The recognition unit 20 is used to input the outline of the target pedestrian into a preset posture recognition network to obtain the key point coordinates and posture category of each target pedestrian;
[0100] A first calculation unit 30 is used to track the target pedestrian through a preset pedestrian tracking model, and calculate the displacement speed and displacement direction of the key points of the target pedestrian according to the changes in the key point coordinates of the target pedestrian during the tracking process;
[0101] The behavior judgment unit 40 is used to determine whether the target pedestrian has a fighting action according to the comparison result between the displacement speed of the key point of the target pedestrian and the preset speed threshold, the pointing relationship between the displacement direction of the key point of the target pedestrian and other target pedestrians, and the regional overlap relationship between the key point of the target pedestrian and other target pedestrians;
[0102] A behavior recording unit 50 is used to record the number of occurrences of fighting actions of the target pedestrian during the tracking process and the number of switching of the posture category of the target pedestrian during the tracking process, and calculate the distance between each target pedestrian;
[0103] The alarm unit 60 is used to issue a fighting warning when the distance between a target pedestrian and another target pedestrian is less than a preset distance threshold, and the number of fighting actions of the target pedestrian during the tracking process is greater than a preset action threshold and the number of times the posture category of the target pedestrian switches during the tracking process is greater than a preset switching threshold.
[0104] See also Fig.10 The present invention provides an electronic device, comprising: a memory 200 and a processor 100, wherein the memory 200 stores a computer program, and when the computer program is executed by the processor 100, the processor 100 executes the steps of the above method.
[0105] In summary, the present invention provides a fighting behavior recognition method, device and electronic device, the method comprising the following steps: acquiring an image to be detected, inputting the image to be detected into a preset target extraction network, and extracting a contour map of each target pedestrian; inputting the contour map of the target pedestrian into a preset posture recognition network, and obtaining the key point coordinates and posture category of each target pedestrian; tracking the target pedestrian through a preset pedestrian tracking model, and calculating the key point displacement speed and displacement direction of the target pedestrian according to the changes in the key point coordinates of the target pedestrian during the tracking process; and comparing the key point displacement speed of the target pedestrian with a preset speed threshold, the directional relationship between the key point displacement direction of the target pedestrian and other target pedestrians, and the key point of the target pedestrian and other target pedestrians. The overlapping relationship between the areas of the target pedestrians is used to determine whether the target pedestrian has a fighting action; the number of fighting actions of the target pedestrian during the tracking process and the number of times the target pedestrian's posture category is switched during the tracking process are recorded, and the distance between each target pedestrian is calculated; when the distance between a target pedestrian and another target pedestrian is less than a preset distance threshold and the number of fighting actions of the target pedestrian during the tracking process is greater than a preset action threshold and the number of times the target pedestrian's posture category is switched during the tracking process is greater than a preset switching threshold, a fighting warning is issued, and the key point coordinates and posture category of the target pedestrian are accurately identified, and the displacement speed of the key points are calculated, and then the combination of the key point coordinates, posture category and displacement speed of the key points is used to complete the fighting behavior recognition, which can effectively improve the recognition accuracy.
[0106] As described above, for ordinary technicians in this field, various other corresponding changes and modifications can be made based on the technical solutions and technical concepts of the present invention, and all these changes and modifications should fall within the protection scope of the claims of the present invention.
Claims
1. A fighting behavior recognition method, characterized in that: The steps include: Step S1, obtaining an image to be detected, inputting the image to be detected into a preset target extraction network, and extracting the contour image of each target pedestrian; Step S2, inputting the outline image of the target pedestrian into a preset posture recognition network to obtain the key point coordinates and posture category of each target pedestrian; Step S3, tracking the target pedestrian through a preset pedestrian tracking model, and calculating the displacement speed and displacement direction of the key points of the target pedestrian according to the changes in the coordinates of the key points of the target pedestrian during the tracking process; Step S4, determining whether the displacement speed of the key point of the target pedestrian exceeds a preset speed threshold, whether the displacement direction points to another target pedestrian, and whether the key point of the target pedestrian overlaps with another target pedestrian; When the displacement speed of the key point of the target pedestrian exceeds the preset speed threshold and the displacement direction points to another target pedestrian and the key point of the target pedestrian overlaps with the appearance area of another target pedestrian, it is determined that the target pedestrian has performed a fighting action; Step S5, recording the number of occurrences of the target pedestrian's fighting actions during the tracking process and the number of times the target pedestrian's posture category switches during the tracking process, and calculating the distance between each target pedestrian; Step S6: When the distance between a target pedestrian and another target pedestrian is less than a preset distance threshold, and the number of fighting actions of the target pedestrian during the tracking process is greater than a preset action threshold and the number of times the target pedestrian switches the posture category during the tracking process is greater than a preset switching threshold, a fighting warning is issued.
2. The fighting behavior recognition method according to claim 1, characterized in that: Calculating the displacement speed and displacement direction of the key points of the target pedestrian according to the changes in the key point coordinates of the target pedestrian during the tracking process in step S3 specifically includes: calculating the displacement speed and displacement direction of the key points of the target pedestrian according to the changes in the key point coordinates of the target pedestrian in the current frame and n frames of images before the current frame, where n is a positive integer.
3. The fighting behavior recognition method according to claim 2, characterized in that: The method of calculating the displacement speed of the key points of the target pedestrian based on the change in the coordinates of the key points of the target pedestrian in the current frame and n frames of images before the current frame and judging whether the displacement speed of the key points of the target pedestrian exceeds a preset speed threshold specifically includes: calculating the displacement speeds of multiple key points of the target pedestrian according to a preset first speed calculation formula; calculating and determining the maximum displacement speed and the average displacement speed among the displacement speeds of the multiple key points; when the maximum displacement speed among the displacement speeds of the multiple key points exceeds the preset maximum speed threshold and the average displacement speed exceeds the preset average speed threshold, judging that the displacement speed of the key points of the target pedestrian exceeds the preset speed threshold.
4. The fighting behavior recognition method according to claim 3, characterized in that: The first speed calculation formula is: Among them, V i represents the displacement velocity of the i-th key point, and is the horizontal and vertical coordinates of the i-th key point in the current frame, and is the horizontal and vertical coordinates of the i-th key point in the image n frames before the current frame, t is the interval between the current frame and the image n frames before the current frame, and i is a positive integer.
5. The fighting behavior recognition method according to claim 3, characterized in that: The multiple key points of the target pedestrian include: a right wrist key point, a left wrist key point, a right leg ankle key point, and a left leg ankle key point.
6. The fighting behavior recognition method according to claim 1, characterized in that: The formula for calculating the distance between each target pedestrian in step S5 is: d jk =sqrt((center j (x)-center k (x)) 2 +(center j (y)-center k (y)) 2 ) Among them, center j (x) and center j (y) is the horizontal and vertical coordinates of the hip joint midpoint of the jth target pedestrian, center k (x) and center k (y) is the horizontal and vertical coordinates of the hip joint midpoint of the kth target pedestrian, d jk is the distance between the jth target pedestrian and the kth target pedestrian, where j and k are both positive integers.
7. The fighting behavior recognition method according to claim 1, characterized in that: The step S1 specifically includes: obtaining the face frame of each target pedestrian and the alignment parameters of each target pedestrian through the target extraction network recognition; establishing the contour map of each target pedestrian according to the face frame of the target pedestrian and the alignment parameters of the target pedestrian; the alignment parameters of each target pedestrian specifically include: the midpoint parameters of the face, the midpoint parameters of the shoulders, the midpoint parameters of the hip joint, and the parameters of the circumscribed circle of the human body; the step S2 specifically includes: identifying the key point coordinates of each target pedestrian according to the contour map of the target pedestrian; classifying the posture of the target pedestrian according to the key point coordinates of the target pedestrian and the contour map of the target pedestrian to obtain the posture category of the target pedestrian; the posture category includes fighting posture and normal posture.
8. The fighting behavior recognition method according to claim 7, characterized in that: In step S5, each time the target pedestrian switches from a normal posture to a fighting posture or from a fighting posture to a normal posture during the tracking process, it is considered that the posture category of the target pedestrian has completed a switch during the tracking process.
9. A fighting behavior recognition device, characterized in that: The steps include: An extraction unit, used to obtain an image to be detected, input the image to be detected into a preset target extraction network, and extract the contour image of each target pedestrian; A recognition unit, used to input the outline of the target pedestrian into a preset posture recognition network to obtain the key point coordinates and posture category of each target pedestrian; A first calculation unit is used to track the target pedestrian through a preset pedestrian tracking model, and calculate the displacement speed and displacement direction of the key points of the target pedestrian according to the changes in the key point coordinates of the target pedestrian during the tracking process; A behavior judgment unit is used to judge whether the displacement speed of the key point of the target pedestrian exceeds a preset speed threshold, whether the displacement direction points to another target pedestrian, and whether the key point of the target pedestrian overlaps with another target pedestrian; When the displacement speed of the key point of the target pedestrian exceeds the preset speed threshold and the displacement direction points to another target pedestrian and the key point of the target pedestrian overlaps with the appearance area of another target pedestrian, it is determined that the target pedestrian has performed a fighting action; A behavior recording unit is used to record the number of fighting actions of the target pedestrian during the tracking process and the number of times the target pedestrian switches its posture category during the tracking process, and calculate the distance between each target pedestrian; An alarm unit is used to issue a fighting warning when the distance between a target pedestrian and another target pedestrian is less than a preset distance threshold, and the number of fighting actions of the target pedestrian during the tracking process is greater than a preset action threshold and the number of times the posture category of the target pedestrian switches during the tracking process is greater than a preset switching threshold.
10. An electronic device, characterized in that: include: A memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the method according to any one of claims 1 to 8.
Citation Information
Patent Citations
Method and system for detecting and monitoring fight behavior
CN102457705A
Personnel relationship determination method and device, electronic equipment and storage medium
CN111324772A
KR20200052418A
