Limb violent behavior detection method and system and electronic equipment

Abstract

The invention is suitable for the technical field of computer analysis, and provides a limb violent behavior detection method and system and electronic device.The method comprises the steps that based on at least one observation individual in a dynamic image, kinetic energy of multiple human body parts of each observation individual is determined; screening suspected violent individuals from the at least one observation individual based on the kinetic energy; and under the condition that the part action postures of the plurality of human body parts of the suspected violent individual accord with a set attack posture, determining that the suspected violent individual is a target individual with a limb violent behavior. According to the scheme, the detection accuracy of limb violent behaviors can be improved.

Description

Technical Field

[0001] This application belongs to the technical field of computer analysis, and particularly relates to a method, system, and electronic device for detecting physical violence behavior. Background Art

[0002] Violent behavior mainly manifests in two categories: verbal violence and physical violence. Currently, verbal violence relies on voiceprint feature extraction and natural language processing technologies to achieve detection through emotion recognition and matching of insulting words. With the development of these technologies, the detection accuracy of verbal violence has been improved. However, the existing detection and determination dimensions of physical violence behavior detection methods are relatively single, resulting in relatively low detection accuracy of physical violence behavior. Summary of the Invention

[0003] Embodiments of this application provide a method, system, and electronic device for detecting physical violence behavior to solve the problem of low detection accuracy of physical violence behavior in the prior art.

[0004] The first aspect of the embodiments of this application provides a method for detecting physical violence behavior, including: Based on at least one observed individual in a dynamic image, determine the kinetic energy of multiple human body parts of each observed individual; Based on the kinetic energy, screen out suspected violent individuals from at least one of the observed individuals; When the action postures of multiple human body parts of the suspected violent individual conform to a set attack posture, determine the suspected violent individual as a target individual with physical violence behavior.

[0005] The second aspect of the embodiments of this application provides a system for detecting physical violence behavior, including: A first determination module, configured to determine the kinetic energy of multiple human body parts of each observed individual based on at least one observed individual in a dynamic image; A screening module, configured to screen out suspected violent individuals from at least one of the observed individuals based on the kinetic energy; A second determination module, configured to determine the suspected violent individual as a target individual with physical violence behavior when the action postures of multiple human body parts of the suspected violent individual conform to a set attack posture.

[0006] The third aspect of the embodiments of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the method described in the first aspect are implemented.

[0007] A fourth aspect of the embodiments of the present application provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the method described in the first aspect.

[0008] A fifth aspect of the embodiments of the present application provides a computer-readable storage medium that stores a computer program which, when executed by a processor, implements the steps of the method described in the first aspect.

[0009] As can be seen from the above, the present application first determines the kinetic energy of multiple human body parts of each observed individual included in the dynamic image, and quantifies the energy characteristics of the actions of the observed individuals. Then, based on the kinetic energy, suspected violent individuals are screened out from at least one observed individual, and when the part action postures of multiple human body parts of the suspected violent individual conform to the set attack postures, the suspected violent individual is determined as the target individual with physical violence behavior. Through kinetic energy preliminary screening and posture verification, multi-dimensional detection and determination are realized. This dual-modal detection and determination mechanism effectively improves the detection accuracy of physical violence behavior. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0011] Figure 1 is a flowchart of a method for detecting physical violence behavior provided by an embodiment of the present application; Figure 2 is a schematic diagram showing that the part action posture of a human body part conforms to a set attack posture provided by an embodiment of the present application Figure 1 ; Figure 3 is a schematic diagram showing that the part action posture of a human body part conforms to a set attack posture provided by an embodiment of the present application Figure 2 ; Figure 4 is a structural diagram of a system for detecting physical violence behavior provided by an embodiment of the present application; Figure 5 is a structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0012] In the following description, specific details such as specific system architectures, technologies, etc. are presented for the purpose of illustration rather than limitation, so as to thoroughly understand the embodiments of the present application. However, those skilled in the art should clearly understand that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from interfering with the description of the present application.

[0013] It should be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, wholes, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.

[0014] It should also be understood that the terms used in this specification of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in this specification of the present application and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an", and "the" are intended to include the plural forms.

[0015] It should be further understood that the term "and / or" used in this specification of the present application and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0016] As used in this specification and the appended claims, the term "if" can be interpreted as "when", "once", "in response to determining", or "in response to detecting" depending on the context. Similarly, the phrase "if determined" or "if [the described condition or event] is detected" can be interpreted as meaning "once determined", "in response to determining", "once [the described condition or event] is detected", or "in response to detecting [the described condition or event]" depending on the context.

[0017] In a specific implementation, the terminals described in the embodiments of the present application include, but are not limited to, other portable devices such as mobile phones, laptop computers, or tablet computers having a touch-sensitive surface (e.g., a touch screen display and / or a touchpad). It should also be understood that in some embodiments, the device is not a portable communication device, but a desktop computer having a touch-sensitive surface (e.g., a touch screen display and / or a touchpad).

[0018] In the following discussion, terminals including a display and a touch-sensitive surface are described. However, it should be understood that the terminal may include one or more other physical user interface devices such as a physical keyboard, a mouse, and / or a joystick.

[0019] The terminal supports various applications, such as one or more of the following: drawing applications, presentation applications, word processing applications, website creation applications, disc burning applications, spreadsheet applications, game applications, phone applications, video conferencing applications, email applications, instant messaging applications, exercise support applications, photo management applications, digital camera applications, digital video camera applications, web browsing applications, digital music player applications, and / or digital video player applications.

[0020] Various applications that can be executed on the terminal can use at least one common physical user interface device such as a touch-sensitive surface. One or more functions of the touch-sensitive surface and the corresponding information displayed on the terminal can be adjusted and / or changed between applications and / or within the respective applications. In this way, the common physical architecture of the terminal (e.g., the touch-sensitive surface) can support various applications with a user interface that is intuitive and transparent to the user.

[0021] It should be understood that the magnitudes of the sequence numbers of the steps in this embodiment do not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

[0022] To illustrate the technical solutions described in the present application, the following will be described through specific embodiments.

[0023] See Figure 1 , Figure 1 is a flowchart of a method for detecting physical violence behavior provided by an embodiment of the present application. As Figure 1 shown, a method for detecting physical violence behavior, the method includes the following steps: Step 101, based on at least one observed individual in the dynamic image, determine the kinetic energy of multiple human body parts of each said observed individual.

[0024] In the observation area, such as school corridors, hospital lobbies, subway platforms and other areas, install image acquisition devices for acquiring dynamic images.

[0025] In some embodiments, the image acquisition device is a high-frame-rate wide-angle camera, a thermal imaging device, a depth sensor, etc. Among them, the depth sensor can be a Red Green Blue Depth (RGBD) camera to more accurately identify the observed individual and determine the kinetic energy of the human body parts of the observed individual in space.

[0026] In some embodiments, the dynamic images of the observation area can also be flexibly acquired by the image acquisition device carried by a drone.

[0027] Dynamic images are image data captured by image acquisition devices, including multi-frame images that are continuous in time, such as video streams and time-series image sequences. Each frame of the image usually contains corresponding timestamp information to support image analysis and processing of time-related information such as speed.

[0028] An observed individual refers to a person who appears in the observation area and is captured by the image acquisition device.

[0029] In some embodiments, dynamic images refer to video streams and / or time-series image sequences captured by image acquisition devices that contain the effective observation data of the observed individual, that is, through pre-screening, the content containing effective observation data is screened out to reduce the amount of data to be processed subsequently.

[0030] In some embodiments, the dynamic image is the image data after image preprocessing. Image preprocessing includes means such as Gaussian filtering, histogram equalization, and contrast enhancement. Through image preprocessing, the image quality of the dynamic image is improved, and the detection accuracy of physical violence behavior is enhanced.

[0031] In some embodiments, the images included in the dynamic image are all images in a set image format, such as images with a resolution of 1280×720 (pixels) or 1920×1080 (pixels).

[0032] Human body parts refer to parts such as hands, feet, heads, elbows, knees, and shoulders that may attack others. It should be noted that when calculating kinetic energy, these unilateral human body parts that distinguish left and right, such as the left hand, right hand, left foot, right foot, left knee, and right knee, are all regarded as independent parts to calculate the corresponding kinetic energy.

[0033] The core essence of physical violence behavior is that human body parts generate high energy output in a short period of time, and kinetic energy is a characteristic quantity that quantifies the energy output of human body parts.

[0034] Through the dynamic image, the kinetic energy of multiple human body parts of the observed individual is determined to detect whether there is high energy output in a short period of time by the observed individual, so as to achieve the detection of physical violence behavior.

[0035] In some embodiments, determining the kinetic energy of multiple human body parts of each of the observed individuals based on at least one observed individual in the dynamic image includes: performing person detection on the dynamic image to determine at least one of the observed individuals in the dynamic image; performing person matching on each of the observed individuals to obtain an image frame sequence of each of the observed individuals in the dynamic image; determining the body part mass of the multiple human body parts of the observed individual and the position data of multiple human body key points in the image frame sequence according to the image frame sequence of each of the observed individuals; calculating the moving speed of the multiple human body parts of the observed individual based on the position change data of the multiple human body key points in the image frame sequence; and calculating the kinetic energy of the multiple human body parts of each of the observed individuals based on the moving speed and the body part mass of each human body part.

[0036] By identifying and matching the observed individuals in the dynamic image and analyzing the key points, accurately calculate the moving speed and kinetic energy of multiple human body parts of each observed individual, convert the limb movements into quantifiable physical energy characteristics, effectively identify abnormal limb behaviors with high energy output, and realize cross-modal limb violence behavior detection from visual data to physical quantity determination.

[0037] In some embodiments, person detection is performed on the dynamic image through object detection algorithms such as YOLOv8, Faster Region-based Convolutional Neural Network (Faster R-CNN) algorithm, Mask Region-based Convolutional Neural Network (Mask R-CNN) algorithm, etc., so as to accurately identify at least one observed individual included in the dynamic image.

[0038] In some embodiments, the observed individuals in multiple frames of images included in the dynamic image are identified through object detection algorithms.

[0039] After determining the observed individuals through the object detection algorithm, person matching is performed through the physical characteristics and movement trajectories of the observed individuals to realize the matching mapping of the same observed individuals in multiple frames of images, and then an image frame sequence of each observed individual in the dynamic image is obtained. Among them, the image frame sequence is image data including at least one frame of image and corresponding timestamp information.

[0040] In some embodiments, the image frame sequence of each observed individual is subjected to local enhancement processing or background blurring processing, so that the observed individual maintains visual salience in each frame of image, facilitating better determination of the quality, moving speed, and kinetic energy of the human body parts of the observed individual. The images included in the image frame sequence of each observed individual also conform to the set image format.

[0041] After clarifying the image frame sequence of each observed individual, based on the image frame sequence, further determine the human body part masses of multiple human body parts of the observed individual and the position data of multiple human body key points in the image frame sequence. Among them, the human body key points refer to the skeletal joint points of the human body parts and serve as reference points for determining the moving speed of the human body parts.

[0042] In some embodiments, multiple frames of images in the dynamic image captured by an image acquisition device at a certain fixed position share a reference coordinate system. For example, the lower left corner of the image is taken as the coordinate origin, the edge of the image length where the coordinate origin is located is taken as the x-axis, and the edge of the image width where the coordinate origin is located is taken as the y-axis. Correspondingly, the image frame sequence of the observed individual in the dynamic image captured by the image acquisition device at the fixed position also shares this reference coordinate system.

[0043] In some embodiments, if the image acquisition device is an RGBD camera, the reference coordinate system is a three-dimensional space coordinate system.

[0044] Introducing the reference coordinate system can quickly obtain the position data, i.e., coordinate values, of each human body key point in the same reference coordinate system, thereby making the subsequent calculated moving speed more accurate and more valuable for use.

[0045] In some embodiments, the determining of the human body part masses of multiple human body parts of the observed individual and the position data of multiple human body key points in the image frame sequence according to the image frame sequence of each observed individual includes: calculating the human body part masses of multiple human body parts of the observed individual according to the image areas and mass calculation coefficients of the respective human body parts of the observed individual in the image frame sequence; identifying the human body key points of the observed individual in each frame of the image frame sequence, and determining the position of the identified human body key points in the image frame sequence as the position data of the human body key points.

[0046] Based on the image frame sequence, calculate the human body part masses of the human body parts and determine the position data of the human body key points, realizing the mapping from image data to biometric data and providing a data basis for kinetic energy calculation.

[0047] For ease of understanding, the image area in the above embodiments is referred to as the first image area. According to the image frame sequence of the observed individual, determine the first image areas corresponding to the respective human body parts of the observed individual.

[0048] In some embodiments, each sequence of image frames of an observed individual includes at least one image frame, and the second image area of each human body part in each image frame is determined. For each human body part, the average value of the second image areas of this human body part in each image frame can be used as the first image area. The units of the first image area and the second image area can be mm 2 or pixel 2 where pixel means pixel.

[0049] In some embodiments, the mass calculation coefficient is a conversion parameter for converting the image area of a human body part into the mass of the human body part. Introducing the mass calculation coefficient to predict the mass of a human body part can adapt to different body types and improve the accuracy of mass prediction. The unit of the mass calculation coefficient can be kg / mm 2 or kg / pixel 2 , for example, the mass calculation coefficient of the arm is 0.05 kg / pixel².

[0050] In some embodiments, the sample mass of each human body part of the observed individual and the sample image area in a set image format are pre - statistically calculated from multiple reference images. Based on the sample mass and the sample image area, the conversion parameter between the sample mass and the sample image area is solved, that is, the mass calculation coefficient is solved. Different types of human body parts correspond to different mass calculation coefficients.

[0051] Multiply the first image area of each human body part by the mass calculation coefficient, that is, human body part mass = first image area × mass calculation coefficient, and then calculate the human body part mass of the human body part, realizing the conversion from the image area to the human body part mass.

[0052] Using algorithms such as the OpenPose algorithm, the DeepCut algorithm, or the AlphaPose algorithm, etc., the human body skeletons in each image frame of the image frame sequence are extracted. Then, based on the human body skeletons, the human body key points of the observed individual in each image frame are obtained, and the coordinates of the human body key points in the reference coordinate system are determined to obtain position data.

[0053] The position change data of the position of the human body key points in the image frame sequence represents the distance s, that is, the coordinate change value of the human body key points in different images is the distance s, and the time difference between the corresponding images is the time t. According to the formula, the moving speeds of multiple human body key points of the observed individual are calculated and used as the moving speeds of the corresponding human body parts.

[0054] In some embodiments, the coordinates of the left hand of the observed individual in the first image frame are , and the coordinates in the second image frame are , the first frame image is taken earlier than the second frame image, and the time difference between the two is , then the moving speed of the left hand of the observed individual is calculated based on these two frames of images, and the moving speed of the left hand of the observed individual is .

[0055] In some embodiments, for any part of the human body, an average value of the moving speeds between multiple frames is taken as the moving speed required for calculating kinetic energy.

[0056] Knowing the moving speed v and mass m of the human body part, according to the kinetic energy calculation formula , calculate the kinetic energy E of each body part of each observed individual.

[0057] In some embodiments, there is a situation where a body part of the observed individual is obscured. To deal with this situation, based on the image frame sequence, body part detection is performed on each of the observed individuals. When it is detected that the observed individual has obscured body parts, the unobstructed body parts of the observed individual are determined; based on the unobstructed body parts, the posture of the observed individual is estimated to obtain the part contour and the body key points of the obscured body part, and the position data of the body key points are determined; and the body part mass of the obscured body part is calculated based on the image area corresponding to the part contour and the mass calculation coefficient.

[0058] That is, the human body parts of the observed individuals in each image in the image frame sequence are detected in advance, and whether there are obscured human body parts in each frame of the image is detected, and the unobstructed human body parts in the image are determined at the same time. In the case of detecting the existence of obscured human body parts, the posture of the observed individual can be estimated based on the determined unobstructed human body parts, and the part contours and human body key points of the obscured human body parts can be generated, and the coordinates of the estimated human body key points, i.e., the position data, can be determined accordingly. After that, the image area and mass calculation coefficient corresponding to the part contour of the obscured human body parts can be used to calculate the human body part mass of the obscured human body parts. At the same time, the coordinates of the human body key points corresponding to the obscured human body parts can also be used to calculate the moving speed of the obscured human body parts.

[0059] When determining the contours and key points of the occluded human body parts, the reasonable positions of the key points of the occluded human body parts are inferred based on the key points of the unoccluded human body parts, combined with kinematic constraints and movement trajectories, that is, the corresponding key points of the human body are generated. By using technologies such as generative adversarial networks, the contours of the occluded human body parts are constructed based on the contours of the unoccluded human body parts, and the contours of the occluded human body parts are generated, that is, highly robust posture recovery is achieved in an occluded environment, and the reconstruction of human body posture and parts in an occluded environment is achieved, effectively supporting kinetic energy calculation.

[0060] Step 102: Based on the kinetic energy, screen out suspected violent individuals from at least one of the observed individuals.

[0061] Compare the kinetic energy of multiple human body parts of each observed individual with a preset kinetic energy, such as 50 joules.

[0062] If the kinetic energy of any human body part of an observed individual is greater than the preset kinetic energy, it is determined that there may be physical violence behavior in the corresponding human body part of the observed individual, and the observed individual is a suspected violent individual.

[0063] If the kinetic energy of all human body parts of an observed individual is less than or equal to the preset kinetic energy, it is determined that there is no physical violence behavior in the observed individual, and the observed individual is an ordinary observed individual.

[0064] Kinetic energy is a physical quantity that quantifies the intensity of an action. Based on kinetic energy, a preliminary screening of physical violence behavior is carried out to screen out suspected violent individuals. Judging based on this objective energy of kinetic energy can quickly identify potential dangers, lock in high-risk individuals, and improve the detection efficiency and detection accuracy.

[0065] Step 103: When the action postures of multiple human body parts of the suspected violent individual conform to the set attack postures, determine the suspected violent individual as the target individual with physical violence behavior.

[0066] Based on kinetic energy, suspected violent individuals are screened out, but relying only on kinetic energy may lead to false detections. Therefore, it is necessary to check whether the action postures of the human body parts of the suspected violent individual conform to known attack postures, that is, set attack postures, such as punching or kicking. When kicking, one foot generates kinetic energy greater than the preset kinetic energy, and the other foot stands still, which can effectively distinguish misjudgments in scenarios such as people running, and achieve further screening and determination, reduce false detections, obtain the target individual with physical violence behavior, that is, the dangerous individual, and improve the detection accuracy.

[0067] Among them, the set attack postures limit the action postures of some or all human body parts. When comparing, it is only necessary to compare with the action postures of the corresponding parts.

[0068] In some embodiments, before determining that the suspected violent individual is the target individual with physical violence behavior when the action postures of multiple human body parts of the suspected violent individual conform to the set attack postures, it further includes: generating the action postures of multiple human body parts of the suspected violent individual based on the position data of multiple human body key points of the suspected violent individual.

[0069] Determine the body part movement postures of each body part of a suspected violent individual to facilitate more accurate identification of aggressive actions and reduce false detections.

[0070] This application can not only identify dangerous individuals who are committing physical violence, but also identify potential dangerous individuals who have the potential to commit physical violence and may harm other observed individuals. It upgrades from seeing physical violence to foreseeing physical violence, achieving an upgrade from passive identification to active identification and defense of physical violence.

[0071] As Figure 2 shown, Figure 2 is a schematic diagram of the body part movement posture of a body part of this application embodiment conforming to a set attack posture Figure 1 . Figure 2 There are two observed individuals, namely observed individual A and observed individual B. Based on the dynamic image, it is calculated that the kinetic energy of body part A1 of observed individual A is greater than the preset kinetic energy, that is, the kinetic energy of punching with body part A1 (left hand) of observed individual A is greater than the preset kinetic energy. Therefore, observed individual A is determined as a suspected violent individual. At the same time, the movement trajectory of body part A1 of observed individual A points to observed individual B, that is, the body part movement posture of body part A1 of observed individual A is moving towards observed individual B, which conforms to the set attack posture of punching. Furthermore, observed individual A, this suspected violent individual, is determined as a target individual with physical violence behavior, and the body part with physical violence behavior is body part A1 of observed individual A. Correspondingly, the target of the physical violence behavior of observed individual A is observed individual B.

[0072] As Figure 3 shown, Figure 3 is a schematic diagram of the body part movement posture of a body part of this application embodiment conforming to a set attack posture Figure 2 . Figure 3 There are two observed individuals, namely observed individual C and observed individual D. Based on the dynamic image, it is calculated that the kinetic energy of body part C1 of observed individual C is greater than the preset kinetic energy, that is, the kinetic energy of kicking with body part C1 (left foot) of observed individual C is greater than the preset kinetic energy. Therefore, observed individual C is determined as a suspected violent individual. At the same time, the movement trajectory of body part C1 of observed individual C points to observed individual D and body part C2 (right foot) stands still on the ground. The standing kicking body part movement posture formed by body part C1 and body part C2 of observed individual C towards observed individual D conforms to the set attack posture of kicking. Furthermore, observed individual C, this suspected violent individual, is determined as a target individual with physical violence behavior, and the body part with physical violence behavior is body part C1 of observed individual C. Correspondingly, the target of the physical violence behavior of observed individual C is observed individual D.

[0073] In some embodiments, after determining that the suspected violent individual is the target individual with physical violence behavior when the posture of the body parts of the suspected violent individual conforms to the set attack posture, the following steps are further included: determining the level of the violent behavior of the target individual according to the kinetic energy range corresponding to the maximum kinetic energy among the kinetic energies of the multiple body parts of the target individual; and / or determining the level of the violent behavior of the target individual according to the contact condition and body size difference condition between the target individual and other observed individuals.

[0074] There are different levels of physical violence behavior, and different levels of violent behavior correspond to different degrees of danger and countermeasures.

[0075] In some embodiments, the size of the kinetic energy of the body part, the contact condition (i.e., distance) between the target individual and other observed individuals, the body size difference condition between the target individual and other observed individuals, etc. will all affect the determination of the level of violent behavior.

[0076] In some embodiments, the level of violent behavior can be determined based on the kinetic energy range corresponding to the maximum kinetic energy of the body part, the degree of contact between the target individual and other observed individuals, the body size difference condition between the target individual and other observed individuals, etc., so as to clarify the degree of danger of the target individual, facilitate the determination of countermeasures according to the level of violent behavior, reduce the risk of personnel injury, and minimize personnel harm.

[0077] In some embodiments, the determination of the level of the violent behavior of the target individual according to the contact condition and body size difference condition between the target individual and other observed individuals includes: if the distance between the target individual and other observed individuals is less than or equal to the set distance, the level of violent behavior is increased; if the height ratio between the target individual and other observed individuals is greater than the set height ratio and / or the mass ratio is greater than the set mass ratio, the level of violent behavior is increased.

[0078] Among them, the other observed individuals used to determine the height ratio and mass ratio are the observed individuals within the attack range of the target individual.

[0079] In some embodiments, if the distance between the target individual and other observed individuals is greater than the set distance, the height ratio between the target individual and other observed individuals is less than or equal to the set height ratio, and the mass ratio is less than or equal to the set mass ratio, the level of violent behavior is determined to be level one. Among them, the set distance is, for example, 0.5 meters, the set height ratio can be 1.3, and the set mass ratio can be 1.5. The values here are only examples and can be adjusted according to the actual situation.

[0080] In some embodiments, if the distance between the target individual and other observed individuals is less than or equal to a set distance, and the height ratio of the target individual to other observed individuals is greater than a set height ratio, or the mass ratio is greater than a set mass ratio, and any one of these three situations is satisfied, then the violence behavior level is determined to be level two.

[0081] In some embodiments, if the distance between the target individual and other observed individuals is less than or equal to a set distance, and the height ratio of the target individual to other observed individuals is greater than a set height ratio, or the mass ratio is greater than a set mass ratio, and any two of these three situations are satisfied, then the violence behavior level is determined to be level three.

[0082] In some embodiments, if the distance between the target individual and other observed individuals is less than or equal to a set distance, and the height ratio of the target individual to other observed individuals is greater than a set height ratio, and the mass ratio is greater than a set mass ratio, and all of these three situations are satisfied, then the violence behavior level is determined to be level four.

[0083] In some embodiments, the scores corresponding to each determination item can be determined based on the kinetic energy interval corresponding to the maximum kinetic energy of the human body part, the degree of contact between the target individual and other observed individuals, the body type difference between the target individual and other observed individuals, etc., and the sum is obtained to get the total score value, and the violence behavior level of the target individual is determined using the total score value.

[0084] In some embodiments, there are multiple kinetic energy intervals, and different kinetic energy intervals correspond to different scores. The kinetic energy unit is joule (J). For example, the score corresponding to [50J, 100J) is 1 point, the score corresponding to [100J, 150J) is 3 points, and the score corresponding to [150J, +∞) is 5 points.

[0085] In some embodiments, the attack risk of the target individual can also be determined according to the moving speed, and the attack risk is used as a determination item for evaluating the violence behavior level. For example, the maximum moving speed and the initial moving speed of the human body part where the target individual is determined to have physical violence behavior are determined, and the speed change rate is calculated based on the maximum moving speed and the initial moving speed to obtain the moving acceleration. If the maximum moving speed is greater than or equal to the first set attack speed and the moving acceleration is greater than or equal to the set attack acceleration, then the attack risk is determined to be a high risk. If the maximum moving speed is greater than or equal to the second set attack speed, then the attack risk is determined to be a medium risk, where the second set attack speed = 0.7 × the first set attack speed. This is an example. If neither of the above two situations is satisfied, then the attack risk is determined to be a low risk. Different attack risks correspond to different scores, which are used to evaluate the violence behavior level.

[0086] In some embodiments, different distance intervals may also be set, and different distance intervals correspond to different scores. Similarly, different height ratio intervals and different mass ratio intervals may be set. Different height ratio intervals correspond to different scores, and different mass ratio intervals correspond to different scores.

[0087] Determine the scores of each judgment item, and then calculate the total score for evaluating the level of violent behavior. Different levels of violent behavior correspond to different total scores, and determine the level of violent behavior corresponding to the total score according to the total score.

[0088] Through the level of violent behavior, clarify the danger level of physical violent behavior, issue graded warnings, and take corresponding measures in a timely manner to reduce the physical violent injuries suffered by the observers.

[0089] In the embodiments of the present application, first, determine the kinetic energy of multiple human body parts of each observed individual included in the dynamic image, and quantify the energy characteristics of the actions of the observed individuals. Then, based on the kinetic energy, screen out suspected violent individuals from at least one observed individual, and determine the suspected violent individual as the target individual with physical violent behavior when the part action postures of multiple human body parts of the suspected violent individual conform to the set attack postures. Through the initial screening of kinetic energy and posture verification, multi-dimensional detection and identification are realized, and this dual-modal detection and identification mechanism effectively improves the detection accuracy of physical violent behavior.

[0090] See Figure 4 , Figure 4 FIG. is a structural diagram of a physical violent behavior detection system provided by an embodiment of the present application. For ease of description, only parts related to the embodiments of the present application are shown.

[0091] The physical violent behavior detection system 400 includes: a first determination module 401, a screening module 402, and a second determination module 403.

[0092] The first determination module 401 is configured to determine the kinetic energy of multiple human body parts of each of the at least one observed individual based on the dynamic image.

[0093] The screening module 402 is configured to screen out suspected violent individuals from at least one of the observed individuals based on the kinetic energy.

[0094] The second determination module 403 is configured to determine the suspected violent individual as the target individual with physical violent behavior when the part action postures of multiple human body parts of the suspected violent individual conform to the set attack postures.

[0095] In some embodiments, the first determination module is specifically configured to: Perform person detection on the dynamic image to determine at least one of the observed individuals in the dynamic image; Perform personnel matching for each of the observed individuals to obtain an image frame sequence of each of the observed individuals in the dynamic image; Based on the image frame sequence of each of the observed individuals, determine the body part quality of multiple body parts of the observed individual and the position data of multiple body key points in the image frame sequence; Based on the position change data of the position of multiple body key points in the image frame sequence, calculate the moving speed of multiple body parts of the observed individual; Based on the moving speed and body part quality of each body part, calculate the kinetic energy of multiple body parts of each observed individual.

[0096] In some embodiments, the first determination module is further configured to: Calculate the body part quality of multiple body parts of the observed individual according to the image area and mass calculation coefficient of each body part of the observed individual in the image frame sequence; Identify the body key points of the observed individual in each image frame of the image frame sequence, and determine the position of the identified body key points in the image frame sequence as the position data of the body key points.

[0097] In some embodiments, the first determination module is further configured to: Based on the image frame sequence, perform body part detection on each of the observed individuals. When it is detected that there is an occluded body part of the observed individual, determine the unoccluded body parts of the observed individual; Based on the unoccluded body parts, perform pose estimation on the observed individual to obtain the part contour and body key points of the occluded body part, and determine the position data of the body key points; Calculate the body part quality of the occluded body part according to the image area and mass calculation coefficient corresponding to the part contour.

[0098] In some embodiments, the system further includes a pose determination module, configured to: Generate the part action poses of multiple body parts of the suspected violent individual based on the position data of multiple body key points of the suspected violent individual.

[0099] In some embodiments, the system further includes a violent behavior level determination module, configured to: Determine the violent behavior level of the target individual according to the kinetic energy interval corresponding to the maximum kinetic energy among the kinetic energies of multiple body parts of the target individual; and / or, Determine the level of violent behavior of the target individual according to the contact status and body size difference status between the target individual and other observed individuals.

[0100] In some embodiments, the violent behavior level determination module is further configured to: If the distance between the target individual and other observed individuals is less than or equal to a set distance, increase the level of violent behavior; If the height ratio of the target individual to other observed individuals is greater than a set height ratio and / or the mass ratio is greater than a set mass ratio, increase the level of violent behavior.

[0101] The limb violent behavior detection system provided by the embodiments of the present application can implement each process of the above-mentioned embodiments of the limb violent behavior detection method, and can achieve the same technical effects. To avoid repetition, it will not be elaborated here.

[0102] Figure 5 It is a structural diagram of an electronic device provided by an embodiment of the present application. As shown in this figure, the electronic device 5 of this embodiment includes: at least one processor 50 ( Figure 5 only one is shown), a memory 51, and a computer program 52 stored in the memory 51 and executable on the at least one processor 50. When the processor 50 executes the computer program 52, the steps in any of the above method embodiments are implemented.

[0103] The electronic device 5 may be a computing device such as a desktop computer, a notebook, a palm computer, and a cloud server. The electronic device 5 may include, but is not limited to, a processor 50 and a memory 51. Those skilled in the art can understand that Figure 5 merely examples of the electronic device 5 do not constitute a limitation to the electronic device 5, and may include more or fewer components than shown in the figure, or combine some components, or different components. For example, the electronic device may further include input / output devices, network access devices, buses, etc.

[0104] The processor 50 may be a Central Processing Unit (CPU), or may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0105] The memory 51 may be an internal storage unit of the electronic device 5, such as the hard disk or memory of the electronic device 5. The memory 51 may also be an external storage device of the electronic device 5, such as a plug-in hard disk equipped on the electronic device 5, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. Further, the memory 51 may also include both the internal storage unit and the external storage device of the electronic device 5. The memory 51 is used to store the computer program and other programs and data required by the electronic device. The memory 51 may also be used to temporarily store data that has been output or is to be output.

[0106] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the above division of each functional unit and module is used as an example. In actual applications, the above functions can be allocated to different functional units and modules according to needs, that is, the internal structure of the system is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiments can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated units can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of each functional unit and module are only for the convenience of mutual distinction and do not limit the protection scope of this application. The specific working processes of the units and modules in the above system can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0107] In the above embodiments, the descriptions of the various embodiments have their own emphases. For parts not detailed or recorded in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0108] Those of ordinary skill in the art will realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of this application.

[0109] In the embodiments provided in this application, it should be understood that the disclosed systems / electronic devices and methods can be implemented in other ways. For example, the system / electronic device embodiments described above are merely illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection to each other can be through some interfaces. The indirect coupling or communication connection of the system or unit can be in an electrical, mechanical or other form.

[0110] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place, or can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0111] In addition, the functional units in each embodiment of this application can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated units can be implemented in the form of hardware or in the form of software functional units.

[0112] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, to implement all or part of the processes in the above-described embodiment methods of this application, it can also be completed by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above-described method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file, or some intermediate form, etc. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content included in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0113] To implement all or part of the processes in the above-described embodiment methods of this application, it can also be achieved through a computer program product. When the computer program product runs on an electronic device, it causes the electronic device to execute and implement the steps in the above-described method embodiments.

[0114] The above-described embodiments are only used to illustrate the technical solutions of this application, rather than to limit it; although this application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included in the protection scope of this application.

Claims

1. A method for detecting physical violence, characterized in that: include: Determining kinetic energy of a plurality of body parts of each of the observed individuals based on at least one observed individual in the dynamic image; Based on the kinetic energy, screening out a suspected violent individual from at least one of the observed individuals; When the movement postures of the multiple body parts of the suspected violent individual meet the set attack postures, the suspected violent individual is determined to be a target individual with physical violent behavior.

2. The method according to claim 1, characterized in that The step of determining the kinetic energy of multiple body parts of each observed individual based on at least one observed individual in the dynamic image comprises: Performing personnel detection on the dynamic image to determine at least one observed individual in the dynamic image; Performing person matching on each of the observed individuals to obtain an image frame sequence of each of the observed individuals in the dynamic image; Determine, according to the image frame sequence of each observed individual, the body part masses of the plurality of body parts of the observed individual and the position data of a plurality of body key points in the image frame sequence; Calculating the moving speeds of the multiple human body parts of the observed individual based on the position change data of the position data of the multiple human body key points in the image frame sequence; The kinetic energy of the multiple human body parts of each of the observed individuals is calculated based on the moving speed and the mass of the human body parts.

3. The method according to claim 2, characterized in that The step of determining the body part masses of the plurality of body parts of the observed individual and the position data of a plurality of body key points in the image frame sequence according to the image frame sequence of each observed individual comprises: Calculating the body part mass of the plurality of body parts of the observed individual according to the image area and mass calculation coefficient of each body part of the observed individual in the image frame sequence; The human body key points of the observed individual in each image frame in the image frame sequence are identified, and the positions of the identified human body key points in the image frame sequence are determined as the position data of the human body key points.

4. The method according to claim 2, characterized in that: The method further comprises: Based on the image frame sequence, performing body part detection on each of the observed individuals, and in the case where it is detected that the observed individual has an obstructed body part, determining an unobstructed body part of the observed individual; Based on the unobstructed human body part, the posture of the observed individual is estimated to obtain the part contour of the obstructed human body part and the human body key points, and determine the position data of the human body key points; The human body part mass of the obscured human body part is calculated according to the image area corresponding to the part contour and the mass calculation coefficient.

5. The method according to claim 2, characterized in that: Before determining that the suspected violent individual is a target individual with physical violence behavior when the action postures of the multiple body parts of the suspected violent individual meet the set attack posture, the method further includes: Based on the position data of the multiple key points of the human body of the suspected violent individual, the part action postures of the multiple parts of the human body of the suspected violent individual are generated.

6. The method according to claim 1, characterized in that After determining that the suspected violent individual is a target individual with physical violence behavior when the action postures of the multiple body parts of the suspected violent individual meet the set attack posture, the method further includes: determining the level of violence of the target individual according to the kinetic energy interval corresponding to the maximum kinetic energy among the kinetic energies of the multiple body parts of the target individual; and / or, The violent behavior level of the target individual is determined based on the contact status and body size difference between the target individual and other observed individuals.

7. The method according to claim 6, characterized in that Determining the level of violence of the target individual based on the contact status and size difference between the target individual and other observed individuals includes: If the distance between the target individual and the other observed individuals is less than or equal to the set distance, the level of the violent behavior is increased; If the height ratio between the target individual and the other observed individuals is greater than a set height ratio and / or the mass ratio is greater than a set mass ratio, the level of violent behavior is increased.

8. A physical violence behavior detection system, characterized in that: include: A first determination module, configured to determine the kinetic energy of multiple body parts of each observed individual based on at least one observed individual in the dynamic image; A screening module, configured to screen out a suspected violent individual from at least one of the observed individuals based on the kinetic energy; The second determination module is used to determine that the suspected violent individual is a target individual with physical violence behavior when the action postures of the multiple body parts of the suspected violent individual meet the set attack posture.

9. An electronic device, characterized in that: The electronic device comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the electronic device implements the method according to any one of claims 1 to 7.

10. A computer program product, characterized in that The invention comprises a computer program which, when executed, causes the method according to any one of claims 1 to 7 to be performed.

Images