Abnormal behavior identification method, device, electronic device, medium and program product

Abstract

The present application provides a method, device, electronic device, medium, and program product for identifying abnormal behavior, belonging to the field of image recognition technology. The method includes: obtaining the coordinates of various parts of a human body in an image in a monitoring video and the movement speed of the parts; determining the distance of the hand part relative to the facial part and the angle of the upper limbs based on the coordinates of the various parts; determining whether the human body is covering its face based on the distances corresponding to adjacent images in the monitoring video, determining whether the human body is striking based on the angles corresponding to the adjacent images, and determining whether the human body is sprinting based on the movement speeds of the various parts in the adjacent images; determining the order in which the face-covering, striking, and sprinting actions occur when the human body is covering its face, striking, and sprinting; and determining that the human body is engaging in abnormal behavior when the order matches a preset order. In the present application, the efficiency of identifying abnormal behavior is improved.

Description

Technical Field

[0001] The present application relates to the field of image recognition technology, and in particular to an abnormal behavior recognition method, device, electronic device, medium and program product. Background Art

[0002] In some places, violent incidents may occur, such as vandalism of ATMs, robbery of cash withdrawal personnel, and other violent incidents.

[0003] In the exemplary technology, surveillance videos of a place are manually reviewed to determine whether people in the place are engaging in abnormal behavior, thereby preventing violent incidents.

[0004] Violent incidents occur in a short period of time, but the number of places that need to be monitored is large. Manual monitoring videos need to be checked regularly, and the efficiency of identifying abnormal behaviors is low. Summary of the Invention

[0005] The present application provides an abnormal behavior identification method, device, electronic device, medium and program product to solve the problem of low efficiency in identifying abnormal behavior.

[0006] In one aspect, the present application provides an abnormal behavior recognition method, which is applied to an abnormal behavior recognition device. The abnormal behavior recognition method includes:

[0007] Obtaining coordinates of various parts of a human body in an image in a monitoring video and movement speeds of the parts, the parts including facial parts and hand parts;

[0008] Determining the distance between the hand part and the facial part according to the coordinates of each part, and determining the angle of the upper limb according to the coordinates of each part;

[0009] determining whether the human body is covering its face based on the distances between adjacent images in the monitoring video, determining whether the human body is striking based on the angles between adjacent images, and determining whether the human body is sprinting based on the movement speeds of the respective parts in the adjacent images;

[0010] When the human body performs the face-covering action, the striking action, and the sprinting action, determining a sequence in which the face-covering action, the striking action, and the sprinting action occur;

[0011] When the sequence matches a preset sequence, it is determined that the human body has abnormal behavior.

[0012] On the other hand, the present application also provides an abnormal behavior identification device, comprising:

[0013] An acquisition module is used to obtain the coordinates of various parts of the human body in the image of the monitoring video and the movement speed of the parts, wherein the parts include the face and the hands;

[0014] a first determining module, configured to determine the distance between the hand part and the facial part according to the coordinates of each part, and to determine the angle of the upper limb according to the coordinates of each part;

[0015] a second determining module, configured to determine whether the human body is covering its face based on the distances between adjacent images in the monitoring video, determine whether the human body is striking based on the angles between the adjacent images, and determine whether the human body is sprinting based on the movement speeds of the respective parts in the adjacent images;

[0016] a third determining module, configured to determine a sequence in which the face-covering action, the striking action, and the sprinting action occur when the human body performs the face-covering action, the striking action, and the sprinting action;

[0017] The fourth determining module is used to determine that the human body has abnormal behavior when the sequence matches a preset sequence.

[0018] On the other hand, the present application also provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor;

[0019] The memory stores computer-executable instructions;

[0020] The processor executes the computer-executable instructions stored in the memory to implement the method described above.

[0021] On the other hand, the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to implement the above-mentioned method when executed by a processor.

[0022] On the other hand, the present application also provides a computer program product, comprising a computer program, which implements the above method when executed by a processor.

[0023] The abnormal behavior identification method, device, electronic device, medium, and program product provided in this application obtain the coordinates of various parts of the human body in the image of the monitoring video and the movement speed of the parts, determine the distance of the hand part relative to the facial part based on the coordinates of each part, and determine the angle of the upper limb based on the coordinates of each part, thereby determining whether the human body has made abnormal movements such as covering the face, hitting, and sprinting based on the distance, angle, and movement speed of adjacent images in the monitoring video. If there are face-covering movements, hitting movements, and sprinting movements, the order in which the face-covering movements, hitting movements, and sprinting movements are sounded is determined. If the order matches a preset order, it is determined that the human body has made abnormal behavior. In this application, the device automatically identifies abnormal movements such as face-covering movements, hitting movements, and sprinting movements in the monitoring video. If there are abnormal movements, it further identifies whether the human body has made abnormal behavior based on the abnormal movements. There is no need to manually review the monitoring video to determine the abnormal behavior of the human body, thereby improving the efficiency of abnormal behavior identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0025] Figure 1 A schematic diagram of a scenario of the abnormal behavior identification method involved in this application;

[0026] Figure 2 This is a flowchart of the first embodiment of the abnormal behavior identification method provided by this application;

[0027] Figure 3 This is a flow chart of the second embodiment of the abnormal behavior identification method provided by this application;

[0028] Figure 4 This is a flowchart of the third embodiment of the abnormal behavior identification method provided by this application;

[0029] Figure 5 This is a flowchart of the fourth embodiment of the abnormal behavior identification method provided by this application;

[0030] Figure 6 This is a module diagram of the abnormal behavior identification device of this application;

[0031] Figure 7 This is a structural diagram of the abnormal behavior identification device of this application.

[0032] The above drawings illustrate specific embodiments of the present application, which will be described in more detail below. These drawings and the textual description are not intended to limit the scope of the present application in any way, but rather to illustrate the concepts of the present application to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0033] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0034] In some places, violent incidents may occur, such as vandalism of ATMs, robbery of cash withdrawal personnel, and other violent incidents.

[0035] In the exemplary technology, surveillance videos of a place are manually reviewed to determine whether people in the place are engaging in abnormal behavior, thereby preventing violent incidents.

[0036] The inventors of the present application have found that violent incidents occur in a short time, while the number of places that need to be monitored is large, and manual monitoring of monitoring videos is required at regular intervals, resulting in low efficiency in identifying abnormal behaviors.

[0037] The inventors of this application therefore came up with the idea of ​​automatically identifying abnormal human movements such as covering the face, hitting, and sprinting in the monitoring video. If the human body has abnormal movements, it can further identify whether the human body has abnormal behavior based on the abnormal movements. There is no need to manually check the monitoring video to determine the abnormal behavior of the human body, which improves the efficiency of identifying abnormal behavior.

[0038] Reference Figure 1 , Figure 1 The following is a schematic diagram of the scenario involved in the abnormal behavior identification method of the present application. The abnormal behavior identification device 100 obtains a monitoring video 200. The monitoring video 200 is composed of multiple images. The abnormal behavior identification device 100 identifies and judges the human body parts in the images in the monitoring video 200, thereby determining whether the human body in the monitoring video 200 has any abnormal actions such as covering the face, hitting, and sprinting. If the human body has any abnormal actions such as covering the face, hitting, and sprinting, the abnormal behavior identification device determines whether the order in which the face-covering, hitting, and sprinting actions occur matches a preset order. If so, it can be determined that the human body has abnormal behavior.

[0039] The following specific embodiments describe in detail the technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0040] In addition, the data involved in this application may be data authorized by the user or fully authorized by all parties. The collection, dissemination, and use of the data shall comply with the requirements of relevant national laws and regulations. The implementation methods / examples of this disclosure may be combined with each other.

[0041] It should be noted that the abnormal behavior identification method, device, electronic device, medium and program product of the present application can be used in the field of image recognition technology for security protection, and can also be used in any field other than security protection. The application field of the abnormal behavior identification method, device, electronic device, medium and program product of the present application is not limited.

[0042] Reference Figure 2 , Figure 2 This is a flowchart of the first embodiment of the abnormal behavior identification method of the present application. The abnormal behavior identification method includes the following steps:

[0043] Step S201 , obtaining the coordinates of various parts of the human body in the image in the monitoring video and the moving speed of the parts, the parts including the face and the hands.

[0044] In this embodiment, the execution subject is an abnormal behavior recognition device. For ease of description, the term "device" is used to refer to the abnormal behavior recognition device. The device can be any device with image recognition capabilities.

[0045] The monitoring video can be input by an external device, which can be an image acquisition device that collects video in the location. After obtaining the monitoring video, the device obtains the coordinates and movement speed of each part of the human body in the image in the monitoring video.

[0046] Exemplarily, the device is provided with a human skeleton recognition model, through which various parts of the human body in the image can be identified. Parts include facial parts and hand parts. Facial parts include the nose, eyes, ears, neck, etc., and hand parts include shoulders, elbows, and wrists, etc. After identifying the parts of the human body, the human skeleton recognition model marks the corresponding parts of the human body in the image, and the image has a corresponding image coordinate system. Through the image coordinate system and the position of the parts in the image, the device can determine the coordinates of each part. Furthermore, the device can set a three-dimensional human body model, and convert the two-dimensional coordinates of the parts in the image coordinate system into three-dimensional coordinates through the three-dimensional human body model.

[0047] The device can determine the movement speed of a part by the positional difference between adjacent images. For example, there is a certain time interval between adjacent images. The device calculates the distance the part has moved within the time interval based on the coordinates of the part in the adjacent images. The movement speed of the part is obtained by dividing the movement distance by the time interval.

[0048] Step S202, determining the distance between the hand part and the face part according to the coordinates of each part, and determining the angle of the upper limb according to the coordinates of each part.

[0049] After determining the movement speed of the hand, the device also needs to determine the distance between the hand and the face to determine whether the person is covering their face. For example, the device includes a 3D human model, and the coordinates of the hand are converted into 3D coordinates using the 3D human model. The coordinates of the face are also converted into 3D coordinates based on the 3D human model. The distance between the hand and face can be calculated using the 3D coordinates of the hand and face.

[0050] In addition, the device also needs to determine the angle of the upper limbs based on the three-dimensional coordinates of each part. Specifically, the angles of the upper limbs include the angle formed by the upper arm, elbow and forearm; the angle formed by the nose, neck and shoulder; and the angle formed by the upper arm, shoulder and neck. Exemplarily, the device obtains the three-dimensional coordinates of the elbow, shoulder and wrist, constructs the vector of the upper arm based on the three-dimensional coordinates of the elbow and shoulder, and constructs the vector of the forearm based on the three-dimensional coordinates of the elbow and wrist, and then calculates the angles of the upper arm, elbow and forearm through the two vectors.

[0051] Step S203, determining whether the human body is covering the face based on the distances between adjacent images in the monitoring video, determining whether the human body is hitting based on the angles between adjacent images, and determining whether the human body is sprinting based on the moving speeds of various parts in the adjacent images.

[0052] The device determines whether a person is obscuring their face by monitoring clusters corresponding to adjacent images in a video. For example, the adjacent images include two adjacent images. If the distance between the hand and the face in one image is less than a preset distance, and the distance between the hand and the face in the other image is also less than a preset distance, then the person is determined to be obscuring their face.

[0053] The device can also determine whether the human body is performing a striking motion based on the angles corresponding to adjacent images. For example, the device calculates the change in the angle of the same upper limb in the vector image. If the change in angle is large, it can be determined that the human body has extended the upper limb in a short period of time, and thus the human body can be determined to have performed a striking motion. For example, if the angle formed by the upper arm, elbow, and forearm shows a large extension movement in a short period of time, it can be determined that the human body has performed a striking motion.

[0054] Furthermore, the device can determine whether a person is sprinting based on the movement speeds of various parts in adjacent images. For example, if the movement speed of various parts in one of the adjacent images is greater than a preset speed, and the movement speed of various parts in the other image is also greater than a preset speed, then it can be determined that the person is sprinting, i.e., the person is sprinting.

[0055] Step S204 , when the human body performs a face-covering action, a striking action, and a sprinting action, determining a sequence in which the face-covering action, the striking action, and the sprinting action occur.

[0056] Step S205: When the sequence matches the preset sequence, it is determined that the human body has abnormal behavior.

[0057] When the device determines that a person has engaged in an abnormal action, such as face covering, striking, or sprinting, the device determines the order in which the face covering, striking, and sprinting actions occurred. The device stores a preset sequence corresponding to the abnormal behavior. For example, if the abnormal behavior is first face covering, then striking, and finally sprinting, the preset sequence is: face covering → striking → sprinting.

[0058] The device can determine the order in which the three actions, namely, the face-covering action, the striking action, and the sprinting action, occur by comparing their respective timings. For example, if the face-covering action occurs earlier than the striking action, which in turn occurs earlier than the sprinting action, then the order in which the face-covering, striking, and sprinting occur is: face-covering → striking → sprinting. The preset order is: face-covering → striking → sprinting. Therefore, if the sequence matches the preset order, it can be determined that an abnormal human movement has occurred.

[0059] If there is abnormal human behavior, the device can output an alarm message, thereby prompting that a violent incident has occurred in the place where the monitoring video is located.

[0060] In this embodiment, the coordinates of various parts of the human body in the image of the monitoring video and the movement speed of the parts are obtained. The distance of the hand part relative to the facial part is determined based on the coordinates of each part, and the angle of the upper limb is determined based on the coordinates of each part. Therefore, based on the distance, angle and movement speed of adjacent images in the monitoring video, it is determined whether the human body has made abnormal movements such as covering the face, striking, and sprinting. If there are face-covering movements, striking movements, and sprinting movements, the order in which the face-covering movements, striking movements, and sprinting movements are sounded is determined. If the order matches the preset order, it is determined that the human body has made abnormal behavior. In this embodiment, the device automatically identifies abnormal movements such as face-covering movements, striking movements, and sprinting movements in the monitoring video. If there are abnormal movements, it further identifies whether the human body has made abnormal behavior based on the abnormal movements. There is no need to manually review the monitoring video to determine the abnormal behavior of the human body, thereby improving the efficiency of identifying abnormal behavior.

[0061] Reference Figure 3 , Figure 3 This is the second embodiment of the abnormal behavior identification method of the present application. Based on the first embodiment, step S203 includes:

[0062] Step S301: Determine the duration of time that the human body maintains a distance in adjacent images according to the frame rate of the monitoring video.

[0063] In this embodiment, the device obtains the frame rate of the monitoring video, and the interval duration between the vector images can be calculated based on the frame rate. The device can determine the duration of the distance between the human body positions in adjacent images based on the interval duration. For example, the interval duration between the adjacent first image and the second image is 0.05 seconds, the playback time point of the first image is earlier than the playback time point of the second image, the distance between the hand part and the face part in the first image is 0.1m, and the distance between the hand part and the face part in the second image is 0.11m. The change in the two distances is 0.01m, which is less than the preset change of 0.02m. It can be considered that the time the human body maintains the 0.1m distance is 0.05 seconds; if the corresponding distance in the second image is 0.13, the change in the two distances is 0.02. This change is greater than or equal to the preset change of 0.02, and it can be considered that the time the human body maintains the 0.1m distance is 0 seconds.

[0064] Step S302 : extracting target durations from durations corresponding to adjacent images of the monitoring video. The target duration is a duration corresponding to a target distance, and the target distance is a distance less than a preset distance.

[0065] After the device obtains the duration corresponding to each distance, it determines the target duration in each duration. The target duration is the duration corresponding to the target distance, and the target distance is the distance less than the preset distance. For example, if the preset distance is 0.13m, then the distance less than 0.13m is the target distance, and the duration corresponding to the target distance is determined as the target duration. It should be noted that when the device extracts the target duration, it extracts it from the various durations of the same hand part relative to the same facial part. For example, each image of the monitoring video has the distance of the right wrist relative to the nose, that is, there are multiple distances, and the target duration is extracted from the durations corresponding to the multiple distances, that is, each target duration is the duration that the human body maintains the distance of the right wrist relative to the nose in different images. Of course, the device also needs to extract the target duration from the duration that the human body maintains the distance of the left wrist relative to the nose in different images, and the two target durations are classified and extracted.

[0066] Step S303: Determine whether the sum of the target durations is greater than a preset duration, wherein when the sum of the target durations is greater than the preset duration, it is determined that the human body is performing a face-covering action.

[0067] The device will calculate the sum of the target durations. If the sum of the target durations is greater than the preset duration, it can be determined that the human body is covering the face for a long time. In this case, the device determines that the human body has a conscious face-covering action, rather than an unintentional face-covering action.

[0068] It should be noted that if the sum of the target durations of the right hand part relative to the face part is greater than the preset duration, or the sum of the target durations of the left hand part relative to the face part is greater than the preset duration, it is considered that the human body has covered the face.

[0069] In this embodiment, the device determines the duration that the human body maintains the distance in adjacent images based on the frame rate of the monitored video, and thus extracts the target duration from each duration. If the sum of each target duration is greater than the preset duration, it can be determined that the human body has a face-covering action. That is, the device accurately determines whether the human body has a face-covering action by the cumulative duration that the human body maintains the distance between the hand part and the facial part.

[0070] Reference Figure 4 , Figure 4 This is the third embodiment of the abnormal behavior identification method of the present application. Based on the first embodiment, step S203 includes:

[0071] Step S401: determining a change in angle according to the angle between the same upper limb in a first image and a second image, where the first image and the second image are adjacent images.

[0072] In this embodiment, the device obtains the angle of the upper limb in the image and obtains the angle change of the same upper limb in the adjacent image. The adjacent image with the earlier playback time point is defined as the first image, and the image with the later playback time point is defined as the second image.

[0073] Step S402 : extracting a target angle variation from the angle variations corresponding to each adjacent image of the monitoring video, where the target angle variation is an angle variation greater than a preset angle.

[0074] The monitoring video includes multiple images, and thus has multiple adjacent images. Each adjacent image has a corresponding angle change. The device extracts a target angle change from each angle change. The target angle change is an angle change greater than a preset angle.

[0075] Step S403, determining whether the amount of target angle change is greater than a preset amount, wherein if the amount of target angle change is greater than the preset amount, it is determined that the human body has a striking action.

[0076] The device counts the number of target angle changes, specifically those of the same upper limb, and determines whether the number is greater than a preset number. If so, it indicates the person has rapidly and repeatedly extended their upper limb, and the device determines that the person has struck.

[0077] In this embodiment, the device accurately determines whether the human body has a striking action based on the change in the angle of the same upper limb in the first image and the second image.

[0078] Reference Figure 5 , Figure 5 This is the fourth embodiment of the abnormal behavior identification method of the present application. Based on the first embodiment, step S203 includes:

[0079] Step S501: construct multiple combined parts based on the various parts.

[0080] In this embodiment, the device needs to determine whether the human body is sprinting. Specifically, the device needs to determine whether the human upper limb combination has moved significantly within a short period of time. If so, it can be determined that the human body is sprinting. Specifically, the device constructs multiple combined parts based on various parts. The combined parts can be any combination of human body parts, for example, the shoulder and elbow can be combined parts, and the nose and glasses can be combined parts.

[0081] Step S502 , determining whether the moving speed of each combined part in each image of the adjacent images is greater than a preset speed.

[0082] Step S503: When the moving speeds of all combined parts are greater than a preset speed, it is determined that the human body is in a sprinting motion.

[0083] The device determines whether the moving speed of each combined part in each image of the adjacent images is greater than a preset speed. If the moving speed of each combined part in each image of the adjacent images is greater than the preset speed, it can be determined that the human body is sprinting.

[0084] For example, various parts can be arbitrarily combined to obtain multiple combined parts, each of which includes multiple parts, each of which corresponds to a movement speed, and the minimum movement speed can be used as the movement speed of the combined part. If the movement speeds of all combined parts in an image are greater than a preset speed, it can be determined that the person is sprinting, rather than just shaking their arms. Furthermore, if the movement speeds of all combined parts in adjacent images are greater than the preset speed, it can be determined that the person is continuously sprinting, rather than occasionally walking briskly, and thus it can be determined that the person is sprinting.

[0085] In this embodiment, the device constructs a combined part based on each part, and determines whether the movement speed of each combined part in each image in adjacent images is greater than a preset speed. If the movement speed of each combined part is greater than the preset speed, it can be determined that the human body is sprinting.

[0086] In one embodiment, the device is provided with a feature information extraction model for extracting the coordinates of each part, the movement speed, and the angle of the upper limb. The feature information extraction model may be an OpenPose model.

[0087] For example, because the distance between people in the monitored video image and the differences in human body shape will affect the final recognition accuracy, the coordinate information of the human body parts is normalized. The human skeleton can be defined as display points in the feature information extraction model: position 0 is the nose, position 1 is the neck, position 2 is the left shoulder, position 3 is the right shoulder, position 4 is the left elbow, position 5 is the right elbow, position 6 is the left wrist, position 7 is the right wrist, position 8 is the left hip joint, position 9 is the right hip joint, position 10 is the left eye, position 11 is the right eye, position 12 is the left ear, and position 13 is the right ear.

[0088] The video is divided into frame images, and the coordinate information of the above parts is obtained from the OpenPose model. The coordinates are normalized based on the nose coordinate point. Assume that the nose coordinate is c0=(c x ,c y ), then the coordinates of the i-th part are The upper body skeleton information can be represented by S: S = (C, E), C = {c0, c1, ... c 13},E={e0,e1,...e 13}, C is the coordinate of the human skeleton in each frame image, and E is the set of limb vectors. According to different time, the frame video position at different time is defined as: c t =(c xt ,y yt ).

[0089] In the OpenPose model, two skeletal parts of the human body are connected to form a limb. The smaller the Euclidean distance between the skeletal points, the closer the two skeletal points are. In actual monitoring scenarios, the length of the limbs of the upper body of the human body does not change significantly, so the distance ratio is used to represent the distance feature. in is the x-coordinate of the ith part, is the x-coordinate of the j-th part, is the y coordinate of the i-th part, is the y-coordinate of the jth part. In each frame, the distance between the left eye 10 and the right eye 11 is taken as an example and recorded as d(10,11). Since abnormal behaviors mostly occur in the arms and other limbs, the other distance features of the upper body are recorded as d(2,4), d(4,6), d(3,5), d(5,7), d(6,8), and d(7,9), for a total of 6 groups of distance features.

[0090] The angle formed between limbs is called limb angle. When the human body is doing any action, the physical manifestation feature is the change of each limb of the body. Therefore, the degree of change and the time of change of the angle between the limbs of the human body contain the characteristic information of the current human behavior. There are 6 groups of upper body limb angles with obvious movement, including the left elbow joint 2-4-6, the right elbow joint 3-5-7, the left shoulder joint 4-2-1, the right shoulder joint 3-5-1, the left neck 0-1-2, and the right neck 0-1-3. In order to measure the limb angle, it is necessary to know the distance between the parts, so the length of the bone parts is regularized in the video frame image. The distance between the parts can be expressed as r i,j , refers to the distance from part i to part j, then the angle of the limbs in the nth group can be expressed as: Then the human body action in an image can be expressed as A = [θ0, ...θ5], with a total of 6 groups of angle features.

[0091] The speed of a part can provide dynamic information about the user's movements. To calculate the speed of a part, the average speed of the part during the time Δt between two frames is used as the instantaneous speed of the part at time t; the distance the part moves from time Δt to time Δt+1 is: d ▽t+1 -d ▽t , then the velocity of the part at Δt is:

[0092] In this embodiment, through the above method, the feature information extraction model can determine the movement speed and coordinates of the parts and the angle of the upper limbs.

[0093] In one embodiment, the abnormal behavior recognition device includes a recognition model that is used to determine whether a person is covering their face based on the distance between adjacent images in the monitored video. The recognition model is also used to determine whether a person is striking based on the angle between adjacent images. The recognition model is also used to determine whether a person is sprinting based on the movement speed of various body parts in adjacent images. It is understood that the recognition model is used to identify whether a person is performing abnormal movements.

[0094] The recognition model is trained through various training samples. The training samples include sample videos and corresponding labels, where the labels are abnormal actions or normal actions.

[0095] For example, videos are collected online, including videos of normal and abnormal movements. Preprocessing is performed on the videos, for example, capturing 20 frames per second as the video to be processed. A feature extraction model is then used to label and detect human body parts in the video. Abnormal or normal movements are then used as labels to generate training samples. The device is then trained using these training samples to generate a recognition model.

[0096] This application also provides an abnormal behavior recognition device, referring to Figure 6 , the abnormal behavior identification device 600 includes:

[0097] An acquisition module 610 is used to obtain the coordinates of various parts of the human body in the image of the monitoring video and the movement speed of the parts, the parts including the face and the hands;

[0098] A first determination module 620 is configured to determine the distance of the hand relative to the facial part based on the coordinates of each part, and to determine the angle of the upper limb based on the coordinates of each part;

[0099] The second determination module 630 is configured to determine whether the human body is covering its face based on the distance between adjacent images in the monitoring video, determine whether the human body is striking based on the angle between adjacent images, and determine whether the human body is sprinting based on the movement speed of each body part in the adjacent images;

[0100] The third determining module 640 is configured to determine the order in which the face-covering action, the striking action, and the sprinting action occur when the human body performs the face-covering action, the striking action, and the sprinting action;

[0101] The fourth determining module 650 is configured to determine if abnormal behavior exists in the human body when the sequence matches the preset sequence.

[0102] In one embodiment, the second determining module 630 includes:

[0103] a first determining unit, configured to determine, based on a frame rate of the monitoring video, a duration for which a human body in adjacent images maintains a distance;

[0104] A first extraction unit is configured to extract each target duration from the durations corresponding to each adjacent image of the monitoring video, where the target duration is the duration corresponding to the target distance, and the target distance is a distance less than a preset distance;

[0105] The second determining unit is used to determine whether the sum of the target durations is greater than a preset duration, wherein when the sum of the target durations is greater than the preset duration, it is determined that the human body has a face-covering action.

[0106] In one embodiment, the second determining module 630 includes:

[0107] a third determining unit, configured to determine an angle variation according to an angle between the same upper limb in the first image and the second image, the first image and the second image being adjacent images;

[0108] A second extraction unit is used to extract a target angle variation from the angle variations corresponding to each adjacent image of the monitoring video, where the target angle variation is an angle variation greater than a preset angle;

[0109] The fourth determining unit is used to determine whether the amount of target angle change is greater than a preset amount, wherein if the amount of target angle change is greater than the preset amount, it is determined that the human body has a striking action.

[0110] In one embodiment, the second determining module 630 includes:

[0111] A construction unit, used for constructing a plurality of combined parts according to the various parts;

[0112] a fifth determining unit, configured to determine whether a moving speed of each combined part in each image of adjacent images is greater than a preset speed;

[0113] The sixth determining unit is configured to determine that the human body is sprinting when the moving speeds of the various combined parts are greater than a preset speed.

[0114] In one embodiment, the acquisition module 610 includes:

[0115] A first acquisition unit, configured to acquire monitoring video;

[0116] The second acquisition unit is used to acquire the coordinates of various parts of the human body in the image in the monitoring video when the human body exists in the image in the monitoring video.

[0117] In one embodiment, the abnormal behavior recognition device is provided with a feature information extraction model, which is used to obtain the coordinates of each part, the movement speed and the angle of the upper limb.

[0118] In one embodiment, the abnormal behavior recognition device is provided with a recognition model, which is used to determine whether a human body is covering its face based on the distances corresponding to adjacent images in the monitoring video;

[0119] The recognition model is also used to determine whether the human body has struck according to the angles corresponding to adjacent images;

[0120] The recognition model is also used to determine whether the human body is sprinting based on the movement speed of each part in adjacent images.

[0121] In one embodiment, the recognition model is obtained by training various training samples, and the training samples include sample videos and corresponding labels, where the labels are abnormal actions or normal actions.

[0122] Figure 7 The figure is a hardware structure diagram of an abnormal behavior recognition device according to an exemplary embodiment.

[0123] The abnormal behavior identification device 700 may include: a processor 71, such as a CPU, a memory 72, and a transceiver 73. Those skilled in the art will appreciate that Figure 7 The structure shown in the figure does not limit the abnormal behavior identification device and may include more or fewer components than shown, or a combination of certain components, or a different arrangement of components. The memory 72 can be implemented by any type of volatile or non-volatile memory device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0124] The processor 71 may call a computer program stored in the memory 72 or execute computer instructions to complete all or part of the steps of the above-mentioned abnormal behavior identification method.

[0125] The transceiver 73 is used to receive information sent by an external device and send information to the external device.

[0126] An electronic device comprises: a processor, and a memory communicatively connected to the processor;

[0127] Memory stores computer-executable instructions;

[0128] The processor executes the computer-executable instructions stored in the memory to implement the abnormal behavior identification method of any of the above embodiments.

[0129] A non-transitory computer-readable storage medium, when the instructions (computer-executable instructions) in the storage medium are executed by a processor of an abnormal behavior recognition device, enables the abnormal behavior recognition device to perform the above-mentioned abnormal behavior recognition method.

[0130] A computer program product includes a computer program. When the computer program is executed by a processor of an abnormal behavior recognition device, the abnormal behavior recognition device is enabled to perform the abnormal behavior recognition method.

[0131] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the following claims.

[0132] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for identifying abnormal behavior, characterized in that: Applied to an abnormal behavior recognition device, the abnormal behavior recognition method includes: Obtaining coordinates of various parts of a human body in an image in a monitoring video and movement speeds of the parts, the parts including facial parts and hand parts; Determining the distance between the hand part and the facial part according to the coordinates of each part, and determining the angle of the upper limb according to the coordinates of each part; determining whether the human body is covering its face based on the distances between adjacent images in the monitoring video, determining whether the human body is striking based on the angles between adjacent images, and determining whether the human body is sprinting based on the movement speeds of the respective parts in the adjacent images; When the human body performs the face-covering action, the striking action, and the sprinting action, determining a sequence in which the face-covering action, the striking action, and the sprinting action occur; When the sequence matches a preset sequence, it is determined that the human body has abnormal behavior.

2. The abnormal behavior identification method according to claim 1, characterized in that: The step of determining whether the human body has a face-covering action according to the distances corresponding to adjacent images in the monitoring video includes: determining, based on a frame rate of the monitoring video, a duration for which the human body in the adjacent images maintains the distance; Extracting each target duration from the durations corresponding to each adjacent image of the monitoring video, wherein the target duration is the duration corresponding to the target distance, and the target distance is the distance less than the preset distance; Determine whether the sum of the target durations is greater than a preset duration, wherein when the sum of the target durations is greater than the preset duration, it is determined that the human body has a face-covering action.

3. The abnormal behavior identification method according to claim 1, characterized in that: determining whether the human body has a striking action according to the angles corresponding to the adjacent images; determining an angle change amount based on an angle between a first image and a second image of the same upper limb, wherein the first image and the second image are adjacent images; Extracting a target angle variation from angle variations corresponding to adjacent images of the monitoring video, wherein the target angle variation is the angle variation greater than a preset angle; Determine whether the target angle variation is greater than a preset quantity, wherein if the target angle variation is greater than the preset quantity, determine that the human body has a striking action.

4. The abnormal behavior identification method according to claim 1, characterized in that: The step of determining whether the human body is sprinting according to the moving speed of each of the parts in the adjacent images comprises: constructing a plurality of combined parts according to each of the parts; determining whether a moving speed of each of the combined parts in each of the adjacent images is greater than a preset speed; When the moving speed of each of the combined parts is greater than the preset speed, it is determined that the human body is in a sprinting motion.

5. The abnormal behavior identification method according to claim 1, characterized in that: The step of obtaining the coordinates of various parts of the human body in the image in the monitoring video includes: Obtaining the monitoring video; When a human body exists in the image in the monitoring video, coordinates of various parts of the human body in the image in the monitoring video are obtained.

6. The abnormal behavior identification method according to any one of claims 1 to 5, characterized in that: The abnormal behavior recognition device is provided with a feature information extraction model, which is used to obtain the coordinates, movement speed and angle of each of the parts.

7. The abnormal behavior identification method according to any one of claims 1 to 5, characterized in that: The abnormal behavior recognition device is provided with a recognition model, and the recognition model is used to determine whether the human body has a face-covering action based on the distances corresponding to adjacent images in the monitoring video; The recognition model is further used to determine whether the human body has a striking action according to the angles corresponding to the adjacent images; The recognition model is further used to determine whether the human body is sprinting according to the moving speed of each of the parts in the adjacent images.

8. The abnormal behavior identification method according to claim 7, characterized in that: The recognition model is obtained by training various training samples, and the training samples include sample videos and corresponding labels, and the labels are abnormal actions or normal actions.

9. An abnormal behavior recognition device, characterized in that: include: An acquisition module is used to obtain the coordinates of various parts of the human body in the image of the monitoring video and the movement speed of the parts, wherein the parts include the face and the hands; a first determining module, configured to determine the distance between the hand part and the facial part according to the coordinates of each part, and to determine the angle of the upper limb according to the coordinates of each part; a second determining module, configured to determine whether the human body is covering its face based on the distances between adjacent images in the monitoring video, determine whether the human body is striking based on the angles between the adjacent images, and determine whether the human body is sprinting based on the movement speeds of the respective parts in the adjacent images; a third determining module, configured to determine a sequence in which the face-covering action, the striking action, and the sprinting action occur when the human body performs the face-covering action, the striking action, and the sprinting action; The fourth determining module is used to determine that the human body has abnormal behavior when the sequence matches a preset sequence.

10. An electronic device, characterized in that: include: a processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 8 when executed by a processor.

12. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images