Method and system for detecting abnormal behavior in a video image

Abstract

The application relates to a method and system for detecting abnormal behavior in a video image, and the method comprises the following steps: based on an obtained monitoring image stream, marking a target person and a plurality of reference persons from images of the monitoring image stream; respectively calculating a plurality of first relative positions between the target person and different reference persons according to relative position distances; calculating a first change sequence according to the monitoring image stream and the plurality of first relative positions corresponding to the monitoring image stream, wherein the first change sequence corresponds to change records of the plurality of first relative positions; calculating change trend data based on a preset normal behavior sequence and the first change sequence, wherein the change trend data corresponds to a trajectory intersection change trend of the target person and the plurality of reference persons; judging the change trend data based on a preset abnormal judgment threshold value, and if a corresponding behavior result is judged to be abnormal behavior, marking the target person as abnormal; and the application has the effect of marking abnormal behavior of a person in monitoring.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a method for detecting abnormal behavior in a video image and a system thereof. BACKGROUND

[0002] At present, with the increasingly stringent requirements for regional security, the requirements for security and protection systems of regional protection are also increasing, for example, the coverage rate of security and protection systems within a unit range needs to reach a full coverage state, so that there is no dead angle in regional protection.

[0003] In order to enable the monitoring network of the security and protection system to meet the above requirements, a huge monitoring network is needed, and at present, the monitoring network is mostly patrolled by security personnel through manual observation of the monitor. The larger the monitoring network is, the more security personnel are needed. Therefore, while the security and protection requirements are improved, the cost is also increased, and the more security personnel there are, the less effective the fault tolerance rate will be improved, and even it will decrease.

[0004] The existing detection method also uses an intelligent algorithm to recognize a face, and marks the recognized face in the monitoring network with a box to improve the monitoring efficiency of the security personnel. In this way, the security personnel can quickly see the number of personnel in the monitor, so as to help the security personnel focus their attention on the marked personnel.

[0005] However, the way of marking the face by the intelligent algorithm cannot mark the abnormal behavior of the personnel, and the security personnel still need to watch the monitor to analyze the dynamic change of the personnel. SUMMARY

[0006] In order to be able to mark the abnormal behavior of the personnel in the monitoring, the present application provides a method for detecting abnormal behavior in a video image and a system thereof.

[0007] In a first aspect, the present application provides a method for detecting abnormal behavior in a video image, which adopts the following technical scheme:

[0008] A method for detecting abnormal behavior in a video image, comprising the following steps:

[0009] Based on the obtained monitoring image stream, marking a target personnel and a plurality of reference personnel from the images of the monitoring image stream;

[0010] Respectively calculating a plurality of first relative positions between the target personnel and different reference personnel;

[0011] According to the monitoring image stream and a plurality of first relative positions corresponding to the monitoring image stream, a first change sequence is calculated, and the first change sequence corresponds to the change record of the plurality of first relative positions;

[0012] calculate a change trend data based on the preset normal behavior sequence and the first change sequence, the change trend data corresponding to a trajectory intersection change trend of the target person and the plurality of reference persons;

[0013] judge the change trend data based on a preset abnormal judgment threshold, if a corresponding behavior result is judged as an abnormal behavior, mark the target person as abnormal.

[0014] By adopting the above technical solution, the monitoring image stream contains a plurality of continuous images, and the plurality of images can also be spliced to restore a large-scale live picture. The target person and the plurality of reference persons around the target person can be marked by using an existing face recognition algorithm. In the process of the person walking, the relative positions between the target person and the reference persons change, and the change of the relative positions has different performance characteristics compared with normal walking and abnormal walking. A first change sequence is obtained through the change of a plurality of first relative positions, and the first change sequence contains the regular characteristics of the position change between the target person and the reference persons. Change trend data is calculated through the normal behavior sequence and the first change sequence to reflect the trajectory intersection change trend of the target person and the reference persons. Finally, it is judged whether the behavior of the target person is abnormal. The more frequently the trajectory of the target person intersects with the trajectories of the reference persons, the more abnormal the behavior of the target person is. The method can mark the abnormal behavior of the person in the monitoring.

[0015] As a preferred, the method further comprises the following steps:

[0016] extract a target trajectory of the target person and reference trajectories of the plurality of reference persons;

[0017] mark intersection points between the target trajectory and the plurality of reference trajectories, and calculate a uniformity of distribution of the intersection points;

[0018] if the uniformity is outside a preset uniformity range value, mark the target person as abnormal.

[0019] By adopting the above technical solution, the distribution uniformity of the trajectory intersection points of the target person and the reference persons is calculated. The behavior of the target person is analyzed according to the distribution uniformity. If the trajectory intersection points are too dense or too concentrated locally, it means that the target person may have abnormal behavior. The target person is marked as abnormal.

[0020] As a preferred, the method further comprises the following steps:

[0021] calculate a flow diffusion path of a plurality of surrounding persons beside the target person;

[0022] calculate a follow-up area of the surrounding persons based on the flow diffusion path;

[0023] If the target person is located in the follow-up area, the target person is marked as abnormal.

[0024] By using the above technical solution, the path of the target person and the surrounding people is calculated. When the target person causes the flow diffusion of the surrounding people, the target person will exist in the follow-up area of the surrounding people and will first drive the follow-up area. If the target person drives multiple surrounding people to follow, it means that the target person may have abnormal behavior, and the target person is marked as abnormal.

[0025] Preferably, the method further comprises the following steps:

[0026] Calculate the flow aggregation path of the multiple surrounding people beside the target person;

[0027] Based on the flow aggregation path, the aggregation area of the surrounding people is calculated;

[0028] If the target person is located at the center of the aggregation area, the target person is marked as abnormal.

[0029] By using the above technical solution, when the target person causes the flow aggregation of the surrounding people, the target person will exist at the center of the aggregation area of the surrounding people and will drive the aggregation area. If multiple surrounding people gather towards the target person, it means that the target person may have abnormal behavior, and the target person is marked as abnormal.

[0030] Preferably, the method further comprises the following steps:

[0031] Extract the first motion endpoint and the second motion endpoint possessed by the target person;

[0032] Calculate the second relative position between the first motion endpoint and the second motion endpoint;

[0033] According to the monitoring image stream and the multiple second relative positions corresponding to the monitoring image stream, a second change sequence is calculated, which corresponds to the change record of the multiple second relative positions;

[0034] Action trend data is calculated through the second change sequence, which corresponds to the relative position change trend of the first motion endpoint and the second motion endpoint;

[0035] Based on the preset action range value, the action trend data is judged. If the corresponding action result is judged to be abnormal action, the target person is marked as abnormal.

[0036] By adopting the technical scheme, the first motion end point and the second motion end point can be two relatively independent motion parts on the target person, when the target person normally walks, the relative motion of the two relatively independent motion parts has regularity, the motion trend data calculated by the third change sequence can reflect the regularity, and if the regularity is judged as abnormal motion, the target person is marked as abnormal.

[0037] As preferred, the method further comprises the following steps:

[0038] extracting a first motion end point and a second motion end point possessed by the target person;

[0039] calculating an included angle between a line connecting the first motion end point and the second motion end point and a gravity line;

[0040] calculating a third change sequence of the included angle according to the monitoring image stream and the plurality of included angles corresponding to the monitoring image stream, the third change sequence corresponding to change records of the plurality of included angles;

[0041] calculating included angle trend data from the third change sequence, the included angle trend data corresponding to a change trend of the included angle;

[0042] judging the included angle trend data based on a preset included angle range value, and marking the target person as abnormal if a corresponding included angle result is judged as abnormal motion.

[0043] By adopting the technical scheme, the first motion end point and the second motion end point can be two relatively independent motion parts on the target person, when the target person normally walks, the relative motion of the two relatively independent motion parts has regularity, the motion trend data calculated by the third change sequence can reflect the regularity, and if the regularity is judged as abnormal motion, the target person is marked as abnormal.

[0044] As preferred, the method further comprises the following steps:

[0045] extracting a first motion end point, a second motion end point and a third motion end point possessed by the target person;

[0046] calculating a region area of a region surrounded by the first motion end point, the second motion end point and the third motion end point;

[0047] calculating a fourth change sequence of the region area according to the monitoring image stream and the plurality of region areas corresponding to the monitoring image stream, the fourth change sequence corresponding to change records of the plurality of region areas;

[0048] The area trend data is calculated by the fourth change sequence, and the area trend data corresponds to the change trend of the area of the region.

[0049] The area trend data is judged based on the preset area range value, and if the corresponding area result is determined to be an abnormal action, the target person is marked as abnormal.

[0050] By adopting the above technical solution, the first motion end point, the second motion end point and the third motion end point can be three relatively independent motion parts on the target person's body. When the target person normally walks, the area between the three relatively independent motion parts will change regularly. The area trend data calculated by the fourth change sequence can reflect this regularity. If the regularity is determined to be an abnormal action, the target person is marked as abnormal.

[0051] As a preferred, the method further comprises the following steps:

[0052] The first motion end point, the second motion end point and the third motion end point possessed by the target person are extracted;

[0053] The infrared temperature data of the first motion end point, the second motion end point and the third motion end point are collected and fused into infrared imaging data;

[0054] The change speed of the infrared imaging data is calculated according to the monitoring image stream;

[0055] If the change speed is greater than a preset infrared value, the target person is marked as abnormal.

[0056] By adopting the above technical solution, the first motion end point, the second motion end point and the third motion end point can be three relatively independent heat parts on the target person's body. When the target person normally walks, the three relatively independent heat parts have relatively stable infrared performance. The calculated change speed can reflect this stability. If the stability is determined to be an abnormal reaction, the target person is marked as abnormal.

[0057] As a preferred, the method further comprises the following steps:

[0058] The infrared imaging data of the reference person is calculated;

[0059] According to the infrared imaging data of the target person and the infrared imaging data of the reference person, an infrared change sequence is calculated, and the infrared change sequence corresponds to the infrared imaging change trend of the target person and a plurality of reference persons;

[0060] The infrared change sequence and the first change sequence are fused into coincidence degree data;

[0061] judge the coincidence degree data based on the preset coincidence judgment threshold, and if the corresponding behavior result is judged to be abnormal behavior, mark the target person as abnormal.

[0062] By using the above technical solution, when the target person is normally walking, the three relatively independent heating parts have relatively stable infrared performance, and the stability of the infrared performance is related to the action amplitude and the body state of the target person. When the infrared performance stability of the target person does not match the action performance regularity, that is, the coincidence degree data is judged to be abnormal, the target person is marked as abnormal.

[0063] In a second aspect, the application provides a video image abnormal behavior detection system using the following technical solution:

[0064] A video image abnormal behavior detection system uses any of the above video image abnormal behavior detection methods.

[0065] In summary, the application has the following at least one beneficial technical effect:

[0066] (1) The relative positions between the target person and the reference person change, and the change of the relative positions has different performance characteristics. The change of the performance characteristics reflects the trajectory crossing trend of the target person and the reference person, and judges whether the behavior of the target person is abnormal behavior, so that the abnormal behavior of the person can be marked in the monitoring;

[0067] (2) The relative positions of the parts of the target person change when the target person is walking, the regular characteristics of the part change are reflected, and whether the behavior of the target person is abnormal behavior is judged according to the regular characteristics, so that the abnormal behavior of the person can be marked in the monitoring;

[0068] (3) The infrared imaging of the parts of the target person and the comparative change of the relative positions of the parts are compared when the target person is walking, the coincidence degree between the infrared and the parts is reflected, and whether the behavior of the target person is abnormal behavior is judged according to the regular coincidence degree, so that the abnormal behavior of the person can be marked in the monitoring. BRIEF DESCRIPTION OF DRAWINGS

[0069] Figure 1 is a method flowchart for judging abnormal behavior using change trend data in an embodiment;

[0070] Figure 2 is an imaging schematic diagram of one image in a monitoring image stream in an embodiment;

[0071] Figure 3 is an imaging schematic diagram of another image in a monitoring image stream in an embodiment;

[0072] Figure 4is a flowchart of a method of using uniformity to judge abnormal behavior in an embodiment;

[0073] Figure 5 is a schematic diagram of normal behavior of an image in an embodiment;

[0074] Figure 6 is a schematic diagram of abnormal behavior of another image in an embodiment;

[0075] Figure 7 is a flowchart of a method of using follow-up area to judge abnormal behavior in an embodiment;

[0076] Figure 8 is a schematic diagram of a flow expansion path and follow-up area of an image in an embodiment;

[0077] Figure 9 is a flowchart of a method of using aggregation area to judge abnormal behavior in an embodiment;

[0078] Figure 10 is a schematic diagram of a flow aggregation path and aggregation area of an image in an embodiment;

[0079] Figure 11 is a flowchart of a method of using motion trend data to judge abnormal behavior in an embodiment;

[0080] Figure 12 is a schematic diagram of monitoring a motion endpoint of an image in an image stream in an embodiment;

[0081] Figure 13 is a flowchart of a method of using angle trend data to judge abnormal behavior in an embodiment;

[0082] Figure 14 is a schematic diagram of monitoring an angle delta of an image in an image stream in an embodiment;

[0083] Figure 15 is a flowchart of a method of using area trend data to judge abnormal behavior in an embodiment;

[0084] Figure 16 is a schematic diagram of an area region of an image in an embodiment;

[0085] Figure 17 is a flowchart of a method of using change speed to judge abnormal behavior in an embodiment;

[0086] Figure 18 is a flowchart of a method of using coincidence degree data to judge abnormal behavior in an embodiment. DETAILED DESCRIPTION

[0087] The following description is made in connection with the accompanying drawings Figures 1-18 The application is further described in detail.

[0088] The embodiment of the application discloses a method for detecting abnormal behavior in a video image, based on a large monitoring system, which comprises a plurality of monitoring cameras, a splicable display screen, a cloud server and an offline server and the like hardware resources. The monitoring cameras shoot live pictures to form a monitoring image stream and transmit the monitoring image stream to the offline server. The offline server can splice the live pictures to restore a large live image to the splicable display screen, and the offline server also uploads the monitoring image stream or the large live image to the cloud server. The cloud server runs its built-in computing program to call an image processing interface to process the image, and the processing content can be face recognition to generate a processing result. The cloud server returns the processing result to the offline server and displays the processing result on the splicable display screen, and the processing result can be a marking box of the recognized face.

[0089] With reference to Figure 1 The method comprises the following steps:

[0090] Based on the obtained monitoring image stream, a target person and a plurality of reference persons are marked in the images of the monitoring image stream. The offline server obtains the monitoring image stream, and the monitoring image stream comprises a plurality of continuous images of a corresponding region. The offline server uploads the monitoring image stream to the cloud server, the cloud server returns the processing result to the offline server, and the offline server marks the face in the image. Then, the offline server can randomly select a person corresponding to one face as a target person, randomly select a plurality of other persons as reference persons, and mark the target person and the reference persons. When the selected target person is processed, the target person will not be selected as a target person in the next time, but can be selected as a reference person. If the offline server is disconnected from the cloud server, the offline server can start its built-in face recognition program to recognize the face.

[0091] A plurality of first relative positions are calculated between the target person and different reference persons. The offline server calculates the relative position distance between the target person and each reference person to obtain a plurality of first relative positions. The content of the first relative position can be a relative distance or a relative distance plus a relative relationship of the position, and the relative relationship of the position can be embodied as a direction vector formed by the two persons.

[0092] A first change sequence is calculated according to the monitoring image stream and a plurality of first relative positions corresponding to the monitoring image stream, and the first change sequence corresponds to the change record of the plurality of first relative positions. For example, Figure 2 And Figure 3As shown, each full-scene image corresponds to a plurality of first relative positions, and there are a plurality of full-scene images in the monitoring image stream, so there are a plurality of sets of the plurality of first relative positions, including the movement process of the two persons in the dynamic scene, so the plurality of sets of the plurality of first relative positions contain the movement process of the relative positions between the target person and the different reference persons, and the movement process is embodied by the first change sequence, for example, the first change sequence can save the cumulative displacement and cumulative rotation angle of each first relative position.

[0093] Returning to Figure 1 , the change trend data corresponding to the trajectory intersection change trend of the target person and the plurality of reference persons is calculated based on the preset normal behavior sequence and the first change sequence. The software in the offline server is built-in or preset with a normal behavior sequence, for example, the normal behavior sequence can be a sequence with the same format as the first change sequence but fixed content. For example, in the case of multiple people walking, the normal behavior of the target person and the reference persons should be to keep the relative position unchanged or the change of the relative position to be small and not large in amplitude, at which time the normal behavior sequence can include a distance component of 1 and a rotation angle component of 0.1, or the normal behavior sequence can include a displacement component of 0.1 and a rotation angle component of 0.1. The normal behavior sequence and the first change sequence are multiplied by a linear algebra method to obtain a value that can be used for judgment. Alternatively, the normal behavior sequence and the plurality of vectors in the first change sequence can be calculated to obtain the number of positive and negative values of the vector calculation result through the change of the angle, so that the number of positive and negative values of the value reflects the change trend of the trajectory intersection.

[0094] The change trend data is judged based on a preset abnormal judgment threshold, and if the corresponding behavior result is judged to be abnormal behavior, the target person is marked as abnormal. The offline server can judge by the size of the value or the number of positive and negative values of the value, for example, the abnormal judgment threshold is a value range, and the value calculated is judged by whether it is located in the range of the abnormal judgment threshold; or the abnormal judgment threshold is a positive and negative value range, and the number of positive and negative values calculated is judged by whether it is located in the range of the abnormal judgment threshold.

[0095] Further, as shown in Figure 4 , the target trajectory of the target person and the reference trajectories of the plurality of reference persons are extracted. The offline server can store the processing results from the cloud server by classification, save the position coordinates of the face of the same person on the scene in the form of a single linked list, so as to form the trajectory of the identified person. After the offline server selects the target person and the reference persons, the target trajectory of the target person and the reference trajectories of the reference persons can be extracted.

[0096] The intersection points between the target trajectory and the reference trajectories are marked, and the uniformity of the distribution of the intersection points is calculated. As shown in Figure 5 With Figure 6 As shown in the figure, the intersection points between the trajectories are calculated by a plurality of chain tables recording the trajectories, and the uniformity of the distribution of the intersection points is calculated based on the calculated distribution positions of the intersection points. The uniformity can be the intersection point density in different regions on the scene, and the uniformity can also be the number of intersection points in a region set with the intersection point as the center.

[0097] If the uniformity is outside the preset uniformity range value, the target person is marked as abnormal. The offline server can store or preset a plurality of uniformity range values. If the intersection point density is outside the corresponding uniformity range value or the number of intersection points is outside the corresponding uniformity range value, the target person is marked as abnormal.

[0098] Further, as shown in Figure 7 With Figure 8 As shown in the figure, the flow diffusion paths of the surrounding persons near the target person are calculated. The offline server calculates the direction vectors of the moving directions of the personnel trajectories, and the direction vectors can represent the diffusion directions of the paths.

[0099] Based on the flow diffusion paths, the following regions of the surrounding persons are calculated. The offline server can calculate the included angles between the direction vectors of adjacent persons, and calculate the following regions of the surrounding persons based on the change trend of the included angles. If the included angles of adjacent persons are gradually increasing, it means that the target person is pushing the diffusion of the surrounding persons in the corresponding region, that is, the target person is causing the shift of the following region. Therefore, the region surrounded by the surrounding persons with gradually increasing included angles can be a more accurate following region. If the target person is in the more accurate following region, the target person is marked as abnormal.

[0100] On this basis, as shown in Figure 9 With Figure 10 As shown in the figure, the offline server can also calculate the flow aggregation paths of the surrounding persons near the target person. The offline server calculates the moving coordinates of the personnel trajectories, and the latest moving coordinates are filtered by a window to obtain the latest positions of the personnel. When the area surrounded by the latest positions of the surrounding persons gradually decreases, it can represent the aggregation characteristics of the surrounding persons, and the set of the latest positions of the related surrounding persons is the flow aggregation path.

[0101] Based on the flow aggregation path, the aggregation region of the surrounding persons is calculated. The region surrounded by the latest positions of the surrounding persons can be the aggregation region, or the region surrounded by the flow aggregation path can be the aggregation region. If the target person is in the center position of the aggregation region, the target person is marked as abnormal.

[0102] Further, as shown in Figure 11 WithFigure 12 As shown, the first motion end point and the second motion end point of the target person are extracted, and the first motion end point and the second motion end point can be parts of the body of the person that are less active and more related to the behavior pattern of the person. For example, the first motion end point can be the arm or the hand, and the second motion end point can be the leg or the foot, wherein the calculation of the center point of the arm is the most stable; or the first motion end point can be the face, and the second motion end point can be the arm, the hand, or the foot.

[0103] A second relative position between the first motion end point and the second motion end point is calculated. Based on the face position coordinates of the target person, the offline server can obtain the position coordinates of the first motion end point and the position coordinates of the second motion end point through an edge detection algorithm or a human body recognition algorithm, and the second relative position is calculated through the position coordinates of the two.

[0104] A second change sequence is calculated according to the monitoring image stream and the plurality of second relative positions corresponding to the monitoring image stream, and the second change sequence corresponds to the change record of the plurality of second relative positions. Each full-scene image corresponds to a plurality of second relative positions, and there are a plurality of full-scene images in the monitoring image stream, so there are a plurality of sets of the plurality of second relative positions, which include the motion process of the two parts in the dynamic scene, so the plurality of sets of the plurality of second relative positions contain the motion process of the relative position between the first motion end point and the second motion end point, and the motion process is embodied through the second change sequence, for example, the cumulative displacement and the cumulative rotation angle of each second relative position can be saved in the second change sequence.

[0105] Action trend data is calculated through the second change sequence, and the action trend data corresponds to the change trend of the relative position between the first motion end point and the second motion end point. The action trend data can be the vector product between the combined vector of the cumulative displacement and the cumulative rotation angle and the unit vector vertically downward.

[0106] The action trend data is judged based on a preset action range value, and if the corresponding action result is judged to be an abnormal action, the target person is marked as abnormal. Under normal circumstances, the vector product is located in a certain range value, and the range value is preset in the offline server, which is the action range value. If the vector product is not within the action range value, the judgment result is an abnormal action.

[0107] Further, as shown in Figure 13 and Figure 14 The angle between the line connecting the first motion end point and the second motion end point and the gravity line is calculated, and the angle can be marked as δ.

[0108] According to the monitoring image stream and the plurality of angles corresponding to the monitoring image stream, a third change sequence of the angles is calculated, and the third change sequence corresponds to change records of the plurality of angles. Under normal actions of the human body in the process of walking, the change of the angles has an extractable regular characteristic, such as regular swinging actions.

[0109] A trend data of the angles is calculated through the third change sequence, and the trend data of the angles corresponds to a change trend of the angles, that is, the extractable regular characteristic. The trend data of the angles can be embodied as a swinging frequency, that is, a change frequency of the angles.

[0110] The trend data of the angles is judged based on a preset angle range value. If it is judged that the corresponding angle result is an abnormal action, the target person is marked as abnormal. Under normal circumstances, the change frequency of the angles is within a certain range value, and the range value is preset in the offline server, that is, the angle range value. If the change frequency of the angles is not within the angle range value, the judgment result is an abnormal action.

[0111] Further, as shown in Figure 15 and Figure 16 , a first motion endpoint, a second motion endpoint and a third motion endpoint possessed by the target person are extracted. The first motion endpoint, the second motion endpoint and the third motion endpoint can be parts of the body of the person with low activity correlation and high behavior pattern correlation, for example, the first motion endpoint can be the face; the second motion endpoint can be the arm, the hand or the foot, wherein the calculation effect of the center point of the arm is the most stable; and the third motion endpoint can be the leg or the foot, wherein the calculation effect of the center point of the thigh is the most stable.

[0112] The area area of the region surrounded by the first motion endpoint, the second motion endpoint and the third motion endpoint is calculated. Based on the face position coordinates of the target person, the offline server can obtain the position coordinates of the first motion endpoint, the second motion endpoint and the third motion endpoint through an edge detection algorithm or a human body recognition algorithm, and the area area is calculated through the position coordinates of the three.

[0113] According to the monitoring image stream and the plurality of angles corresponding to the monitoring image stream, a third change sequence of the angles is calculated, and the third change sequence corresponds to change records of the plurality of angles. Under normal actions of the human body in the process of walking, the change of the angles has an extractable regular characteristic, such as regular swinging actions.

[0114] The area trend data is calculated by the fourth change sequence, and corresponds to the change trend of the area. The area trend data can be embodied as the body movement amplitude, i.e. the change amplitude or change frequency of the whole body movement amplitude.

[0115] The area trend data is judged based on the preset area range value. If the corresponding area result is determined to be an abnormal action, the target person is marked as abnormal. Under normal circumstances, the change amplitude or change frequency of the area is within a certain range value, which is preset in the offline server as the area range value. If the area trend data is not within the area range value, the result is determined to be an abnormal action.

[0116] In addition, as shown in Figure 17 As shown in Figure 18 The infrared temperature data of the first, second and third movement endpoints can also be collected by the infrared sensor and fused into infrared imaging data. The infrared sensor can use infrared imaging temperature measuring instruments on the market, such as large-scale, multi-target identification infrared imaging temperature measuring instruments used at high-speed rail stations and airport entrances, which can output images with temperature marks and temperature isotherm data.

[0117] The change speed of the infrared imaging data is calculated according to the monitoring image stream. The change speed of the infrared imaging data can be the temperature change speed of the corresponding body part or the intensity of the body's exhaled gas.

[0118] If the change speed is greater than the preset infrared value, the target person is marked as abnormal. If the temperature change speed of the body part is too intense or the body's exhalation is too intense, it represents that the target person has an abnormal behavior.

[0119] Next, the infrared imaging data of the reference person can also be calculated. According to the infrared imaging data of the target person and the infrared imaging data of the reference person, the infrared change sequence is calculated, which corresponds to the infrared imaging change trend of the target person and the plurality of reference persons. The change of the infrared imaging data of the target person and the reference person is recorded for subsequent matching with the corresponding person's action.

[0120] The infrared change sequence and the first change sequence are fused into coincidence degree data. The matching data is the coincidence degree data, which is the corresponding degree of body infrared change and action change, i.e. the more intense the action, the more intense the infrared change. The coincidence degree data is judged based on the preset coincidence judgment threshold. If the corresponding behavior result is determined to be an abnormal behavior, i.e. the action features and infrared features of the target person and the reference person cannot be matched or cannot be corresponded, the target person is marked as abnormal.

[0121] In the process of applying the above method, the image stream contains a plurality of continuous images, and the plurality of images can be spliced to restore a large-scale live image. Through the existing face recognition algorithm, the target person and the surrounding reference person can be marked. In the process of the person's movement, the posture of the target person and the relative position between the target person and the reference person will change. Compared with the normal movement process, the change of the posture of the target person and the relative position between the target person and the reference person will have different performance characteristics.

[0122] In order to utilize the performance characteristics, a first change sequence can be obtained through the changes of the plurality of first relative positions, and the first change sequence contains the regularity of the position changes between the target person and the reference person. The change trend data is calculated through the normal behavior sequence and the first change sequence to reflect the trajectory intersection change trend of the target person and the reference person, or the distribution uniformity of the trajectory intersection points of the target person and the reference person is calculated, and the behavior of the target person is analyzed according to the distribution uniformity. Based on this, it is determined whether the behavior of the target person is an abnormal behavior, and then the abnormal behavior of the person is marked in the monitoring. The more frequently the trajectory of the target person intersects with the trajectory of the reference person in the movement process, the more abnormal the behavior of the target person is. In addition, if the trajectory intersection points are too dense or too concentrated locally, it means that the target person may have an abnormal behavior.

[0123] In addition to calculating the trajectory intersection change trend or the trajectory intersection point distribution, the flow diffusion or aggregation of the surrounding people can also be reflected by calculating the path of the target person and the surrounding people. When the flow diffusion or aggregation occurs, the target person will first drive the follow-up area or the aggregation area, and the target person may have an abnormal behavior.

[0124] In terms of utilizing the posture of the target person, the first motion endpoint and the second motion endpoint can be selected for analysis. In the posture change of the target person, the relative positions of two relatively independent moving parts have regularity. The action trend data calculated through the second change sequence or the included angle trend data calculated through the third change sequence can reflect this regularity. If the regularity is determined to be an abnormal action, the target person is marked as abnormal. The identification part on the target person can also be added. The area of the region between three or more relatively independent moving parts will have regular changes, and the heat of the part movement will also have regular accumulation or emission. The change speed or the area trend data calculated through the fourth change sequence can reflect the regularity, so as to determine whether the target person has an abnormal action.

[0125] After calculating the area trend data and the change speed, the stability of infrared performance and the correlation of action amplitude and body state can also be utilized to calculate the coincidence degree data. When the coincidence degree data is determined to be abnormal, the target person is marked as abnormal.

[0126] The embodiment of the present application discloses a system for detecting abnormal behavior in a video image, wherein the above-mentioned method for detecting abnormal behavior in a video image is used.

[0127] The above are preferred embodiments of the present application, and are not intended to limit the protection scope of the present application, so: all equivalent changes made according to the structure, shape, principle of the present application should be covered in the protection scope of the present application.

Claims

1. A method of detecting abnormal behavior in a video image, characterized by: The method comprises the following steps: Based on the obtained monitoring image stream, mark the target person and multiple reference persons from the images of the monitoring image stream; Respectively calculate the relative position distances between the target person and different reference persons to obtain multiple first relative positions; According to the monitoring image stream and multiple first relative positions corresponding to the monitoring image stream, calculate a first change sequence, which corresponds to the change records of multiple first relative positions; Based on the preset normal behavior sequence and the first change sequence, calculate change trend data, which corresponds to the trajectory intersection change trend of the target person and multiple reference persons; The software in the offline server is built-in or preset with a normal behavior sequence, which is selected as a sequence with the same format as the first change sequence but fixed content. The normal behavior sequence is multiplied with the first change sequence by linear algebra method, or multiple vectors in the normal behavior sequence and the first change sequence are calculated; Based on the preset abnormal judgment threshold, judge the change trend data. If the corresponding behavior result is determined to be abnormal behavior, the target person is marked as abnormal.

2. The method of claim 1, wherein: The method further comprises the following steps: Extract the target trajectory of the target person and the reference trajectories of multiple reference persons; Mark the intersection points between the target trajectory and multiple reference trajectories, and calculate the uniformity of the intersection point distribution; If the uniformity is outside the preset uniformity range value, the target person is marked as abnormal.

3. The method of claim 1, wherein: The method further comprises the following steps: Calculate the flow diffusion paths of multiple surrounding persons beside the target person; Based on the flow diffusion paths, calculate the following areas of the surrounding persons; If the target person is located in the following area, the target person is marked as abnormal.

4. The method of claim 1, wherein: The method further comprises the following steps: Calculate the flow diffusion paths of multiple surrounding persons beside the target person; Based on the flow diffusion paths, calculate the following areas of the surrounding persons; If the target person is located in the center of the following area, the target person is marked as abnormal.

5. The method of claim 1, wherein: The method further comprises the following steps: Extract the first motion endpoint and the second motion endpoint of the target person; Calculate the second relative position between the first motion endpoint and the second motion endpoint; According to the monitoring image stream and multiple second relative positions corresponding to the monitoring image stream, calculate a second change sequence, which corresponds to the change records of multiple second relative positions; Calculate action trend data from the second change sequence, which corresponds to the relative position change trend of the first motion endpoint and the second motion endpoint; Based on the preset action range value, judge the action trend data. If the corresponding action result is determined to be abnormal action, the target person is marked as abnormal.

6. The method of claim 1, wherein: The method further comprises the following steps: Extract the first motion endpoint and the second motion endpoint of the target person; Calculate the included angle between the line connecting the first motion endpoint and the second motion endpoint and the gravity line; According to the monitoring image stream and the plurality of angles corresponding to the monitoring image stream, a third change sequence of the angles is calculated, the third change sequence corresponding to a change record of the plurality of angles; An angle trend data is calculated through the third change sequence, the angle trend data corresponding to a change trend of the angles; The angle trend data is judged based on a preset angle range value, and if a corresponding angle result is judged to be an abnormal action, the target person is marked as abnormal.

7. The method of claim 1, wherein: The method further comprises the following steps: First, second and third motion end points of the target person are extracted; An area area of a region surrounded by the first, second and third motion end points is calculated; According to the monitoring image stream and the plurality of area areas corresponding to the monitoring image stream, a fourth change sequence of the area areas is calculated, the fourth change sequence corresponding to a change record of the plurality of area areas; An area trend data is calculated through the fourth change sequence, the area trend data corresponding to a change trend of the area areas; The area trend data is judged based on a preset area range value, and if a corresponding area result is judged to be an abnormal action, the target person is marked as abnormal.

8. The method of claim 1, wherein: The method further comprises the following steps: First, second and third motion end points of the target person are extracted; Infrared temperature data of the first, second and third motion end points is collected and fused into infrared imaging data; A change speed of the infrared imaging data is calculated according to the monitoring image stream; If the change speed is greater than a preset infrared value, the target person is marked as abnormal.

9. The method of claim 8, wherein: The method further comprises the following steps: The infrared imaging data of the reference person is calculated; According to the infrared imaging data of the target person and the infrared imaging data of the reference person, an infrared change sequence is calculated, the infrared change sequence corresponding to an infrared imaging change trend of the target person and the plurality of reference persons; The infrared change sequence and the first change sequence are fused into coincidence degree data; The coincidence degree data is judged based on a preset coincidence judgment threshold, and if a corresponding behavior result is judged to be an abnormal behavior, the target person is marked as abnormal.

10. A system for detecting abnormal behavior in a video image, characterized by: The method for detecting abnormal behavior in a video image according to any one of claims 1-9 is used.

Images