A method and system for monitoring abnormal behavior of personnel based on machine vision

Abstract

The present invention discloses a method and system for monitoring abnormal behavior of personnel based on machine vision, which relates to the technical field of monitoring abnormal behavior of personnel, and includes the following steps: collecting video information of personnel in a grid area in advance, and extracting personnel feature information from the video information, wherein the personnel feature information at least includes: the centroid information, facial information and identity information corresponding to the facial information of the current person. The present invention realizes a real-time monitoring and early warning mechanism, and can provide important scientific basis and decision-making support for the safety management of personnel in public places by accurately identifying and predicting potential unsafe behaviors such as strangers following, gathering of people during abnormal periods, and unusual stays, greatly improving the level of safety management in public places, effectively preventing possible safety accidents, and having a high degree of real-time, accuracy and predictability, providing new and effective technical means for safety management in modern society.

Description

Technical Field

[0001] The present invention relates to the technical field of abnormal behavior monitoring of personnel, and in particular to a method and system for abnormal behavior monitoring of personnel based on machine vision. Background Art

[0002] In recent years, with the development of big data, cloud computing, artificial intelligence and other fields, the rapid growth of data has brought severe challenges and valuable opportunities to many industries. Big data contains huge value. It not only has a significant role and significance in different fields such as society, economy, and scientific research, but also has penetrated into people's lives, providing sufficient information for people to more thoroughly understand and understand the material world. At the same time, with the improvement of modernization and the development of 5G and Internet of Things technology, residents are also paying more and more attention to the safety of public areas.

[0003] At present, more and more cameras are deployed for public area security, mostly used to monitor key areas in public areas. These facilities make it more likely that abnormal situations in public areas will be discovered in the first place. Even if they are not discovered in the first place, surveillance videos can provide strong evidence for later investigations, but there will still be some irreparable losses to people in public areas. Only by being able to discover potential dangers in time and issuing alarms to security personnel for early investigation can we prevent them before they happen. Improving the security level of public areas, increasing security prevention and control measures, and discovering or even preventing illegal and criminal acts in advance will help maintain a healthy public area environment, safeguard the interests of the masses, and give the masses more sense of security and confidence.

[0004] However, most of the existing public area network surveillance cameras only start monitoring functions, and cannot realize the intelligent distinction between people in public areas and the monitoring of abnormal behavior of people. Therefore, it is impossible to warn and prevent some illegal and dangerous behaviors in time, resulting in people in public areas often being harmed before they can get compensation and protection. However, some injuries are irreversible and cannot be compensated. Therefore, the existing public area network surveillance cameras cannot effectively prevent dangerous and illegal behaviors before they happen.

[0005] Currently, no effective solution has been proposed for the problems in the related technologies. Summary of the invention

[0006] In view of the problems in the related art, the present invention proposes a method and system for monitoring abnormal behavior of personnel based on machine vision to overcome the above-mentioned technical problems existing in the existing related art.

[0007] The technical solution of the present invention is achieved in this way:

[0008] In one aspect, the present invention:

[0009] A method and system for monitoring abnormal behavior of personnel based on machine vision, comprising the following steps:

[0010] Step S1, collecting video information of people in a grid area in advance, and extracting person feature information from the video information, wherein the person feature information at least includes: centroid information, facial information, and identity information corresponding to the facial information of the current person;

[0011] Step S2, predicting abnormal behavior based on personnel feature information, and tracking and calibrating personnel with abnormal behavior, including: detecting personnel movement trajectory for a single centroid information in a key frame image, detecting personnel tracking trajectory for two centroid information in a key frame image, or detecting personnel movement trajectory for multiple centroid information in a key frame image;

[0012] Step S3, performing facial expression recognition on the calibrated abnormal person to obtain the current emotional state of the abnormal person.

[0013] The step of collecting video information of personnel in the grid area includes the following steps:

[0014] Step S101, pre-arranging cameras in a grid in a selected area, and calibrating the geographical location of the camera coverage area;

[0015] Step S102, binding the person currently captured by the camera with the calibrated geographic location to obtain the current geographic location of the person.

[0016] Wherein, step S201, the step of detecting the movement trajectory of a person on the single centroid information in the key frame image, comprises the following steps:

[0017] The centroid positions of the current person in the calibrated key frame image and the next key frame image are , get personnel speed , expressed as:

[0018] ;

[0019] Get speed variance , expressed as:

[0020] ;

[0021] like ,and , and , it means that the current movement trajectory of the personnel is abnormal, and an early warning is output;

[0022] in, Indicates the frame rate, k represents the speed threshold, m is the vertical axis threshold, represents the average speed, D represents the threshold of velocity variance, represents the time threshold, Indicates the total number of centroid information velocities between multiple keyframes.

[0023] Wherein, step S202, performing personnel tracking trajectory detection on two centroid information in the key frame image, comprises the following steps:

[0024] The center of mass position of the current person in the calibrated key frame image is and , respectively represent the centroid point P 1 and P 2 In time t 1 The coordinate positions of the two centroids are expressed as:

[0025] ;

[0026] Elapsed time t 2 back, P 1 and P 2 The coordinate position is expressed as and , the centroid position is expressed as:

[0027] ;

[0028] Get the relative distance difference, expressed as: , including:

[0029] like d Keep constant and time T > T t When , it means that the relative distance between P1 and P2 has not changed, the current movement trajectory of the personnel is abnormal, and an early warning is output;

[0030] like d >0 and T < T t When , it means that the relative distance between P1 and P2 becomes larger, and the personnel are in a normal walking state;

[0031] like d＜0 When , it indicates that P1 and P2 are gradually decreasing, including:

[0032] If P1 is constant, P2 accelerates until , P2 maintains the same speed as P1 and T < T t , indicating the normal situation;

[0033] If P1 accelerates and P2 keeps constant, ,and T < T t , indicating normal behavior;

[0034] If P1 keeps constant speed, P2 accelerates and keeps accelerating, and T > T t , indicating that the current movement trajectory of the personnel is abnormal, and an early warning is output;

[0035] If P1 accelerates, P2 accelerates, and T > T t ， Indicates that the current movement trajectory of the personnel is abnormal and outputs a warning.

[0036] Among them, step S203, the step of performing personnel gathering trajectory detection on multiple centroid information in the key frame image, includes the following steps:

[0037] The center of mass position of the current person in the calibrated key frame image is ,in , Represents the centroid P 1 and P n In time t 1 The coordinate position of its centroid is expressed as d n ;

[0038] Elapsed time t 2 back, P 1 and P n The coordinate position is expressed as and , and its centroid position is expressed as ;

[0039] Get the relative distance difference, expressed as: , including:

[0040] like Keep constant and time T＜ Aggregation Threshold T m When P1 and P n The relative distance does not change and the current gathering time is a normal period;

[0041] like Keep constant and time T≥ Aggregation Threshold T m When P1 and P n If the relative distance does not change and the current gathering time is an abnormal period, an early warning is output.

[0042] Wherein, the step of obtaining the emotional state of the current abnormal person includes the following steps:

[0043] Step S301, pre-set multiple micro-expression areas, calibrate key frame images as intermediate time frames, and collect start frame images and end frame images;

[0044] Step S302, intercepting feature point images corresponding to a plurality of micro-expression regions set respectively from the start frame image and the end frame image;

[0045] Step S303, graying the feature point images of each micro-expression region to obtain a grayscale image of the micro-expression region;

[0046] Step S304, comparing the grayscale images of the same micro-expression region of the start frame image and the end frame image, which includes the following steps:

[0047] If there is a change in the grayscale image of the same micro-expression area, it is determined that there is micro-expression in the micro-expression area, and the key frame image of the feature point image with the change is selected as the first feature output;

[0048] Compare the first feature with the preset expression threshold value to obtain the emotion value;

[0049] The emotion values ​​corresponding to all current first features are combined as the second feature output, and the maximum value of the same emotion value in the second feature is taken as the current facial expression.

[0050] Wherein, the micro-expression area includes: mouth, nose, eyes and eyebrows.

[0051] The emotion values ​​include: happy, angry, surprised and natural.

[0052] Another aspect of the present invention is:

[0053] A personnel abnormal behavior monitoring system based on machine vision, comprising:

[0054] A collection module, used to collect video information of people in the grid area, and extract person feature information from the video information, wherein the person feature information at least includes: centroid information, facial information and identity information corresponding to the facial information of the current person;

[0055] The abnormal behavior prediction module is used to predict abnormal behavior based on personnel feature information and track and calibrate personnel with abnormal behavior, including: performing personnel movement trajectory detection on a single centroid information in a key frame image, performing personnel tracking trajectory detection on two centroid information in a key frame image, or performing personnel movement trajectory detection on multiple centroid information in a key frame image;

[0056] The expression recognition module is used to recognize the facial expressions of the calibrated abnormal persons and obtain the emotional state of the current abnormal persons.

[0057] Beneficial effects of the present invention:

[0058] The present invention is a machine vision-based personnel abnormal behavior monitoring method and system, which collects personnel video information in a grid area in advance, extracts personnel feature information from the video information, performs personnel movement trajectory detection on a single centroid information in a key frame image according to the personnel feature information, performs personnel tracking trajectory detection on two centroid information in the key frame image, or performs personnel movement trajectory detection on multiple centroid information in the key frame image, performs abnormal behavior prediction on the collected personnel, tracks and calibrates the personnel with abnormal behavior, and simultaneously performs facial expression recognition on the calibrated abnormal personnel to obtain the emotional state of the current abnormal personnel, realizes real-time monitoring and early warning mechanism, and can provide important scientific basis and decision-making support for personnel safety management and control in public places by accurately identifying and predicting potential unsafe behaviors such as strangers following, personnel gathering during abnormal time periods, and unusual stays, greatly improves the level of safety management in public places, effectively prevents possible safety accidents, has high real-time, accuracy and predictability, and provides new and effective technical means for safety management in modern society. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0060] Figure 1 is a flow chart of a method for monitoring abnormal behavior of personnel based on machine vision according to an embodiment of the present invention;

[0061] Figure 2 The present invention is a block diagram of a system for monitoring abnormal human behavior based on machine vision according to an embodiment of the present invention. DETAILED DESCRIPTION

[0062] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the present invention.

[0063] According to an embodiment of the present invention, a method for monitoring abnormal behavior of personnel based on machine vision is provided.

[0064] like Figure 1 As shown, the method for monitoring abnormal behavior of personnel based on machine vision according to an embodiment of the present invention includes the following steps:

[0065] Step S1, collecting video information of people in a grid area in advance, and extracting person feature information from the video information, wherein the person feature information at least includes: centroid information, facial information, and identity information corresponding to the facial information of the current person;

[0066] The process of collecting video information of people in the grid area includes the following steps:

[0067] Step S101, pre-arranging cameras in a grid in a selected area, and calibrating the geographical location of the camera coverage area;

[0068] Specifically, by deploying high-precision, intelligent facial recognition terminal equipment, such as high-definition cameras, at key locations such as community entrances, workplaces and public areas, a variety of key information can be captured and recorded in real time and uninterruptedly.

[0069] Step S102, binding the person currently captured by the camera with the calibrated geographic location to obtain the current geographic location of the person.

[0070] Specifically, the current person's location information is obtained based on the coverage area of ​​the camera.

[0071] With the help of the above scheme, the key frame images can be extracted by using K-means clustering to obtain the characteristics of the difference between adjacent images, and the frames with larger entropy values ​​can be selected as key frame images. In addition, for the above centroid information, the center of the human body contour can be simplified as a mass point.

[0072] Step S2, predicting abnormal behavior based on personnel feature information, and tracking and calibrating personnel with abnormal behavior, including: detecting personnel movement trajectory for a single centroid information in a key frame image, detecting personnel tracking trajectory for two centroid information in a key frame image, or detecting personnel movement trajectory for multiple centroid information in a key frame image;

[0073] Among them, step S201, performing personnel movement trajectory detection on a single centroid information in a key frame image, includes the following steps:

[0074] The centroid positions of the current person in the calibrated key frame image and the next key frame image are , get personnel speed , expressed as:

[0075] ;

[0076] Get speed variance , expressed as:

[0077] ;

[0078] like ,and , and , it means that the current movement trajectory of the personnel is abnormal, and an early warning is output;

[0079] in, Indicates the frame rate, k Indicates the speed threshold, the value is 1. m is the vertical axis threshold, the value is 10, represents the average speed, D represents the threshold of velocity variance, Indicates the time threshold, Indicates the total number of centroid information velocities between multiple keyframes.

[0080] In addition, step S202, performing personnel tracking trajectory detection on two centroid information in the key frame image, includes the following steps:

[0081] The center of mass position of the current person in the calibrated key frame image is and , respectively represent the centroid point P 1 and P 2 In time t 1 The coordinate positions of the two centroids are expressed as:

[0082] ;

[0083] Elapsed time t 2 back, P 1 and P 2 The coordinate position is expressed as and , the centroid position is expressed as:

[0084] ;

[0085] Get the relative distance difference, expressed as: , including:

[0086] like d Keep constant and time T > T t When , it means that the relative distance between P1 and P2 has not changed, that is, P2 always follows P1 at a certain distance, indicating that the current movement trajectory of the personnel is abnormal, and an early warning is output;

[0087] like d >0 and T < T t When , it means that the relative distance between P1 and P2 becomes larger, that is, the person is in a normal walking state;

[0088] like d＜0 When , it indicates that P1 and P2 are gradually decreasing, including:

[0089] Among them, if P1 is constant, P2 accelerates until , P2 maintains the same speed as P1 and T < T t , indicating a normal situation and no threat;

[0090] If P1 accelerates and P2 keeps constant, ,and T < T t , indicating normal behavior and not threatening;

[0091] If P1 keeps constant speed, P2 accelerates and keeps accelerating, and T > T t , which means that tailgating robbery may occur, indicating that the current movement trajectory of the person is abnormal, and outputting a warning;

[0092] If P1 accelerates, P2 accelerates, and T > T t ， If the person is being followed or chased, it means that the person's current movement trajectory is abnormal, and an early warning is output;

[0093] In addition, step S203, performing personnel gathering trajectory detection on multiple centroid information in the key frame image, includes the following steps:

[0094] The center of mass position of the current person in the calibrated key frame image is ,in , Represents the centroid P 1 andP n In time t 1 The coordinate position of its centroid is expressed as d n ;

[0095] Elapsed time t 2 back, P 1 and P n The coordinate position is expressed as and , and its centroid position is expressed as ;

[0096] Get the relative distance difference, expressed as: , including:

[0097] like Keep constant and time T＜ Aggregation Threshold T m When P1 and P n The relative distance does not change and the current gathering time is a normal period, that is, the current gathering of people is normal;

[0098] like Keep constant and time T≥ Aggregation Threshold T m When P1 and P n The relative distance does not change and the current gathering time is an abnormal period, that is, the current personnel gathering is abnormal, and an early warning is output.

[0099] Step S3, performing facial expression recognition on the calibrated abnormal person to obtain the current emotional state of the abnormal person, includes the following steps:

[0100] Step S301, pre-set multiple micro-expression areas, calibrate key frame images as intermediate time frames, and collect start frame images and end frame images;

[0101] Specifically, the micro-expression area includes: mouth, nose, eyes and eyebrows. The mouth, nose, eyes and eyebrows with micro-expression expression functions are selected as the micro-expression area.

[0102] At the same time, according to the intermediate time frame, a certain moment before the key frame image is used as the start frame image, and a certain moment after the key frame image is used as the end frame image, so as to obtain at least three expression images containing expressions.

[0103] Step S302, intercepting feature point images corresponding to a plurality of micro-expression regions set respectively from the start frame image and the end frame image;

[0104] This technical solution calibrates the mouth, nose, eyes and eyebrows for the starting frame image and the ending frame image respectively. For example, the mouth of the starting frame image is represented by 1A, and the mouth of the ending frame image is represented by 1B. And so on. The micro-expression areas of the mouth, nose, eyes and eyebrows are all calibrated to obtain four groups of corresponding feature point images.

[0105] Step S303, graying the feature point images of each micro-expression region to obtain a grayscale image of the micro-expression region;

[0106] Step S304, comparing the grayscale images of the same micro-expression region of the start frame image and the end frame image, which includes the following steps:

[0107] If there is a change in the grayscale image of the same micro-expression area, it is determined that there is micro-expression in the micro-expression area, and the key frame image of the feature point image with the change is selected as the first feature output;

[0108] Specifically, the four groups of corresponding feature point images processed by grayscale are compared respectively. For example, if there is a micro-expression area in any group at present, the key frame image is selected as the recognition object.

[0109] Compare the first feature with the preset expression threshold respectively to obtain the emotion value;

[0110] Among them, the emotion values ​​include: happy, angry, surprised and natural.

[0111] This technical solution selects the key frame image according to the micro-expression area, and compares the selected area with the corresponding preset expression threshold to obtain the emotion value of each selected area, such as: "happy, happy, happy and natural".

[0112] The emotion values ​​corresponding to all current first features are combined as the second feature output, and the maximum value of the same emotion value in the second feature is taken as the current facial expression.

[0113] With the aid of the above scheme, for example, "happy, happy, happy and natural" is output as the second feature, and the one with the highest repetition rate is selected as "happy", and the facial expression of the current key frame image is "happy".

[0114] With the help of the above solution, people with abnormal behaviors are tracked and calibrated, and facial expressions of the calibrated abnormal people are recognized simultaneously to obtain the emotional state of the current abnormal people, and realize real-time monitoring and early warning mechanism. By accurately identifying and predicting potential unsafe behaviors such as strangers following, gathering of people during abnormal time periods, and unusual stays, it can provide important scientific basis and decision-making support for the safety management of people in public places.

[0115] According to an embodiment of the present invention, a system for monitoring abnormal behavior of personnel based on machine vision is provided.

[0116] like Figure 2 As shown, the abnormal behavior monitoring system based on machine vision according to an embodiment of the present invention includes:

[0117] The acquisition module 1 is used to collect video information of people in the grid area and extract person feature information from the video information, wherein the person feature information at least includes: centroid information, facial information and identity information corresponding to the facial information of the current person;

[0118] Abnormal behavior prediction module 2, used for predicting abnormal behavior based on personnel feature information, and tracking and calibrating personnel with abnormal behavior, including: performing personnel movement trajectory detection on single centroid information in key frame images, performing personnel tracking trajectory detection on two centroid information in key frame images, or performing personnel movement trajectory detection on multiple centroid information in key frame images;

[0119] The expression recognition module 3 is used to perform facial expression recognition on the calibrated abnormal person and obtain the current emotional state of the abnormal person.

[0120] In summary, with the help of the above-mentioned technical scheme of the present invention, by pre-collecting video information of personnel in a grid area, and extracting personnel feature information from the video information, the single centroid information in the key frame image is detected according to the personnel feature information, and the two centroid information in the key frame image is detected for personnel tracking trajectory detection, or the multiple centroid information in the key frame image is detected for personnel movement trajectory detection, and abnormal behavior prediction of the collected personnel is performed, and the abnormal behavior personnel are tracked and calibrated, and the facial expression recognition of the calibrated abnormal personnel is performed simultaneously to obtain the emotional state of the current abnormal personnel, and realize real-time monitoring and early warning mechanism. By accurately identifying and predicting potential unsafe behaviors such as strangers following, gathering of people during abnormal time periods, and unusual stays, it can provide important scientific basis and decision-making support for personnel safety management and control in public places, greatly improve the level of safety management in public places, and effectively prevent possible safety accidents. It has a high degree of real-time, accuracy and predictability, and provides new and effective technical means for safety management in modern society.

[0121] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. After considering the disclosure of the specification and the examples, those skilled in the art will easily think of other embodiments of the present disclosure. This application is intended to cover any modification, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The specification and the examples are only regarded as exemplary, and the true scope and spirit of the present disclosure are indicated by the claims.

[0122] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for monitoring abnormal behavior of personnel based on machine vision, characterized in that: The following steps are involved: Collecting video information of people in the grid area in advance, and extracting person feature information from the video information, wherein the person feature information at least includes: centroid information, facial information and identity information corresponding to the facial information of the current person; Predict abnormal behavior based on personnel feature information, and track and calibrate personnel with abnormal behavior, including: detecting personnel movement trajectory for single centroid information in key frame images, detecting personnel tracking trajectory for two centroid information in key frame images, or detecting personnel movement trajectory for multiple centroid information in key frame images; Perform facial expression recognition on the calibrated abnormal persons to obtain the emotional state of the current abnormal persons; The detection of the movement trajectory of a person based on the single centroid information in the key frame image includes the following steps: calibrating the centroid positions of the current person in the key frame image and the next key frame image as follows: , get personnel speed , expressed as: ； Get speed variance , expressed as: ； like ,and , and , it means that the current movement trajectory of the personnel is abnormal, and an early warning is output; in, Indicates the frame rate, k represents the speed threshold, m is the vertical axis threshold, represents the average speed, D represents the threshold of velocity variance, represents the time threshold, Represents the total number of centroid information velocities between multiple keyframes; The method of performing personnel tracking trajectory detection on two centroid information in the key frame image includes the following steps: calibrating the centroid position of the current person in the key frame image as and , respectively represent the centroid point P 1 and P 2 In time t 1 The coordinate positions of the two centroids are expressed as: ； Elapsed time t 2 back, P 1 and P 2 The coordinate position is expressed as and , the centroid position is expressed as: ； Get the relative distance difference, expressed as: , including: like d Keep constant and time T>T t When , it means that the relative distance between P1 and P2 has not changed, the current movement trajectory of the personnel is abnormal, and an early warning is output; like d >0 and T<T t When , it means that the relative distance between P1 and P2 becomes larger, and the personnel are in a normal walking state; like d＜0 When , it indicates that P1 and P2 are gradually decreasing, including: If P1 is constant, P2 accelerates until , P2 maintains the same speed as P1 and T<T t , indicating the normal situation; If P1 accelerates and P2 keeps constant, ,and T<T t , indicating normal behavior; If P1 keeps constant speed, P2 accelerates and keeps accelerating, and T>T t , indicating that the current movement trajectory of the personnel is abnormal, and an early warning is output; If P1 accelerates, P2 accelerates, and T>T t ， Indicates that the current movement trajectory of the personnel is abnormal and outputs an early warning; The method of detecting the gathering trajectory of people based on the multiple centroid information in the key frame image includes the following steps: calibrating the centroid position of the current person in the key frame image as ,in , Represents the centroid P 1 and P n In time t 1 The coordinate position of the center of mass is expressed as d n ; Elapsed time t 2 back, P 1 and P n The coordinate position is expressed as and , and its centroid position is expressed as ; Get the relative distance difference, expressed as: , including: like Keep constant and time T＜ Aggregation Threshold T m When P1 and P n The relative distance does not change and the current gathering time is a normal period; like Keep constant and time T≥ Aggregation Threshold T m When P1 and P n If the relative distance does not change and the current gathering time is an abnormal period, an early warning is output.

2. The method for monitoring abnormal behavior of personnel based on machine vision according to claim 1, characterized in that: The method of collecting video information of personnel in the grid area includes the following steps: Arrange cameras in a grid in the selected area in advance and calibrate the geographical location of the camera coverage area; Bind the person captured by the current camera with the calibrated geographic location to obtain the current person's geographic location.

3. The method for monitoring abnormal behavior of personnel based on machine vision according to claim 1, characterized in that: The step of obtaining the emotional state of the current abnormal person includes the following steps: Preset multiple micro-expression areas, calibrate key frame images as intermediate time frames, and collect start frame images and end frame images; The starting frame image and the ending frame image are respectively intercepted to obtain feature point images corresponding to the set multiple micro-expression areas; Grayscale the feature point images of each micro-expression area to obtain a grayscale image of the micro-expression area; Comparing the grayscale images of the same micro-expression region of the starting frame image and the ending frame image, wherein the following steps are included: If there is a change in the grayscale image of the same micro-expression area, it is determined that there is micro-expression in the micro-expression area, and the key frame image of the feature point image with the change is selected as the first feature output; Compare the first feature with the preset expression threshold value to obtain the emotion value; The emotion values ​​corresponding to all current first features are combined as the second feature output, and the maximum value of the same emotion value in the second feature is taken as the current facial expression.

4. The method for monitoring abnormal behavior of personnel based on machine vision according to claim 3, characterized in that: The micro-expression area includes: mouth, nose, eyes and eyebrows.

5. The method for monitoring abnormal behavior of personnel based on machine vision according to claim 4, characterized in that: The emotion values ​​include: happy, angry, surprised and natural.

6. A system for monitoring abnormal behavior of personnel based on machine vision, used in the system of the method for monitoring abnormal behavior of personnel based on machine vision according to any one of claims 1 to 5, characterized in that: include: A collection module (1) is used to collect video information of people in a grid area and extract person feature information from the video information, wherein the person feature information at least includes: centroid information, facial information and identity information corresponding to the facial information of the current person; The abnormal behavior prediction module (2) is used to predict abnormal behavior based on personnel feature information and track and calibrate personnel with abnormal behavior, including: detecting the movement trajectory of personnel based on single centroid information in key frame images, detecting the movement trajectory of personnel based on two centroid information in key frame images, or detecting the movement trajectory of personnel based on single centroid information in key frame images; The expression recognition module (3) is used to perform facial expression recognition on the calibrated abnormal person and obtain the current emotional state of the abnormal person.

Images