Personnel evaluation method and system

Abstract

The invention relates to the field of image analysis, in particular to a personnel evaluation method and system, and the method comprises the steps: obtaining real-time video data of a target region of a production site; dynamic frame extraction is carried out on the real-time video data, and when a target frame containing personnel information is obtained, face recognition and posture feature extraction are carried out on the target frame; as the motion tracks of the production field are relatively fixed, the target equipment of the current personnel is deduced based on the personnel identity information, and the motion direction and the inertial attitude characteristics of the personnel are determined according to the attitude characteristics, so that the prediction track of the current personnel is comprehensively predicted. And judging whether a risk area exists in front through the predicted trajectory, if so, analyzing whether a risk exists when the current person passes through the risk area according to the current feature in combination with the inertial attitude feature, and carrying out risk warning in advance. According to the invention, the risk when the worker passes through the risk area is assessed, and the problem of safety accidents caused by unintentional contact with the production equipment is avoided.

Description

Technical Field

[0001] The present invention relates to the field of image analysis, and in particular to a personnel assessment method and system. Background Art

[0002] Safety in production is the top priority of enterprise management. Although the introduction of more automated equipment has rapidly improved production efficiency, it has also brought more safety issues. More safety issues often occur in automated equipment.

[0003] Existing safety management is more reflected in safety training and safety awareness. However, the on-site situation is often complicated, and there are always times when personnel are careless. For example, when carrying objects, on-site personnel often keep their hands or elbows in a stable and outward posture. When passing through production equipment, if they are careless, their hands or elbows may come into contact with the equipment, causing safety accidents. Existing safety management and risk assessment methods often fail to warn of the above situations. Summary of the invention

[0004] In view of this, an object of the present invention is to provide a personnel assessment method and system to solve the problems in the background technology.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] A personnel evaluation method of the present invention comprises the steps of:

[0007] Acquire real-time video data of a target area of ​​a production site, wherein the real-time video data is collected by a camera preset at the target location;

[0008] Extracting frames from the real-time video data based on motion characteristics to obtain multiple frames of images; and identifying the multiple frames of images, and when the multiple frames of images contain personnel information, taking the image frame containing the personnel information as the target frame;

[0009] Performing face recognition on the target frame to obtain person identity information; extracting posture features in each target frame to obtain a posture feature set; analyzing the posture feature set to obtain an inertial posture analysis result;

[0010] Determine the target device of the current person based on the person identity information, determine the current position of the current person based on the person identity information and the real-time video data, and generate a predicted trajectory based on the current position of the current person, the posture feature set and the position of the target device; and determine the warning risk area of ​​the current person based on the predicted trajectory and the production site;

[0011] A posture risk assessment is performed on the current person based on the inertial posture analysis result and the path risk area.

[0012] In one embodiment of the present application, the real-time video data is frame extracted based on motion characteristics to obtain multiple frames of images, including:

[0013] With the initial time interval T 0 Extracting frames from the real-time video data to obtain two frames of reference images;

[0014] Filter and grayscale the reference image to obtain two frames of pre-processed images;

[0015] Subtract two frames of pre-processed images to obtain a difference image;

[0016] Calculate the absolute value of the sum of the grayscale values ​​of all pixels in the difference image, and when the absolute value of the sum of the grayscale values ​​of all pixels in the difference image is greater than a preset threshold, adjust the sampling time interval to the time interval T 1 , where T 1 <T 0 .

[0017] In an embodiment of the present application, posture features in each target frame are extracted to obtain a posture feature set, including:

[0018] Preprocess each target frame to obtain, wherein the preprocessing method includes noise reduction and contrast enhancement;

[0019] Call the human key point extraction model to extract the human skeleton joint points of each target frame;

[0020] Connect the human skeleton joints of each target frame to obtain the posture features of each target frame;

[0021] A pose feature set is constructed based on the pose features of multiple target frames.

[0022] In one embodiment of the present application, the posture feature set is analyzed to obtain an inertial posture analysis result, including:

[0023] Determine the reference point in each pose feature and target point The reference point is the center of the line connecting the left hip joint and the right hip joint, and the target point is is one of the key points of the head, the left elbow joint, the right elbow joint, the left wrist joint and the right wrist joint, and i is the sequence number of the posture feature;

[0024] Based on the reference point and the target point The relative position vector is constructed by connecting the lines in, The reference point and the target point The distance, θ i The reference point and the target point The angle between the line connecting the two reference lines;

[0025] Multiple relative position vectors Mapping to a pre-constructed two-dimensional coordinate system, and performing a density-based clustering algorithm on multiple relative position vectors in the two-dimensional coordinate system to obtain one or more clusters;

[0026] When there is a target cluster with a data quantity greater than a preset threshold, it is determined that the current person has an inertial posture, and based on the relative position vector in the target cluster Constructing relative position vector template The relative position vector template As an inertial attitude feature.

[0027] In one embodiment of the present application, determining the current position of the current person based on the person identity information and the real-time video data includes:

[0028] Acquire a plan view of the production site, and capture a current frame image at a current time point from the real-time video data; wherein the plan view includes a plurality of production equipment and personnel passages surrounding the plurality of production equipment;

[0029] Extract multiple human reference points and ground reference points of posture features in the current frame image The ground reference point is the center point of the line connecting the left ankle joint and the right ankle joint, and the multiple human body reference points include the center point of the line connecting the left ankle joint and the right ankle joint, the midpoint of the line connecting the left knee joint and the right knee joint, the midpoint of the line connecting the left hip joint and the right hip joint, the midpoint of the line connecting the left shoulder joint and the right shoulder joint, and the key point of the head;

[0030] For multiple human reference points Connect the lines to obtain the human body reference line;

[0031] Extracting a human body contour line in the current frame image, and intercepting the reference line based on the human body contour line to obtain a human body height dimension h in the image;

[0032] The actual height H of the current person is obtained from the person identity information, and the distance S between the camera and the current person is calculated based on the human body height h in the image, the actual height H, and the camera focal length j, where the distance S is:

[0033]

[0034] Connect the ground reference point and the camera to obtain a direction line, and determine the angle γ between the camera orientation and the direction line from the current frame image;

[0035] Constructing the relative position (S, γ) of the current person and the camera based on the distance S between the camera and the current person and the angle γ between the camera orientation and the direction line;

[0036] The camera position is determined in the plan view, and the current position of the current person is determined based on the relative position (S, γ) between the current person and the camera.

[0037] In one embodiment of the present application, generating a predicted trajectory based on the current position of the current person, the posture feature set, and the position of the target device includes:

[0038] Generate one or more unidirectional candidate trajectories L in the plan view, starting from the current position and ending at the position of the target device. n , where n is a positive integer;

[0039] Acquire the posture feature at the current time point and the posture feature at the previous time point from the posture feature set, and determine the current displacement direction of the current person based on the ground reference point of the posture feature at the current time point and the ground reference point of the posture feature at the previous time point;

[0040] Determine the current position of the current person and each one-way candidate trajectory L n The shortest distance S between n , and determine the extension line of the current displacement direction of the current person and each unidirectional candidate trajectory L n The angle θ between n ;

[0041] Based on the shortest distance S n and the angle θ n Calculate each one-way candidate trajectory L n The probability P n , where each unidirectional candidate trajectory L n The probability P n for:

[0042]

[0043] In the formula, θ max is the maximum possible angle, θ min is the minimum possible angle, S max is the maximum possible distance, S min is the minimum possible distance, W 1 is the first weight, W2 is the second weight;

[0044] The probability P n The maximum number of candidate unidirectional trajectories L n as a predicted trajectory.

[0045] In one embodiment of the present application, determining the posture risk based on the inertial posture analysis result and the path risk area includes:

[0046] Determining an aisle width w of the pathway risk area based on the predicted trajectory;

[0047] The center line of the aisle is determined based on the aisle width w, wherein the center line is located at x=0, x is the horizontal coordinate of the center line, and the left and right boundaries of the aisle are located at and location;

[0048] Setting a Gaussian distribution function of the position where a pedestrian passes through the aisle;

[0049] For the Gaussian distribution function The segment is normalized so that the Gaussian distribution function is The sum of the probabilities within a segment is 1;

[0050] Determine a segment (-X, +X) in which the sum of probabilities in the normalized Gaussian distribution function is greater than a preset probability threshold;

[0051] Substitute the ground reference point in the inertial posture feature into the segment (-X, +X), and determine whether any human key point in the inertial posture feature contacts the boundary of the risk area when the ground reference point in the inertial posture feature is located at any position in the segment (-X, +X); if so, it is determined that the current person has a posture risk; if not, it is determined that the current person does not have a posture risk.

[0052] In one embodiment of the present application, it also includes:

[0053] When the current person is at risk of security, an early warning message is sent to the current person.

[0054] In one embodiment of the present application, the target device is determined based on historical work records of multiple employees.

[0055] The present application also provides a personnel evaluation system, comprising:

[0056] An acquisition module, used to acquire real-time video data of a target area of ​​a production site, wherein the real-time video data is collected by a camera preset at the target location;

[0057] A frame extraction module is used to extract frames from the real-time video data based on motion characteristics to obtain multiple frames of images; and to identify the multiple frames of images, and when the multiple frames of images contain personnel information, the image frame containing the personnel information is used as the target frame;

[0058] An image recognition module is used to perform face recognition on the target frame to obtain personal identity information; extract posture features in each target frame to obtain a posture feature set; analyze the posture feature set to obtain an inertial posture analysis result;

[0059] A trajectory analysis module, for determining a target device of the current person based on the person identity information, determining a current position of the current person based on the person identity information and the real-time video data, and generating a predicted trajectory based on the current position of the current person, the posture feature set, and the position of the target device; and determining a warning risk area of ​​the current person based on the predicted trajectory and the production site;

[0060] The risk assessment module is used to perform a fixed posture risk assessment on the current person based on the inertial posture analysis result and the path risk area.

[0061] The beneficial effects of the present invention are as follows: a personnel assessment method and system of the present invention obtains real-time video data of a target area of ​​a production site; then dynamically extracts frames of the real-time video data, and when a target frame containing personnel information is obtained, face recognition and posture feature extraction are performed on the target frame; since the motion trajectories of the production site are relatively fixed, the present application infers the target device of the current personnel based on the personnel identity information, and determines the movement direction and inertial posture features of the personnel based on the posture features, and then comprehensively predicts the predicted trajectory of the current personnel. It is determined by the predicted trajectory whether there is a risk area ahead, and if so, it is analyzed in combination with the inertial posture features whether there is a greater risk probability when the current personnel passes through the risk area with the current features, and a risk warning is issued in advance. The present application can effectively assess the risks of staff when passing through risk areas, and avoid the problem of inadvertent contact with production equipment and causing safety accidents. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] The present invention will be further described below in conjunction with the accompanying drawings and embodiments:

[0063] Figure 1 is an application scenario diagram of a personnel assessment method shown in an embodiment of the present application;

[0064] Figure 2 is a flow chart of a personnel evaluation method shown in one embodiment of the present application;

[0065] Figure 3A schematic diagram of the posture features in this application;

[0066] Figure 4 A schematic plan view of a production site in one embodiment of the present application;

[0067] Figure 5 A schematic diagram of inertial posture characteristics and risk areas in an embodiment of the present application;

[0068] Figure 6 is a structural diagram of a personnel evaluation system shown in one embodiment of the present application;

[0069] Figure 7 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application is shown. DETAILED DESCRIPTION

[0070] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0071] It should be noted that the illustrations provided in the following embodiments are only used to schematically illustrate the basic concept of the present invention. Therefore, the drawings only show the layers related to the present invention rather than being drawn according to the number, shape and size of the layers in actual implementation. In actual implementation, the type, quantity and proportion of each layer may be changed arbitrarily, and the layer layout may also be more complicated.

[0072] In the following description, numerous details are discussed to provide a more thorough explanation of embodiments of the present invention; however, it is apparent to one skilled in the art that embodiments of the present invention may be practiced without these specific details.

[0073] The information of the relevant persons involved in this application has been obtained with full consent and authorization, and the collection, use and processing of relevant information must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0074] Figure 1 is an application scenario diagram of a personnel evaluation method shown in an embodiment of the present application, such as Figure 1As shown, the present application sets a camera 110 at the entrance area or aisle area of ​​the production site. The camera 110 collects video data in real time and sends the video data to the host computer 120. The host computer 120 performs personnel risk assessment through a built-in algorithm, or the host computer 120 manages real-time video data and uploads images containing personnel data to the cloud server 130 for personnel risk assessment.

[0075] Figure 2 is a flow chart of a personnel evaluation method shown in an embodiment of the present application. Figure 2 As shown: A personnel evaluation method of this embodiment may include steps S210 to S250:

[0076] S210, acquiring real-time video data of a target area of ​​the production site, wherein the real-time video data is collected by a camera preset at the target location;

[0077] The camera at the target location is ceiling mounted or bracket mounted, which can better collect real-time monitoring images of the target area from top to bottom.

[0078] S220, extracting frames from the real-time video data based on motion characteristics to obtain multiple frames of images; and identifying the multiple frames of images, and when the multiple frames of images contain personnel information, taking the image frame containing the personnel information as the target frame;

[0079] In order to reduce the amount of subsequent calculations, this application uses a frame extraction method to determine whether there are moving objects in the target area. That is, based on the motion characteristics of the picture, when the picture is static, the picture is analyzed at a frequency of 2 frames per second. During the analysis process, once dynamic features appear, the frame extraction frequency is increased to 10 frames per second.

[0080] Specifically, the real-time video data is frame extracted based on the motion characteristics to obtain multiple frames of images, including:

[0081] S221, with the initial time interval T 0 Extracting frames from the real-time video data to obtain two frames of reference images;

[0082] In this embodiment, under normal circumstances, the initial time interval T 0 (i.e. every 0.5S) the real-time video data is extracted. In order to perform dynamic analysis, at least two reference images are required, i.e. the reference image at the current time point. t and the reference image at the previous time point t-1 .

[0083] S222, filtering and graying the reference image to obtain two frames of pre-processed images;

[0084] Specifically, the reference image image at the current time point is t and the reference image at the previous time point t-1 Perform grayscale conversion to obtain the grayscale image gray at the current time point t and the grayscale image gray at the previous time point t-1 ; In terms of filtering method, high-pass filtering is used to retain more contour details.

[0085] S223, performing subtraction on the two pre-processed images to obtain a subtraction image Dif t ;

[0086] Find the difference image Dif t = | gray t -gray t-1 |, find the difference image Dif t It is an image obtained by subtracting the grayscale values ​​of pixels at the same position in two grayscale images.

[0087] S224, calculating the absolute value of the sum of the grayscale values ​​of all the pixels of the difference image, and when the absolute value of the sum of the grayscale values ​​of all the pixels of the difference image is greater than a preset threshold, adjusting the sampling time interval to the time interval T 1 , where T 1 <T 0 .

[0088] Finally, the difference image Dif t The grayscale value of the last pixel in the image is summed, that is, ∑Dif(i,j). The sum of the grayscale values ​​can reflect the dynamic difference between the two images. If the difference is small, it means that the target area is static at the current time point. If the difference is large, it means that the target area has dynamic characteristics. At this time, the frame rate is increased to 10 frames per second.

[0089] For the extracted image frames, it is also necessary to ensure that the image contains person information. Therefore, the present application identifies the image frames based on the pre-trained person recognition model to check whether there are human features or facial features inside.

[0090] The personnel recognition model in this application is based on a CNN convolutional neural network. The CNN convolutional neural network is trained using pre-prepared training data. The personnel recognition model filters the image frames to retain the target frames containing personnel information.

[0091] S230, performing face recognition on the target frame to obtain person identity information; extracting posture features in each target frame to obtain a posture feature set; analyzing the posture feature set to obtain an inertial posture analysis result;

[0092] The face recognition in step S230 is also based on the personnel recognition model constructed in the above process. The camera in this application is facing the entrance or set in the corner of the aisle to ensure that the face of the person can be extracted in the picture.

[0093] After identification, the personnel’s personal information, including name, position, height, department, etc., can be retrieved from the production management system.

[0094] Then, since the target frame has been ensured to contain personnel information, in order to further verify whether the person is a risk, we need to extract the posture features in each target frame.

[0095] Specifically, the process of acquiring the posture feature set includes:

[0096] S2311, preprocessing each target frame to obtain, wherein the preprocessing method includes noise reduction and contrast enhancement;

[0097] S2312, calling a human key point extraction model to extract human skeleton joint points of each target frame;

[0098] In this application, OpenPose is used to collect the key points of the human body in the target frame. Thus, 25 key points of the human body in the target frame are marked, which are: nose, left eye, right eye, left ear, and right ear belonging to the head. Since this application does not involve the analysis and processing of facial expressions, the nose key point can be used as the head key point; left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hand (fingertips), and right hand (fingertips) belonging to the upper limbs; neck (usually the midpoint of the line connecting the two shoulders) and center point (the center of the chest or abdomen, sometimes also called the base of the spine or the center of the pelvis) belonging to the torso; left hip, right hip, left knee, right knee, left ankle, right ankle, left heel, right heel, left toe, and right toe belonging to the lower limbs.

[0099] S2313, connecting the human skeleton joints of each target frame to obtain the posture features of each target frame;

[0100] After marking the key points of the human body, the posture features of each target frame can be obtained by connecting them according to the human body features. In actual application, some unimportant key points can be omitted to reduce the amount of calculation. Figure 3 is a schematic diagram of the posture features in this application. The human posture features extracted in this embodiment are as follows Figure 3 shown.

[0101] S2314, constructing a posture feature set based on the posture features of multiple target frames.

[0102] Finally, the pose features of multiple target frames are placed into the set Pose in chronological order to obtain a temporal pose feature set.

[0103] When people are moving, their postures are constantly changing within the normal range. However, if a person is carrying items, or has some unusual habitual movements (such as swinging hands with a large amplitude, holding the head with both hands and stretching when walking, etc.), their postures will change. If these postures are continued in the risk area, it is possible that the wrists, elbows and other areas will come into contact with the risk area (equipment with electricity, or with cutting and transmission functions). Therefore, after obtaining the posture feature set Pose, it is first necessary to analyze the inertial posture features of the person and view the inertial posture features of the current person. This will facilitate the subsequent inertial posture features to be combined with the analysis of the risk area.

[0104] Specifically, the inertial posture analysis process in this application includes:

[0105] S2321, determine the reference point in each posture feature and target point The reference point is the center of the line connecting the left hip joint and the right hip joint, and the target point is is one of the key points of the head, the left elbow joint, the right elbow joint, the left wrist joint and the right wrist joint, and i is the sequence number of the posture feature;

[0106] Reference Points It is a point where a person remains relatively still during walking, such as the waist. Therefore, this application uses the center of the line connecting the left hip joint and the right hip joint as a reference point, and analyzes the stability of other human feature points in the time axis from the perspective of the reference point, so as to determine whether there is an inertial posture feature.

[0107] S2322, based on the reference point and the target point The relative position vector is constructed by connecting the lines in, The reference point and the target point The distance, θ i The reference point and the target point The angle between the line connecting the two reference lines;

[0108] This application constructs reference points on a two-dimensional plane and target point The relative position vector includes the distance parameter and the angle parameter. The distance and angle are used to determine whether other key points of the human body have changed relative to the reference point.

[0109] S2323, multiple relative position vectors Mapping to a pre-constructed two-dimensional coordinate system, and performing a density-based clustering algorithm on multiple relative position vectors in the two-dimensional coordinate system to obtain one or more clusters;

[0110] Since it is necessary to perform inertial analysis on multiple key points of the human body, that is, if a person maintains the same posture, the relative position vectors generated are similar or identical. Therefore, it is necessary to cluster the large number of relative position vectors of various key points of the human body and reference points constructed on the time axis in the previous article to obtain the same or similar clusters. If the number of samples in these clusters is large enough, they can be used as corresponding inertial posture feature clusters.

[0111] This application uses the DBSCAN (Density-Based Spatial Clustering of Applications with Noise) clustering algorithm. When clustering, you need to pay attention to setting the maximum distance within the cluster. A smaller maximum distance within the cluster is required to obtain the desired cluster of inertial posture features.

[0112] S2324, when there is a target cluster whose data quantity is greater than a preset threshold, it is determined that the current person has an inertial posture, and based on the relative position vector in the target cluster Constructing relative position vector template The relative position vector template As an inertial attitude feature.

[0113] Finally, if there are one or more target clusters whose data quantity is greater than the preset threshold, it means that the current person has one or more inertial postures in the video. For example, when carrying an object, both hands are still relative to the reference point; when both hands are swinging, multiple inertial postures will also be obtained.

[0114] S240, determining a target device of the current person based on the person identity information, determining a current position of the current person based on the person identity information and the real-time video data, and generating a predicted trajectory based on the current position of the current person, the posture feature set, and the position of the target device; and determining a path risk area of ​​the current person based on the predicted trajectory and the production site;

[0115] The purpose of extracting inertial posture features is to combine with subsequent areas for risk analysis. Therefore, step S240 is mainly to determine the risk area that the current person may pass through. The idea is roughly to determine the target device that the current person may go to, predict the trajectory based on the current position and the position of the target device, and combine the predicted trajectory with the actual situation of the production site to determine the possible risk area of ​​the current person's path.

[0116] The target device is determined based on the historical work records of multiple employees, and the frequently operated device of the current employee can be used as the target device.

[0117] The general idea of ​​locating the current position of the current person is to determine the relative position of the person and the camera. Since the position of the camera is known, the current position of the person can also be determined when the relative position of the person and the camera is determined. The specific process includes:

[0118] S2411, obtaining a plan view of the production site, and capturing a current frame image at a current time point from the real-time video data; wherein the plan view includes a plurality of production equipment and personnel passages surrounding the plurality of production equipment;

[0119] Figure 4 Schematic diagram of a production site in an embodiment of the present application. Figure 4 As shown, the production equipment is surrounded by personnel passages. If personnel want to reach the target equipment, they must go through the personnel passages.

[0120] S2412, extracting multiple human reference points and ground reference points of posture features in the current frame image The ground reference point is the center point of the line connecting the left ankle joint and the right ankle joint, and the multiple human body reference points include the center point of the line connecting the left ankle joint and the right ankle joint, the midpoint of the line connecting the left knee joint and the right knee joint, the midpoint of the line connecting the left hip joint and the right hip joint, the midpoint of the line connecting the left shoulder joint and the right shoulder joint, and the key point of the head;

[0121] This embodiment needs to extract the image frame at the current time point for analysis, and the image frame at the current time point represents the latest position of the current person. This application uses the zoom principle of the camera to infer the distance between the camera and the current person. Therefore, multiple human body reference points and ground reference points are first extracted.

[0122] S2413, for multiple human reference points Connect the lines to obtain the human body reference line;

[0123] Multiple human body reference points are located on the central axis of the human body, so the line connecting them can basically represent the posture of the human body on the axis.

[0124] S2414, extracting a human body contour line in the current frame image, and intercepting the reference line based on the human body contour line to obtain a human body height dimension h in the image;

[0125] The human body reference line portion intercepted by the contour line can be used as the height dimension in the image, and the height dimension in the image is obtained by counting the pixel points.

[0126] S2415, obtaining the actual height H of the current person from the person identity information, and calculating the distance S between the camera and the current person based on the human height dimension h in the image, the actual height H, and the camera focal length j, wherein the distance S is:

[0127]

[0128] S2416, connect the ground reference point and the camera to obtain a direction line, and determine the angle γ between the camera orientation and the direction line from the current frame image; wherein the camera orientation is the orientation on the horizontal plane, and the vertical angle is not considered here. Therefore, the angle γ is actually the angle on the two-dimensional horizontal plane.

[0129] S2417, constructing a relative position (S, γ) between the current person and the camera based on a distance S between the camera and the current person and an angle γ between the camera orientation and the direction line;

[0130] S2418, determining the camera position in the plan view, and determining the current position of the current person based on the relative position (S, γ) between the current person and the camera.

[0131] This application uses angle and distance to construct relative position, which can be regarded as a polar coordinate system. However, the coordinates of the camera in the plane are XY two-dimensional coordinates. Therefore, the current position of the current person can be determined by coordinate conversion. The relative position (S, γ) can also be directly mapped in the plane to obtain the current position of the current person.

[0132] The process of generating a predicted trajectory based on the current position of the current person, the posture feature set, and the position of the target device includes:

[0133] S2421, generating one or more one-way candidate trajectories L in the plan view, starting from the current position and ending at the position of the target device. n , where n is a positive integer;

[0134] Trajectory generation can call existing path planning algorithms, such as heuristic search algorithms, shortest path algorithms, etc.

[0135] S2422, acquiring the posture feature at the current time point and the posture feature at the previous time point from the posture feature set, and determining the current displacement direction of the current person based on the ground reference point of the posture feature at the current time point and the ground reference point of the posture feature at the previous time point;

[0136] The current displacement direction of the personnel is determined based on the ground reference points at two time points, namely

[0137] S2423, determining the current position of the current person and each one-way candidate trajectory L n The shortest distance S between n , and determine the extension line of the current displacement direction of the current person and each unidirectional candidate trajectory L n The angle θ between n ;

[0138] In this application, since a large number of candidate trajectories may be generated in the previous text, in order to find the most likely trajectory, this application considers the two aspects of the confluence distance and the angle. The confluence distance is the distance between the current position of the current person and each one-way candidate trajectory L n The shortest distance S between n The angle is used to reflect whether the current person's movement direction is consistent with the candidate trajectory. If it reflects a high consistency, that is, the angle is small, then it means that the current person is more likely to move along the candidate path. Therefore, the following formula is constructed to calculate the possibility.

[0139] S2424, based on the shortest distance S n and the angle θ n Calculate each one-way candidate trajectory L n The probability P n , where each unidirectional candidate trajectory I n The probability P n for:

[0140]

[0141] In the formula, θ max is the maximum possible angle, θ min is the minimum possible angle, S max is the maximum possible distance, S min is the minimum possible distance, W 1 is the first weight, W 2 is the second weight;

[0142] in, In fact, it is the reciprocal of the normalized angle. In fact, it is the reciprocal of the normalized shortest distance. Normalization is used to eliminate the dimension effect, and weighted summation is performed to obtain the possibility evaluation value of each candidate trajectory. The larger the evaluation value, the more likely the current person is to move along the candidate path.

[0143] S2425, the possibility P n The maximum number of candidate unidirectional trajectories L nSpecifically, 1-2 candidate trajectories with the highest likelihood evaluation values ​​are generally selected as the predicted trajectories.

[0144] After obtaining the predicted trajectory, combined with the floor plan of the production site, the equipment areas where risks may exist can be marked. If the predicted trajectory passes through this area, then this risk area is the pathway risk area predicted by this application.

[0145] Specifically, risk equipment may be high-voltage power equipment, cutting equipment, conveyor belt equipment, fans, etc. The boundaries of risk equipment are pre-delineated in the plan view as the basis for subsequent risk analysis. If the inertial posture of the current person may cause a part of the current person's body to touch the boundary of the risk equipment during the subsequent analysis, it is defined as a safety risk.

[0146] S250: Performing a fixed posture risk assessment on the current person based on the inertial posture analysis result and the path risk area.

[0147] The specific analysis process is as follows:

[0148] S251, determining the aisle width w of the path risk area based on the predicted trajectory;

[0149] Aisle width directly affects the risk assessment of current personnel. Generally speaking, pedestrians will walk in the middle of the aisle or avoid the side of the equipment. However, if the aisle width is narrow, even if pedestrians walk in the middle, their raised wrists or elbows may touch the boundary of the risk area, which will lead to a greater risk probability.

[0150] S252, determining the center line of the aisle based on the aisle width w, wherein the center line is located at x=0, x is the horizontal coordinate of the center line, and the left and right boundaries of the aisle are respectively located at and location;

[0151] The center line is the reference line and the trajectory line with the highest probability. Based on the center line and the boundary, a general probability density distribution model can be constructed to define the probability distribution characteristics of the pedestrian's passing position.

[0152] S253, setting a Gaussian distribution function of the position where the pedestrian passes through the aisle;

[0153] Specifically, the present application adopts the classic Gaussian distribution model to construct the probability distribution function of the position of pedestrians passing through a corridor with known width and make it conform to the Gaussian distribution. This process involves abstracting the actual problem into a mathematical model and selecting appropriate parameters to describe the position distribution of pedestrians.

[0154] The Gaussian distribution function is determined by the mean and standard deviation. In this application, we can assume that the position distribution of pedestrians in the aisle is symmetrical about the center line of the aisle, so the mean μ should be set to 0.

[0155] The standard deviation determines the extent to which pedestrians deviate from the center line. A reasonable standard deviation σ can be set based on empirical data or observations. If specific data is lacking, σ can be adjusted by trial and error until the simulated results look reasonable. Usually σ should be less than To ensure that most pedestrians remain in the aisle.

[0156] Therefore, the probability density function of the composite Gaussian distribution in this application should be:

[0157]

[0158] S254, for the Gaussian distribution function The segment is normalized so that the Gaussian distribution function is The sum of the probabilities within a segment is 1;

[0159] Since pedestrians cannot appear outside the aisle, we need to truncate the Gaussian distribution, that is, only consider In order to keep the sum of probabilities equal to 1, the truncated distribution needs to be renormalized. The calculation method is as follows:

[0160] Calculate the cumulative distribution function (CDF) of the original Gaussian distribution in the aisle range, that is, from arrive 's points.

[0161] Use this integral value to normalize the distribution so that the total area under the new distribution is equal to 1.

[0162] S255, determining a segment (-X, +X) in which the sum of probabilities in the normalized Gaussian distribution function is greater than a preset probability threshold;

[0163] By integration, we obtain a section (-X, +X) with a symmetry axis of x=0 and a total probability greater than 80%. This section can be used as a location section that pedestrians will pass through with a high probability.

[0164] S256, substitute the ground reference point in the inertial posture feature into the segment (-X, +X), and determine whether any human key point in the inertial posture feature contacts the boundary of the risk area when the ground reference point in the inertial posture feature is located at any position in the segment (-X, +X); if so, determine that the current person has a posture risk; if not, determine that the current person does not have a posture risk.

[0165] Figure 5 Schematic diagram of inertial posture characteristics and risk areas in an embodiment of the present application, such as Figure 5 As shown in the figure, if the ground reference point in the inertial posture feature falls into the side of the segment (-X, +X) closest to the risk area, the body just touches the boundary of the risk area, which means that if the current person passes through the risk area in this posture, there is a certain risk of triggering a safety accident. At this time, it is determined that the current person has a safety risk. When the current person has a safety risk, an early warning message is sent to the current person. The early warning notification can be issued through an intercom or an on-site loudspeaker.

[0166] A personnel assessment method of the present invention obtains real-time video data of a target area of ​​a production site; then dynamically extracts frames of the real-time video data, and when a target frame containing personnel information is obtained, face recognition and posture feature extraction are performed on the target frame; since the motion trajectories of the production site are relatively fixed, the present application infers the target device of the current personnel based on the personnel identity information, and determines the movement direction and inertial posture features of the personnel based on the posture features, and then comprehensively predicts the predicted trajectory of the current personnel. It is determined by the predicted trajectory whether there is a risk area ahead, and if so, it is analyzed in combination with the inertial posture features whether there is a greater risk probability when the current personnel passes through the risk area with the current features, and a risk warning is issued in advance. The present application can effectively assess the risks of staff members when passing through risk areas, and avoid the problem of inadvertent contact with production equipment and causing safety accidents.

[0167] like Figure 6 As shown, the present application also provides a personnel evaluation system, comprising:

[0168] An acquisition module, used to acquire real-time video data of a target area of ​​a production site, wherein the real-time video data is collected by a camera preset at the target location;

[0169] A frame extraction module is used to extract frames from the real-time video data based on motion characteristics to obtain multiple frames of images; and to identify the multiple frames of images, and when the multiple frames of images contain personnel information, the image frame containing the personnel information is used as the target frame;

[0170] An image recognition module is used to perform face recognition on the target frame to obtain personal identity information; extract posture features in each target frame to obtain a posture feature set; analyze the posture feature set to obtain an inertial posture analysis result;

[0171] A trajectory analysis module, for determining a target device of the current person based on the person identity information, determining a current position of the current person based on the person identity information and the real-time video data, and generating a predicted trajectory based on the current position of the current person, the posture feature set, and the position of the target device; and determining a warning risk area of ​​the current person based on the predicted trajectory and the production site;

[0172] The risk assessment module is used to perform a fixed posture risk assessment on the current person based on the inertial posture analysis result and the path risk area.

[0173] A personnel assessment system of the present invention obtains real-time video data of a target area of ​​a production site; then dynamically extracts frames of the real-time video data, and when a target frame containing personnel information is obtained, face recognition and posture feature extraction are performed on the target frame; since the motion trajectories of the production site are relatively fixed, the present application infers the target device of the current person based on the personnel identity information, and determines the movement direction and inertial posture features of the person based on the posture features, and then comprehensively predicts the predicted trajectory of the current person. By predicting the trajectory, it is determined whether there is a risk area ahead. If so, the inertial posture features are combined to analyze whether there is a high risk probability when the current person passes through the risk area with the current features, and a risk warning is issued in advance. The present application can effectively assess the risks of staff members when passing through risk areas, and avoid the problem of inadvertent contact with production equipment and causing safety accidents.

[0174] Figure 7 The structure diagram of the computer system of the electronic device suitable for implementing the embodiment of the present invention is shown. It should be noted that: Figure 7 The computer system of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0175] like Figure 7 As shown, the computer system includes a CPU 701 (Central Processing Unit), which can perform various appropriate actions and processes according to the program stored in a ROM 702 (Read-Only Memory) or the program loaded from a storage part 708 to a RAM 703 (Random Access Memory), such as executing the method in the above embodiment. In the RAM 703, various programs and data required for system operation are also stored. The CPU 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An I / O interface 705 is also connected to the bus 704.

[0176] The following components are connected to the I / O interface 705: an input section 706 including a keyboard, a mouse, etc.; an output section 707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 710 as needed so that a computer program read therefrom is installed into the storage section 708 as needed.

[0177] In particular, according to an embodiment of the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a computer readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through the communication part 709, and / or installed from a removable medium 711. When the computer program is executed by the CPU 701, various functions defined in the system of the present invention are executed.

[0178] It should be noted that the computer-readable medium shown in the embodiment of the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, an electrical, magnetic, optical, electromagnetic, infrared or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device or any suitable combination of the above. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable computer program. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. A computer program contained on a computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0179] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. Among them, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and the above-mentioned module, program segment or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart and the combination of boxes in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0180] The units involved in the embodiments of the present invention may be implemented by software or hardware, and the units described may also be arranged in a processor. The names of these units do not, in some cases, limit the units themselves.

[0181] Another aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor of a computer, causes the computer to execute the above method. The computer-readable storage medium may be included in the electronic device described in the above embodiment, or may exist independently without being assembled into the electronic device.

[0182] Another aspect of the present invention further provides a computer program product or a computer program, the computer program product or the computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method provided in each of the above embodiments.

[0183] The above embodiments are only preferred embodiments for fully illustrating the present invention, and the protection scope of the present invention is not limited thereto. Any equivalent substitution or change made by a person skilled in the art based on the present invention is within the protection scope of the present invention.

Claims

1. A personnel evaluation method, characterized in that: Includes steps: Acquire real-time video data of a target area of ​​a production site, wherein the real-time video data is collected by a camera preset at the target location; Extracting frames from the real-time video data based on motion characteristics to obtain multiple frames of images; and identifying the multiple frames of images, and when the multiple frames of images contain personnel information, taking the image frame containing the personnel information as the target frame; Performing face recognition on the target frame to obtain person identity information; extracting posture features in each target frame to obtain a posture feature set; analyzing the posture feature set to obtain an inertial posture analysis result; Determine the target device of the current person based on the person identity information, determine the current position of the current person based on the person identity information and the real-time video data, and generate a predicted trajectory based on the current position of the current person, the posture feature set and the position of the target device; and determine the warning risk area of ​​the current person based on the predicted trajectory and the production site; A posture risk assessment is performed on the current person based on the inertial posture analysis result and the path risk area.

2. A personnel assessment method according to claim 1, characterized in that: The real-time video data is frame extracted based on the motion characteristics to obtain multiple frames of images, including: At initial time interval Extracting frames from the real-time video data to obtain two frames of reference images; Filter and grayscale the reference image to obtain two frames of pre-processed images; Subtract two frames of pre-processed images to obtain a difference image; Calculate the absolute value of the sum of the grayscale values ​​of all pixels of the difference image, and when the absolute value of the sum of the grayscale values ​​of all pixels of the difference image is greater than a preset threshold, adjust the sampling time interval to the time interval ,in, .

3. A personnel evaluation method according to claim 1, characterized in that: Extract the posture features in each target frame to obtain a posture feature set, including: Preprocess each target frame to obtain, wherein the preprocessing method includes noise reduction and contrast enhancement; Call the human key point extraction model to extract the human skeleton joint points of each target frame; Connect the human skeleton joints of each target frame to obtain the posture features of each target frame; A pose feature set is constructed based on the pose features of multiple target frames.

4. A personnel assessment method according to claim 1, characterized in that: Analyze the posture feature set to obtain an inertial posture analysis result, including: Determine the reference point in each pose feature and target point , where the reference point is the center of the line connecting the left hip joint and the right hip joint, and the target point is one of the head keypoint, left elbow joint, right elbow joint, left wrist joint and right wrist joint, is the serial number of the posture feature; Based on the reference point and the target point The relative position vector is constructed by connecting the lines , ,in, The reference point and the target point The distance The reference point and the target point The angle between the line connecting the two reference lines; Multiple relative position vectors Mapping to a pre-constructed two-dimensional coordinate system, and performing a density-based clustering algorithm on multiple relative position vectors in the two-dimensional coordinate system to obtain one or more clusters; When there is a target cluster with a data quantity greater than a preset threshold, it is determined that the current person has an inertial posture, and based on the relative position vector in the target cluster Constructing relative position vector template , , the relative position vector template As an inertial attitude feature.

5. A personnel assessment method according to claim 1, characterized in that: Determining the current position of the current person based on the person identity information and the real-time video data includes: Acquire a plan view of the production site, and capture a current frame image at a current time point from the real-time video data; wherein the plan view includes a plurality of production equipment and personnel passages surrounding the plurality of production equipment; Extract multiple human reference points and ground reference points of posture features in the current frame image , wherein the ground reference point is the center point of the line connecting the left ankle joint and the right ankle joint, and the multiple human body reference points include the center point of the line connecting the left ankle joint and the right ankle joint, the midpoint of the line connecting the left knee joint and the right knee joint, the midpoint of the line connecting the left hip joint and the right hip joint, the midpoint of the line connecting the left shoulder joint and the right shoulder joint, and the key point of the head; For multiple human reference points Connect the lines to obtain the human body reference line; Extract the human body contour line in the current frame image, and intercept the reference line based on the human body contour line to obtain the height size of the human body in the image ; Obtaining the actual height of the current person from the person's identity information , and based on the height dimensions of the human body in the image , Actual height And the camera focal length Calculate the distance between the camera and the current person , where the distance for: Connect the ground reference point and the camera to obtain a direction line, and determine the angle between the camera orientation and the direction line from the current frame image. ; Based on the distance between the camera and the current person and the angle between the camera direction and the direction line Construct the relative position of the current person and the camera ; Determine the camera position in the plan view, and based on the relative position of the current person and the camera A current location of the current person is determined.

6. A personnel assessment method according to claim 5, characterized in that: Generate a predicted trajectory based on the current position of the current person, the posture feature set, and the position of the target device, including: Generate one or more one-way candidate trajectories in the plan view, starting from the current position and ending at the position of the target device. ,in, is a positive integer; Acquire the posture feature at the current time point and the posture feature at the previous time point from the posture feature set, and determine the current displacement direction of the current person based on the ground reference point of the posture feature at the current time point and the ground reference point of the posture feature at the previous time point; Determine the current position of the current person and each one-way candidate trajectory The shortest distance between , and determine the extension line of the current displacement direction of the current person and each one-way candidate trajectory The angle between ; Based on the shortest distance and the angle Calculate each one-way candidate trajectory Possibility , where each of the one-way candidate trajectories Possibility for: In the formula, is the maximum possible angle, is the minimum possible angle, is the maximum possible distance, is the minimum possible distance, is the first weight, is the second weight; The possibility Maximum number of candidate unidirectional trajectories or target number as a predicted trajectory.

7. A personnel assessment method according to claim 4, characterized in that: Determining posture risk based on the inertial posture analysis result and the path risk area includes: Determining the aisle width of the pathway risk area based on the predicted trajectory ; Based on the aisle width Determine the centerline of the aisle, wherein the centerline is located at location, is the horizontal coordinate of the center line, and the left and right boundaries of the aisle are located at and location; Setting a Gaussian distribution function of the position where a pedestrian passes through the aisle; For the Gaussian distribution function The segment is normalized so that the Gaussian distribution function is The sum of the probabilities within a segment is 1; Determine the segments in the normalized Gaussian distribution function where the sum of probabilities is greater than a preset probability threshold ; Substitute the ground reference point in the inertial attitude feature into the segment In the example, it is determined that when the ground reference point in the inertial attitude feature is located in the segment When any position is reached, any key point of the human body in the inertial posture feature contacts the boundary of the risk area; if so, it is determined that the current person has a posture risk; if not, it is determined that the current person does not have a posture risk.

8. A personnel assessment method according to claim 7, characterized in that: Also includes: When the current person is at risk of security, an early warning message is sent to the current person.

9. A personnel assessment method according to claim 1, characterized in that: The target devices are determined based on historical work records of a plurality of employees.

10. A personnel evaluation system, characterized in that: include: An acquisition module, used to acquire real-time video data of a target area of ​​a production site, wherein the real-time video data is collected by a camera preset at the target location; A frame extraction module is used to extract frames from the real-time video data based on motion characteristics to obtain multiple frames of images; and to identify the multiple frames of images, and when the multiple frames of images contain personnel information, the image frame containing the personnel information is used as the target frame; An image recognition module is used to perform face recognition on the target frame to obtain personal identity information; extract posture features in each target frame to obtain a posture feature set; analyze the posture feature set to obtain an inertial posture analysis result; A trajectory analysis module, for determining a target device of the current person based on the person identity information, determining a current position of the current person based on the person identity information and the real-time video data, and generating a predicted trajectory based on the current position of the current person, the posture feature set, and the position of the target device; and determining a warning risk area of ​​the current person based on the predicted trajectory and the production site; The risk assessment module is used to perform a fixed posture risk assessment on the current person based on the inertial posture analysis result and the path risk area.

Images