A method and system for personnel assessment

Abstract

The present application relates to the field of image analysis, and particularly relates to a personnel evaluation method and system, real-time video data of a target area of a production site is acquired; then dynamic frame extraction is performed on the real-time video data to obtain a target frame containing personnel information, face recognition and posture feature extraction are performed on the target frame; since the motion track of the production site is relatively fixed, the present application infers the target device of the current personnel based on the personnel identity information, and determines the motion direction and inertial posture feature of the personnel according to the posture feature, and further comprehensively predicts the predicted track of the current personnel. It is judged whether there is a risk area in front through the predicted track, if there is, whether there is a risk when the current personnel passes through the risk area with the current feature is analyzed in combination with the inertial posture feature, and a risk warning is performed in advance. The present application evaluates the risk when the staff passes through the risk area, avoiding the problem of unintentional contact with production equipment and causing safety accidents.

Description

Technical Field

[0001] This invention relates to the field of image analysis, specifically a method and system for personnel assessment. Background Technology

[0002] Safety in production is of paramount importance to enterprise management. Although the introduction of more automated equipment has rapidly improved production efficiency, it has also brought more safety problems, which often occur on automated equipment.

[0003] Current safety management focuses primarily on safety training and building safety awareness. However, on-site situations are often complex, and personnel can be negligent. For example, when moving objects, personnel often maintain a stable, outward-facing hand or elbow position. If they are not careful when passing production equipment, their hands or elbows may come into contact with the equipment, causing an accident. Existing safety management and risk assessment methods often fail to provide adequate warnings for such situations. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a personnel assessment method and system to solve the problems in the prior art.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] A personnel assessment method of the present invention includes the following steps:

[0007] Acquire real-time video data of a target area in the production site, wherein the real-time video data is collected by a camera preset at the target location;

[0008] The real-time video data is frame-stripped based on motion characteristics to obtain multiple frames of images; and the multiple frames of images are then identified. When the multiple frames of images contain personnel information, the image frame containing the personnel information is taken as the target frame.

[0009] Face recognition is performed on the target frame to obtain the person's identity information; and posture features are extracted from each target frame to obtain a posture feature set; the posture feature set is analyzed to obtain the inertial posture analysis results;

[0010] Based on the personnel identification information, the target device of the current personnel is determined; based on the personnel identification information and the real-time video data, the current position of the current personnel is determined; and based on the current position of the current personnel, the set of posture features, and the position of the target device, a predicted trajectory is generated; and based on the predicted trajectory and the production site, the early warning risk area of ​​the current personnel is determined.

[0011] Based on the inertial attitude analysis results and the path risk area, a fixed attitude risk assessment is conducted on the current personnel.

[0012] In one embodiment of this application, the real-time video data is frame-stripped based on motion characteristics to obtain multiple frames of images, including:

[0013] The real-time video data is sampled at an initial time interval T0 to obtain two reference images;

[0014] The reference image is filtered and converted to grayscale to obtain two preprocessed images;

[0015] The difference between the two preprocessed images is calculated to obtain the difference image;

[0016] Calculate the absolute value of the sum of gray values ​​of all pixels in the difference image, and when the absolute value of the sum of gray values ​​of all pixels in the difference image is greater than a preset threshold, adjust the sampling time interval to time interval T1, where T1 <T0。

[0017] In one embodiment of this application, pose features are extracted from each target frame to obtain a pose feature set, including:

[0018] Each target frame is preprocessed to obtain the result, where the preprocessing methods include noise reduction and contrast enhancement;

[0019] The human body key point extraction model is invoked to extract the human skeletal joints of each target frame;

[0020] Connect the human skeleton joints of each target frame to obtain the pose features of each target frame.

[0021] A pose feature set is constructed based on the pose features of multiple target frames.

[0022] In one embodiment of this application, the attitude feature set is analyzed to obtain inertial attitude analysis results, including:

[0023] Determine the reference point in each pose feature and target point The reference point is the center of the line connecting the left and right hip joints, and the target point is... It is one of the head key points, left elbow joint, right elbow joint, left wrist joint, and right wrist joint, where i is the sequence number of the posture feature;

[0024] Based on the reference point and the target point The lines connecting them construct a relative position vector. in, The reference point and the target point The distance, θ iThe reference point and the target point The angle between the line connecting the reference lines;

[0025] Multiple relative position vectors Map the data to a pre-constructed two-dimensional coordinate system, and perform a density-based clustering algorithm on multiple relative position vectors in the two-dimensional coordinate system to obtain one or more clusters;

[0026] When there is a target cluster with a data quantity greater than a preset threshold, it is determined that the current person has an inertial posture, and the relative position vector within the target cluster is used as the basis for the determination. Constructing a relative position vector template The relative position vector template As an inertial attitude feature.

[0027] In one embodiment of this application, determining the current location of a person based on the person's identity information and the real-time video data includes:

[0028] Obtain a floor plan of the production site and extract the current frame image at the current time point from the real-time video data; wherein, the floor plan includes multiple production devices and personnel passageways surrounding the multiple production devices;

[0029] Extract multiple human reference points and ground reference points from the pose features of the current frame image. Among them, the ground reference point is the center point of the line connecting the left and right ankle joints, and the multiple human body reference points include the center point of the line connecting the left and right ankle joints, the midpoint of the line connecting the left and right knee joints, the midpoint of the line connecting the left and right hip joints, the midpoint of the line connecting the left and right shoulder joints, and the key points of the head.

[0030] For multiple human reference points Connect the lines to obtain the human body reference line;

[0031] Extract the human body contour line from the current frame image, and cut the reference line based on the human body contour line to obtain the human body height dimension h in the image;

[0032] The actual height H of the current person is obtained from the person's identity information, and the distance S between the camera and the current person is calculated based on the human body height dimension h in the image, the actual height H, and the camera focal length j, where the distance S is:

[0033]

[0034] A direction line is obtained by connecting the ground reference point with the camera, and the angle γ between the camera orientation and the direction line is determined from the current frame image.

[0035] The relative position (S,γ) between the current person and the camera is constructed based on the distance S between the camera and the current person and the angle γ between the camera's orientation and the direction line.

[0036] The camera position is determined in the plan view, and the current position of the current person is determined based on the relative position (S,γ) between the current person and the camera.

[0037] In one embodiment of this application, generating a predicted trajectory based on the current location of the person, the set of posture features, and the location of the target device includes:

[0038] One or more unidirectional candidate trajectories L are generated on the plan view, starting from the current position and ending at the position of the target device. n where n is a positive integer;

[0039] The current person's current displacement direction is determined based on the ground reference point of the current person's attitude feature set and the attitude feature of the previous person's attitude feature set.

[0040] Determine the current position of the current person and each unidirectional candidate trajectory L n The shortest distance S between them n And determine the extension line of the current displacement direction of the current person and each unidirectional candidate trajectory L. n The included angle θ n ;

[0041] Based on the shortest distance S n and the included angle θ n Calculate each unidirectional candidate trajectory L n Possibility P n Each unidirectional candidate trajectory L n Possibility P n for:

[0042]

[0043] In the formula, θ max θ is the maximum possible angle. min S is the smallest possible angle. max S is the maximum possible distance. min To determine the minimum possible distance, W1 is the first weight and W2 is the second weight.

[0044] The probability P n The largest or target number of unidirectional candidate trajectories Ln As a predicted trajectory.

[0045] In one embodiment of this application, determining attitude risk based on the inertial attitude analysis results and the path risk region includes:

[0046] The passage width w of the risk area along the route is determined based on the predicted trajectory.

[0047] The centerline of the passageway is determined based on the passageway width w, where the centerline is located at x = 0, x is the abscissa of the centerline, and the left and right boundaries of the passageway are respectively located at... and Location;

[0048] Define the Gaussian distribution function of the pedestrian crossing position;

[0049] For the Gaussian distribution function The segments are normalized so that the Gaussian distribution function is within the range of... The sum of probabilities within the segment is 1;

[0050] Identify the segment (-X, +X) in the normalized Gaussian distribution function where the sum of probabilities is greater than a preset probability threshold;

[0051] Substitute the ground reference point in the inertial attitude feature into the segment (-X, +X), and determine that when the ground reference point in the inertial attitude feature is located at any position in the segment (-X, +X), there is any human body key point in the inertial attitude feature that is in contact with the boundary of the risk area; if so, it is determined that the current person has an attitude risk; if not, it is determined that the current person does not have an attitude risk.

[0052] In one embodiment of this application, it further includes:

[0053] When the current person poses a security risk, a warning message is sent to the current person.

[0054] In one embodiment of this application, the target device is determined based on the historical work records of multiple employees.

[0055] This application also provides a personnel assessment system, including:

[0056] The acquisition module is used to acquire real-time video data of a target area in the production site, wherein the real-time video data is collected by a camera preset at the target location;

[0057] The frame extraction module is used to extract frames from the real-time video data based on motion characteristics to obtain multiple frames of images; and to identify the multiple frames of images, and when the multiple frames of images contain personnel information, the image frame containing the personnel information is taken as the target frame.

[0058] The image recognition module is used to perform face recognition on the target frame to obtain the person's identity information; and to extract the posture features in each target frame to obtain a posture feature set; and to analyze the posture feature set to obtain the inertial posture analysis result.

[0059] The trajectory analysis module is used to determine the target device of the current person based on the personnel identity information, determine the current position of the current person based on the personnel identity information and the real-time video data, generate a predicted trajectory based on the current position of the current person, the posture feature set and the position of the target device; and determine the early warning risk area of ​​the current person based on the predicted trajectory and the production site.

[0060] The risk assessment module is used to conduct a fixed-posture risk assessment of the current personnel based on the inertial attitude analysis results and the path risk area.

[0061] The beneficial effects of this invention are as follows: The personnel assessment method and system of this invention acquires real-time video data of the target area in the production site; then, it dynamically extracts frames from the real-time video data to obtain target frames containing personnel information, and performs facial recognition and posture feature extraction on the target frames. Since the movement trajectories in the production site are relatively fixed, this application infers the target equipment of the current personnel based on their identity information, and determines the personnel's movement direction and inertial posture characteristics based on their posture features, thereby comprehensively predicting the current personnel's trajectory. The predicted trajectory is used to determine whether there is a risk area ahead. If so, the inertial posture characteristics are combined to analyze whether there is a high probability of risk when the current personnel pass through the risk area with their current characteristics, and a risk warning is issued in advance. This application can effectively assess the risks when workers pass through risk areas, avoiding the problem of unintentional contact with production equipment leading to safety accidents. Attached Figure Description

[0062] The present invention will be further described below with reference to the accompanying drawings and embodiments:

[0063] Figure 1 This is a diagram illustrating an application scenario of a personnel assessment method in one embodiment of this application;

[0064] Figure 2 This is a flowchart illustrating a personnel assessment method in one embodiment of this application;

[0065] Figure 3 This is a schematic diagram of the posture features in this application;

[0066] Figure 4 This is a plan view of the production site according to one embodiment of this application;

[0067] Figure 5 This is a schematic diagram of the inertial attitude features and risk areas in one embodiment of this application;

[0068] Figure 6 This is a structural diagram of a personnel assessment system shown in one embodiment of this application;

[0069] Figure 7 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application is shown. Detailed Implementation

[0070] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, unless otherwise specified, the following embodiments and features described therein can be combined with each other.

[0071] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the layers related to the present invention and are not drawn according to the actual number, shape and size of the layers in the actual implementation. In the actual implementation, the form, number and proportion of each layer can be arbitrarily changed, and the layer layout may also be more complex.

[0072] Numerous details are explored in the following description to provide a more thorough explanation of embodiments of the invention; however, it will be apparent to those skilled in the art that embodiments of the invention may be practiced without these specific details.

[0073] All information concerning the individuals involved in this application has been obtained with full consent and authorization, and the collection, use, and processing of such information must comply with the relevant laws, regulations, and standards of the relevant countries and regions.

[0074] Figure 1 This is a diagram illustrating an application scenario of a personnel assessment method in one embodiment of this application, such as... Figure 1As shown, this application installs a camera 110 in the entrance area or corridor area of ​​the production site. The camera 110 collects video data in real time and sends the video data to the host computer 120. The host computer 120 performs personnel risk assessment through a built-in algorithm, or the host computer 120 manages the real-time video data and uploads images containing personnel data to the cloud server 130 for personnel risk assessment.

[0075] Figure 2 This is a flowchart illustrating a personnel assessment method in one embodiment of this application, such as... Figure 2 As shown: A personnel assessment method in this embodiment may include steps S210 to S250:

[0076] S210, acquire real-time video data of the target area of ​​the production site, wherein the real-time video data is collected by a camera preset at the target location;

[0077] The camera at the target location can be ceiling-mounted or bracket-mounted, allowing for better real-time monitoring of the target area from top to bottom.

[0078] S220, based on motion characteristics, the real-time video data is extracted to obtain multiple frames of images; and the multiple frames of images are identified, and when the multiple frames of images contain personnel information, the image frame containing personnel information is taken as the target frame;

[0079] To reduce subsequent computational load, this application employs frame-sampling to determine the presence of moving objects in the target area. Specifically, based on the motion characteristics of the image, when the image is static, it is analyzed at a frequency of 2 frames per second. During the analysis, once dynamic features are detected, the frame-sampling frequency is increased to 10 frames per second.

[0080] Specifically, the real-time video data is frame-segmented based on motion characteristics to obtain multiple frames of images, including:

[0081] S221, the real-time video data is sampled at an initial time interval T0 to obtain two reference images;

[0082] In this embodiment, under normal circumstances, frames of real-time video data are extracted at an initial time interval T0 (i.e., every 0.5 seconds). For dynamic analysis, at least two reference frames are required, namely the reference image at the current time point. t Image and reference image from the previous time point t-1 .

[0083] S222, the reference image is filtered and grayscaled to obtain two preprocessed images;

[0084] Specifically, for the reference image at the current time point...t Image and reference image from the previous time point t-1 Perform grayscale conversion to obtain the grayscale image at the current time point. t grayscale image from the previous time point t-1 In terms of filtering method, a high-pass filter is used to retain more contour details.

[0085] S223, calculate the difference between the two preprocessed images to obtain the difference image Dif. t ;

[0086] Difference image Dif t =|gray t -gray t-1 |, Difference image Dif t It is an image obtained by subtracting the gray values ​​of pixels at the same position in two grayscale images.

[0087] S224, calculate the absolute value of the sum of gray values ​​of all pixels in the difference image, and when the absolute value of the sum of gray values ​​of all pixels in the difference image is greater than a preset threshold, adjust the sampling time interval to time interval T1, where T1 <T0。

[0088] Finally, the difference image Dif t The grayscale values ​​of the last pixels are summed, i.e., ∑Dif(i,j). The resulting sum of grayscale values ​​reflects the dynamic differences between the two images. If the difference is small, it indicates that the target region is static at the current time point; if the difference is large, it indicates that the target region exhibits dynamic characteristics. In this case, the frame rate is increased to 10 frames per second.

[0089] For the extracted image frames, it is also necessary to ensure that the images contain human information. Therefore, this application uses a pre-trained human recognition model to identify the image frames and check whether human or facial features are present.

[0090] The person recognition model in this application is based on a CNN convolutional neural network. The CNN is trained using pre-prepared training data. The person recognition model filters image frames to retain target frames containing person information.

[0091] S230, perform face recognition on the target frame to obtain personnel identity information; and extract posture features from each target frame to obtain a posture feature set; analyze the posture feature set to obtain inertial posture analysis results;

[0092] The facial recognition in step S230 is also based on the personnel recognition model constructed through the above process. The camera in this application is directed towards the entrance or positioned in a corner of the corridor to ensure that the image can capture the faces of the personnel.

[0093] After identification, the personnel's personal information, including name, position, height, and department, can be retrieved from the production management system.

[0094] Then, since we have already ensured that the target frame contains personnel information, in order to further verify whether there is any risk associated with the personnel, we need to extract the pose features from each target frame.

[0095] Specifically, the process of obtaining the pose feature set includes:

[0096] S2311, preprocess each target frame to obtain, wherein the preprocessing methods include noise reduction and contrast enhancement;

[0097] S2312, call the human body key point extraction model to extract the human skeletal joints of each target frame;

[0098] In this application, OpenPose is used to collect human body key points in the target frame. This results in the marking of 25 human body key points in the target frame, which are: for the head: nose, left eye, right eye, left ear, right ear (since this application does not involve facial expression analysis, the nose key point can be considered as a head key point); for the upper limbs: left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hand (fingertips), right hand (fingertips); for the trunk: neck (usually the midpoint of the line connecting the two shoulders), center point (center of the chest or abdomen, sometimes also called the base of the spine or pelvic center); for the lower limbs: left hip, right hip, left knee, right knee, left ankle, right ankle, left heel, right heel, left toe, right toe.

[0099] S2313, connect the human skeleton joints of each target frame to obtain the pose features of each target frame.

[0100] After marking the key points of the human body, the pose features of each target frame can be obtained by connecting them according to the human body characteristics. In practical applications, some less important key points can be omitted to reduce the amount of computation. Figure 3 This is a schematic diagram of the posture features in this application. The human posture features extracted in this embodiment are as follows: Figure 3 As shown.

[0101] S2314, construct a pose feature set based on the pose features of multiple target frames.

[0102] Finally, the pose features of multiple target frames are placed into the set Pose in chronological order to obtain the temporal pose feature set.

[0103] During activity, a person's posture changes continuously within a normal range. However, if a person is carrying objects or has unusual habitual movements (such as large hand swings, habitually holding their head with their hands while walking, or stretching), their posture will change. If these postures are carried over to a risk area, it could lead to the wrists, elbows, or other areas coming into contact with the risk area (electrified equipment or equipment with cutting or transmission functions). Therefore, after obtaining the posture feature set Pose, it is first necessary to analyze the person's inertial posture characteristics to examine the current inertial posture features. This will facilitate the subsequent integration of inertial posture characteristics into the risk area analysis.

[0104] Specifically, the inertial attitude analysis process in this application includes:

[0105] S2321, Determine the reference point in each pose feature. and target point The reference point is the center of the line connecting the left and right hip joints, and the target point is... It is one of the head key points, left elbow joint, right elbow joint, left wrist joint, and right wrist joint, where i is the sequence number of the posture feature;

[0106] Reference point This application considers points that remain relatively stationary during walking, such as the waist. Therefore, it uses the center of the line connecting the left and right hip joints as a reference point to analyze the stability of other human feature points along the time axis from the perspective of this reference point. This allows for the determination of whether inertial posture features exist.

[0107] S2322, based on the reference point and the target point The lines connecting them construct a relative position vector. in, The reference point and the target point The distance, θ i The reference point and the target point The angle between the line connecting the reference lines;

[0108] This application constructs a reference point on a two-dimensional plane. and target point The relative position vector includes distance and angle parameters. These two parameters are used to determine whether other key points on the human body have changed relative to the reference point.

[0109] S2323, multiple relative position vectors Map the data to a pre-constructed two-dimensional coordinate system, and perform a density-based clustering algorithm on multiple relative position vectors in the two-dimensional coordinate system to obtain one or more clusters;

[0110] Since it is necessary to perform inertial analysis on multiple human key points, that is, if a person maintains the same posture, the relative position vectors they generate will be similar or identical, it is necessary to cluster the large number of relative position vectors of various human key points and reference points constructed on the time axis in the previous text to obtain the same or similar clusters. If there are enough samples in these clusters, they can be used as the corresponding inertial posture feature clusters.

[0111] This application employs the DBSCAN (Density-Based Spatial Clustering of Applications with Noise) clustering algorithm. During clustering, the key consideration is setting the maximum distance within each cluster. A relatively small maximum intra-cluster distance is necessary to obtain clusters representing the desired inertial attitude features.

[0112] S2324, when there is a target cluster with a data quantity greater than a preset threshold, it is determined that the current person has an inertial posture, and based on the relative position vector within the target cluster... Constructing a relative position vector template The relative position vector template As an inertial attitude feature.

[0113] Finally, if there are one or more target clusters with a data quantity greater than a preset threshold, it indicates that the person in the video has one or more inertial postures. For example, when carrying an item, the hands are stationary relative to a reference point; when the hands swing, multiple inertial postures will also be obtained.

[0114] S240, determine the target device of the current person based on the personnel identity information, determine the current position of the current person based on the personnel identity information and the real-time video data, and generate a predicted trajectory based on the current position of the current person, the posture feature set and the position of the target device; and determine the path risk area of ​​the current person based on the predicted trajectory and the production site.

[0115] The purpose of extracting inertial attitude features is to combine them with subsequent area analysis for risk assessment. Therefore, step S240 mainly involves determining the risk areas that the current personnel may traverse. The general approach is to identify the target equipment that the current personnel might visit, predict their trajectory based on their current location and the target equipment's location, and then, by combining the predicted trajectory with the actual conditions of the production site, determine the risk areas that the current personnel might pass through.

[0116] The target equipment is determined based on the historical work records of multiple employees. The equipment most frequently operated by the current employee can be used as the target equipment.

[0117] The general idea for locating a person's current position is to determine the person's relative position to the camera. Since the camera's position is known, the person's current position can be determined once the relative position is established. The specific process includes:

[0118] S2411, Obtain a floor plan of the production site and extract the current frame image at the current time point from the real-time video data; wherein, the floor plan includes multiple production devices and personnel passages surrounding the multiple production devices;

[0119] Figure 4 This is a schematic plan view of the production site according to one embodiment of this application, as shown below. Figure 4 As shown, the production equipment is surrounded by personnel passages. If personnel want to reach the target equipment, they must go through the personnel passages.

[0120] S2412, Extract multiple human reference points and ground reference points for pose features from the current frame image. Among them, the ground reference point is the center point of the line connecting the left and right ankle joints, and the multiple human body reference points include the center point of the line connecting the left and right ankle joints, the midpoint of the line connecting the left and right knee joints, the midpoint of the line connecting the left and right hip joints, the midpoint of the line connecting the left and right shoulder joints, and the key points of the head.

[0121] This embodiment requires extracting image frames at the current time point for analysis, as these frames represent the latest position of the person. This application utilizes the zoom principle of the camera to estimate the distance between the camera and the person. Therefore, multiple human reference points and ground reference points are first extracted.

[0122] S2413, for multiple human reference points Connect the lines to obtain the human body reference line;

[0123] Multiple human reference points are located on the central axis of the human body; therefore, the line connecting them can basically represent the posture of the human body on the axis.

[0124] S2414, Extract the human body contour line in the current frame image, and cut the reference line based on the human body contour line to obtain the human body height h in the image;

[0125] The human body reference line portion extracted by the outline can be used as the height dimension in the image, which is obtained by counting the pixels.

[0126] S2415, obtain the actual height H of the current person from the person's identity information, and calculate the distance S between the camera and the current person based on the human body height dimension h in the image, the actual height H, and the camera focal length j, where the distance S is:

[0127]

[0128] S2416, connect the ground reference point to the camera to obtain a direction line, and determine the angle γ between the camera orientation and the direction line from the current frame image; wherein, the camera orientation is the orientation on the horizontal plane, and the vertical angle is not considered here. Therefore, the angle γ is actually the angle on the two-dimensional horizontal plane.

[0129] S2417, Construct the relative position (S,γ) between the current person and the camera based on the distance S between the camera and the current person and the angle γ between the camera's orientation and the direction line;

[0130] S2418, determine the camera position in the plan view, and determine the current position of the current person based on the relative position (S,γ) between the current person and the camera.

[0131] This application constructs relative position using angles and distances, which can be viewed as a polar coordinate system. However, the camera's coordinates in the planar diagram are two-dimensional XY coordinates. Therefore, the current position of the person can be determined through coordinate transformation. Alternatively, the relative position (S, γ) can be directly mapped onto the planar diagram to obtain the current position of the person.

[0132] The process of generating a predicted trajectory based on the current location of the personnel, the set of posture features, and the location of the target device includes:

[0133] S2421, Generate one or more unidirectional candidate trajectories L on the plan view, starting from the current position and ending at the position of the target device. n where n is a positive integer;

[0134] Trajectory generation can call existing path planning algorithms, such as heuristic search algorithms and shortest path algorithms.

[0135] S2422, Obtain the posture features of the current time point and the posture features of the previous time point from the posture feature set, and determine the current displacement direction of the current person based on the ground reference point of the posture features of the current time point and the ground reference point of the posture features of the previous time point.

[0136] The current displacement direction of personnel is determined based on ground reference points at two points in time, namely...

[0137] S2423, determine the current position of the current person and each unidirectional candidate trajectory L. n The shortest distance S between them n And determine the extension line of the current displacement direction of the current person and each unidirectional candidate trajectory L. n The included angle θ n ;

[0138] In this application, since a large number of candidate trajectories may be generated as described above, this application considers both the merging distance and the included angle in order to find the most likely trajectory. The merging distance is the distance between the current position of the current person and each unidirectional candidate trajectory L. n The shortest distance S between them n The included angle is used to indicate whether the current person's direction of movement is consistent with the candidate trajectory. If there is a high degree of consistency, i.e., the included angle is very small, then it means that the current person is more likely to move along the candidate path. Therefore, the following formula is constructed to calculate the probability.

[0139] S2424, based on the shortest distance S n and the included angle θ n Calculate each unidirectional candidate trajectory L n Possibility P n Each unidirectional candidate trajectory I n Possibility P n for:

[0140]

[0141] In the formula, θ max θ is the maximum possible angle. min S is the smallest possible angle. max S is the maximum possible distance. min To determine the minimum possible distance, W1 is the first weight and W2 is the second weight.

[0142] in, It is actually the reciprocal of the angle after normalization. In fact, it is the reciprocal of the shortest distance after normalization. Normalization is used to eliminate the influence of dimensions, and weighted summation is performed to obtain the probability assessment value of each candidate trajectory. The larger the assessment value, the more likely the current personnel are to move along that candidate path.

[0143] S2425, the probability P n The largest or target number of unidirectional candidate trajectories L n As the predicted trajectory. Specifically, generally, 1-2 candidate trajectories with the highest probability assessment value are selected as the predicted trajectory.

[0144] After obtaining the predicted trajectory, combined with the floor plan of the production site, the equipment areas that may pose risks can be marked. If the predicted trajectory passes through the area, then this risk area is the path risk area predicted in this application.

[0145] Specifically, the risky equipment may be high-voltage electrical equipment, cutting equipment, conveyor belt equipment, fans, etc. The boundaries of the risky equipment are pre-defined on the floor plan as the basis for subsequent risk analysis. If, during the subsequent analysis, the inertial posture of the personnel may cause a part of their body to touch the boundary of the risky equipment, then it is defined as a safety risk.

[0146] S250, based on the inertial attitude analysis results and the path risk area, conduct a fixed attitude risk assessment for the current personnel.

[0147] The specific analysis process is as follows:

[0148] S251, determine the passage width w of the risk area along the route based on the predicted trajectory;

[0149] The width of a passageway directly affects the risk assessment of pedestrians. Generally, pedestrians will walk in the center of the passageway or avoid the side with equipment. However, if the passageway is narrow, even walking in the center can lead to a greater probability of a pedestrian's raised wrist or elbow touching the boundary of the risk area.

[0150] S252, determine the centerline of the passageway based on the passageway width w, wherein the centerline is located at x = 0, x is the abscissa of the centerline, and the left and right boundaries of the passageway are respectively located at... and Location;

[0151] The centerline serves as a reference line and represents the trajectory line with the highest probability. Based on the centerline and boundaries, a general probability density distribution model can be constructed to define the probability distribution characteristics of pedestrian path locations.

[0152] S253, Set the Gaussian distribution function of the pedestrian's position when crossing the passageway;

[0153] Specifically, this application employs the classic Gaussian distribution model to construct a probability distribution function for the positions of pedestrians passing through passageways of known width, ensuring that it conforms to a Gaussian distribution. This process involves abstracting the practical problem into a mathematical model and selecting appropriate parameters to describe the distribution of pedestrian positions.

[0154] The Gaussian distribution function is determined by the mean and standard deviation. In this application, we can assume that the position distribution of pedestrians in the passageway is symmetrical about the center line of the passageway, so the mean μ should be set to 0.

[0155] The standard deviation determines the degree to which a pedestrian deviates from the center line. A reasonable standard deviation σ can be set based on empirical data or observations. If specific data is lacking, σ can be adjusted through trial and error until the simulated results appear reasonable. Typically, σ should be less than [a certain value]. To ensure that most pedestrians remain inside the passageway.

[0156] Therefore, the probability density function of the composite Gaussian distribution in this application should be:

[0157]

[0158] S254, regarding the Gaussian distribution function The segments are normalized so that the Gaussian distribution function is within the range of... The sum of probabilities within the segment is 1;

[0159] Since pedestrians cannot appear outside the passageway, we need to truncate the Gaussian distribution, that is, only consider... The values ​​within the interval. To keep the sum of probabilities equal to 1, the truncated distribution needs to be renormalized. The calculation method is as follows:

[0160] Calculate the cumulative distribution function (CDF) of the original Gaussian distribution within the aisle area, i.e., from... arrive The points.

[0161] Use this integral value to normalize the distribution so that the total area under the new distribution is equal to 1.

[0162] S255, determine the segment (-X,+X) in the normalized Gaussian distribution function where the sum of probabilities is greater than a preset probability threshold;

[0163] By integrating, we obtain the segment (-X, +X) with x = 0 as the axis of symmetry and a total probability greater than 80%. This segment can be used as the location segment that pedestrians are likely to pass through when crossing the road.

[0164] S256, Substitute the ground reference point in the inertial attitude feature into the segment (-X, +X), and determine that when the ground reference point in the inertial attitude feature is located at any position in the segment (-X, +X), there is any human body key point in the inertial attitude feature that is in contact with the boundary of the risk area; if so, it is determined that the current person has an attitude risk; if not, it is determined that the current person does not have an attitude risk.

[0165] Figure 5 This is a schematic diagram of the inertial attitude characteristics and risk areas in one embodiment of this application, as shown below. Figure 5As shown, if the ground reference point in the inertial attitude characteristics falls to the side of the segment (-X, +X) closest to the risk area, and the body just touches the boundary of the risk area, this indicates that if the person passes through the risk area in this posture, there is a certain risk of triggering a safety accident. In this case, a safety risk is determined for the person, and a warning message is sent to the person. The warning can be issued via walkie-talkie or on-site loudspeaker.

[0166] This invention discloses a personnel assessment method that acquires real-time video data of a target area in a production site. Then, it dynamically extracts frames from the real-time video data to obtain target frames containing personnel information. These target frames are then used for facial recognition and posture feature extraction. Since movement trajectories in a production site are relatively fixed, this application infers the target equipment of the current personnel based on their identity information and determines their movement direction and inertial posture characteristics based on these characteristics. This allows for a comprehensive prediction of the personnel's trajectory. The predicted trajectory is used to determine if a risk area exists ahead. If so, the inertial posture characteristics are analyzed to assess the probability of a significant risk when the personnel pass through the risk area, and a risk warning is issued in advance. This application can effectively assess the risks faced by workers passing through risk areas, preventing accidental contact with production equipment and resulting safety accidents.

[0167] like Figure 6 As shown, this application also provides a personnel assessment system, including:

[0168] The acquisition module is used to acquire real-time video data of a target area in the production site, wherein the real-time video data is collected by a camera preset at the target location;

[0169] The frame extraction module is used to extract frames from the real-time video data based on motion characteristics to obtain multiple frames of images; and to identify the multiple frames of images, and when the multiple frames of images contain personnel information, the image frame containing the personnel information is taken as the target frame.

[0170] The image recognition module is used to perform face recognition on the target frame to obtain the person's identity information; and to extract the posture features in each target frame to obtain a posture feature set; and to analyze the posture feature set to obtain the inertial posture analysis result.

[0171] The trajectory analysis module is used to determine the target device of the current person based on the personnel identity information, determine the current position of the current person based on the personnel identity information and the real-time video data, generate a predicted trajectory based on the current position of the current person, the posture feature set and the position of the target device; and determine the early warning risk area of ​​the current person based on the predicted trajectory and the production site.

[0172] The risk assessment module is used to conduct a fixed-posture risk assessment of the current personnel based on the inertial attitude analysis results and the path risk area.

[0173] This invention discloses a personnel assessment system that acquires real-time video data of a target area in a production site. The system then dynamically extracts frames from the real-time video data to obtain target frames containing personnel information. These target frames are then used for facial recognition and posture feature extraction. Since movement trajectories in a production site are relatively fixed, this application infers the target equipment of the current personnel based on their identity information and determines their movement direction and inertial posture characteristics based on these characteristics. This allows for a comprehensive prediction of the personnel's trajectory. The predicted trajectory is used to determine if a risk area exists ahead. If so, the system analyzes the inertial posture characteristics to assess the probability of a significant risk when the personnel pass through the risk area, and issues an early risk warning. This system effectively assesses the risks faced by workers traversing risk areas, preventing accidental contact with production equipment and potential safety accidents.

[0174] Figure 7 A schematic diagram of a computer system suitable for implementing embodiments of the present invention is shown. It should be noted that... Figure 7 The computer system of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of the present invention.

[0175] like Figure 7 As shown, the computer system includes a CPU 701 (Central Processing Unit), which can perform various appropriate actions and processes based on programs stored in ROM 702 (Read-Only Memory) or programs loaded from storage portion 708 into RAM 703 (Random Access Memory), such as performing the methods described in the above embodiments. RAM 703 also stores various programs and data required for system operation. The CPU 701, ROM 702, and RAM 703 are interconnected via bus 704. I / O interface 705 is also connected to bus 704.

[0176] The following components are connected to the I / O interface 705: an input section 706 including a keyboard, mouse, etc.; an output section 707 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 710 as needed so that computer programs read from it can be installed into the storage section 708 as needed.

[0177] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing computer programs for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 709, and / or installed from removable medium 711. When the computer program is executed by CPU 701, it performs various functions defined in the system of the present invention.

[0178] It should be noted that the computer-readable medium shown in the embodiments of the present invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In the present invention, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0179] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. Each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0180] The units described in the embodiments of the present invention can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0181] Another aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a computer's processor, causes the computer to perform the method as described above. This computer-readable storage medium may be included in the electronic device described in the above embodiments, or it may exist independently and not assembled into the electronic device.

[0182] Another aspect of the present invention provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the various embodiments described above.

[0183] The above embodiments are merely preferred embodiments provided to fully illustrate the present invention, and the scope of protection of the present invention is not limited thereto. Equivalent substitutions or modifications made by those skilled in the art based on the present invention are all within the scope of protection of the present invention.

Claims

1. A personnel evaluation method, characterized in that, Including the following steps: Acquire real-time video data of a target area in the production site, wherein the real-time video data is collected by a camera preset at the target location; The real-time video data is extracted based on motion characteristics to obtain multiple frames of images; and the multiple frames of images are identified. When the multiple frames of images contain personnel information, the image frame containing the personnel information is taken as the target frame. Face recognition is performed on the target frame to obtain the person's identity information; and posture features are extracted from each target frame to obtain a posture feature set; the posture feature set is analyzed to obtain the inertial posture analysis results; Based on the personnel identification information, the target device of the current personnel is determined; based on the personnel identification information and the real-time video data, the current position of the current personnel is determined; and based on the current position of the current personnel, the set of posture features, and the position of the target device, a predicted trajectory is generated; and based on the predicted trajectory and the production site, the early warning risk area of ​​the current personnel is determined. Based on the inertial attitude analysis results and the path risk area, a fixed attitude risk assessment is conducted on the current personnel. The analysis of the attitude feature set yields inertial attitude analysis results, including: Determine the reference point in each pose feature and target point The reference point is the center of the line connecting the left and right hip joints, and the target point is... It is one of the following: a key point on the head, the left elbow joint, the right elbow joint, the left wrist joint, and the right wrist joint. This is the sequence number of the posture feature; Based on the reference point and the target point The lines connecting them construct a relative position vector. , ,in, The reference point and the target point distance, The reference point and the target point The angle between the line connecting the reference lines; Multiple relative position vectors Map the data to a pre-constructed two-dimensional coordinate system, and perform a density-based clustering algorithm on multiple relative position vectors in the two-dimensional coordinate system to obtain one or more clusters; When there is a target cluster with a data quantity greater than a preset threshold, it is determined that the current person has an inertial posture, and the relative position vector within the target cluster is used as the basis for the determination. Constructing a relative position vector template , The relative position vector template As an inertial attitude feature.

2. The personnel assessment method according to claim 1, characterized in that, Based on motion characteristics, the real-time video data is frame-by-frame extracted to obtain multiple frames of images, including: With the initial time interval Frames are extracted from the real-time video data to obtain two reference frames; The reference image is filtered and converted to grayscale to obtain two preprocessed images; The difference between the two preprocessed images is calculated to obtain the difference image; Calculate the absolute value of the sum of gray values ​​of all pixels in the difference image, and when the absolute value of the sum of gray values ​​of all pixels in the difference image is greater than a preset threshold, adjust the sampling time interval to a time interval. ,in, .

3. The personnel assessment method according to claim 1, characterized in that, Extract pose features from each target frame to obtain a pose feature set, including: Each target frame is preprocessed to obtain the result, where the preprocessing methods include noise reduction and contrast enhancement; The human body key point extraction model is invoked to extract the human skeletal joints of each target frame; Connect the human skeleton joints of each target frame to obtain the pose features of each target frame. A pose feature set is constructed based on the pose features of multiple target frames.

4. The personnel assessment method according to claim 1, characterized in that, Determining the current location of a person based on their identity information and the real-time video data includes: Obtain a floor plan of the production site and extract the current frame image at the current time point from the real-time video data; wherein, the floor plan includes multiple production devices and personnel passageways surrounding the multiple production devices; Extract multiple human reference points and ground reference points from the pose features of the current frame image. The ground reference point is the center point of the line connecting the left and right ankle joints. The multiple human body reference points include the center point of the line connecting the left and right ankle joints, the midpoint of the line connecting the left and right knee joints, the midpoint of the line connecting the left and right hip joints, the midpoint of the line connecting the left and right shoulder joints, and the key points of the head. For multiple human reference points Connect the lines to obtain the human body reference line; Extract the human body contour lines from the current frame image, and crop the reference line based on the human body contour lines to obtain the human body height dimensions in the image. ; The actual height of the current person is obtained from the person's identity information. And based on the human height dimensions in the image Actual height and camera focal length Calculate the distance between the camera and the person currently in the scene. Among them, distance for: A direction line is obtained by connecting the ground reference point to the camera, and the angle between the camera's orientation and the direction line is determined from the current frame image. ; Based on the distance between the camera and the current person and the angle between the camera orientation and the direction line. Construct the relative position of the current person and the camera. ; The camera position is determined in the plan view, based on the current relative position of the person and the camera. Determine the current location of the person in question.

5. The personnel assessment method according to claim 4, characterized in that, Generate a predicted trajectory based on the current location of the personnel, the set of posture features, and the location of the target device, including: One or more unidirectional candidate trajectories are generated on the plan view, starting from the current position and ending at the position of the target device. ,in, It is a positive integer; The current time point's posture features and the previous time point's posture features are obtained from the posture feature set, and the current displacement direction of the current person is determined based on the ground reference point of the current time point's posture features and the ground reference point of the previous time point's posture features. Determine the current location of the current person and each unidirectional candidate trajectory. shortest distance between And determine the extension line of the current displacement direction of the current person and each unidirectional candidate trajectory. The angle between ; Based on the shortest distance and the included angle Calculate each unidirectional candidate trajectory Possibility Each unidirectional candidate trajectory Possibility for: In the formula, The maximum possible angle, For the smallest possible angle, The maximum possible distance. The minimum possible distance. As the first weight, As the second weight; Possibility The largest or target number of unidirectional candidate trajectories As a predicted trajectory.

6. The personnel assessment method according to claim 1, characterized in that, Based on the inertial attitude analysis results and the path risk region, attitude risk is determined, including: The width of the passageway in the risk area is determined based on the predicted trajectory. ; Based on the width of the passageway Determine the centerline of the passageway, wherein the centerline is located at... Location, The x-coordinate of the centerline is [value], and the left and right boundaries of the passageway are located at [locations]. and Location; Define the Gaussian distribution function of the pedestrian crossing position; For the Gaussian distribution function The segments are normalized so that the Gaussian distribution function is within the range of... The sum of probabilities within the segment is 1; Identify the segments in the normalized Gaussian distribution function where the sum of probabilities exceeds a preset probability threshold. ; Substitute the ground reference point from the inertial attitude characteristics into the segment. In the process, it is determined that when the ground reference point in the inertial attitude feature is located in the segment... If, at any position, any key point of the human body in the inertial posture features comes into contact with the boundary of the risk area, then the current person is determined to have a posture risk; otherwise, the current person is determined not to have a posture risk.

7. A personnel assessment method according to claim 6, characterized in that, Also includes: When the current person poses a security risk, a warning message is sent to the current person.

8. The personnel assessment method according to claim 1, characterized in that, The target equipment was determined based on the historical work records of multiple employees.

9. A personnel evaluation system, characterized in that, include: The acquisition module is used to acquire real-time video data of a target area in the production site, wherein the real-time video data is collected by a camera preset at the target location; The frame extraction module is used to extract frames from the real-time video data based on motion characteristics to obtain multiple frames of images; and to identify the multiple frames of images, and when the multiple frames of images contain personnel information, the image frame containing the personnel information is taken as the target frame. The image recognition module is used to perform face recognition on the target frame to obtain the person's identity information; and to extract the posture features in each target frame to obtain a posture feature set; and to analyze the posture feature set to obtain the inertial posture analysis result. The trajectory analysis module is used to determine the target device of the current person based on the personnel identity information, determine the current position of the current person based on the personnel identity information and the real-time video data, generate a predicted trajectory based on the current position of the current person, the posture feature set and the position of the target device; and determine the early warning risk area of ​​the current person based on the predicted trajectory and the production site. The risk assessment module is used to conduct a fixed-posture risk assessment of the current personnel based on the inertial attitude analysis results and the path risk area; The analysis of the attitude feature set yields inertial attitude analysis results, including: Determine the reference point in each pose feature and target point The reference point is the center of the line connecting the left and right hip joints, and the target point is... It is one of the following: a key point on the head, the left elbow joint, the right elbow joint, the left wrist joint, and the right wrist joint. This is the sequence number of the posture feature; Based on the reference point and the target point The lines connecting them construct a relative position vector. , ,in, The reference point and the target point distance, The reference point and the target point The angle between the line connecting the reference lines; Multiple relative position vectors Map the data to a pre-constructed two-dimensional coordinate system, and perform a density-based clustering algorithm on multiple relative position vectors in the two-dimensional coordinate system to obtain one or more clusters; When there is a target cluster with a data quantity greater than a preset threshold, it is determined that the current person has an inertial posture, and the relative position vector within the target cluster is used as the basis for the determination. Constructing a relative position vector template , The relative position vector template As an inertial attitude feature.

Images