Human body posture detection method, device and computer equipment

Abstract

The present application relates to a method, device and computer equipment for detecting human postures, wherein the method for detecting human postures comprises: obtaining shooting information of human targets in a plurality of video frames, obtaining a posture recognition algorithm matching the shooting information according to the shooting information, performing posture recognition on the human targets in the corresponding video frames according to the matching posture recognition algorithm, determining the posture recognition results of the human targets in the video frames, and determining the posture type of the human targets according to the posture recognition results of the plurality of video frames, thereby solving the problem of misjudgment in the related art of judging whether the employee's behavior is compliant by comparing the employee's actual behavior with preset data, and improving the accuracy of human posture recognition.

Description

Technical Field

[0001] The present application relates to the technical field of video surveillance, and in particular to a method, device and computer equipment for detecting human posture. Background Art

[0002] Human posture detection is the most challenging research direction in the field of computer vision and is widely used in human-computer interaction, intelligent monitoring, virtual reality, and human behavior analysis. The behavior of corporate employees at work greatly affects customer experience and corporate image; currently, the supervision of employee behavior is mainly in the form of manual on-site supervision and video manual supervision. On the one hand, manual supervision is insufficient and easy to relax; on the other hand, human judgment and supervision have certain subjective factors, which are difficult to review and standardize in the later stage.

[0003] In the related art, the supervision of employee behavior is generally carried out by using cameras and microphones to collect video and voice of designated areas respectively, and comparing the collected video with preset scene images, and comparing the collected voice with preset keywords. When the collected video and voice are inconsistent with the preset scene images and preset keywords, it is judged that the employee's current behavior is not compliant; however, the same behavior varies greatly from person to person, so it is easy to misjudge whether the actual behavior of the employee is compliant by comparing it with the preset data.

[0004] Currently, in the relevant technology, whether the employee's behavior is compliant is judged by comparing the employee's actual behavior with the preset data, but there is a problem of misjudgment, and no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present application provide a method, apparatus and computer device for detecting human posture, which at least solve the problem of misjudgment in the related art when comparing the actual behavior of employees with preset data to determine whether the behavior of employees is compliant.

[0006] In a first aspect, an embodiment of the present application provides a method for detecting a human body posture, the method comprising:

[0007] Obtain shooting information of human targets in multiple video frames;

[0008] Acquire a gesture recognition algorithm matching the shooting information according to the shooting information;

[0009] Performing posture recognition on a human target in a corresponding video frame according to a matched posture recognition algorithm, and determining a posture recognition result of the human target in the video frame;

[0010] The posture type of the human target is determined according to the posture recognition results of the plurality of video frames.

[0011] In some embodiments, the posture recognition algorithm includes a key point-based detection algorithm and a neural network feature extraction-based algorithm, and the shooting information includes a shooting angle; if the shooting angle meets a preset angle, the key point-based detection algorithm is used to perform posture recognition of the human target, otherwise the neural network feature extraction-based algorithm is used to perform posture recognition of the human target.

[0012] In some embodiments, according to the matched posture recognition algorithm, posture recognition is performed on the human target in the corresponding video frame, and determining the posture recognition result of the human target in the video frame includes:

[0013] Obtaining key point positions and angles between limbs of the human target in the video frame;

[0014] The posture recognition result of the human target in the video frame is determined according to the key point positions and the angle between the limbs.

[0015] In some embodiments, according to the matched posture recognition algorithm, posture recognition is performed on the human target in the corresponding video frame, and determining the posture recognition result of the human target in the video frame includes:

[0016] Segmenting a contour image of the human target from the video frame, and determining an edge information feature vector of the contour image according to the contour image;

[0017] Determine a depth image according to the video frame, and perform key point detection on a human target in the depth image to determine key point position information based on the depth image;

[0018] Determine a feature vector of the key point position information according to the key point position information;

[0019] Adding the edge information feature vector of the contour image and the feature vector of the key point position information in corresponding dimensions to determine a fused feature vector;

[0020] The fused feature vector is input into a preset support vector machine classifier for classification to determine the posture recognition result of the human target in the video frame.

[0021] In some embodiments, the method further comprises:

[0022] If the posture type of the human target is an unqualified type, identifying the human target, and determining the identity information and region information of the human target;

[0023] The identity information of the human target and the posture type are transmitted to a background database, and a prompt device of a corresponding area is triggered according to the area information.

[0024] In some embodiments, before acquiring the shooting information of the human target in the plurality of video frames, the method further includes:

[0025] judging whether the human target is an employee according to the clothing of the human target in the video frame;

[0026] If so, the shooting information of the human target in the video frame is obtained.

[0027] In some embodiments, judging whether the human target is an employee based on the clothing of the human target in the video frame includes:

[0028] Acquire a target frame image of the human target in the video frame;

[0029] According to the target frame image, determine a color channel map and a grayscale map of the target frame image, wherein the color channel map includes: an R channel map, a G channel map, and a B channel map;

[0030] Counting the distribution of each pixel value in the color channel image and the grayscale image respectively, and determining a pixel statistical vector of the color channel image and a pixel statistical vector of the grayscale image;

[0031] Splicing the pixel statistical vector of the color channel image and the pixel statistical vector of the grayscale image to determine a color representation image of the target frame image;

[0032] Inputting the color representation image into a pre-trained color convolution model for feature extraction to determine a color feature vector;

[0033] The feature vector is classified by a preset classifier to determine whether the human target is a staff member.

[0034] In a second aspect, an embodiment of the present application provides a human posture detection device, the device comprising an acquisition module, a posture recognition algorithm module, and a posture type recognition module;

[0035] The acquisition module is used to acquire the shooting information of the human target in multiple video frames;

[0036] The gesture recognition algorithm module is used to obtain a gesture recognition algorithm matching the shooting information according to the shooting information;

[0037] The posture type recognition module is used to perform posture recognition on the human target in the corresponding video frame according to the matching posture recognition algorithm, and determine the posture recognition result of the human target in the video frame; the posture type recognition module is also used to determine the posture type of the human target according to the posture recognition results of multiple video frames.

[0038] In a third aspect, an embodiment of the present application provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the human body posture detection method as described in the first aspect above is implemented.

[0039] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the human body posture detection method as described in the first aspect above.

[0040] Compared with the related art, the human posture detection method, device and computer equipment provided in the embodiments of the present application obtain shooting information of human targets in multiple video frames, obtain a posture recognition algorithm matching the shooting information according to the shooting information, perform posture recognition on the human targets in the corresponding video frames according to the matching posture recognition algorithm, determine the posture recognition result of the human targets in the video frames, and determine the posture type of the human target according to the posture recognition results of the multiple video frames. This at least solves the problem of misjudgment in the related art of judging whether the employee's behavior is compliant by comparing the employee's actual behavior with preset data, thereby improving the accuracy of human posture recognition.

[0041] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0043] Figure 1 The process of the human body posture detection method according to the embodiment of the present application is as follows Figure 1 ;

[0044] Figure 2a The method flow for determining the posture recognition result of a human target in a video frame according to an embodiment of the present application is as follows Figure 1 ;

[0045] Figure 2b is a schematic diagram of the distribution of various key points of a human body according to an embodiment of the present application;

[0046] Figure 2c This is a schematic diagram of key points of a human body in an unqualified posture according to an embodiment of the present application. Figure 1 ;

[0047] Figure 2d is a second schematic diagram of key points of a human body in an irregular posture according to an embodiment of the present application;

[0048] Figure 2e This is a schematic diagram of key points of a human body in an unqualified posture according to an embodiment of the present application. Figure 3 ;

[0049] Figure 2f This is a schematic diagram of key points of a human body in an unqualified posture according to an embodiment of the present application. Figure 4 ;

[0050] Figure 3 is a second flow chart of a method for determining a posture recognition result of a human target in a video frame according to an embodiment of the present application;

[0051] Figure 4 is a second flowchart of the human body posture detection method according to an embodiment of the present application;

[0052] Figure 5 The process of the human body posture detection method according to the embodiment of the present application is as follows Figure 3 ;

[0053] Figure 6a is a flow chart of a method for determining whether a human target is an employee according to an embodiment of the present application;

[0054] Figure 6b is a schematic diagram of determining whether a human target is an employee based on the clothing of the human target according to an embodiment of the present application;

[0055] Figure 7 is a structural block diagram of a human body posture detection device according to an embodiment of the present application;

[0056] Figure 8 It is a schematic diagram of the internal structure of a computer device according to an embodiment of the present application. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments provided in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0058] Obviously, the drawings described below are only some examples or embodiments of the present application. For ordinary technicians in this field, the present application can also be applied to other similar scenarios based on these drawings without creative work. In addition, it can also be understood that although the efforts made in this development process may be complicated and lengthy, for ordinary technicians in this field related to the content disclosed in this application, some changes in design, manufacturing or production based on the technical content disclosed in this application are just conventional technical means, and should not be understood as insufficient content disclosed in this application.

[0059] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those of ordinary skill in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0060] Unless otherwise defined, the technical terms or scientific terms involved in this application should be understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a kind of", "the" and the like involved in this application do not indicate a quantitative limitation and may represent the singular or plural. The terms "include", "comprise", "have" and any of their variations involved in this application are intended to cover non-exclusive inclusions; for example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include other steps or units inherent to these processes, methods, products or devices. The "multiple" involved in this application refers to two or more.

[0061] The behavior of corporate employees at work greatly affects customer experience and corporate image. In the prior art, the supervision of employees is mainly in the form of manual on-site supervision and video manual on-duty. On the one hand, manual supervision is insufficient and easy to relax; on the other hand, there are certain subjective factors in human judgment and supervision, which makes it difficult to review and standardize in the later stage. In order to solve the problem of insufficient supervision in manual on-site supervision and video manual on-duty, in the relevant technology, the supervision of employee behavior is generally to use cameras and microphones to collect video and voice in the designated area respectively, and compare the collected video with the preset scene image, and compare the collected voice with the preset keywords. When the collected video and voice are inconsistent with the preset scene image and the preset keywords, it is judged that the current behavior of the employee is not compliant; but for human behavior, the same behavior is quite different, so by comparing the actual behavior of the employee with the preset data to judge whether the actual behavior is compliant, it is easy to misjudge.

[0062] A human posture detection method provided by the present application obtains multiple video frames through a camera, and can obtain a posture recognition algorithm matching the shooting information of the human target in the video frame. Further, the posture of the human target in the corresponding video frame is identified according to the posture recognition algorithm matching the shooting information of the human target, and the posture recognition result of the human target in the video frame is determined. According to the posture recognition results of the multiple video frames, the posture type of the human target is determined, wherein the posture type includes a compliant posture type and an non-compliant posture type, and the non-compliant posture type includes but is not limited to bending over, holding the head with both hands, holding the chest with both hands, and having both hands on the hips.

[0063] This embodiment provides a method for detecting human body posture. Figure 1 The process of the human body posture detection method according to the embodiment of the present application is as follows Figure 1 ,like Figure 1 As shown, the human body posture detection method includes the following steps:

[0064] Step S101, obtaining shooting information of a human target in a plurality of video frames;

[0065] It should be noted that the multiple video frames are acquired by an acquisition device, the acquisition device is not limited to a monocular camera or a binocular camera, and the acquired video frames are not limited to original images and depth images. Human target detection is first performed on the video frames acquired by the acquisition device. When a human target is detected in the video frame, shooting information of the human target is acquired, wherein the shooting information may include a shooting angle of the human target and a shooting range of the human target. The shooting range of the human target may be understood as an area in which the human target appears in the video frame, for example, the whole body, upper body or lower body of the human target.

[0066] Step S102, acquiring a gesture recognition algorithm matching the shooting information according to the shooting information;

[0067] It should be noted that at least two different types of posture recognition algorithms are included; for example, if the shooting angle of the human target in the video frame is frontal or approximately frontal, the posture type of the human target in the video frame can be accurately obtained by a simple key point detection algorithm;

[0068] Step S103, performing posture recognition on a human target in a corresponding video frame according to the matched posture recognition algorithm, and determining a posture recognition result of the human target in the video frame;

[0069] It should be noted that, for multiple video frames, shooting information of a certain number of video frames can be obtained, and the corresponding posture recognition algorithm can be obtained according to the shooting information of each video frame, and posture recognition is performed on the human target in the corresponding video frame to determine whether the posture of the human target in each video frame is compliant. If the number of video frames with non-compliant postures is greater than a preset threshold, the posture type of the human target can be considered to be a non-compliant type, otherwise it is a compliant type;

[0070] Step S104, determining the posture type of the human target according to the posture recognition results of the multiple video frames;

[0071] Specifically, it can be to take shooting information of a certain video frame from multiple video frames, and obtain a corresponding posture recognition algorithm according to the shooting information to perform posture recognition of a human target in the video frame to determine the posture type of the human target in the video frame. If the posture of the human target in the video frame is an irregular posture, the human target is tracked, and a matching posture recognition algorithm is used for other tracked multiple video frames to perform posture recognition of the human target, and the posture of the human target in other video frames is determined; if the duration of the irregular posture of the human target is greater than a preset time, the posture type of the human target is determined to be an irregular posture type, otherwise continuous posture detection is continued, wherein the duration of the irregular posture is equal to the number of frames N*(1 / frame rate) of the human target being tracked;

[0072] Through the above steps S101 to S104, firstly, the shooting information of the human target in the video frame is obtained, and then the matching posture recognition algorithm is obtained according to the shooting information of the human target in each video frame. Finally, the matching posture recognition algorithm is used to perform posture recognition on the human target in the corresponding video frame to determine the posture recognition result of the human target in the video frame. Further, according to the posture recognition results of multiple video frames, the posture type of the human target is determined, that is, the posture recognition algorithm matching the human target in each current video frame is used for recognition. Firstly, compared with the existing manual on-site supervision, video manual duty and other schemes, the patent of the present invention can reduce labor costs, realize intelligent real-time video supervision, and avoid the subjectivity of manual supervision and duty. In addition, considering the presence of different shooting information on the monitoring site, different posture recognition algorithms are used to perform posture recognition on the human target in each video frame, so as to solve the problem of misjudgment in the related technology of judging whether the employee's behavior is compliant by comparing the actual behavior of the employee with the preset data, and improve the accuracy of human posture recognition.

[0073] In some embodiments, the posture recognition algorithm includes a key point detection algorithm and a neural network feature extraction algorithm, and the shooting information includes a shooting angle; if the shooting angle meets the preset angle, the key point detection algorithm is used to perform posture recognition of the human target, otherwise the neural network feature extraction algorithm is used to perform posture recognition of the human target;

[0074] It should be noted that if the posture recognition algorithm only includes an algorithm based on key point detection and an algorithm based on neural network feature extraction, step S102 can also be to determine whether the shooting angle of the human target in each video frame meets the preset angle. If so, the algorithm based on key point detection is used to perform posture recognition on the human target in the video frame; otherwise, the algorithm based on neural network feature extraction is used to perform posture recognition on the human target in the video frame.

[0075] In some of these embodiments, Figure 2a The method flow for determining the posture recognition result of a human target in a video frame according to an embodiment of the present application is as follows Figure 1 ,like Figure 2a As shown, according to the matched posture recognition algorithm, the posture recognition of the human target in the corresponding video frame is performed, and the posture recognition result of the human target in the video frame is determined, which includes the following steps:

[0076] Step S201, obtaining key point positions and angles between limbs of a human target in a video frame;

[0077] It should be noted that when the shooting angle of the human target in the video frame is frontal or approximately frontal, the posture of the human target is identified by a key point-based detection algorithm, and the shooting angle of the human target refers to the angle at which the human target is shot; wherein, Figure 2b is a schematic diagram of the distribution of each key point of the human body according to an embodiment of the present application. As Figure 2b shown, the black solid dots represent the positions of the main key points of the human body, including the left and right ears, the left and right shoulders, the left and right elbows, the left and right wrists, the left and right hips, the left and right knees, and the left and right ankles. That is, except for the key points on the head, the remaining key points are all distributed at each joint of the human body. Therefore, logical judgments can be made based on the position information of the key points and the included angles between the limbs, so as to determine whether the human target in each video frame belongs to a non-compliant posture;

[0078] Step S202: Determine the posture recognition result of the human target in the video frame according to the key point positions and the included angles between the limbs;

[0079] Specifically, the posture judgment is taken as an example of the right side of the human body, and the calculation method for the left side is the same. Among them, the coordinate value symbols are uniformly defined from top to bottom as: shoulder (x1, y1), elbow (x2, y2), right wrist (x3, y3), left wrist (x3_1, y3_1), hip (x4, y4), knee (x5, y5);

[0080] Figure 2c is a schematic diagram of the key points of the human body in a non-compliant posture according to an embodiment of the present application Figure 1 , such as Figure 2c shown, this schematic diagram shows the judgment method for the bending posture. The included angle α between one side of the torso and the leg can be calculated through the coordinates of the three key points of the shoulder, hip, and knee. When α < 145°, it is judged as a non-compliant posture of bending;

[0081] Figure 2d is the second schematic diagram of the key points of the human body in a non-compliant posture according to an embodiment of the present application, such as Figure 2d shown, this schematic diagram shows the judgment method for holding the head with both hands. On the one hand, when y3 > y1, that is, the wrist is higher than the shoulder, and when the included angle β of the elbow arm calculated through the wrist, elbow, and shoulder is < 145°, and both sides of the human body meet the above conditions, it is judged as a non-compliant posture of holding the head with both hands;

[0082] Figure 2e is a schematic diagram of the key points of the human body in a non-compliant posture according to an embodiment of the present application Figure 3 , such as Figure 2e shown, this schematic diagram shows the judgment method for holding the chest with both hands. When x3 < x3_1, that is, the hands are crossed, and when the included angle β of the elbow arm calculated through the positions of the shoulder, elbow, and wrist is less than 100°, and both the left and right sides meet the above conditions, it is judged as a non-compliant posture of holding the chest with both hands;

[0083] Figure 2f is a schematic diagram of the key points of the human body in a non-compliant posture according to an embodiment of the present application Figure 4 , such as Figure 2f As shown, this schematic diagram shows the judgment method of hands on the hips. When y3 < y1, that is, the wrist is below the shoulder, and when the elbow-arm angle β calculated through the shoulder, elbow and wrist is < 150°, and both the left and right sides meet the above conditions, it is determined as a non-compliant posture of hands on the hips;

[0084] Through the above steps S201 to S202, when the shooting angle of the detected human target is frontal or approximately frontal, directly perform human key point detection on the human target, and then judge the category of the human target posture by calculating the positional relationship between the key points. On the one hand, the recognition accuracy of the human target posture is improved, and on the other hand, the calculation and processing efficiency are also improved.

[0085] In some embodiments, Figure 3 is the second method flowchart for determining the posture recognition result of the human target in the video frame according to the embodiment of the present application. As Figure 3 shown, according to the matched posture recognition algorithm, perform posture recognition on the human target in the corresponding video frame, and the steps for determining the posture recognition result of the human target in the video frame are as follows:

[0086] Step S301, segment the contour image of the human target from the video frame, and determine the edge information feature vector of the contour image according to the contour image;

[0087] It should be noted that when the shooting angle of the human target in the video frame is not a frontal angle, an algorithm based on neural network feature extraction is used for the posture recognition of the human target. Specifically, the video frame with the target box of the human target is input into the trained FCN network for segmentation to obtain the contour image of the human target. The full name of the FCN network is Fully Convolutional Networks for Semantic Segmentation, which is a method of applying deep learning to image segmentation; further, the contour image of the human target obtained by segmentation is input into the trained lightweight first convolutional model for feature extraction to obtain the edge information feature vector of the contour image;

[0088] Step S302, determine the depth image according to the video frame, and perform key point detection on the human target in the depth image to determine the key point position information based on the depth image;

[0089] Specifically, according to the position of the target box of the human target in the video frame, find the position of the target box of the human target in the corresponding depth image to determine the depth image corresponding to the target box of the human target; further, perform key point detection on the human target in the depth image to obtain the key point position information based on the depth image;

[0090] Step S303, determining a feature vector of the key point position information according to the key point position information;

[0091] Specifically, the key point position information based on the depth image is input into the trained lightweight second convolution model to obtain a feature vector of the key point position information, wherein the lightweight first convolution model and the lightweight second convolution model are both without a fully connected layer;

[0092] Step S304, adding the corresponding dimensions of the feature vector of the edge information of the contour image and the feature vector of the key point position information to determine a fused feature vector;

[0093] Specifically, the feature vector of the contour image is fused with the feature vector of the key point position information. The fusion method is to add the corresponding dimensions of the two vectors. The premise of feature fusion is that the two feature vectors have the same dimension. Therefore, in the embodiment of the present application, a 1x1 convolution layer method is used to control the dimension of the feature vector. The feature vector of the contour image is transformed in dimension through a layer of 1x1 convolution so that the dimension of the feature vector of the contour image is equal to the dimension of the feature vector of the key point position information. The two vectors are added according to the corresponding dimensions to obtain the fused feature vector.

[0094] Step S305, inputting the fused feature vector into a preset support vector machine classifier for classification to determine the posture recognition result of the human target in the video frame;

[0095] Specifically, the fused feature vector is input into the trained support vector machine classifier SVC for classification, and it is determined whether there is an irregular posture of the human target in the video frame according to the classification result;

[0096] Through the above steps S301 to S305, when the shooting angle of the human target in the video frame does not conform to the frontal shooting angle, a neural network feature extraction algorithm based on the fusion of the edge information feature vector of the contour image and the feature vector of the key point position information is used to perform posture recognition of the human target, thereby improving the recognition accuracy.

[0097] In some of these embodiments, Figure 4 Flowchart 2 of the human body posture detection method according to an embodiment of the present application, such as Figure 4 As shown, the human body posture detection method also includes the following steps:

[0098] Step S401, if the posture type of the human target is an unqualified type, the human target is identified to determine the identity information and region information of the human target;

[0099] Step S402, transmitting the identity information and posture type of the human target to the background database, and triggering the prompt device of the corresponding area according to the area information;

[0100] Specifically, the identity information of the human target with the identified non-compliant posture type, the category of the non-compliant posture (bending over, holding the head with both hands, holding the chest with both hands, hands on the hips, etc.), the date and time information of the occurrence, and the video clip of the non-compliant posture can be matched and stored in the background database, and can be exported later to facilitate incorporation into the evaluation system of the employee, which is conducive to the standardized and normalized management of employees. The present application is not limited to bank employees, but can also be extended to various industries that have requirements for the etiquette posture of employees at work; the prompt device of the corresponding area is triggered according to the area information, so as to prompt the employees in the corresponding area to correct their non-compliant posture in time.

[0101] In some of these embodiments, Figure 5 The process of the human body posture detection method according to the embodiment of the present application is as follows Figure 3 ,like Figure 5 As shown, before obtaining the shooting information of the human target in multiple video frames, the human posture detection method also includes the following steps:

[0102] Step S501, judging whether the human target is an employee by the clothing of the human target in the video frame; if so, obtaining the shooting information of the human target in the video frame;

[0103] Through the above step S501, considering that there may be non-staff target interference factors at the monitoring site, it is first determined whether the target person is an employee, and if so, further posture recognition is performed, thereby avoiding analysis and processing of many unnecessary targets, thereby greatly improving the efficiency of calculation and processing.

[0104] In some of these embodiments, Figure 6a is a flow chart of a method for determining whether a human target is an employee according to an embodiment of the present application. Figure 6a As shown, judging whether a human target is an employee by the clothing of the human target in the video frame includes the following steps:

[0105] Step S601, obtaining a target frame image of a human target in a video frame;

[0106] In order to avoid the influence of angle, background, lighting and other factors on the recognition results, the video frames of the detected human targets are first scored and optimized. Figure 6b FIG. 1 is a schematic diagram of determining whether a human target is an employee by the clothing of the human target according to an embodiment of the present application. Figure 6bAs shown, rectangles numbered 1-4 represent video frames collected in a short period of time. They are scored based on shooting angle, lighting, background and other factors, and the frame with the highest score is selected as the preferred target frame image;

[0107] Step S602, determining a color channel map and a grayscale map of the target frame image according to the target frame image, wherein the color channel map includes: an R channel map, a G channel map, and a B channel map;

[0108] like Figure 6b As shown, the three RGB color channels of the preferred target frame image are separated to obtain a color channel map of the target frame image, and the target frame image is grayed to obtain a grayscale map;

[0109] Step S603, respectively counting the distribution of each pixel value in the color channel image and the grayscale image, and determining the pixel statistical vector of the color channel image and the pixel statistical vector of the grayscale image;

[0110] Step S604, concatenating the pixel statistical vector of the color channel image and the pixel statistical vector of the grayscale image to determine a color representation image of the target frame image;

[0111] Specifically, Figure 6b As shown in the figure, [1][2][3][4] represent the pixel statistical vectors obtained by statistically analyzing the pixel values ​​of the R channel image, the G channel image, the B channel image, and the grayscale image, respectively, and the four vectors are concatenated to obtain a color representation image with a size of 4x256;

[0112] It should be noted that the advantages of the above-mentioned image processing method are: on the one hand, clothing recognition relies heavily on the color feature information of the image to distinguish between different clothing, and fusing the pixel statistical information of different color channels of the image can better represent the color distribution of the target, so that the subsequent color convolution model can better learn important features and distinguish different features; on the other hand, the target frames of the detected human targets will always be different in size. The above-mentioned method can convert images of different sizes into feature maps of the same size, that is, no matter how large the input target frame size is, after statistics and splicing of pixel values ​​of different channels, the size is 4x256, which is conducive to improving the recognition accuracy of the subsequent color convolution model and accelerating the convergence speed of the model;

[0113] Step S605, inputting the color representation image into a pre-trained color convolution model for feature extraction to determine a color feature vector;

[0114] It should be noted that the color convolution model is a convolution model with trained parameters for feature extraction of color representation images; specifically, the color representation image is input into the pre-trained convolution model for feature extraction, and the convolution model is a convolution model without a fully connected layer;

[0115] Step S606, classifying the color feature vector by a preset classifier to determine whether the human target is an employee;

[0116] Specifically, the trained support vector machine classifier SVC is used to classify the color feature vector obtained after feature extraction by the color convolution model, and the result of clothing recognition is obtained, that is, whether the target person is an employee. Among them, the feature extraction and classification method of the above-mentioned color convolution model + support vector machine classifier SVC has two main advantages. First, the color convolution model, as the optimal representative lightweight convolutional neural network, has good feature extraction capabilities while greatly reducing the model parameters and calculation amount, and is more suitable for use in embedded devices; second, for convolutional neural networks, most of their parameters are mainly concentrated in the fully connected layer. Therefore, the fully connected layer is not used here, but the support vector machine SVC is used as the feature classifier instead, which can not only ensure good recognition accuracy, but also greatly reduce the parameters and calculation amount of the convolutional neural network.

[0117] It should be noted that the steps shown in the above process or the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0118] The present embodiment also provides a human posture detection device, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the terms "module", "unit", "subunit" and the like can implement a combination of software and / or hardware of a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware is also possible and conceived.

[0119] In some embodiments, Figure 7 is a structural block diagram of a human body posture detection device according to an embodiment of the present application, such as Figure 7 As shown, the human body posture detection device includes: an acquisition module 71, a posture recognition algorithm module 72 and a posture type recognition module 73;

[0120] An acquisition module 71 is used to acquire shooting information of a human target in a plurality of video frames;

[0121] A gesture recognition algorithm module 72, used to obtain a gesture recognition algorithm matching the shooting information according to the shooting information;

[0122] The posture type recognition module 73 is used to perform posture recognition on the human target in the corresponding video frame according to the matched posture recognition algorithm, and determine the posture recognition result of the human target in the video frame; the posture type recognition module is also used to determine the posture type of the human target according to the posture recognition results of multiple video frames.

[0123] In some of the embodiments, the acquisition module 71, the posture recognition algorithm module 72, and the posture type recognition module 73 are also used to implement the steps in the human posture detection method provided in the above embodiments, which will not be repeated here.

[0124] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0125] In one embodiment, a computer device is provided, which may be a terminal. The computer device includes a processor, a memory, a network interface, a display screen, and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a method for detecting a human posture is implemented. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covered on the display screen, or a key, trackball, or touchpad provided on the housing of the computer device, or an external keyboard, touchpad, or mouse, etc.

[0126] In one embodiment, Figure 8 is a schematic diagram of the internal structure of a computer device according to an embodiment of the present application, such as Figure 8 As shown, a computer device is provided, which may be a server, and its internal structure diagram may be as shown in Figure 8As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a human posture detection method is implemented.

[0127] Those skilled in the art will understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0128] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps in the human body posture detection method provided in the above embodiments are implemented.

[0129] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the human body posture detection method provided in the above embodiments are implemented.

[0130] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0131] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0132] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A human body posture detection method, characterized in that: The method comprises: Obtain shooting information of human targets in multiple video frames; Acquire a gesture recognition algorithm matching the shooting information according to the shooting information; Performing posture recognition on a human target in a corresponding video frame according to a matched posture recognition algorithm, and determining a posture recognition result of the human target in the video frame; Determining the posture type of the human target according to the posture recognition results of the plurality of video frames; Wherein, the posture recognition algorithm includes a key point-based detection algorithm and a neural network-based feature extraction algorithm, and the shooting information includes a shooting angle; if the shooting angle meets a preset angle, the key point detection algorithm is used to perform posture recognition of the human target, otherwise the neural network feature extraction algorithm is used to perform posture recognition of the human target; The step of adopting the key point detection algorithm to perform posture recognition on a human target in a corresponding video frame and determining a posture recognition result of the human target in the video frame comprises: Obtaining key point positions and angles between limbs of the human target in the video frame; Determine a posture recognition result of the human target in the video frame according to the key point positions and the angle between the limbs; The adopting of the neural network feature extraction algorithm to perform posture recognition of the human target includes: adopting a neural network feature extraction algorithm based on the fusion of edge information feature vectors of contour images and feature vectors of key point position information to perform posture recognition of the human target.

2. The human body posture detection method according to claim 1, characterized in that: Using the neural network feature extraction algorithm to perform posture recognition on a human target in a corresponding video frame, determining a posture recognition result of the human target in the video frame includes: Segmenting a contour image of the human target from the video frame, and determining an edge information feature vector of the contour image according to the contour image; Determine a depth image according to the video frame, and perform key point detection on a human target in the depth image to determine key point position information based on the depth image; Determine a feature vector of the key point position information according to the key point position information; Adding the edge information feature vector of the contour image and the feature vector of the key point position information in corresponding dimensions to determine a fused feature vector; The fused feature vector is input into a preset support vector machine classifier for classification to determine the posture recognition result of the human target in the video frame.

3. The human body posture detection method according to claim 1, characterized in that: The method further comprises: If the posture type of the human target is an unqualified type, identifying the human target, and determining the identity information and region information of the human target; The identity information of the human target and the posture type are transmitted to a background database, and a prompt device of a corresponding area is triggered according to the area information.

4. The human body posture detection method according to claim 1, characterized in that: Before acquiring the shooting information of the human target in the multiple video frames, the method further includes: judging whether the human target is an employee according to the clothing of the human target in the video frame; If so, the shooting information of the human target in the video frame is obtained.

5. The human body posture detection method according to claim 4, characterized in that: Judging whether the human target is an employee by the clothing of the human target in the video frame includes: Acquire a target frame image of the human target in the video frame; According to the target frame image, determine a color channel map and a grayscale map of the target frame image, wherein the color channel map includes: an R channel map, a G channel map, and a B channel map; Counting the distribution of each pixel value in the color channel image and the grayscale image respectively, and determining a pixel statistical vector of the color channel image and a pixel statistical vector of the grayscale image; Splicing the pixel statistical vector of the color channel image and the pixel statistical vector of the grayscale image to determine a color representation image of the target frame image; Inputting the color representation image into a pre-trained color convolution model for feature extraction to determine a color feature vector; The feature vector is classified by a preset classifier to determine whether the human target is a staff member.

6. A human body posture detection device, characterized in that: The device comprises an acquisition module, a posture recognition algorithm module and a posture type recognition module; The acquisition module is used to acquire the shooting information of the human target in multiple video frames; The gesture recognition algorithm module is used to obtain a gesture recognition algorithm matching the shooting information according to the shooting information; The posture type recognition module is used to perform posture recognition on the human target in the corresponding video frame according to the matched posture recognition algorithm, and determine the posture recognition result of the human target in the video frame; the posture type recognition module is also used to determine the posture type of the human target according to the posture recognition results of multiple video frames; Wherein, the posture recognition algorithm includes a key point-based detection algorithm and a neural network-based feature extraction algorithm, and the shooting information includes a shooting angle; if the shooting angle meets a preset angle, the key point detection algorithm is used to perform posture recognition of the human target, otherwise the neural network feature extraction algorithm is used to perform posture recognition of the human target; The step of adopting the key point detection algorithm to perform posture recognition on a human target in a corresponding video frame and determining a posture recognition result of the human target in the video frame comprises: Obtaining key point positions and angles between limbs of the human target in the video frame; Determine a posture recognition result of the human target in the video frame according to the key point positions and the angle between the limbs; The adopting of the neural network feature extraction algorithm to perform posture recognition of the human target includes: adopting a neural network feature extraction algorithm based on the fusion of edge information feature vectors of contour images and feature vectors of key point position information to perform posture recognition of the human target.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the human body posture detection method according to any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the human body posture detection method as described in any one of claims 1 to 5 is implemented.

Images