Motion compliance detection method, device, equipment and medium
By establishing a binding relationship between the human body, movement and face image area markers in motion compliance detection, the problem of inaccurate detection results in the prior art is solved, and accurate analysis and real-time detection of multiple people washing their hands at the same time are realized.
Patent Information
- Application Number
- CN202411196586.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-28
- Publication Date
- 2024-11-19
AI Technical Summary
The existing motion compliance detection methods are difficult to accurately match faces and hand washing movements in the problem of inaccurate detection results, especially when multiple people wash their hands at the same time, resulting in inaccurate detection results.
By using human body recognition and action recognition based on the continuously acquired human body images, the binding relationship between human body and action recognition results and image area marks is established, and face recognition is carried out based on the continuously collected human face images to establish face recognition results and image areas The binding relationship of the tag determines the target action recognition results and the target face recognition results when someone leaves the target position area, thereby achieving action compliance detection.
It realizes real-time analysis of multiple people washing their hands simultaneously in the case of two cameras, improving the accuracy and efficiency of motion compliance detection, and ensuring the correctness and efficiency of user hand washing.
Smart Images

Figure CN118982872A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of motion detection, and in particular to a motion compliance detection method, device, equipment and medium. Background Art
[0002] Following reasonable hand washing steps is particularly important in industries such as food, medical and pharmaceuticals. However, without supervision, hand washing methods often require a certain degree of self-discipline, and the correctness and effectiveness of hand washing cannot always be guaranteed.
[0003] In the current solution, a camera is used to capture images of hand washing scenes, and face recognition is performed based on the images of hand washing scenes to detect whether the hand washing actions of the corresponding person are compliant. In some scenarios, when the above solution is used to perform action compliance detection, there is a problem of inaccurate detection results. Summary of the invention
[0004] The present application provides an action compliance detection method, device, equipment and medium to solve the problem of inaccurate detection results when performing action compliance detection in existing solutions.
[0005] In a first aspect, the present application provides an action compliance detection method, the method comprising:
[0006] Based on the continuously collected human images, human body recognition and tracking are performed to obtain human body tracking results, and based on the continuously collected human body images, action recognition is performed to obtain action recognition results, and a first binding relationship between the human body, the action recognition result and the image area mark is established according to the human body tracking result and the action recognition result, and the image area mark is used to mark the same position area in the human body image and the face image corresponding to the implementation of the action;
[0007] Performing face recognition based on continuously collected face images, and establishing a second binding relationship between the face recognition result and the image region mark, wherein the face images and the body images are collected in the same time period;
[0008] In response to a person leaving the target location area, determining a target action recognition result and a target face recognition result of the image area mark corresponding to the target location area based on the first binding relationship and the second binding relationship;
[0009] According to the target action recognition result and the target face recognition result, the action compliance detection result of the person leaving the target location area is determined.
[0010] In an embodiment of the present application, human body recognition and tracking are performed based on continuously acquired human body images to obtain human body tracking results, and action recognition is performed based on continuously acquired human body images to obtain action recognition results, and a first binding relationship between a human body, an action recognition result, and an image area mark is established based on the human body tracking results and the action recognition results, including:
[0011] Determine a preset human body acquisition device, and determine the continuously acquired human body images sent by the human body acquisition device;
[0012] Perform human body recognition frame by frame on human body images to obtain recognition results;
[0013] If the recognition result is that a human body appears in the human body image, then the human body image in which the human body appears is determined to be the initial human body image;
[0014] Determine the acquisition order of the continuously acquired human body images, and perform human body recognition on the human body images after the initial human body image according to the acquisition order and the human body, to obtain a human body tracking result;
[0015] Perform action recognition on the human body in the human body image to obtain the corresponding action recognition result.
[0016] In an embodiment of the present application, human body recognition and tracking are performed based on continuously acquired human body images to obtain human body tracking results, and action recognition is performed based on continuously acquired human body images to obtain action recognition results, and a first binding relationship between a human body, an action recognition result, and an image area mark is established based on the human body tracking results and the action recognition results, including:
[0017] Determine a preset image area mark, and determine an image area corresponding to the image area mark;
[0018] Determine a human body tracking result for each frame of the human body image, and determine a human body in the human body tracking result;
[0019] An image region and an action recognition result corresponding to a human body are determined, and the human body, the corresponding action recognition result and an image region mark corresponding to the image region are bound to obtain a first binding relationship between the human body, the action recognition result and the image region mark.
[0020] In an embodiment of the present application, in response to a person leaving the target location area, a target action recognition result and a target face recognition result of an image area mark corresponding to the target location area are determined based on the first binding relationship and the second binding relationship, including:
[0021] Determine the target human body that leaves the target location area;
[0022] According to the human body tracking result, determine the image area mark corresponding to the target human body in each frame of the human body image;
[0023] Determine the number of times each image region mark corresponds, and determine the image region mark with the largest number of times as the target image region mark;
[0024] According to the target image area mark, the target action recognition result and the target face recognition result are determined.
[0025] In the embodiment of the present application, the target action recognition result and the target face recognition result are determined according to the target image area mark, including:
[0026] Determine a target action recognition result corresponding to the target human body, and determine a face image area corresponding to the target image area mark in the face image;
[0027] Determine the face image at the acquisition time corresponding to the human body tracking result, and determine the face that appears most times in the face image area according to the face recognition result of the face image, and determine the face as the target face;
[0028] Determine the recognition result corresponding to the target face as the target face recognition result corresponding to the target image area mark.
[0029] In the embodiment of the present application, the action compliance detection result of the person leaving the target location area is determined according to the target action recognition result and the target face recognition result, including:
[0030] Determine the action time corresponding to the person leaving the target location area, and determine the time difference between the action time and the preset action time;
[0031] Determine the action similarity between the target continuous action corresponding to the target action recognition result and the preset continuous action;
[0032] Compare the time difference with a preset difference threshold range to obtain a difference comparison result, and compare the action similarity with a preset similarity threshold range to obtain a similarity comparison result;
[0033] If the difference comparison result is that the time difference does not exceed the difference threshold range, and the similarity comparison result is that the action similarity does not exceed the similarity threshold range, then the action compliance detection result is determined to be the standard;
[0034] If the difference comparison result is that the time difference exceeds the difference threshold range, or the similarity comparison result is that the action similarity exceeds the similarity threshold range, then the action compliance detection result is determined to be non-standard.
[0035] In the embodiment of the present application, after determining the compliance detection result of the action of the person leaving the target location area according to the target action recognition result and the target face recognition result, the method further includes:
[0036] Output action compliance detection results.
[0037] In a second aspect, the present application provides an action compliance detection device, the device comprising:
[0038] A first binding module is used to perform human body recognition and tracking based on continuously collected human body images to obtain human body tracking results, and to perform action recognition based on continuously collected human body images to obtain action recognition results, and to establish a first binding relationship between the human body, the action recognition result and the image area mark based on the human body tracking result and the action recognition result, wherein the image area mark is used to mark the same position area in the human body image and the face image where the corresponding action is performed;
[0039] A second binding module is used to perform face recognition based on continuously collected face images and establish a second binding relationship between the face recognition result and the image area mark, and the collection time period of the face image and the body image is the same;
[0040] A recognition result determination module, configured to determine, in response to a person leaving the target location area, a target action recognition result and a target face recognition result corresponding to the image area mark of the target location area based on the first binding relationship and the second binding relationship;
[0041] The detection result determination module is used to determine the action compliance detection result of the person leaving the target location area based on the target action recognition result and the target face recognition result.
[0042] In a third aspect, the present application provides a computer device, comprising: a processor, and a memory communicatively connected to the processor;
[0043] Memory stores computer-executable instructions;
[0044] The processor executes the computer-executable instructions stored in the memory to implement the method of the present application.
[0045] In a fourth aspect, the present application provides a computer-readable storage medium, in which program code is stored, and when the program code is executed by a processor, it is used to implement the method of the present application.
[0046] The motion compliance detection method, apparatus, equipment and medium provided in the present application obtain human tracking results by performing human body recognition and tracking based on continuously acquired human body images, and obtain motion recognition results by performing motion recognition based on continuously acquired human body images, and establish a first binding relationship between the human body, the motion recognition result and the image area mark based on the human body tracking result and the motion recognition result; perform face recognition based on continuously acquired face images, and establish a second binding relationship between the face recognition result and the image area mark; in response to someone leaving the target position area, determine the target action recognition result and the target face recognition result of the image area mark corresponding to the target position area based on the first binding relationship and the second binding relationship; determine the motion compliance detection result of the person leaving the target position area based on the target action recognition result and the target face recognition result.
[0047] In this way, the problem of matching faces from moving humans and hand-washing movements can be solved through regional joint calibration, target tracking, and matching algorithms, thereby achieving real-time analysis of multiple people washing their hands at the same time with only two cameras. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0049] Figure 1 A flowchart of an action compliance detection method provided in an embodiment of the present application;
[0050] Figure 2 A flowchart of another action compliance detection method provided in an embodiment of the present application;
[0051] Figure 3 A method diagram of an action compliance detection method provided in an embodiment of the present application;
[0052] Figure 4 A schematic diagram of the structure of an action compliance detection device provided in an embodiment of the present application;
[0053] Figure 5 This is a structural block diagram of a device for executing an action compliance detection method according to an embodiment of the present application.
[0054] The above-mentioned drawings have shown clear embodiments of the present application, which will be described in more detail below; these drawings and textual descriptions are not intended to limit the scope of the concept of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION
[0055] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.
[0056] In the existing technology, some solutions assume that the same camera is used to identify hand washing movements and faces. However, the installation conditions near the sink are very limited. The distance between people and the camera is less than 1 meter. The field of view of a camera cannot take into account both hand washing movement detection and face detection, which is difficult to implement in actual applications. In addition, some solutions only describe how to more accurately identify hand washing movements, but cannot bind users to hand washing completion status. Some solutions require users to approach a separate face camera to activate the entire process. When multiple people wash their hands at the same sink at the same time, there is a lack of correct handling methods. If multiple such systems are installed for simultaneous hand washing, additional costs are added.
[0057] In order to solve the above problems, the embodiment of the present application provides a method for detecting compliance with actions, which can be based on simple hardware devices, such as two ordinary surveillance cameras, a voice broadcast and an industrial computer device, without the need for personnel to activate separately, and can support multiple people washing hands at the same time, and make real-time voice broadcasts for non-compliant hand-washing personnel. In addition, the algorithm also uses machine learning methods to learn and improve user behavior to improve the accuracy and efficiency of detection.
[0058] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.
[0059] Figure 1 A flow chart of an action compliance detection method provided in an embodiment of the present application. Figure 1 As shown, the action compliance detection method may include the following steps:
[0060] S110, performing human body recognition and tracking based on continuously acquired human body images to obtain human body tracking results, and performing action recognition based on continuously acquired human body images to obtain action recognition results, and establishing a first binding relationship between the human body, the action recognition result and the image area mark based on the human body tracking results and the action recognition results, wherein the image area mark is used to mark the same position area in the human body image and the face image where the corresponding action is performed.
[0061] Among them, the continuously captured human body images are images corresponding to the human body area obtained by continuously capturing images of a preset human body area within a time period. For example, the camera may capture images of the human body area in real time and transmit the images to the terminal so that the terminal obtains the human body images corresponding to the human body area. At the same time, the human body area is a preset area where the human body image of the person washing hands will appear. For example, when the system is deployed, the human body camera is mainly focused on the hand movements of the person washing hands, and the image captured by the human body camera is the human body image of the person washing hands, so that the corresponding human body tracking result can be determined later based on the human body image.
[0062] Human body recognition is a technology that uses computer vision and artificial intelligence algorithms to identify and analyze human features. It has a wide range of applications in many fields, including security monitoring, medical diagnosis, virtual reality, and human-computer interaction. Through human body recognition, the human body information in the image can be determined so that the hand-washing personnel's movements can be analyzed based on the human body information.
[0063] Tracking is human body tracking. Human body tracking (or human body tracking) technology refers to the real-time detection and tracking of the position and movement trajectory of the human body in the video sequence through computer vision and artificial intelligence algorithms. In this way, by tracking the human body in the image, the movement trajectory of the human body in the continuous images can be subsequently determined.
[0064] Action recognition is a technology that uses computer vision and artificial intelligence algorithms to detect and identify human actions in video or image sequences. In this way, by performing action recognition on the human body in the human image, the corresponding hand-washing action of the human body can be determined, so as to subsequently determine whether the hand-washing action of the human body is compliant; for example, hand-washing action recognition includes the common six-step hand-washing method. Furthermore, since the system uses a separate camera to focus on the hand-washing action, and since the resolution of the hand-washing action in the image can be improved, we can use the deep learning target detection algorithm to distinguish between the left and right hands, such as rubbing the left palm and the right palm.
[0065] Image area marking can be understood as the image label corresponding to the image. Image labeling refers to the process of adding descriptive labels to objects, scenes or features in the image, which is used to determine the preset area in the image; for example, by creating a set of image labels in advance, the number of labels is the same as the number of people supported by the washbasin for hand washing, and the same labels are used to mark the areas corresponding to the same hand washing position in the face image and the body image. The image area marking only needs to be configured once before the entire system is run, so that the body information and face information corresponding to the same hand washing person and the area where the hand washing action is completed can be determined based on the image area corresponding to the image area label, thereby achieving accurate binding of the user and the hand washing completion status.
[0066] The first binding relationship can be understood as the correspondence between the human body, action recognition results and the image area marks. By determining the first binding relationship, the image area marks corresponding to the human body and action recognition results can be subsequently determined, so that the corresponding facial information can be determined according to the image area corresponding to the image area mark, thereby achieving matching of facial information and human body movements, thereby improving the accuracy and efficiency of detecting and ensuring the user's hand washing.
[0067] Based on this, by performing human body recognition tracking and action recognition on continuously collected human body images, the tracking results and action recognition results corresponding to the human body are determined, and the image area mark corresponding to the human body's action implementation area is determined, so as to determine the first binding relationship between the human body action and the image area mark, so as to subsequently determine the target image area mark corresponding to the human body action according to the first binding relationship, thereby achieving matching of human body information and facial information.
[0068] S120, performing face recognition based on the continuously collected face images, and establishing a second binding relationship between the face recognition results and the image area marks, wherein the face images and the body images are collected in the same time period.
[0069] Among them, the continuously captured facial images are images of a preset facial area that are continuously captured within a time period, thereby obtaining images corresponding to the facial area. For example, the face can be focused on by a face camera to determine the facial information corresponding to the hand-washing person. At the same time, the acquisition time of the body camera and the face camera is corresponding, that is, the facial information and body information of the hand-washing person are collected at the same time, thereby completely recording the overall hand-washing action of the hand-washing person.
[0070] Face recognition is a technology that identifies an individual by analyzing and comparing facial features. It can be achieved through deep learning technology. Face recognition is performed on images acquired by a face camera, and the facial information corresponding to the person washing hands is determined based on the recognition results. The human body movements and facial information can be matched subsequently, so as to determine the personal information corresponding to the person washing hands based on the facial information, such as the person's name and other information.
[0071] The second binding relationship can be understood as the correspondence between the face recognition result and the image area mark. By determining the second binding relationship, the face information corresponding to the image area mark can be subsequently determined, so that the face information corresponding to the image area mark corresponding to the image area corresponding to the human body movement can be determined, thereby achieving the matching of human body movement and facial information, and determining the corresponding hand washing person based on the facial information.
[0072] Based on this, face recognition is performed on the continuously collected face images to determine the corresponding face recognition result, and a binding relationship between the face recognition result and the image area mark is established, so that the face information corresponding to the target image area mark can be determined according to the second binding relationship, thereby achieving the matching of body information and face information.
[0073] S130 , in response to a person leaving the target location area, determining a target action recognition result and a target face recognition result of the image area mark corresponding to the target location area based on the first binding relationship and the second binding relationship.
[0074] The target location area is the location area where the current person performs the action to be inspected for compliance, such as the hand washing area where the person washing hands performs the hand washing action.
[0075] The response to someone leaving the target location area means that the person currently performing the action has completed the action and left the target location area. For example, a person washing hands has completed the hand washing action and is about to leave the hand washing area. The response can be through body tracking to determine that the image corresponding to the human body of the action no longer exists in the human body image transmitted by the human body camera, and then it is considered that the current person has completed the action. It can also be that the corresponding face image no longer exists in the face camera transmitted by the face camera. As long as it can be determined that the current person performing the action has completed the action, this embodiment does not limit this.
[0076] The target action recognition result is the recognition result corresponding to the action to be detected performed by the person to be tested for compliance within the target position area within the time period. For example, if a person washing hands performs the hand washing action, the corresponding target action recognition result is the hand washing action of the person washing hands during the hand washing process.
[0077] The target face recognition result is the face recognition result corresponding to the person performing the action of the target action recognition result. For example, if a person washing hands performs the hand washing action, the target face recognition result is the face recognition result corresponding to the person washing hands who performs the hand washing action in the target position area.
[0078] Based on this, by determining that the current actor has completed the action to be detected, the image area mark corresponding to the target position area corresponding to the actor is determined, and based on the first binding relationship and the second binding relationship, the target action recognition result and the target face recognition result corresponding to the image area mark are determined, thereby achieving a match between the human body action and face information corresponding to the actor, thereby improving the accuracy of the match.
[0079] S140: Determine the action compliance detection result of the person leaving the target location area according to the target action recognition result and the target face recognition result.
[0080] Among them, the action compliance detection result is used to characterize whether the human body action corresponding to the person doing the action is compliant, that is, by detecting the target action recognition result, it is determined whether the person doing the action is compliant. For example, it is determined whether the hand washing action of the person washing hands is compliant, and the result is reported, so that the person washing hands can clearly understand the completion status of the hand washing.
[0081] Based on this, the target action recognition result is used to determine the action detection result of the person who performs the action, and the face information and personnel information corresponding to the person who performs the action are determined based on the target face recognition result, so as to report the action compliance detection results and let users know whether their behavior complies with regulations.
[0082] In order to solve the problem of inaccurate matching results in the existing solutions for action compliance detection, the continuously collected human body images and face images transmitted by the human body camera and the face camera can be obtained to perform human body recognition and tracking on the human body image, and perform action recognition on the human body to obtain human body tracking results and action recognition results, and perform face recognition on the face image to obtain face recognition results, so as to bind the human body with the image area mark of the corresponding area according to the image area corresponding to the human body and the face to obtain a first binding relationship, and bind the face with the image area mark of the corresponding area to obtain a second binding relationship, so that when it is determined that a person in action has left the target position area, the target image area mark corresponding to the body of the person in action is determined according to the first binding relationship, and the corresponding face area and the corresponding target face recognition result in the area are determined in the face image according to the mark, so as to realize the determination of the corresponding personnel information of the person in action.
[0083] In this way, the human body movements and facial information of the actors can be matched according to the preset image area marks, so as to obtain accurate information about the actors. At the same time, the hand washing system can be composed of two ordinary surveillance cameras, a voice broadcast and an industrial computer device. Based on simple hardware equipment, it does not require separate activation by personnel, can support multiple people to wash their hands at the same time, and make real-time voice broadcasts for non-compliant hand washing personnel; in addition, the algorithm also uses machine learning methods to learn and improve user behavior to improve the accuracy and efficiency of detection, thereby improving the detection and ensuring the correctness and efficiency of user hand washing.
[0084] Based on the feasible implementation of S110 above, the present application further provides a process of obtaining a human body tracking result by continuously collecting human body images according to a preset human body collection device and performing human body recognition frame by frame on the human body images:
[0085] Determine a preset human body acquisition device, and determine the continuously acquired human body images sent by the human body acquisition device;
[0086] Perform human body recognition frame by frame on human body images to obtain recognition results;
[0087] If the recognition result is that a human body appears in the human body image, then the human body image in which the human body appears is determined to be the initial human body image;
[0088] Determine the acquisition order of the continuously acquired human body images, and perform human body recognition on the human body images after the initial human body image according to the acquisition order and the human body, to obtain a human body tracking result;
[0089] Perform action recognition on the human body in the human body image to obtain the corresponding action recognition result.
[0090] Among them, the preset human body acquisition device is used to obtain continuously acquired human body images in a preset human body area and transmit them to the terminal, so that the terminal can perform human body recognition and tracking based on the human body images. It can be a human body camera installed in advance.
[0091] The recognition result is used to characterize whether there is a human body in each frame of the human body image. If a human body appears in a frame of the human body image, it indicates that a person has entered the target position area and may start to perform an action. The human body image in which the person appears is determined as the initial human body image, so that the action time of the person can be determined based on the initial human body image, and the facial information and human body action can be matched.
[0092] Based on this, image acquisition and transmission are performed through the preset human body device, so that the terminal obtains continuously acquired human body images and performs frame-by-frame human body recognition, thereby determining the initial human body image in which the human body appears, and performing frame-by-frame human body recognition and human body tracking on the human body images after the initial human body image, thereby determining the human body tracking result corresponding to the human body, and at the same time performing action recognition on the human body in each frame, thereby determining the corresponding action of the human body in each frame of the human body image, so as to subsequently determine the continuous action of the human body within the action execution time period, thereby realizing compliance detection of the action, and determining whether the action performed by the human body is compliant.
[0093] Based on the feasible implementation of S110 above, the present application further provides a process of determining a first binding relationship between a human action recognition result of each frame of human image and an image area mark corresponding to an image area where the action is performed by determining a human tracking result of the human image:
[0094] Determine a preset image area mark, and determine an image area corresponding to the image area mark;
[0095] Determine a human body tracking result for each frame of the human body image, and determine a human body in the human body tracking result;
[0096] An image region and an action recognition result corresponding to a human body are determined, and the human body, the corresponding action recognition result and an image region mark corresponding to the image region are bound to obtain a first binding relationship between the human body, the action recognition result and the image region mark.
[0097] The image region corresponding to the image region mark is a preset mark region, that is, the image region corresponding to the image label in the human body image and the face image.
[0098] Based on this, by determining the image area corresponding to the human body and the human body action recognition result in each frame of the human body image, the human body and action recognition results in each frame of the human body image are bound to the image area mark corresponding to the image area, thereby obtaining a first binding relationship. When the human body action is subsequently subjected to compliance detection, the target image area corresponding to the target action of the human body is determined according to the binding relationship between the human body action and the image area mark in each frame of the human body image, thereby determining the target face recognition result corresponding to the target image area in the face image, thereby achieving matching of human body action and face recognition information.
[0099] Based on the feasible implementation of S130 above, the present application further provides a process of determining the corresponding target action recognition result and target face recognition result according to the target image area mark by determining the image area mark corresponding to the target human body in each frame of the human body image, and determining the target image area mark corresponding to the target human body during the execution of the entire action:
[0100] Determine the target human body that leaves the target location area;
[0101] According to the human body tracking result, determine the image area mark corresponding to the target human body in each frame of the human body image;
[0102] Determine the number of times each image region mark corresponds, and determine the image region mark with the largest number of times as the target image region mark;
[0103] According to the target image area mark, the target action recognition result and the target face recognition result are determined.
[0104] Among them, the target human body can be understood as the person who has currently performed the action, that is, the terminal determines the human body that has disappeared from the human body image through frame-by-frame human body recognition and tracking. At this time, the terminal binds the human body action and human body information of the target human body, and detects the action to determine whether the human body action of the target human body is compliant.
[0105] Based on this, by determining the target human body that has completed the current action and left the target position area, and based on the initial human body image corresponding to the target human body and the target human body image that has currently left the target position area, the corresponding image area mark in each frame of the human body image of the target human body within the action time period is determined, and based on the number of times each image area mark corresponds to, the target image area mark with the largest number of corresponding times is determined, so that the corresponding target action recognition result and target face recognition result can be determined based on the target area mark later. In this way, accurate recognition of the actions of the executors is achieved, inaccurate recognition caused by people moving around or occluding each other is avoided, and the accuracy of action compliance detection is improved.
[0106] Based on the feasible implementation of S130 above, the present application further provides a process of determining the target face recognition result by determining the face recognition result of the target image region mark corresponding to the region in the face image:
[0107] Determine a target action recognition result corresponding to the target human body, and determine a face image area corresponding to the target image area mark in the face image;
[0108] Determine the face image at the acquisition time corresponding to the human body tracking result, and determine the face that appears most times in the face image area according to the face recognition result of the face image, and determine the face as the target face;
[0109] Determine the recognition result corresponding to the target face as the target face recognition result corresponding to the target image area mark.
[0110] Among them, since the image area mark is used to mark the area corresponding to the same hand washing position in the face image and the body image, that is, the image area corresponding to the same image area mark corresponds to each other in the face image and the body image. For example, the image area mark corresponds to position 1 in the face image, and also corresponds to position 1 in the body image, thereby realizing the correspondence between the face and body of the actor in the target position area.
[0111] Based on this, by determining the target image area mark corresponding to the human body action of the target human body, the image area corresponding to the target image area mark is determined in the face image, and the face that appears most times in the area during the action execution time period is determined, so as to obtain the target face, that is, the face information of the person performing the action, thereby determining the match between the target face recognition result and the target face recognition result.
[0112] In this embodiment, in order to solve the problem of inaccurate detection results when performing action compliance detection in the existing scheme, the preset human body acquisition device and face acquisition device can be used to obtain the corresponding continuously acquired human body images and face images, so as to realize human body tracking and action recognition based on the human body image, and realize face recognition based on the face image, and determine the image area corresponding to each mark through the preset image area mark, so as to realize the image area mark corresponding to the human body action and establish a first binding relationship according to the image areas corresponding to the human body and the face respectively, and determine the image area mark corresponding to the face and establish a second binding relationship. In this way, by establishing the first binding relationship and the second binding relationship, the correspondence between the human body and the image area and the correspondence between the face and the image area are determined, so as to determine the target image area mark with the most corresponding times in all human body images according to the corresponding relationship, so as to determine the target face corresponding to the target human body according to the target face with the most corresponding times in the face image by the target image area mark, and realize the matching between the human body action and the target face.
[0113] In this way, the collection of human body images and human face images is realized through the preset human body collection equipment and face collection equipment. It is based on simple hardware equipment and does not require personnel to activate it separately. It can support multiple people to wash their hands at the same time. The field of view cannot take into account both hand washing action detection and face detection. It realizes matching of the human body action and face information of the person who performs the action according to the preset image area marking, and binds the user to the completion status of hand washing, which improves the accuracy and efficiency of detection, thereby ensuring the correctness and efficiency of detection and ensuring the correctness and efficiency of user hand washing.
[0114] Figure 2 A flowchart of another action compliance detection method provided in an embodiment of the present application. Figure 2 As shown, the action compliance detection method may include the following steps:
[0115] S210, performing human body recognition and tracking based on continuously collected human body images to obtain human body tracking results, and performing action recognition based on continuously collected human body images to obtain action recognition results, and establishing a first binding relationship between the human body, the action recognition result and the image area mark based on the human body tracking results and the action recognition results, wherein the image area mark is used to mark the same position area in the human body image and the face image where the corresponding action is performed.
[0116] S220, performing face recognition based on the continuously collected face images, and establishing a second binding relationship between the face recognition results and the image area marks, wherein the face images and the body images are collected in the same time period.
[0117] S230 , in response to a person leaving the target location area, determining a target action recognition result and a target face recognition result of the image area mark corresponding to the target location area based on the first binding relationship and the second binding relationship.
[0118] In this embodiment, the specific implementation of step S210 to step S230 can refer to the content of the above-mentioned embodiment, which will not be repeated here.
[0119] S240: Determine the action time corresponding to the person leaving the target location area, and determine the time difference between the action time and the preset action time.
[0120] Among them, the action time is the overall time of performing the action in the target position area, such as the hand washing time of a person washing hands; by performing frame-by-frame human body recognition on the human body images captured by the human body camera, the initial moment when the action person first appears in the target position area can be determined, that is, the initial human body image is determined, and by tracking the human body images after the initial human body image, the target moment when the action person's body does not appear in the target position area can be determined, that is, the time of leaving the target position area. In this way, based on the initial moment and the target moment, the action time corresponding to the person leaving the target position area can be determined.
[0121] The preset action time is a preset time, which is used to determine whether the action time of the action personnel is compliant, that is, by comparing the action time and the preset action time, the time difference between the two is obtained, so as to subsequently determine whether the action execution time of the action personnel meets the standard based on the time difference, for example, whether the hand washing time of the hand washing personnel is sufficient.
[0122] Based on this, the action execution time of the actor is used to determine the time difference between the action execution time and the preset action time, so that the corresponding action compliance detection result can be determined based on the time difference.
[0123] S250: Determine the action similarity between the target continuous action corresponding to the target action recognition result and the preset continuous action.
[0124] The target continuous action is the overall action of the actor within the action time, such as the entire continuous hand washing action of the hand washing person within the hand washing time; by determining the action recognized by the actor in each frame of the human body image, the target continuous action within the entire action time is determined.
[0125] The preset continuous action is a preset, standard action used to detect whether the action performed by the action personnel is compliant, that is, by comparing the target continuous action with the preset continuous action, it is determined whether the action performed by the action personnel is compliant based on the similarity of the two actions.
[0126] Based on this, by determining the similarity between the target continuous action of the action person and the preset action, it is possible to subsequently determine whether the action performed by the person is compliant based on the similarity.
[0127] S260 , comparing the time difference with a preset difference threshold range to obtain a difference comparison result, and comparing the action similarity with a preset similarity threshold range to obtain a similarity comparison result.
[0128] Among them, the preset difference threshold range is a pre-set compliance range of the time difference, that is, if the difference between the action time and the preset action time is within the threshold range, it is determined that the action time of the action personnel is compliant.
[0129] The preset similarity threshold range is a pre-set, compliance range of similarity, that is, if the similarity between the target continuous action and the preset continuous action is within the threshold range, it is determined that the action performed by the action person is compliant.
[0130] Based on this, by comparing the time difference and the preset difference threshold range, the action similarity and the preset similarity threshold range, the action time and the action performed by the actor are determined to be compliant based on the difference comparison results and similarity comparison results obtained by comparison, thereby obtaining the action compliance detection result.
[0131] S270: If the difference comparison result is that the time difference does not exceed the difference threshold range, and the similarity comparison result is that the action similarity does not exceed the similarity threshold range, then determine that the action compliance detection result is the standard.
[0132] Based on this, by determining whether the time difference and the action similarity are within the corresponding threshold range, it is determined whether the action compliance detection result is standard or non-standard. That is, if the time difference does not exceed the difference threshold range, and the action similarity does not exceed the similarity threshold range, then the action compliance detection result corresponding to the action personnel is determined to be standard.
[0133] S280: If the difference comparison result is that the time difference exceeds the difference threshold range, or the similarity comparison result is that the action similarity exceeds the similarity threshold range, then the action compliance detection result is determined to be non-standard.
[0134] Based on this, if the time difference exceeds the difference threshold range, or the action similarity exceeds the similarity threshold range, it is considered that the action execution of the person doing the action is not compliant, that is, the action compliance detection result is not standard.
[0135] In order to determine the result of compliance detection, the action time of the person leaving the target position area can be determined through the initial human image and the target human image, and the target continuous action of the person can be determined based on the action recognition result of the human image, so as to determine whether the action of the person is compliant based on the action time and the target continuous action, that is, by determining the time difference between the action time and the preset action time, and the action similarity between the target continuous action and the preset continuous action, thereby comparing the time difference and the difference threshold range, and comparing the action similarity and the similarity threshold range, and determining the action compliance result of the person according to the comparison result.
[0136] In this way, by determining the action time of the action personnel and the target continuous action, the compliance detection of the action personnel's action is realized according to the time difference and the similarity of the executed action, so as to better determine whether the action personnel's action is compliant, thereby improving the accuracy of action detection. Based on the feasible implementation method of S280 above, the present application further provides a process for notifying the result after determining the action compliance detection result:
[0137] Output action compliance detection results.
[0138] Among them, the output method of the action compliance detection result can be voice broadcast, and the function can be used to report the compliance judgment result. The voice broadcast can be a simple, intuitive and easy-to-understand way to let the action executors know whether their behavior complies with the regulations; the output method can also be to set an electronic screen in the target position area, and display the action compliance detection results through the electronic screen, such as static restrictions or scrolling restrictions; this embodiment does not limit the specific output method of the action compliance detection results.
[0139] Figure 3 A method diagram of an action compliance detection method provided in an embodiment of the present application is shown as follows: Figure 3 The first stage of the action compliance detection method can be to start collecting images and marking image areas, the second stage is to perform human body recognition, face recognition and hand washing action recognition, the third stage is to match the hand washing action with the person, and time each action completed by each person, and determine the action recognition result, so as to know the completion status of each person's hand washing action, and the fourth stage is to report the compliance detection status and broadcast the results.
[0140] In this embodiment, in order to achieve compliance detection of the action performed by the action person, the human body recognition tracking result and the action recognition result of the action person in the human body image can be determined, so as to obtain the corresponding action time from the time when the action person appears in the initial human body image in the target position area to the time when he leaves the target position area, and determine the corresponding target continuous action according to the action recognition result, so as to determine the time difference between the action time and the preset action time, and the action similarity between the target continuous action and the preset continuous action. The time difference and the action similarity can characterize the difference between the action performed by the action person and the standard action, and by determining that the difference is within the preset range, it can be characterized that the action performed by the person is compliant.
[0141] In this way, compliance detection of the actions of the acting personnel is achieved, so as to obtain the corresponding action compliance detection results, and the detection results are reported, thereby realizing action detection, personnel matching, and result notification of the acting personnel, and solving the problem of matching the face of the acting person and the hand washing action, thereby realizing real-time analysis of multiple people washing hands at the same time with only two cameras.
[0142] Figure 4 Schematic diagram of the structure of an action compliance detection device 400 provided in an embodiment of the present application. Figure 4 As shown, the action compliance detection device 400 includes: a first binding module 410 , a second binding module 420 , a recognition result determination module 430 , and a detection result determination module 440 .
[0143] A first binding module 410 is used to perform human body recognition and tracking based on continuously collected human body images to obtain human body tracking results, and to perform action recognition based on continuously collected human body images to obtain action recognition results, and to establish a first binding relationship between the human body, the action recognition result and the image area mark based on the human body tracking result and the action recognition result, wherein the image area mark is used to mark the same position area in the human body image and the face image corresponding to the implementation of the action;
[0144] A second binding module 420, configured to perform face recognition based on continuously collected face images and establish a second binding relationship between the face recognition result and the image region mark, wherein the face images and the body images are collected in the same time period;
[0145] A recognition result determination module 430 is used to determine a target action recognition result and a target face recognition result of an image area mark corresponding to the target location area based on the first binding relationship and the second binding relationship in response to a person leaving the target location area;
[0146] The detection result determination module 440 is used to determine the action compliance detection result of the person leaving the target location area based on the target action recognition result and the target face recognition result.
[0147] In the embodiment of the present application, the first binding module 410 may also be specifically used for:
[0148] Determine a preset human body acquisition device, and determine the continuously acquired human body images sent by the human body acquisition device;
[0149] Perform human body recognition frame by frame on human body images to obtain recognition results;
[0150] If the recognition result is that a human body appears in the human body image, then the human body image in which the human body appears is determined to be the initial human body image;
[0151] Determine the acquisition order of the continuously acquired human body images, and perform human body recognition on the human body images after the initial human body image according to the acquisition order and the human body, to obtain a human body tracking result;
[0152] Perform action recognition on the human body in the human body image to obtain the corresponding action recognition result.
[0153] In the embodiment of the present application, the first binding module 410 may also be specifically used for:
[0154] Determine a preset image area mark, and determine an image area corresponding to the image area mark;
[0155] Determine a human body tracking result for each frame of the human body image, and determine a human body in the human body tracking result;
[0156] An image region and an action recognition result corresponding to a human body are determined, and the human body, the corresponding action recognition result and an image region mark corresponding to the image region are bound to obtain a first binding relationship between the human body, the action recognition result and the image region mark.
[0157] In the embodiment of the present application, the recognition result determination module 430 may also be specifically used for:
[0158] Determine the target human body that leaves the target location area;
[0159] According to the human body tracking result, determine the image area mark corresponding to the target human body in each frame of the human body image;
[0160] Determine the number of times each image region mark corresponds, and determine the image region mark with the largest number of times as the target image region mark;
[0161] According to the target image area mark, the target action recognition result and the target face recognition result are determined.
[0162] In the embodiment of the present application, the recognition result determination module 430 may also be specifically used for:
[0163] Determine a target action recognition result corresponding to the target human body, and determine a face image area corresponding to the target image area mark in the face image;
[0164] Determine the face image at the acquisition time corresponding to the human body tracking result, and determine the face that appears most times in the face image area according to the face recognition result of the face image, and determine the face as the target face;
[0165] Determine the recognition result corresponding to the target face as the target face recognition result corresponding to the target image area mark.
[0166] In the embodiment of the present application, the detection result determination module 440 may also be specifically used for:
[0167] Determine the action time corresponding to the person leaving the target location area, and determine the time difference between the action time and the preset action time;
[0168] Determine the action similarity between the target continuous action corresponding to the target action recognition result and the preset continuous action;
[0169] Compare the time difference with a preset difference threshold range to obtain a difference comparison result, and compare the action similarity with a preset similarity threshold range to obtain a similarity comparison result;
[0170] If the difference comparison result is that the time difference does not exceed the difference threshold range, and the similarity comparison result is that the action similarity does not exceed the similarity threshold range, then the action compliance detection result is determined to be the standard;
[0171] If the difference comparison result is that the time difference exceeds the difference threshold range, or the similarity comparison result is that the action similarity exceeds the similarity threshold range, then the action compliance detection result is determined to be non-standard.
[0172] In the embodiment of the present application, the action compliance detection device 400 may also be specifically used for:
[0173] Output action compliance detection results.
[0174] Figure 5 This is a schematic diagram of the structure of the device provided in the embodiment of the present application. Figure 5 As shown, the device 500 includes:
[0175] The device 500 may include a processor 501 with one or more processing cores, a memory 502 with one or more computer-readable storage media, a communication component 503 and other components. The processor 501 , the memory 502 and the communication component 503 are connected via a bus 504 .
[0176] In the specific implementation process, at least one processor 501 executes the computer execution instructions stored in the memory 502, so that at least one processor 501 executes the above action compliance detection method.
[0177] The specific implementation process of the processor 501 can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.
[0178] In the above Figure 5 In the illustrated embodiment, it should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the present application may be directly implemented as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.
[0179] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (Non-volatile Memory, NVM), such as at least one disk memory.
[0180] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of the present application is not limited to only one bus or one type of bus.
[0181] In some embodiments, a computer program product is also proposed, including a computer program or instructions, which implement the steps in any of the above-mentioned action compliance detection methods when executed by a processor.
[0182] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.
[0183] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.
[0184] To this end, an embodiment of the present application provides a computer-readable storage medium, in which multiple program codes are stored. The program codes can be loaded by a processor to execute the steps in any action compliance detection method provided in the embodiment of the present application.
[0185] The storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.
[0186] According to one aspect of the present application, a computer program product or a computer program is provided. The computer program product or the computer program comprises computer instructions stored in a computer-readable storage medium.
[0187] Since the instructions stored in the storage medium can execute the steps in any action compliance detection method provided in the embodiments of the present application, the beneficial effects that can be achieved by any action compliance detection method provided in the embodiments of the present application can be achieved. Please see the previous embodiments for details and will not be repeated here.
[0188] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.
[0189] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.
Claims
1. A method for detecting action compliance, characterized in that: The method comprises: Performing human body recognition and tracking based on continuously collected human body images to obtain human body tracking results, and performing action recognition based on the continuously collected human body images to obtain action recognition results, and establishing a first binding relationship between the human body, the action recognition result and the image area mark based on the human body tracking result and the action recognition result, wherein the image area mark is used to mark the same position area in the human body image and the face image where the corresponding action is performed; Performing face recognition based on continuously collected face images, and establishing a second binding relationship between the face recognition result and the image area mark, wherein the face images and the body images are collected in the same time period; In response to a person leaving the target location area, determining a target action recognition result and a target face recognition result corresponding to the image area mark of the target location area based on the first binding relationship and the second binding relationship; The action compliance detection result of the person leaving the target location area is determined according to the target action recognition result and the target face recognition result.
2. The method according to claim 1, characterized in that The method includes performing human body recognition and tracking based on the continuously collected human body images to obtain human body tracking results, and performing action recognition based on the continuously collected human body images to obtain action recognition results, and establishing a first binding relationship between the human body, the action recognition result and the image area mark according to the human body tracking result and the action recognition result, including: Determining a preset human body acquisition device, and determining the continuously acquired human body images sent by the human body acquisition device; Performing human body recognition on the human body image frame by frame to obtain a recognition result; If the recognition result is that a human body appears in the human body image, determining that the human body image in which the human body appears is an initial human body image; Determining a collection order of the continuously collected human body images, and performing the human body recognition on the human body images after the initial human body image according to the collection order and the human body, to obtain the human body tracking result; The action recognition is performed on the human body in the human body image to obtain the corresponding action recognition result.
3. The method according to claim 1, characterized in that The method includes performing human body recognition and tracking based on the continuously collected human body images to obtain human body tracking results, and performing action recognition based on the continuously collected human body images to obtain action recognition results, and establishing a first binding relationship between the human body, the action recognition result and the image area mark according to the human body tracking result and the action recognition result, including: Determine the preset image area mark, and determine the image area corresponding to the image area mark; Determine the human body tracking result of each frame of the human body image, and determine the human body in the human body tracking result; Determine the image area and the action recognition result corresponding to the human body, and bind the human body, the corresponding action recognition result and the image area mark corresponding to the image area to obtain the first binding relationship among the human body, the action recognition result and the image area mark.
4. The method according to any one of claims 1 to 3, characterized in that In response to a person leaving the target location area, determining a target action recognition result and a target face recognition result corresponding to the image area mark of the target location area based on the first binding relationship and the second binding relationship, including: Determine the target human body that leaves the target location area; Determine the image region mark corresponding to the target human body in each frame of the human body image according to the human body tracking result; Determine the number of times each of the image area marks corresponds, and determine the image area mark with the largest number of times as the target image area mark; The target action recognition result and the target face recognition result are determined according to the target image area mark.
5. The method according to claim 4, characterized in that The step of determining the target action recognition result and the target face recognition result according to the target image area mark includes: Determine the target action recognition result corresponding to the target human body, and determine the facial image area corresponding to the target image area mark in the facial image; Determine the face image at the acquisition time corresponding to the human body tracking result, and determine the face that appears most times in the face image area according to the face recognition result of the face image, and determine the face as the target face; Determine that the recognition result corresponding to the target face is the target face recognition result corresponding to the target image area mark.
6. The method according to any one of claims 1 to 3, characterized in that The step of determining the action compliance detection result of the person leaving the target location area according to the target action recognition result and the target face recognition result includes: Determine the action time corresponding to the person leaving the target location area, and determine the time difference between the action time and a preset action time; Determine the action similarity between the target continuous action corresponding to the target action recognition result and the preset continuous action; Comparing the time difference with a preset difference threshold range to obtain a difference comparison result, and comparing the action similarity with a preset similarity threshold range to obtain a similarity comparison result; If the difference comparison result is that the time difference does not exceed the difference threshold range, and the similarity comparison result is that the action similarity does not exceed the similarity threshold range, then determining that the action compliance detection result is the standard; If the difference comparison result is that the time difference exceeds the difference threshold range, or the similarity comparison result is that the action similarity exceeds the similarity threshold range, then it is determined that the action compliance detection result is non-standard.
7. The method according to any one of claims 1 to 3, characterized in that After determining the action compliance detection result of the person leaving the target location area according to the target action recognition result and the target face recognition result, the method further includes: Output the compliance detection result of the action.
8. An action compliance detection device, characterized in that: The device comprises: A first binding module is used to perform human body recognition and tracking based on continuously collected human body images to obtain human body tracking results, and to perform action recognition based on the continuously collected human body images to obtain action recognition results, and to establish a first binding relationship between the human body, the action recognition result and the image area mark based on the human body tracking result and the action recognition result, wherein the image area mark is used to mark the same position area in the human body image and the face image where the corresponding action is performed; A second binding module is used to perform face recognition based on continuously collected face images and establish a second binding relationship between the face recognition result and the image area mark, wherein the face images and the body images are collected in the same time period; A recognition result determination module, configured to determine, in response to a person leaving a target location area, a target action recognition result and a target face recognition result corresponding to an image area mark of the target location area based on the first binding relationship and the second binding relationship; The detection result determination module is used to determine the action compliance detection result of the person leaving the target location area according to the target action recognition result and the target face recognition result.
9. A device, characterized in that: include: one or more processors; Memory; One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the method according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program code, which can be called by a processor to execute the method according to any one of claims 1 to 7.
