Behavior recognition method and device, electronic equipment and storage medium

Abstract

The invention provides a behavior recognition method and device, electronic equipment and a storage medium. The behavior recognition method comprises the steps that a single-frame image which is collected by a target camera and comprises at least one person is acquired; intercepting each figure area in the single-frame image, and determining the intercepted single-frame image as a figure image; and for each character image, identifying a plurality of features in the character image for determining the character in the character image, and determining the compliance of the character in the character image based on each feature of the character in the character image. By adopting the technical scheme provided by the invention, the real-time performance of the system is improved, and the calculation redundancy is reduced.

Description

Technical Field

[0001] This application relates to the field of behavior recognition, and in particular to a behavior recognition method, apparatus, electronic device, and storage medium. Background Technology

[0002] In production workshops, employee misconduct is a key factor affecting production safety and hindering production efficiency. Currently, with the rapid development of computer vision technology, real-time monitoring of employee misconduct through workshop surveillance cameras has become a viable method.

[0003] However, existing violation detection technologies typically detect multiple violations independently, which not only increases the demand for computing resources but also leads to a certain degree of computational redundancy. Summary of the Invention

[0004] In view of this, embodiments of this application provide a behavior recognition method, device, electronic device, and storage medium, which, based on multimodal behavior detection technology and combined with time-domain information, overcomes the high demand for computing resources in existing systems, improves the real-time performance of the system, and reduces computational redundancy.

[0005] This application mainly includes the following aspects: In a first aspect, embodiments of this application provide a behavior recognition method, the behavior recognition method comprising: Acquire a single-frame image containing at least one person captured by the target camera; Each person region in the single frame image is cropped, and the cropped single frame image is determined as the person image; For each image of a person, multiple features of the person in the image are identified to determine the compliance of the person in the image based on each feature.

[0006] Furthermore, the features of each person in the image include: facial feature vector, hair region, and mobile phone usage status; The process of determining the compliance of a person in an image based on each feature of that person includes: Based on the facial feature vector of the person in the image, the compliance of the person's identity is determined; based on the hair area of ​​the person in the image, the compliance of the person's image is determined; and based on the person's mobile phone usage status, the compliance of the person's mobile phone usage is determined.

[0007] Furthermore, determining the identity compliance of the person in the image based on the facial feature vector of the person in the image includes: The facial feature vector of the person in the image is compared with the facial feature vectors of multiple people in the face database to obtain the similarity between the facial feature vector of the person in the image and each facial feature vector in the face database. If the maximum similarity is greater than the preset similarity threshold, then the person in the image is determined to be a non-intruder. If the maximum similarity is not greater than the preset similarity threshold, then the person in the image is determined to be an intruder.

[0008] Furthermore, determining the compliance of the hair in the image based on the hair region of the person in the image includes: Identify the hair region of the person in the image; Based on the pixel coordinates of the left shoulder bone point and the right shoulder bone point in the image of the person, determine the pixel coordinates of the back neck bone point in the image of the person. If the pixel coordinates of the back of the neck bone point of the person in the image are located in the hair area of ​​the person in the image, then the image of the person in the image is determined to be compliant. If the pixel coordinates of the back of the neck bone point of the person in the image are not located in the hair area of ​​the person in the image, then the image of the person is determined to be in violation of the rules.

[0009] Furthermore, determining the compliance of a person's mobile phone use based on the person's mobile phone usage status in the image includes: Determine whether a mobile phone is detected in the image of the person; If no mobile phone is detected in the image of the person, it is determined that the person in the image is not using a mobile phone; if a mobile phone is detected in the image of the person, it is determined whether the distance between the hand bone point of the person in the image and the mobile phone is less than a first preset distance threshold. If the distance between the hand bone point and the mobile phone is not less than a first preset distance threshold, it is determined that the person in the image is not using a mobile phone; if the distance between the hand bone point and the mobile phone is less than the first preset distance threshold, it is determined whether the distance between the head bone point of the person in the image and the mobile phone is less than a second preset distance threshold. If the distance between the head bone point and the mobile phone is less than the second preset distance threshold, it is determined that the person in the image is using a mobile phone to make a call, and it is determined that the person in the image is using the mobile phone legally; if the distance between the head bone point and the mobile phone is not less than the second preset distance threshold, it is determined that the person in the image is using the mobile phone illegally.

[0010] Furthermore, the behavior recognition method also includes: When it is determined that a person in the image is using a mobile phone in violation of regulations, the cumulative number of frames in the image corresponding to the person's violation of regulations is counted based on the person's identity identifier in the image. The ratio of the cumulative number of frames in the image to the preset total number of frames is determined as the frame rate ratio of the person's illegal use of the mobile phone in the image. If the frame rate ratio is greater than a preset ratio threshold, it is determined that the person in the image has been using a mobile phone illegally for an extended period of time.

[0011] Furthermore, the behavior recognition method also includes: Based on a series of multiple frames of images containing at least one person captured by the target camera, the identity of each person in the multiple frames of images is determined. For each identity identifier, the fall status of the person corresponding to that identity identifier is determined based on multiple consecutive frames of images.

[0012] Secondly, embodiments of this application also provide a behavior recognition device, the behavior recognition device comprising: The acquisition module is used to acquire a single-frame image containing at least one person captured by the target camera; The cropping module is used to crop each human figure region in the single frame image and determine the cropped single frame image as the human figure image; The recognition module is used to identify multiple features of the person in each image to determine the compliance of the person in the image based on each feature.

[0013] Thirdly, embodiments of this application also provide an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory through the bus. The machine-readable instructions are executed by the processor to perform the steps of the behavior recognition method described in the first aspect or any possible implementation of the first aspect.

[0014] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the behavior recognition method described in the first aspect or any possible implementation of the first aspect.

[0015] This application provides a behavior recognition method, device, electronic device, and storage medium, which acquires a single-frame image containing at least one person captured by a target camera; crops each person region in the single-frame image and determines the cropped single-frame image as a person image; for each person image, identifies multiple features of the person in the person image to determine the compliance of the person in the person image based on each feature of the person in the person image.

[0016] This improves the system's real-time performance and reduces computational redundancy.

[0017] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This document shows one of the flowcharts of a behavior recognition method provided in an embodiment of this application; Figure 2 A second flowchart of a behavior recognition method provided in an embodiment of this application is shown; Figure 3 This illustration shows a structural schematic diagram of a behavior recognition device provided in an embodiment of this application; Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.

[0021] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0022] The methods, apparatus, electronic devices, or computer-readable storage media described in this application can be applied to any scenario requiring behavior recognition. This application does not limit specific application scenarios, and any scheme using the behavior recognition methods and apparatus provided in this application is within the protection scope of this application.

[0023] It is worth noting that employee violations are a key factor affecting production safety and hindering production efficiency in production workshops. Currently, with the rapid development of computer vision technology, real-time monitoring of employee violations through workshop surveillance cameras has become a feasible method. However, existing violation detection technologies typically detect multiple violations independently, which not only increases the demand for computing resources but also leads to a certain degree of computational redundancy.

[0024] To address the aforementioned issues, this application proposes a behavior recognition method, device, electronic device, and storage medium. Based on multimodal behavior detection technology and combined with temporal information, it overcomes the high computational resource requirements of existing systems, improves the real-time performance of the system, and reduces computational redundancy.

[0025] To facilitate understanding of this application, the technical solutions provided in this application will be described in detail below with reference to specific embodiments.

[0026] In the embodiments of this application, existing violation detection technologies typically detect each type of violation independently. Furthermore, these technologies primarily rely on the analysis of single-frame images, without incorporating temporal information from the video stream for comprehensive judgment.

[0027] Please see Figure 1 , Figure 1 This is one of the flowcharts for a behavior recognition method provided in an embodiment of this application.

[0028] like Figure 1 As shown in the figure, the behavior recognition method provided in this application includes the following steps: Step S101: Acquire a single-frame image containing at least one person captured by the target camera.

[0029] Here, the target camera captures images within its field of view in real time and obtains single-frame images containing at least one person.

[0030] Step S102: Extract each person region from the single frame image and determine the extracted single frame image as the person image.

[0031] In this embodiment, the human figure region can be determined using the YOLOV8-POSE model (a skeletal point detection model). Specifically, a single-frame image is input into the YOLOV8-POSE model to obtain the bounding box, skeletal points, bounding box confidence score, and skeletal point confidence score for each human figure in the single-frame image output by the YOLOV8-POSE model. The bounding box defines the boundary and position of the human figure region. Based on the bounding box of each human figure in the single-frame image, each human figure is cropped. Cropping the human figure reduces the detection range, removes useless information, and improves the detection rate and accuracy.

[0032] In this embodiment of the application, the YOLOV8-POSE model is trained using the COCO dataset (Common Objects in Context). The human skeleton points in the COCO dataset include: nose, right eye, left eye, right ear, left ear, right shoulder, left shoulder, right elbow, left elbow, right wrist, left wrist, right hip, left hip, right knee, left knee, right ankle, and right ankle.

[0033] Step S103: For each person image, identify multiple features of the person in the person image to determine the compliance of the person in the person image based on each feature of the person in the person image.

[0034] Here, the features of each person in the image include: facial feature vector, hair area, and mobile phone usage status.

[0035] Regarding step S103, determining the compliance of a person in an image based on each feature of the person in the image, as an example, may include the following steps: determining the identity compliance of the person in the image based on the facial feature vector of the person in the image; determining the image compliance of the person in the image based on the hair region of the person in the image; and determining the mobile phone usage compliance of the person in the image based on the mobile phone usage status of the person in the image.

[0036] Regarding the determination of the identity compliance of a person in an image based on the facial feature vector, as an example, the specific implementation may include the following steps: S11. Compare the facial feature vector of the person in the image with multiple facial feature vectors in the face database to obtain the similarity between the facial feature vector of the person in the image and each facial feature vector in the face database.

[0037] Here, an electronic fence is set up within the field of view of the target camera. Face detection is triggered when a person crosses the electronic fence. The electronic fence is a virtual boundary composed of multiple preset pixel coordinate points. In this application, a face detection model is used to detect faces in the image, and the model is trained using a publicly available face dataset. For the detected faces, a face feature vector is obtained using a face feature extraction model. As an example, the face feature vector can be obtained using a ResNet-based method.

[0038] S12. If the maximum similarity is greater than the preset similarity threshold, then the person in the image is determined to be a non-intruder.

[0039] S13. If the maximum similarity is not greater than the preset similarity threshold, then the person in the image is determined to be an intruder.

[0040] Regarding determining the compliance of a person's image based on the hair region in the image, as an example, the specific implementation may include the following steps: S21. Identify the hair area of ​​the person in the image.

[0041] Here, a semantic segmentation model is used to identify the hair region in a person's image. Specifically, a public hair semantic segmentation dataset is used to train the semantic segmentation model for human hair.

[0042] S22. Based on the pixel coordinates of the left shoulder bone point and the right shoulder bone point in the image of the person, determine the pixel coordinates of the back neck bone point in the image of the person.

[0043] Here, based on the image of the person, the midpoint between the left and right shoulder bones, i.e., the back of the neck bone, is calculated. Specifically, assuming the pixel coordinates of the left shoulder bone are (xl, yl) and the pixel coordinates of the right shoulder bone are (xr, yr), then the midpoint is (xc, yc) = ((xl+xr) / 2, (yl+yr) / 2), which is the pixel coordinate of the back of the neck bone.

[0044] It should be noted that, due to potential fluctuations in the accuracy of skeletal point localization, a skeletal point confidence threshold is set to evaluate the reliability of the currently detected left and right shoulder skeletal points. Only when the confidence levels of both skeletal points exceed the skeletal point confidence threshold will the detection of the person's long, flowing hair feature be triggered. If the confidence level of either skeletal point does not reach the skeletal point confidence threshold, the long, flowing hair feature will not be determined.

[0045] S23. If the pixel coordinates of the back of the neck bone point of the person in the image are located in the hair area of ​​the person in the image, then the image of the person in the image is determined to be compliant.

[0046] Analyze the segmented areas of the hair to determine if the hair area covers the posterior cervical sac. If the hair area does not cover the posterior cervical sac, it can be determined that the person does not have long, flowing hair.

[0047] S24. If the pixel coordinates of the back of the neck bone point of the person in the image are not located in the hair area of ​​the person in the image, then the image of the person in the image is determined to be in violation of the rules.

[0048] If the hair area covers the skeletal points at the back of the neck, it can be determined that the person has long, shoulder-length hair.

[0049] Regarding determining the compliance of a person's mobile phone use based on the person's mobile phone usage status in the image, as an example, the specific implementation may include the following steps: S31. Determine whether a mobile phone is detected in the image of the person.

[0050] Here, a mobile phone detection model is used to identify whether a mobile phone is present in the image of the person. Specifically, a publicly available mobile phone dataset is used to train the face detection model.

[0051] S32. If no mobile phone is detected in the image of the person, it is determined that the person in the image is not using a mobile phone.

[0052] S33. If a mobile phone is detected in the image of the person, determine whether the distance between the hand bone point of the person in the image and the mobile phone is less than a first preset distance threshold.

[0053] Here, the hand bone points include at least one of the following: left hand bone points and right hand bone points. Whether the phone is in the person's hand is determined by whether the person's hand bone points are located near the phone.

[0054] S34. If the distance between the hand bone point and the mobile phone is not less than the first preset distance threshold, then it is determined that the person in the image is not using a mobile phone.

[0055] Here, if the distance between the hand bone point and the mobile phone is not less than the first preset distance threshold, it means that the mobile phone is placed on a platform or table.

[0056] S35. If the distance between the hand bone point and the mobile phone is less than a first preset distance threshold, then determine whether the distance between the head bone point of the person in the image and the mobile phone is less than a second preset distance threshold.

[0057] Here, if the distance between the hand skeletal point and the mobile phone is less than a first preset distance threshold, it indicates that the mobile phone is in the person's hand. By determining whether the distance between the head skeletal point of the person in the image and the mobile phone is less than a second preset distance threshold, it is determined whether the person is looking at their hand or making a phone call.

[0058] S36. If the distance between the head bone point and the mobile phone is less than the second preset distance threshold, it is determined that the person in the image is using the mobile phone to make a call, and it is determined that the person in the image is using the mobile phone in compliance with regulations.

[0059] Here, if the distance between the head bone point and the mobile phone is less than the second preset distance threshold, it indicates that the person is making a call.

[0060] S37. If the distance between the head bone point and the mobile phone is not less than the second preset distance threshold, then it is determined that the person in the image is illegally using the mobile phone.

[0061] If the distance between the head bone point and the mobile phone is not less than the second preset distance threshold, it indicates that the person is playing with the mobile phone.

[0062] It should be noted that the detection of the person's long, flowing hair feature will only be triggered when the confidence scores of both the head and hand skeletal points exceed the skeletal point confidence threshold. If the confidence score of any skeletal point does not reach the skeletal point confidence threshold, no determination of unauthorized mobile phone use will be made.

[0063] In one possible implementation, this application determines whether a mobile phone has been used improperly for an extended period of time, specifically: S41. When it is determined that a person in the image is using a mobile phone in violation of regulations, based on the identity identifier of the person in the image, count the cumulative number of frames in the image corresponding to the person's violation of regulations.

[0064] Here, to determine whether a person has been using their phone illegally for an extended period, temporal information is required. Specifically, a tracking model is used to identify individuals in the images, ensuring that the same person has the same identification in consecutive frames. A temporal counter is then established to accumulate the number of frames in which each identified person has been illegally using their phone.

[0065] To accurately determine whether an individual has been using their phone improperly for an extended period, a temporal information analysis method is employed. Specifically, a tracking model is used to continuously track each individual and assign them a unique identification number. This ensures that the same individual maintains the same identification number across consecutive frames. Simultaneously, a temporal statistician is used to record the cumulative number of frames during which each identified individual improperly uses their phone.

[0066] S42. The ratio of the cumulative number of frames in the image to the preset total number of frames is determined as the frame rate ratio of the person's illegal use of the mobile phone in the image.

[0067] The specific criteria for determining prolonged unauthorized use of a mobile phone are as follows: if the proportion of frames in which a person uses a mobile phone in violation of regulations exceeds a preset threshold, then the person is determined to have used a mobile phone in violation of regulations for an extended period of time.

[0068] The specific criteria for determining prolonged unauthorized use of a mobile phone are as follows: if the proportion of frames a person uses while using the phone exceeds a preset threshold, then that person will be considered to have used the phone for an extended period of time without authorization.

[0069] S43. If the frame rate ratio is greater than a preset ratio threshold, it is determined that the person in the image has been using a mobile phone illegally for an extended period of time.

[0070] Here, as an example, the preset ratio threshold is 60%.

[0071] like Figure 2 As shown, in one possible implementation, this application determines whether a person has fallen, specifically: S51. Based on a series of multiple frames of images containing at least one person captured by the target camera, determine the identity of each person in the multiple frames of images.

[0072] Here, time-domain information is needed to determine whether a person has fallen. A tracking model is used to continuously track each person and assign a unique identifier, continuously tracking individuals with the identified identifiers.

[0073] S52. For each identity identifier, based on the consecutive multiple frames of images corresponding to that identity identifier, determine the fall status of the person corresponding to that identity identifier.

[0074] Here, by analyzing multiple consecutive frames of images, all detection boxes for individuals with the same identity are obtained. These detection boxes are then input into an action recognition model to obtain the individual's fall state as output by the model. As an example, the action recognition model can use the FastSlow model.

[0075] The present application provides a behavior recognition method that overcomes the high demand for computing resources in existing systems, improves the real-time performance of the system, and reduces computational redundancy.

[0076] Based on the same application concept, this application also provides a behavior recognition device corresponding to the behavior recognition method provided in the above embodiments. Since the principle of the device in this application is similar to the behavior recognition method in the above embodiments of this application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0077] Please see Figures 3 to 4 , Figure 3 This is one of the structural schematic diagrams of a behavior recognition device provided in an embodiment of this application. Figure 4 This is a second schematic diagram of the structure of a behavior recognition device provided in an embodiment of this application.

[0078] like Figure 3 As shown in the figure, the behavior recognition device 310 provided in this application embodiment includes: The acquisition module 311 is used to acquire a single-frame image containing at least one person captured by the target camera; The cropping module 312 is used to crop each human figure region in the single frame image and determine the cropped single frame image as the human figure image. The recognition module 313 is used to identify multiple features of the person in each person image to determine the compliance of the person in the person image based on each feature.

[0079] Furthermore, the features of each person in the image include: facial feature vector, hair region, and mobile phone usage status; The identification module 313 is specifically used for: Based on the facial feature vector of the person in the image, the compliance of the person's identity is determined; based on the hair area of ​​the person in the image, the compliance of the person's image is determined; and based on the person's mobile phone usage status, the compliance of the person's mobile phone usage is determined.

[0080] Furthermore, when determining the identity compliance of a person in an image based on the facial feature vector of that person, the recognition module 313 is also specifically used for: The facial feature vector of the person in the image is compared with the facial feature vectors of multiple people in the face database to obtain the similarity between the facial feature vector of the person in the image and each facial feature vector in the face database. If the maximum similarity is greater than the preset similarity threshold, then the person in the image is determined to be a non-intruder. If the maximum similarity is not greater than the preset similarity threshold, then the person in the image is determined to be an intruder.

[0081] Furthermore, when determining the compliance of a person's hair in an image based on the hair region in the image, the recognition module 313 is specifically used for: Identify the hair region of the person in the image; Based on the pixel coordinates of the left shoulder bone point and the right shoulder bone point in the image of the person, determine the pixel coordinates of the back neck bone point in the image of the person. If the pixel coordinates of the back of the neck bone point of the person in the image are located in the hair area of ​​the person in the image, then the image of the person in the image is determined to be compliant. If the pixel coordinates of the back of the neck bone point of the person in the image are not located in the hair area of ​​the person in the image, then the image of the person is determined to be in violation of the rules.

[0082] Furthermore, when determining the compliance of a person's mobile phone use based on the person's mobile phone usage status in the image, the recognition module 313 is also specifically used for: Determine whether a mobile phone is detected in the image of the person; If no mobile phone is detected in the image of the person, it is determined that the person in the image is not using a mobile phone; if a mobile phone is detected in the image of the person, it is determined whether the distance between the hand bone point of the person in the image and the mobile phone is less than a first preset distance threshold. If the distance between the hand bone point and the mobile phone is not less than a first preset distance threshold, it is determined that the person in the image is not using a mobile phone; if the distance between the hand bone point and the mobile phone is less than the first preset distance threshold, it is determined whether the distance between the head bone point of the person in the image and the mobile phone is less than a second preset distance threshold. If the distance between the head bone point and the mobile phone is less than the second preset distance threshold, it is determined that the person in the image is using a mobile phone to make a call, and it is determined that the person in the image is using the mobile phone legally; if the distance between the head bone point and the mobile phone is not less than the second preset distance threshold, it is determined that the person in the image is using the mobile phone illegally.

[0083] Furthermore, the behavior recognition device also includes: The statistics module is used to count the cumulative number of frames in an image when a person is found to be illegally using a mobile phone, based on the person's identity in the image. The ratio determination module is used to determine the ratio of the cumulative number of frames in the image to the preset total number of frames as the frame rate of the person's illegal use of the mobile phone in the image of the person. The long-term violation determination module is used to determine that if the frame rate ratio is greater than a preset ratio threshold, the person in the image has been illegally using a mobile phone for an extended period of time.

[0084] Furthermore, the behavior recognition device also includes: The identification determination module is used to determine the identity of each person in a series of multiple frames of images containing at least one person, based on images captured by a target camera. The fall status determination module is used to determine the fall status of the person corresponding to each identity based on multiple consecutive frames of images corresponding to that identity.

[0085] The present application provides a behavior recognition device that overcomes the high demand for computing resources in existing systems, improves the real-time performance of the system, and reduces computational redundancy.

[0086] Please see Figure 4 , Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0087] like Figure 4 As shown, the electronic device 400 includes a processor 410, a memory 420, and a bus 430.

[0088] The memory 420 stores machine-readable instructions executable by the processor 410. When the electronic device 400 is running, the processor 410 communicates with the memory 420 via the bus 430. When the machine-readable instructions are executed by the processor 410, they can perform the operations described above. Figure 1 and Figure 2 The steps of the behavior recognition method in the illustrated method embodiment can be found in the method embodiment for specific implementation, and will not be repeated here.

[0089] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can perform the above-described actions. Figure 1 and Figure 2 The steps of the behavior recognition method in the illustrated method embodiment can be found in the method embodiment for specific implementation, and will not be repeated here.

[0090] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and devices described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0091] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0092] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0093] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0094] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A behavior recognition method, characterized by, The behavior recognition method comprises: acquiring a single-frame image containing at least one person collected by a target camera; cutting each person region in the single-frame image, and determining the cut single-frame image as a person image; for each person image, recognizing a plurality of features of the person in the person image to determine compliance of the person in the person image based on each feature of the person in the person image.

2. The behavior recognition method of claim 1, wherein, The features of the person in each person image comprise a face feature vector, a hair region and a mobile phone use state; the determination of the compliance of the person in the person image based on each feature of the person in the person image comprises: determination of identity compliance of the person in the person image based on the face feature vector of the person in the person image, determination of image compliance of the person in the person image based on the hair region of the person in the person image, and determination of mobile phone use compliance of the person in the person image based on the mobile phone use state of the person in the person image.

3. The behavior recognition method of claim 2, wherein, the determination of the identity compliance of the person in the person image based on the face feature vector of the person in the person image comprises: comparing the face feature vector of the person in the person image with a plurality of face feature vectors in a face library respectively to obtain a similarity of the face feature vector of the person in the person image and each face feature vector in the face library; if the maximum similarity is greater than a preset similarity threshold, the person in the person image is determined as a non-intruding person; if the maximum similarity is not greater than the preset similarity threshold, the person in the person image is determined as an intruding person.

4. The behavior recognition method of claim 2, wherein, the determination of the hair compliance of the person in the person image based on the hair region of the person in the person image comprises: recognizing the hair region of the person in the person image; determining a pixel coordinate of a back neck skeletal point of the person in the person image according to a pixel coordinate of a left shoulder skeletal point and a pixel coordinate of a right shoulder skeletal point of the person in the person image; if the pixel coordinate of the back neck skeletal point of the person in the person image is located in the hair region of the person in the person image, the person in the person image is determined as image compliance; if the pixel coordinate of the back neck skeletal point of the person in the person image is not located in the hair region of the person in the person image, the person in the person image is determined as image violation.

5. The behavior recognition method of claim 2, wherein, the determination of the mobile phone use compliance of the person in the person image based on the mobile phone use state of the person in the person image comprises: determination of whether a mobile phone is recognized in the person image; if the mobile phone is not detected in the person image, the person in the person image is determined as not using the mobile phone; if the mobile phone is detected in the person image, determination of whether a distance between a hand skeletal point of the person in the person image and the mobile phone is less than a first preset distance threshold; if the distance between the hand skeletal point and the mobile phone is not less than the first preset distance threshold, the person in the person image is determined as not using the mobile phone; if the distance between the hand skeletal point and the mobile phone is less than the first preset distance threshold, determination of whether a distance between a head skeletal point of the person in the person image and the mobile phone is less than a second preset distance threshold; If the distance between the head skeleton point and the mobile phone is less than a second preset distance threshold, it is determined that the person in the person image uses the mobile phone to make a call, and it is determined that the person in the person image uses the mobile phone in compliance with regulations; if the distance between the head skeleton point and the mobile phone is not less than the second preset distance threshold, it is determined that the person in the person image uses the mobile phone in violation of regulations.

6. The behavior recognition method of claim 5, wherein, The behavior recognition method further includes: When it is determined that the person in the person image uses the mobile phone in violation of regulations, based on the identity of the person in the person image, the cumulative frame number of the image corresponding to the person in the person image when using the mobile phone in violation of regulations is counted; The quotient of the cumulative frame number and a preset total frame number is determined as the frame number ratio of the person in the person image using the mobile phone in violation of regulations; If the frame number ratio is greater than a preset ratio threshold, it is determined that the person in the person image uses the mobile phone in violation of regulations for a long time.

7. The behavior recognition method of claim 1, wherein, The behavior recognition method further includes: Based on the continuous multiple frames of images containing at least one person collected by a target camera, the identity of each person in the multiple frames of images is determined; For each identity, based on the continuous multiple frames of images corresponding to the identity, the falling state of the person corresponding to the identity is determined.

8. A behavior recognition apparatus, characterized by comprising: The behavior recognition device includes: An acquisition module configured to acquire a single frame of image containing at least one person collected by a target camera; An intercepting module configured to intercept each person region in the single frame of image, and determine the intercepted single frame of image as a person image; An identification module configured to, for each person image, identify multiple features of the person in the person image, and determine the compliance of the person in the person image based on each feature of the person in the person image.

9. An electronic device, comprising: It includes: A processor, a memory and a bus, the memory stores machine readable instructions executable by the processor, when the electronic device is running, the processor and the memory communicate through the bus, the machine readable instructions are executed by the processor to perform the steps of the behavior recognition method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to perform the steps of the behavior recognition method according to any one of claims 1 to 7.

Images