Personnel behavior compliance detection method, device and electronic device

Through image processing technology, the behavior of bank tellers is analyzed, and the problem of inability to detect tellers' behavior norms in the existing technology is solved, and accurate compliance judgment of bank tellers' behavior is achieved.

CN112668398BActive Publication Date: 2024-11-19ZHEJIANG DAHUA TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202011419238.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-12-07
Publication Date
2024-11-19
Estimated Expiration
2040-12-07

AI Technical Summary

Technical Problem

The existing technology cannot effectively analyze whether the behavior of bank tellers is standardized, and can only analyze service quality through recording information, and cannot directly detect the behavioral compliance of tellers.

Method used

By obtaining images containing people's activities, tracking and detecting human targets, obtaining target node information, analyzing whether the body movements comply with the behavioral norms in the preset activity scenario, and if they do not comply, an alarm indication of irregular behavior is output.

Benefits of technology

It realizes an accurate analysis of bank teller behavior, improves the accuracy of judgment, and can effectively determine whether teller behavior is in compliance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112668398B_ABST
    Figure CN112668398B_ABST
Patent Text Reader

Abstract

The present application relates to a method, device and electronic device for detecting compliance of personnel behavior, which obtains an image containing personnel activity information, tracks and detects a first human target in the image, obtains target joint point information of the first human target in a first detection area, and performs motion analysis on the target joint point information in the first detection area when a preset activity scene appears in the image, obtains target limb movements, and determines whether the target limb movements comply with the behavioral norms under the preset activity scene. If not, an alarm indication of irregular behavior is output. This method of analyzing personnel behavior by integrating multiple image processing technologies can achieve a high judgment accuracy, thereby effectively determining whether personnel behavior is compliant.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of image recognition, and in particular to methods, devices and electronic devices for detecting compliance of human behavior. Background Art

[0002] With the in-depth research and development of deep learning in academia, artificial intelligence products have been implemented and promoted in all walks of life in society, providing convenience for people's work and life. For the financial industry, tellers in business halls directly face customers, and good teller service quality is very important to improve customer satisfaction and brand awareness. The use of artificial intelligence technology to detect the compliance of bank tellers' behavior has also become a common solution in the current financial industry to determine whether tellers' behavior is compliant.

[0003] In the current existing technology, voice recognition technology is often used to analyze the acquired bank teller service recording information to determine whether the teller's service quality is compliant. This method can only analyze the teller service information through the recording information, but cannot effectively analyze whether the teller's own behavior is standardized.

[0004] There is no effective solution to the above problems yet. Summary of the invention

[0005] The embodiments of the present application provide a method, device and electronic device for detecting compliance of personnel behavior, so as to at least solve the problem in the related art that it is impossible to effectively analyze whether the behavior of the personnel themselves is standardized.

[0006] In a first aspect, an embodiment of the present application provides a method for detecting compliance of personnel behavior, the method comprising the following steps:

[0007] Acquire images containing information about human activities;

[0008] Tracking and detecting a first human target in the image to obtain target joint point information of the first human target in a first detection area;

[0009] When a preset activity scene appears in the image, performing motion analysis on the target joint point information in the first detection area to obtain a target limb motion, and determining whether the target limb motion conforms to the behavior specification under the preset activity scene;

[0010] If not, an alarm indication of irregular behavior is output.

[0011] In some embodiments, performing motion analysis on the target joint point information in the first detection area to obtain the target limb motion includes:

[0012] Fitting the target joint point information to obtain a number of straight lines;

[0013] The geometric relationship between the plurality of straight lines is analyzed, and the target limb movement of the first human target is obtained according to the analysis result.

[0014] In some embodiments, fitting the target joint point information to obtain a plurality of straight lines includes:

[0015] The left elbow joint point, the left wrist joint point and the left shoulder joint point are fitted into a first straight line, and the right elbow joint point, the right wrist joint point and the right shoulder joint point are fitted into a second straight line;

[0016] The step of analyzing the geometric relationship between the plurality of straight lines and obtaining the target limb movement of the first human target according to the analysis result comprises:

[0017] When the first straight line is parallel to the second straight line, it is determined that the target limb movement of the first human target is a two-handed handover movement.

[0018] In some embodiments, fitting the target joint point information to obtain a plurality of straight lines further comprises:

[0019] Fitting the wrist joint point and the elbow joint point on the same side into a third straight line, and fitting the elbow joint point and the shoulder joint point on the same side as the third straight line into a fourth straight line;

[0020] The step of analyzing the geometric relationship between the plurality of straight lines and obtaining the target limb movement of the first human target according to the analysis result further comprises:

[0021] When the angle between the third straight line and the fourth straight line is smaller than a set threshold, and the third straight line is in a vertical direction, it is determined that the target limb movement of the first human target is a hand-raising movement.

[0022] In some embodiments, the following steps are also included: obtaining the shoulder joint point, hip joint point and knee joint point on the same side, and when the shoulder joint point, the hip joint point and the knee joint point can be fitted into a straight line, determining that the target limb movement of the first human target is a standing movement.

[0023] In some of the embodiments, the following steps are also included: tracking a second human target in the image to obtain a motion trajectory of the second human target in a second detection area; if the motion trajectory intersects with the second detection area, determining that the preset activity scene occurs.

[0024] In some of the embodiments, before tracking and detecting the first human target in the image to obtain target joint point information of the first human target within a first detection area, the following steps are also included: dividing the image into functional areas to obtain the first detection area where the first human target is located and the second detection area where the second human target is located.

[0025] In some embodiments, the method also includes: the image is an image containing activity information of business hall personnel; the first human target is a teller human target, and the second human target is a customer human target; the first detection area includes a transaction area and a teller area, and the second detection area includes a customer area; the judging whether the preset activity scene appears in the image includes judging whether people enter and exit the customer area.

[0026] In some of the embodiments, the following steps are also included: obtaining a background image of the first detection area through background modeling; comparing the background image of the first detection area with the current image for similarity to determine whether a preset activity scene appears in the image.

[0027] In some of the embodiments, the method further includes: acquiring a background image of a transaction area in a business hall; performing a similarity comparison between the background image of the transaction area and a current image of the transaction area to determine whether an object transfer has occurred in the transaction area.

[0028] In some embodiments, the determination of whether the target body movement complies with the behavioral norms under the preset activity scenario includes: when the preset activity scenario is that an object is passed in a transaction area, the behavioral norm is that the teller in the teller area must perform a two-handed handover action.

[0029] In some embodiments, the determination of whether the target body movement conforms to the behavioral norms under the preset activity scenario further includes: when the preset activity scenario is that a person enters the customer area, the behavioral norm is that the teller must raise his hand to signal.

[0030] In some embodiments, the determination of whether the target body movement conforms to the behavioral norms under the preset activity scenario also includes: when the preset activity scenario is that a person leaves the customer area, the behavioral norm is that the tellers in the teller area must stand up and see the person off.

[0031] In a second aspect, an embodiment of the present application provides a personnel behavior compliance detection device, including: an image acquisition module, a joint point information acquisition module, and a compliance behavior judgment module:

[0032] The image acquisition module is used to acquire images containing personnel activity information;

[0033] The joint point information acquisition module is used to track and detect the first human target in the image to obtain target joint point information of the first human target in the first detection area;

[0034] The compliance behavior judgment module is used to perform motion analysis on the target joint point information in the first detection area to obtain target limb movements and judge whether the target limb movements comply with the behavior norms under the preset activity scene when a preset activity scene appears in the image;

[0035] If not, an alarm indication of irregular behavior is output.

[0036] In a third aspect, an embodiment of the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for detecting compliance of personnel behavior as described in the first aspect above is implemented.

[0037] The above-mentioned personnel behavior compliance detection method, device and electronic device obtain an image containing personnel activity information, track and detect the first human target in the image, obtain the target joint point information of the first human target in the first detection area, and when a preset activity scene appears in the image, perform motion analysis on the target joint point information in the first detection area to obtain the target limb movement, and determine whether the target limb movement conforms to the behavior norms under the preset activity scene. If so, it is confirmed that the behavior of the first human target is compliant. This method of analyzing personnel behavior by integrating multiple image processing technologies can achieve a high judgment accuracy, thereby effectively determining whether personnel behavior is compliant. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0039] Figure 1 is an application environment diagram of a method for detecting compliance of personnel behavior according to an embodiment of the present invention;

[0040] Figure 2 is a flow chart of a method for detecting compliance of personnel behavior according to one embodiment of the present invention;

[0041] Figure 3 is a business hall area division diagram under video surveillance according to an embodiment of the present invention;

[0042] Figure 4is a schematic diagram of the structure of a device for detecting compliance of human behavior according to an embodiment of the present invention;

[0043] Figure 5 is a schematic structural diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments provided in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application. In addition, it can also be understood that although the efforts made in this development process may be complex and lengthy, for ordinary technicians in the field related to the contents disclosed in the present application, some changes such as design, manufacturing or production based on the technical contents disclosed in the present application are only conventional technical means, and should not be understood as insufficient contents disclosed in the present application.

[0045] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those of ordinary skill in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0046] Unless otherwise defined, the technical terms or scientific terms involved in this application should be understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the" and the like involved in this application do not indicate a quantitative limitation, and may represent the singular or plural. The terms "include", "comprise", "have" and any of their variations involved in this application are intended to cover non-exclusive inclusions; for example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include other steps or units inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to greater than or equal to two. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships, for example, "A and / or B" can represent: A exists alone, A and B exist at the same time, and B exists alone. The terms "first", "second", "third" and the like involved in the present application are merely used to distinguish similar objects and do not represent a specific ordering of the objects.

[0047] Figure 1 This is an application environment diagram of a method for detecting compliance of personnel behavior in one embodiment of the present application. Figure 1 As shown, in this application environment, a monitoring terminal 101 and a server 102 are included. The monitoring terminal 101 can be any video monitoring device. The server 102 can be implemented as an independent server or a server cluster composed of multiple servers.

[0048] Specifically, the monitoring terminal 101 collects an image containing the information of the person's activities and sends the image to the server 102. After acquiring the image containing the information of the person's activities, the server 102 tracks and detects the first human target in the image, obtains the target joint point information of the first human target in the first detection area, and performs motion analysis on the target joint point information in the first detection area when a preset activity scene appears in the image, obtains the target limb movement, and determines whether the target limb movement conforms to the behavior norm under the preset activity scene. If not, an alarm indication of irregular behavior is output.

[0049] This application embodiment provides a method for detecting compliance of personnel behavior. Figure 2 is a flow chart of a method for detecting compliance of personnel behavior according to an embodiment of the present application. Figure 2 As shown, the process includes the following steps:

[0050] Step S201, obtaining an image containing personnel activity information.

[0051] Specifically, the above-mentioned image containing the information of personnel activities may be an image collected by a video surveillance device, containing the movement or body movements of personnel. The personnel here may be personnel in a bank business hall under video surveillance or personnel in a street office, specifically, tellers and customers. For the sake of brevity, the following will directly take a bank business hall under video surveillance as an example to explain the embodiment of the present application.

[0052] Step S202: Track and detect the first human target in the image to obtain target joint point information of the first human target in a first detection area.

[0053] The first human target may be an employee in the video surveillance, such as a teller in the business hall, or other personnel who need to be tested for behavioral compliance. To track and detect the first human target in the image, a deep learning target detection algorithm may be used to first detect the human frame in the image, and then bone joint point detection may be performed within the human frame area to obtain target joint point information representing human limb movement information.

[0054] The first detection area here refers to the area where the first human target is active and the people in it need to be subject to behavioral compliance detection. Specifically, it can be the working area of ​​employees under video surveillance, such as the teller area or transaction area in a bank business hall.

[0055] Step S203, when a preset activity scene appears in the image, motion analysis is performed on the target joint point information in the first detection area to obtain the target limb motion, and determine whether the target limb motion conforms to the behavioral norms under the preset activity scene.

[0056] The preset activity scene here refers to a specific event that occurs in the scene under video surveillance, which can be the entry and exit of customers in the bank business hall, the delivery of items generated by business, etc. The motion analysis of the target joint point information here includes processing the extracted target joint point information, for example, fitting the target joint point information into other geometric shapes to determine the body movements formed by the joint points.

[0057] Step S204: If not, output an alarm indication of irregular behavior.

[0058] Specifically, the target body movement is judged to be inconsistent with the behavioral norms under the preset activity scenario. When events such as customers entering or leaving the business hall or items being passed between the teller and the customer occur, the teller fails to perform the actions prescribed by the bank based on the corresponding events, and the teller's behavior is considered to be non-compliant.

[0059] The above-mentioned personnel behavior compliance detection method obtains an image containing personnel activity information, tracks and detects the first human target in the image, obtains the target joint point information of the first human target in the first detection area, and when a preset activity scene appears in the image, performs motion analysis on the target joint point information in the first detection area to obtain the target limb movement, and determines whether the target limb movement conforms to the behavior norms under the preset activity scene. If not, outputs an alarm indication of irregular behavior. This method of analyzing personnel behavior by integrating multiple image processing technologies can achieve a high judgment accuracy, thereby effectively determining whether personnel behavior is compliant.

[0060] In one embodiment, based on the above step S203, performing motion analysis on the target joint point information in the first detection area to obtain the target limb motion includes the following steps:

[0061] Step S301, fitting the target joint point information to obtain a number of straight lines.

[0062] Among them, the target joint point information refers to the position information of each joint in the first human target extracted from the image, specifically including shoulder joint points, elbow joint points, wrist joint points, hip joint points, etc. By fitting these target joint point information, one or more straight lines can be obtained.

[0063] Step S302, analyzing the geometric relationship between a plurality of straight lines, and obtaining the target limb movement of the first human target according to the analysis result.

[0064] When the first human target in the image performs a certain body movement, a corresponding geometric relationship will be generated between the straight lines fitted according to the joint points, such as intersection or parallelism. Therefore, the target body movement of the first human target can be determined by analyzing the geometric relationship.

[0065] In one embodiment, based on the above step S301, the target joint point information is fitted to obtain several straight lines, including: fitting the left elbow joint point, the left wrist joint point and the left shoulder joint point into a first straight line, and fitting the right elbow joint point, the right wrist joint point and the right shoulder joint point into a second straight line. Based on the above step S302, the geometric relationship between the several straight lines is analyzed, and the target limb movement of the first human target is obtained according to the analysis result, including: when the first straight line is parallel to the second straight line, determining that the limb movement of the first human target is a two-handed handover movement.

[0066] Among them, the first straight line indicates that the left arm is in a straight state, and the second straight line indicates that the right arm is in a straight state. When the two straight lines are parallel, it means that the two arms in the straight state are parallel to each other. Usually, the body movement in this state is generally the submission of documents. Therefore, it can be judged that the body movement of the first human target is the submission of both hands.

[0067] In one embodiment, based on the above step S301, the target joint point information is fitted to obtain several straight lines, including: fitting the wrist joint point and elbow joint point on the same side into a third straight line, and fitting the elbow joint point and shoulder joint point on the same side as the third straight line into a fourth straight line. Based on the above step S302, the geometric relationship between the several straight lines is analyzed, and the target limb movement of the first human target is obtained according to the analysis result, including: when the angle between the third straight line and the fourth straight line is less than a set threshold, and the third straight line is in a vertical direction, it is determined that the limb movement of the first human target is a hand-raising movement.

[0068] It is understandable that the wrist joint point can be a left wrist joint point or a right wrist joint point, the elbow joint can be a left elbow joint point or a right elbow joint point, and the shoulder joint point can be a left shoulder joint point or a right shoulder joint point. The same side here means that the acquired wrist joint points, elbow joint points and shoulder joint points should all belong to the same arm. For example, the left wrist joint point and the left elbow joint point are fitted into a third straight line, and the left elbow joint point and the left shoulder joint point are fitted into a fourth straight line. When the angle between the third straight line and the fourth straight line is less than a certain threshold value, and the third straight line is in a vertical direction relative to the ground, it is determined that the limb movement of the first human target is a left hand raising movement.

[0069] In one embodiment, the shoulder joint point, hip joint point and knee joint point on the same side are obtained, and when the shoulder joint point, hip joint point and knee joint point can be fitted into a straight line, it is determined that the limb movement of the first human target is a standing movement.

[0070] In one embodiment, the personnel behavior compliance detection method further includes:

[0071] Step S401: Track a second human target in the image to obtain a motion trajectory of the second human target in a second detection area.

[0072] The second human target refers to other human targets that do not require behavioral compliance detection but are related to the formation of a preset activity scene and are different from the first human target. The second human target can specifically be a customer in a business hall. Tracking the second human target includes using a target detection algorithm to detect a human frame for the second detection target in the image and then using a tracking algorithm to track the human body, thereby obtaining a tracking trajectory of the second detection target, that is, a motion trajectory.

[0073] Similarly, the second detection area here refers to the area where the second human target mainly moves, such as the customer area in a business hall under video surveillance, such as Figure 3 The client area in is the main activity area of ​​the client who is the second human target.

[0074] Specifically, tracking the second human target in the image and obtaining the motion trajectory of the second human target in the second detection area can be performed by performing target detection and human tracking on customers in the customer area of ​​the business hall, thereby obtaining the motion trajectory of the customers.

[0075] Step S402: If the motion trajectory intersects with the second detection area, it is determined that a preset activity scene occurs.

[0076] Specifically, the intersection here can be that the customer's movement trajectory intersects with the client area, for example, a trajectory shows that the target moves outside the area, or a new trajectory extends from outside the area, thereby judging that a customer has entered or left the client area.

[0077] In one embodiment, before tracking and detecting the first human target in the image to obtain target joint point information of the first human target within the first detection area, the method further includes:

[0078] Step S501 , dividing the image into regions according to functions, to obtain a first detection region where a first human target is located and a second detection region where a second human target is located.

[0079] The division of image into regions according to function refers to the division of regions according to the differences in human activities caused by functional differences in the image. Figure 3 As shown, the human activity information image in the bank business hall under video surveillance is divided into three areas: transaction area, teller area and customer area. Among them, the transaction area and teller area are the main activity areas of the tellers, so the first detection area includes the transaction area and teller area where the tellers as the first human detection target are located, and the second detection area includes the customer area where the customers as the second human target are located.

[0080] In one embodiment, the personnel behavior compliance detection method further includes: the image is an image containing personnel activity information in the business hall, the first human target is a teller human target, the second human target is a customer human target, the first detection area includes the transaction area and the teller area of ​​the business hall, and the second detection area includes the customer area of ​​the business hall. Determining whether a preset activity scene appears in the image includes determining whether a person enters or exits the customer area.

[0081] Specifically, the activity information of business hall personnel includes the entry, exit and movement of customers in the customer area, a series of physical movements made by customers and tellers in the transaction area when conducting business transactions, and various physical movements made by tellers in the teller area.

[0082] Whether a person enters or leaves the customer area can be determined by, for example, judging whether a customer enters or leaves the customer area based on the customer's motion trajectory in the customer area in the above manner. It should be noted that judging whether a person enters or leaves the customer area is not limited to using the motion trajectory as a feature, and other image features of the area can also be obtained through other deep learning algorithms, and a single feature can be used, or multiple features can be fused to analyze whether a customer enters or leaves the customer area.

[0083] In one embodiment, the personnel behavior compliance detection method further includes:

[0084] Step S601, obtaining a background image of a first detection area through background modeling.

[0085] Step S602: perform a similarity comparison between the background image of the first detection area and the current image to determine whether a preset activity scene appears in the image.

[0086] If an event occurs in the same area within a time period, the different image frames acquired by the video surveillance of this area during this time period will be different. Therefore, by obtaining the similarity between the background image of the first detection area and the current image, it can be determined whether an event has occurred in the area.

[0087] The similarity can be obtained as shown in Formula 1:

[0088]

[0089] Among them I 1 Indicates the current image, I 2 Represents the background image. When the NCC value is greater than the set threshold, it means that a preset activity scene appears in the first detection area.

[0090] In one embodiment, the personnel behavior compliance detection method further includes:

[0091] Step S701, obtaining a background image of a transaction area in a business hall.

[0092] Step S702, comparing the background image of the transaction area with the current image of the transaction area for similarity to determine whether an object transfer has occurred in the transaction area.

[0093] The similarity here can be obtained by using the formula 1 in the above step S602, or by other similarity calculation methods in the field of deep learning. The object transfer in the transaction area can specifically be the transfer of documents, certificates and other business-related items between the teller and the customer. When the current image of the transaction area is obtained and the similarity with the background image is obtained, and the similarity exceeds a certain threshold, it can be determined that the object transfer or other interactive behaviors between the customer and the teller have occurred in the transaction area.

[0094] In one embodiment, determining whether the target body movement conforms to the behavior norm under the preset activity scene includes: when the preset activity scene is the object transfer in the transaction area, the behavior norm is that the teller in the teller area must perform the handover action with both hands.

[0095] In one embodiment, determining whether the target body movement conforms to the behavior norm under the preset activity scene includes: when the preset activity scene is that a person enters the customer area, the behavior norm is that the teller must raise his hand to signal.

[0096] In one embodiment, determining whether the target body movement conforms to the behavior norm in the preset activity scene includes: when the preset activity scene is that a person leaves the customer area, the behavior norm is that tellers in the teller area must stand up and see the person off.

[0097] In a specific embodiment, a method for detecting compliance of personnel behavior is provided, comprising the following steps:

[0098] 1. Obtain images containing information about personnel activities.

[0099] 1-1. Divide the image into regions according to function to obtain a first detection region where the first human target is located and a second detection region where the second human target is located.

[0100] 2. Track and detect the first human target in the image to obtain target joint point information of the first human target in the first detection area.

[0101] 3. When a preset activity scene appears in the image, the target joint point information in the first detection area is subjected to motion analysis to obtain the target limb movement, and it is determined whether the target limb movement conforms to the behavioral norms under the preset activity scene.

[0102] 3-1. Tracking a second human target in the image to obtain a motion trajectory of the second human target in a second detection area;

[0103] 3-2. If the motion trajectory intersects with the second detection area, it is determined that the preset activity scene occurs.

[0104] 3-2-1. The image contains the activity information of the personnel in the business hall. The first human target is the teller human target, and the second human target is the customer human target. The first detection area includes the transaction area and the teller area, and the second detection area includes the customer area. It is determined whether a preset activity scene appears in the image, including whether there are people entering and exiting the customer area.

[0105] 3-3. Obtain the background image of the first detection area through background modeling.

[0106] 3-3-1. Obtain the background image of the transaction area in the business hall.

[0107] 3-4. Compare the background image of the first detection area with the current image for similarity to determine whether a preset activity scene appears in the image.

[0108] 3-4-1. Compare the similarity between the background image of the transaction area and the current image of the transaction area to determine whether an object transfer has occurred in the transaction area.

[0109] 3-5. Fit the target joint point information to obtain several straight lines.

[0110] 3-5-1. Fit the left elbow joint, left wrist joint and left shoulder joint into a first straight line, and fit the right elbow joint, right wrist joint and right shoulder joint into a second straight line.

[0111] 3-5-2. Fit the wrist joint points and elbow joint points on the same side into a third straight line, and fit the elbow joint points and shoulder joint points on the same side as the third straight line into a fourth straight line.

[0112] 3-6. Analyze the geometric relationship between the above-mentioned straight lines, and obtain the target limb movement of the first human target based on the analysis results.

[0113] 3-6-1. When the first straight line is parallel to the second straight line, the target limb movement of the first human target is determined to be a two-handed handover movement.

[0114] 3-6-2. When the angle between the third straight line and the fourth straight line is less than the set threshold, and the third straight line is in a vertical direction, it is determined that the target limb movement of the first human target is a hand-raising movement.

[0115] 3-6-3. Obtain the shoulder joint point, hip joint point and knee joint point on the same side. When the shoulder joint point, hip joint point and knee joint point can be fitted into a straight line, determine that the target limb movement of the first human target is a standing movement.

[0116] 3-7. When the preset activity scenario is that an object is passed in the transaction area, the behavioral norm is that the teller in the teller area must use both hands to hand over the object.

[0117] 3-8. When the preset activity scenario is that someone enters the customer area, the behavioral norm is that the teller must raise his hand to signal.

[0118] 3-9. When the preset activity scenario is that someone leaves the customer area, the behavioral norm is that tellers in the teller area must stand up and see them off.

[0119] 4. If not, output an alarm indication of irregular behavior.

[0120] It should be understood that, although the various steps in the above-mentioned flow chart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the above-mentioned flow chart may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0121] In one embodiment, Figure 4 As shown, a device 400 for detecting compliance of human behavior is provided, comprising: an image acquisition module 401, a joint point information acquisition module 402, a compliance behavior judgment module 403 and an alarm module 404, wherein:

[0122] The image acquisition module 401 is used to acquire images containing personnel activity information.

[0123] The joint point information acquisition module 402 is used to track and detect the first human target in the image to obtain target joint point information of the first human target in the first detection area.

[0124] The compliance behavior judgment module 403 is used to perform motion analysis on the target joint point information in the first detection area when a preset activity scene appears in the image, obtain the target limb movement, and judge whether the target limb movement complies with the behavior norms under the preset activity scene.

[0125] The alarm module 404 is used to output an alarm indication of irregular behavior when the target body movement does not conform to the behavior norms in the preset activity scene.

[0126] The above-mentioned personnel behavior compliance detection device obtains an image containing personnel activity information, tracks and detects the first human target in the image, obtains the target joint point information of the first human target in the first detection area, and performs motion analysis on the target joint point information in the first detection area when a preset activity scene appears in the image to obtain the target limb movement, and outputs an alarm indication of irregular behavior when the target limb movement does not conform to the behavior norms under the preset activity scene. The device integrates multiple image processing technologies to analyze personnel behavior, and can obtain a high judgment accuracy, thereby effectively determining whether personnel behavior is compliant.

[0127] In one embodiment, the compliance behavior judgment module 403 is also used to fit the target joint point information to obtain a number of straight lines, analyze the geometric relationship between the above-mentioned straight lines, and obtain the target limb movement of the first human target according to the analysis results.

[0128] In one embodiment, the compliance behavior judgment module 403 is also used to fit the left elbow joint, left wrist joint and left shoulder joint into a first straight line, and fit the right elbow joint, right wrist joint and right shoulder joint into a second straight line. When the first straight line is parallel to the second straight line, it is determined that the target limb movement of the first human target is a two-handed handover movement.

[0129] In one embodiment, the compliance behavior judgment module 403 is also used to fit the wrist joint points and elbow joint points on the same side into a third straight line, and to fit the elbow joint points and shoulder joint points on the same side of the third straight line into a fourth straight line. When the angle between the third straight line and the fourth straight line is less than a set threshold and the third straight line is in a vertical direction, it is determined that the target limb movement of the first human target is a hand-raising movement.

[0130] In one embodiment, the compliance behavior judgment module 403 is also used to obtain the shoulder joint point, hip joint point and knee joint point on the same side. When the shoulder joint point, hip joint point and knee joint point can be fitted into a straight line, it is determined that the target limb movement of the first human target is a standing action.

[0131] In one embodiment, the compliance detection device 400 also tracks the second human target in the image through the motion trajectory acquisition module to obtain the motion trajectory of the second human target in the second detection area. Based on the motion trajectory of the second human target in the second detection area, if the motion trajectory intersects with the second detection area, it is determined that a preset activity scene has occurred.

[0132] In one embodiment, the compliance detection device 400 further divides the image into regions according to functions through a region division module to obtain a first detection region where the first human target is located and a second detection region where the second human target is located.

[0133] In one embodiment, the compliance detection device 400 is also used to obtain images containing activity information of business hall personnel, divide the image area into a transaction area, a teller area, and a customer area, obtain target joint point information of the teller human body target, obtain the motion trajectory of the customer human body target, and use the entry and exit of personnel in the customer area as a preset activity scene.

[0134] In one embodiment, the similarity comparison module in the compliance detection device 400 obtains the background image of the first detection area through background modeling, compares the background image of the first detection area with the current image for similarity, and determines whether a preset activity scene appears in the image.

[0135] In one embodiment, the compliance detection device 400 also obtains the background image of the transaction area in the business hall through the similarity comparison module, compares the background image of the transaction area with the current image of the transaction area for similarity, and determines whether object transfer has occurred in the transaction area.

[0136] In one embodiment, the compliance behavior determination module 403 is also used to determine whether the teller has two hands handing over the object when the object is transferred in the transaction area.

[0137] In one embodiment, the compliance behavior determination module 403 is also used to determine whether the teller raises his hand to signal when a person enters the customer area.

[0138] In one embodiment, the compliance behavior judgment module 403 is also used to judge whether the tellers in the teller area stand up and watch the person leave the customer area when the person leaves the customer area.

[0139] For the specific definition of the personnel behavior compliance detection device, please refer to the definition of the personnel behavior compliance detection method above, which will not be repeated here. Each module in the above-mentioned personnel behavior compliance detection device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0140] In one embodiment, Figure 5 As shown, an electronic device is provided, including a memory and a processor. The memory stores a computer program, and the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium.

[0141] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0142] Optionally, in this embodiment, the processor implements the following steps when executing the computer program: acquiring an image containing personnel activity information; tracking and detecting a first human target in the image to obtain target joint point information of the first human target in a first detection area; when a preset activity scene appears in the image, performing motion analysis on the target joint point information in the first detection area to obtain target limb movements, and determining whether the target limb movements conform to the behavioral norms under the preset activity scene; if not, outputting an alarm indication of irregular behavior.

[0143] In one embodiment, when the processor executes the computer program, the following steps are also implemented: fitting the target joint point information to obtain a number of straight lines; analyzing the geometric relationship between the several straight lines, and obtaining the target limb movement of the first human target based on the analysis results.

[0144] In one embodiment, the processor also implements the following steps when executing the computer program: fitting the left elbow joint, left wrist joint and left shoulder joint into a first straight line, and fitting the right elbow joint, right wrist joint and right shoulder joint into a second straight line; when the first straight line is parallel to the second straight line, determining that the target limb movement of the first human target is a two-handed handover movement.

[0145] In one embodiment, when the processor executes the computer program, the following steps are also implemented: fitting the wrist joint point and elbow joint point on the same side into a third straight line, and fitting the elbow joint point and shoulder joint point on the same side of the third straight line into a fourth straight line; when the angle between the third straight line and the fourth straight line is less than a set threshold and the third straight line is in a vertical direction, determining that the target limb movement of the first human target is a hand-raising movement.

[0146] In one embodiment, when the processor executes the computer program, the following steps are also implemented: obtaining the shoulder joint point, hip joint point and knee joint point on the same side, and when the shoulder joint point, hip joint point and knee joint point can be fitted into a straight line, determining that the target limb movement of the first human target is a standing movement.

[0147] In one embodiment, when the processor executes the computer program, the following steps are also implemented: tracking a second human target in the image to obtain a motion trajectory of the second human target in a second detection area; if the motion trajectory intersects with the second detection area, determining that the preset activity scene occurs.

[0148] In one embodiment, when the processor executes the computer program, the following steps are further implemented: dividing the image into regions according to functions to obtain a first detection region.

[0149] In one embodiment, the processor also implements the following steps when executing the computer program: the image is an image containing activity information of personnel in the business hall; the first human target is a teller human target, and the second human target is a customer human target; the first detection area includes the transaction area and the teller area, and the second detection area includes the customer area; and determining whether a preset activity scene appears in the image, including determining whether personnel enter or exit the customer area.

[0150] In one embodiment, when the processor executes the computer program, the following steps are also implemented: obtaining a background image of the first detection area through background modeling; comparing the background image of the first detection area with the current image for similarity to determine whether a preset activity scene appears in the image.

[0151] In one embodiment, when the processor executes the computer program, the following steps are also implemented: obtaining a background image of the transaction area in the business hall; comparing the background image of the transaction area with the current image of the transaction area for similarity to determine whether an object transfer has occurred in the transaction area.

[0152] In one embodiment, when the processor executes the computer program, the following steps are also implemented: when the preset activity scene is that an object is passed in the transaction area, the behavioral specification is that the teller in the teller area must perform a two-handed handover action.

[0153] In one embodiment, when the processor executes the computer program, the following steps are also implemented: when the preset activity scene is that a person enters the customer area, the behavioral norm is that the teller must raise his hand to signal.

[0154] In one embodiment, when the processor executes the computer program, the following steps are also implemented: when the preset activity scenario is that a person leaves the customer area, the behavioral norm is that the tellers in the teller area must stand up and see the person off.

[0155] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing related hardware through a computer program, wherein the computer program can be stored in a non-volatile computer-readable storage medium, and the computer program, when executed, can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0156] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0157] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for detecting compliance of personnel behavior, It is characterized in that include: Acquire images containing information about human activities; Tracking and detecting a first human target in the image to obtain target joint point information of the first human target in a first detection area; When a preset activity scene appears in the image, performing motion analysis on the target joint point information in the first detection area to obtain a target limb motion, and determining whether the target limb motion conforms to the behavior specification under the preset activity scene; If not, an alarm indication of irregular behavior is output; Tracking a second human target in the image to obtain a motion trajectory of the second human target in a second detection area; If the motion trajectory intersects with the second detection area, determining that the preset activity scene occurs; including: if the trajectory shows that the second human target moves outside the second detection area, determining that the second human target leaves the second detection area; Alternatively, a new trajectory extends from outside the second detection area, and it is determined that the second human target has entered the second detection area.

2. The method according to claim 1, It is characterized in that The performing motion analysis on the target joint point information in the first detection area to obtain the target limb motion includes: Fitting the target joint point information to obtain a number of straight lines; The geometric relationship between the plurality of straight lines is analyzed, and the target limb movement of the first human target is obtained according to the analysis result.

3. The method according to claim 2, It is characterized in that The target joint point information is fitted to obtain a plurality of straight lines including: The left elbow joint point, the left wrist joint point and the left shoulder joint point are fitted into a first straight line, and the right elbow joint point, the right wrist joint point and the right shoulder joint point are fitted into a second straight line; The step of analyzing the geometric relationship between the plurality of straight lines and obtaining the target limb movement of the first human target according to the analysis result comprises: When the first straight line is parallel to the second straight line, it is determined that the target limb movement of the first human target is a two-handed handover movement.

4. The method according to claim 2, It is characterized in that The fitting of the target joint point information to obtain a plurality of straight lines also includes: Fitting the wrist joint point and the elbow joint point on the same side into a third straight line, and fitting the elbow joint point and the shoulder joint point on the same side as the third straight line into a fourth straight line; The step of analyzing the geometric relationship between the plurality of straight lines and obtaining the target limb movement of the first human target according to the analysis result further comprises: When the angle between the third straight line and the fourth straight line is smaller than a set threshold, and the third straight line is in a vertical direction, it is determined that the target limb movement of the first human target is a hand-raising movement.

5. The method according to claim 2, It is characterized in that Also includes: The ipsilateral shoulder joint point, hip joint point and knee joint point are obtained, and when the shoulder joint point, the hip joint point and the knee joint point can be fitted into a straight line, it is determined that the target limb action of the first human target is a standing action.

6. The method according to claim 1, It is characterized in that Before tracking and detecting the first human target in the image to obtain target joint point information of the first human target within the first detection area, the following steps are also included: The image is divided into regions according to functions to obtain the first detection region where the first human target is located and the second detection region where the second human target is located.

7. The method according to claim 6, It is characterized in that The method further comprises: The image is an image containing activity information of business hall personnel; The first human target is a teller human target, and the second human target is a customer human target; The first detection area includes the transaction area and the teller area, and the second detection area includes the customer area; The determining whether the preset activity scene appears in the image includes determining whether a person enters or exits a customer area.

8. The method according to claim 1, It is characterized in that The following steps are also included: Acquire a background image of the first detection area through background modeling; The background image of the first detection area is compared with the current image for similarity to determine whether a preset activity scene appears in the image.

9. The method according to claim 8, It is characterized in that Also includes: Obtain the background image of the transaction area in the business hall; A similarity comparison is performed between the background image of the transaction area and the current image of the transaction area to determine whether an object transfer occurs in the transaction area.

10. The method according to any one of claims 1 to 9, It is characterized in that The determining whether the target body movement conforms to the behavior specification in the preset activity scene includes: When the preset activity scene is that an object is passed in the transaction area, the behavior specification is that the teller in the teller area must hand over the object with both hands.

11. The method according to any one of claims 1 to 9, It is characterized in that The determining whether the target body movement conforms to the behavior specification in the preset activity scene also includes: When the preset activity scene is that a person enters the customer area, the behavior norm is that the tellers in the teller area must raise their hands to signal.

12. The method according to any one of claims 1 to 9, It is characterized in that The determining whether the target body movement conforms to the behavior specification in the preset activity scene also includes: When the preset activity scenario is that someone leaves the customer area, the behavioral norm is that the tellers in the teller area must stand up and see the person off.

13. A device for detecting compliance of personnel behavior, It is characterized in that include: Image acquisition module, joint point information acquisition module, and compliance behavior judgment module; The image acquisition module is used to acquire images containing personnel activity information; The joint point information acquisition module is used to track and detect the first human target in the image to obtain target joint point information of the first human target in the first detection area; The compliance behavior judgment module is used to perform motion analysis on the target joint point information in the first detection area to obtain target limb movements and judge whether the target limb movements comply with the behavior norms under the preset activity scene when a preset activity scene appears in the image; If not, an alarm indication of irregular behavior is output; The compliance detection device also tracks the second human target in the image through a motion trajectory acquisition module to obtain the motion trajectory of the second human target in the second detection area. If the motion trajectory intersects with the second detection area, it is determined that the preset activity scene occurs; including: a trajectory shows that the second human target moves outside the second detection area, determining that the second human target has left the second detection area; or a new trajectory extends from outside the second detection area, determining that the second human target has entered the second detection area.

14. An electronic device comprising a memory and a processor, It is characterized in that A computer program is stored in the memory, and the processor is configured to run the computer program to execute the personnel behavior compliance detection method according to any one of claims 1 to 12.