A method and system for identifying suspected customer operation behavior of employees in a bank

Abstract

The application provides a bank employee suspected customer operation behavior identification method and system, through directly interfacing with the real-time picture of the bank site camera, using a deep learning model to process the picture data in seconds, real-time output of key information of personnel, posture and sensitive articles, no data storage delay, through dynamic analysis based on real-time output information, the time consumption from sensitive article ownership judgment to customer operation behavior identification is controlled in milliseconds, the problem can be found in the process of employee customer operation instead of after completion, through the alarm mechanism and the identification result synchronous triggering, the waiting time of manual review is avoided, the key window for stopping illegal behavior on site is won, finally, the deep learning image target detection, posture analysis, face recognition and other means are used to realize real-time, automatic and accurate identification of bank employee customer operation behavior, effectively reducing the financial risk caused by customer operation.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method and system for identifying suspected proxy transactions by bank employees. Background Technology

[0002] As vital financial service institutions, banks bear the important responsibility of safeguarding customer funds and maintaining financial order. With the rapid development of the financial industry, banking services are becoming increasingly diversified and complex, leading to more interactions between customers and bank employees. The phenomenon of employees acting on behalf of customers, such as operating mobile phones or signing documents on their behalf, is becoming increasingly prominent. Such actions, whether against the customer's will or intentionally inducing them to perform certain actions, pose challenges to customer rights protection and the stable operation of banking services.

[0003] Currently, the supervision of bank employees' proxy transactions mainly adopts a post-event review approach. For example, random checks of the previous day's surveillance video at bank branches are used to screen for such transactions. This method has a significant time lag, making it difficult to detect and stop such transactions in a timely manner. Furthermore, random manual review is inefficient, easily influenced by subjective factors leading to missed detections. The multi-branch nature of banks places a high demand on manual review, creating significant review pressure. By the time manual review discovers proxy transactions, losses may have already occurred, damaging the bank's economic interests and reputation, resulting in low regulatory effectiveness. Therefore, this invention proposes a method and system for identifying bank employees' actions on behalf of customers by utilizing deep learning models and algorithms to analyze video surveillance footage, providing strong support for bank risk management and customer rights protection. Summary of the Invention

[0004] This invention provides a method and system for identifying suspected proxy transactions by bank employees, in order to solve the problems mentioned in the background art.

[0005] A method for identifying suspected customer transaction activities by bank employees, comprising: S1: Processes real-time video data transmitted from bank branch cameras and outputs personnel location and identity information, human posture information, and location information of sensitive items; S2: By analyzing the location and identity information of personnel, human posture information and location information of sensitive items through preset judgment logic, the behavior of operating a mobile phone on behalf of a customer and signing on behalf of a customer can be identified. S3: When the system detects that a customer is operating a mobile phone or signing documents on behalf of another customer, it will trigger both on-site and remote alarms.

[0006] Preferably, in step S1, the real-time video data transmitted from the bank branch camera is processed to output personnel location and identity information, human posture information, and sensitive item location information, including: Based on a pre-trained recognition model, personnel detection, personnel tracking, and uniform recognition are performed on real-time video data, and personnel location and identity information are obtained based on the recognition results. Based on a pre-trained pose estimation model, key point coordinates are inferred and calculated from human images in real-time video data. Based on the coordinate integration results, human pose information is obtained. Based on a pre-trained sensitive object recognition model, sensitive objects are identified in real-time video data to determine their location information.

[0007] Preferably, based on a pre-trained recognition model, personnel detection, personnel tracking, and uniform identification are performed on real-time video data. Based on the recognition results, personnel location and identity information are obtained, including: Based on a pre-trained personnel detection model, the location of personnel in real-time video data is detected, and the coordinates of the personnel are determined by the minimum bounding rectangle of the personnel location. Based on the personnel tracking algorithm, the personnel coordinates are tracked in real time, and a unique index ID is assigned to the personnel coordinates of each person; Based on the index ID and a pre-trained uniform recognition model, uniform recognition is performed on the rectangular image corresponding to the personnel coordinates to determine whether the personnel are bank employees. The pre-trained face recognition model identifies the rectangular image corresponding to the coordinates of the person, matches the face recognition result with a pre-established database of bank employees' faces, and determines whether the person is a bank employee based on the matching result. Based on personnel coordinates, uniform recognition results, and facial recognition results, the personnel's location and identity information are obtained; Among them, facial recognition has a higher priority. When the face is obscured or unclear, the identity is determined by the recognition result of the work clothes.

[0008] Preferably, a pre-trained sensitive object recognition model is used to identify sensitive objects in real-time video data and determine the location information of sensitive objects in the real-time video, including: Based on publicly available internet data, we collected and labeled image data of mobile phones, paper, and pens, and trained a YOLOV8 object detection model to obtain a sensitive item recognition model. Based on the sensitive item recognition model, sensitive items are identified in real-time video data to determine the location information of sensitive items in the real-time video. Based on the sensitive item tracking algorithm, an item type index ID is assigned to the location information of sensitive items.

[0009] Preferably, in step S2, the location and identity information of the person, the posture information of the person, and the location information of sensitive items are analyzed by a preset judgment logic to identify the behavior of operating a mobile phone on behalf of another person and the behavior of signing on behalf of another person, including: Based on personnel location and identity information, human posture information, and sensitive item location information, the ownership of sensitive items for operation is determined, specifically as follows: The current person to be identified is determined from their location and identity information. The bounding box of the sensitive item is determined based on the location information of the sensitive item. The coordinates of the elbow and wrist are obtained from the human posture information, and the hand coordinates are calculated according to the following formula. ; ; in, Indicates the x-coordinate point of the hand. Represents the vertical coordinate point of the hand. Indicates the x-coordinate of the elbow point. Represents the ordinate of the elbow point. Represents the x-coordinate of the wrist point. Represents the ordinate of the wrist point. This indicates the preset extension factor 'a', which defaults to 0.4. Continuously determine whether the hand's coordinates are within the bounding box of the item to which it belongs; If so, confirm that the sensitive items belong to the currently identified personnel; Otherwise, determine that the sensitive items do not belong to the currently identified personnel; When the sensitive item is a mobile phone, a judgment will be made on the act of operating the mobile phone on behalf of the customer. When the sensitive items are paper or pen, a judgment will be made regarding the act of signing on behalf of a customer.

[0010] Preferably, when the sensitive item is a mobile phone, the judgment of the act of operating the mobile phone on behalf of another person includes: When the sensitive item is a mobile phone, the mobile phone's ID is determined to be ID0, and the operation ownership information of ID0 is continuously tracked; If the attribution information changes from employee to customer or vice versa, it is determined that a phone operation on behalf of another customer has occurred. If the attribution is determined to be both customer and employee, and the employee is identified as operating the phone near the customer, it is determined that a phone operation on behalf of another customer has occurred.

[0011] Preferably, when the sensitive items are paper or pen, the determination of proxy signature behavior includes: When the sensitive items are paper and pen, determine whether there are customers in the image; If it does not exist, no judgment will be made regarding signing on behalf of another person; If such a situation exists, a determination will be made regarding the act of signing on behalf of another person, specifically as follows: The paper is assigned ID1 and the pen is assigned ID2. The operation attribution information of ID1 and ID2 is continuously tracked. When the operation attribution of ID1 and ID2 belongs to the same staff member to be analyzed, it is determined that the staff member to be analyzed is in the signing state. Set the ID of the staff member to be analyzed as ID3, obtain the first position coordinates of the staff member when in the signing state, set the ID of the customer in the screen as ID4, and obtain the second position coordinates of the customer; The distance between the staff member and the customer to be analyzed is calculated using the following formula; ; in, This indicates the distance between the staff and the customer to be analyzed. This represents the x-coordinate of the top-left corner of the staff member to be analyzed. This represents the lower right x-coordinate of the staff member to be analyzed. This represents the top-left ordinate of the staff member to be analyzed. This represents the bottom right ordinate of the staff member to be analyzed. Represents the x-coordinate of the customer's top-left corner. Represents the x-coordinate of the customer's bottom right corner. Represents the top-left vertical coordinate of the customer. Represents the bottom right vertical coordinate of the customer; Based on the coordinate information of the staff and customers to be analyzed, the average body width of the staff and customers to be analyzed is calculated. ; in, This indicates the average body width of the staff and customers being analyzed; Determine the product of the average body width and a preset comparison coefficient, and determine whether the distance is less than the product; If so, it indicates that the staff member being analyzed was in close proximity to the customer and interacted with them; When the staff member being analyzed is in the signing state and interacts with the customer, it is determined that a signing on behalf of the customer has occurred; Otherwise, it is determined that no signing on behalf of another person occurred.

[0012] Preferably, in step S1, the processing of the real-time video data transmitted from the bank branch camera includes: The system performs preliminary identification on the real-time video data transmitted from the bank branch cameras to determine the personnel density, scene dynamics, and lighting stability. Based on the pre-set weighting of personnel density, scene dynamics, and lighting stability, the system calculates the complexity of the real-time video. The system acquires the range of human movements, the proportion of sensitive items, and the degree of human-item interaction in real-time video data. Based on the pre-set weighted weights of these parameters, it calculates the matching degree between the real-time video and suspected surrogate operation behavior. The overall recognition difficulty of real-time images is determined based on complexity and matching degree; When the overall recognition difficulty is less than the preset recognition difficulty, a lightweight model is matched for the corresponding real-time image; otherwise, a high-precision model is matched for the corresponding real-time image. Once the high-precision model is determined to match the corresponding real-time image, the risk level of the corresponding real-time image is calculated based on the matching degree between the corresponding real-time image and the suspected proxy operation behavior, combined with the dynamic degree of the corresponding real-time image. Based on the rule that the higher the risk level, the higher the task priority, the task processing order for the corresponding real-time images is determined, and further information identification is performed on the corresponding real-time images according to the task processing order.

[0013] Preferably, the comprehensive recognition difficulty of real-time images is determined based on complexity and matching degree, including: Assign a first weight to complexity and a second weight to the matching degree. Obtain the first product of complexity and first weight, and the second product of matching degree and second weight; Adding the first and second products together gives the overall recognition difficulty of the real-time image.

[0014] A system for identifying suspected customer transaction activities by bank employees includes: The deep learning computing module is used to process real-time video data transmitted from bank branch cameras and output personnel location and identity information, human posture information, and location information of sensitive items. The proxy operation analysis module is used to analyze personnel location and identity information, human posture information and sensitive item location information through preset judgment logic to identify proxy mobile phone operation behavior and proxy signature behavior. The alarm module is used to trigger on-site alarms and remote alarms when it detects that the user is operating the phone on behalf of another user or signing on behalf of another user.

[0015] Compared with the prior art, the present invention has achieved the following beneficial effects: By directly connecting to real-time footage from bank branch cameras, and utilizing deep learning models to process the image data in seconds, the system outputs key information about personnel, postures, and sensitive items in real time, with no data storage delay. Through dynamic analysis based on the real-time output information, the time from determining the ownership of sensitive items to identifying proxy behavior is controlled in milliseconds. Problems can be detected during, rather than after, employee proxy operations. An alarm mechanism is triggered synchronously with the identification results, avoiding the waiting time for manual review and providing a crucial window for on-site intervention to stop violations. Ultimately, this system utilizes deep learning image target detection, posture analysis, facial recognition, and other methods to identify bank employees' proxy operations in real time, automatically, and accurately, effectively reducing the financial risks caused by proxy operations.

[0016] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in this application.

[0017] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0018] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart illustrating a method for identifying suspected proxy transactions by bank employees, as described in an embodiment of the present invention. Figure 2 This is a flowchart illustrating the output of personnel location and identity information, human posture information, and sensitive item location information in an embodiment of the present invention. Figure 3 This is a structural diagram of a bank employee suspected of performing transactions on behalf of customers, as described in an embodiment of the present invention. Detailed Implementation

[0019] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0020] Example 1: This embodiment of the invention provides a method for identifying suspected proxy transactions by bank employees, such as... Figure 1 As shown, it includes: S1: Processes real-time video data transmitted from bank branch cameras and outputs personnel location and identity information, human posture information, and location information of sensitive items; S2: By analyzing the location and identity information of personnel, human posture information and location information of sensitive items through preset judgment logic, the behavior of operating a mobile phone on behalf of a customer and signing on behalf of a customer can be identified. S3: When the system detects that a customer is operating a mobile phone or signing documents on behalf of another customer, it will trigger both on-site and remote alarms.

[0021] In this embodiment, the real-time video data captured by the bank branch camera is authorized by employees and other personnel.

[0022] In this embodiment, the personnel location and identity information includes the personnel's coordinate information, work uniform information, and facial recognition information.

[0023] In this embodiment, sensitive items include mobile phones, pens, and paper.

[0024] In this embodiment, when the aforementioned operations and logic detect proxy operations on the monitoring screen, including proxy phone operation and proxy signatures, warnings can be issued to on-site personnel and off-site management personnel through various methods. For example: (1) IoT lighting fixtures can be placed on-site. The fixtures will light up when the monitoring screen detects valet operation. Preferably, the fixtures are natural-looking, soft-lighting, or relatively small indicator lights. They only serve to remind on-site staff and avoid disturbing customers when the lights are on. (2) When valet services are detected on the screen, the current screen image should be saved in a timely manner. The branch manager and other management personnel can be notified through WeChat official account, SMS or other channels to intervene and stop the valet services in a timely manner and reduce the financial risk of the branch.

[0025] The beneficial effects of the above design scheme are as follows: By directly connecting to the real-time images of bank branch cameras, the deep learning model processes the image data in seconds, outputting key information about personnel, postures, and sensitive items in real time without data storage delay. Through dynamic analysis based on the real-time output information, the time from determining the ownership of sensitive items to identifying the act of acting on behalf of others is controlled in milliseconds. Problems can be detected during the process of employees acting on behalf of others, rather than after it is completed. An alarm mechanism is triggered synchronously with the identification results, avoiding the waiting time for manual review and gaining a critical window for stopping violations on-site. Ultimately, it achieves real-time, automatic, and accurate identification of bank employees acting on behalf of others by using deep learning image target detection, posture analysis, facial recognition, and other methods, effectively reducing the financial risks caused by such actions.

[0026] Example 2: Based on Example 1, this embodiment of the invention provides a method for identifying suspected proxy transactions by bank employees, such as... Figure 2As shown, in step S1, the real-time video data transmitted from the bank branch camera is processed to output personnel location and identity information, human posture information, and sensitive item location information, including: Based on a pre-trained recognition model, personnel detection, personnel tracking, and uniform recognition are performed on real-time video data, and personnel location and identity information are obtained based on the recognition results. Based on a pre-trained pose estimation model, key point coordinates are inferred and calculated from human images in real-time video data. Based on the coordinate integration results, human pose information is obtained. Based on a pre-trained sensitive object recognition model, sensitive objects are identified in real-time video data to determine their location information.

[0027] In this embodiment, based on a pre-trained pose estimation model, key point coordinate inference calculations are performed on human images in real-time video data. Based on the coordinate integration results, human pose information is obtained, including: Based on a pre-trained pose estimation model, key point coordinates are inferred and calculated from human images in real-time video data to obtain key coordinate points. The key coordinate points are integrated to obtain human body key point position data, which is then used as human body posture information.

[0028] In this embodiment, key points include the elbow, wrist, etc.

[0029] The beneficial effects of the above design scheme are: by directly connecting to the real-time images of bank branch cameras, the deep learning model processes the image data in seconds and outputs key information about personnel, posture, and sensitive items in real time, with no data storage delay, providing an information foundation for further identification of suspected employee proxy operations.

[0030] Example 3: Based on Example 2, this embodiment of the invention provides a method for identifying suspected proxy transactions by bank employees. It uses a pre-trained recognition model to perform personnel detection, personnel tracking, and uniform identification on real-time video data. Based on the recognition results, it obtains the personnel's location and identity information, including: Based on a pre-trained personnel detection model, the location of personnel in real-time video data is detected, and the coordinates of the personnel are determined by the minimum bounding rectangle of the personnel location. Based on the personnel tracking algorithm, the personnel coordinates are tracked in real time, and a unique index ID is assigned to the personnel coordinates of each person; Based on the index ID and a pre-trained uniform recognition model, uniform recognition is performed on the rectangular image corresponding to the personnel coordinates to determine whether the personnel are bank employees. The pre-trained face recognition model identifies the rectangular image corresponding to the coordinates of the person, matches the face recognition result with a pre-established database of bank employees' faces, and determines whether the person is a bank employee based on the matching result. Based on personnel coordinates, uniform recognition results, and facial recognition results, the personnel's location and identity information are obtained; Among them, facial recognition has a higher priority. When the face is obscured or unclear, the identity is determined by the recognition result of the work clothes.

[0031] In this embodiment, the personnel coordinates of the minimum bounding rectangle are the coordinates of the upper left corner and the lower right corner of the minimum bounding rectangle.

[0032] In this embodiment, the pre-trained person detection model is trained based on the YOLOv8 architecture model.

[0033] In this embodiment, the pre-trained uniform recognition model is trained based on the MobileNetV4 architecture model. Specifically, data on the uniforms worn by employees in the industry during their daily work are collected in advance. Images of employees working are collected under different cameras, angles, lighting, and time points, and labeled. Combined with data on people not wearing uniforms in the public dataset, the MobileNetV4 architecture model is trained, which can distinguish people in the picture as employees and customers.

[0034] In this embodiment, the personnel tracking algorithm is designed based on the ByteTrack algorithm.

[0035] In this embodiment, face recognition has a higher priority. When the face is obscured or unclear, the result of the work clothes classification is used to make a judgment.

[0036] The beneficial effects of the above design scheme are as follows: by prioritizing identity determination with high-precision facial recognition, misjudgments caused by similar work clothes are avoided; when the face is occluded or unclear, the system automatically switches to work clothes recognition to fill the gap, solving the recognition blind spots in scenarios such as people looking down or wearing masks, significantly improving the accuracy of identity judgment; by reusing pre-trained models, the system eliminates the need for large-scale on-site data collection, reducing system deployment time and costs; and assigning unique IDs to personnel for tracking avoids identity confusion when personnel move around, providing stable identity data for subsequent customer operation analysis.

[0037] Example 4: Based on Example 2, this embodiment of the invention provides a method for identifying suspected proxy transactions by bank employees. Based on a pre-trained sensitive item recognition model, it identifies sensitive items in real-time video data and determines the location information of sensitive items in the real-time video, including: Based on publicly available internet data, we collected and labeled image data of mobile phones, paper, and pens, and trained a YOLOV8 object detection model to obtain a sensitive item recognition model. Based on the sensitive item recognition model, sensitive items are identified in real-time video data to determine the location information of sensitive items in the real-time video. Based on the sensitive item tracking algorithm, an item type index ID is assigned to the location information of sensitive items.

[0038] In this embodiment, the sensitive item tracking algorithm is designed based on the ByteTrack algorithm.

[0039] The beneficial effects of the above design scheme are: by using a pre-trained sensitive item recognition model to identify sensitive items in real-time video data, the location information of sensitive items in the real-time video can be determined, providing a basis for item information for subsequent customer operation analysis.

[0040] Example 5: Based on Example 1, this embodiment of the invention provides a method for identifying suspected proxy operations by bank employees. In step S2, the location and identity information of the personnel, human posture information, and location information of sensitive items are analyzed through preset judgment logic to identify proxy mobile phone operation and proxy signature behavior, including: Based on personnel location and identity information, human posture information, and sensitive item location information, the ownership of sensitive items for operation is determined, specifically as follows: The current person to be identified is determined from their location and identity information. The bounding box of the sensitive item is determined based on the location information of the sensitive item. The coordinates of the elbow and wrist are obtained from the human posture information, and the hand coordinates are calculated according to the following formula. ; ; in, Indicates the x-coordinate point of the hand. Represents the vertical coordinate point of the hand. Indicates the x-coordinate of the elbow point. Represents the ordinate of the elbow point. Represents the x-coordinate of the wrist point. Represents the ordinate of the wrist point. This indicates the preset extension factor 'a', which defaults to 0.4. Continuously determine whether the hand's coordinates are within the bounding box of the item to which it belongs; If so, confirm that the sensitive items belong to the currently identified personnel; Otherwise, determine that the sensitive items do not belong to the currently identified personnel; When the sensitive item is a mobile phone, a judgment will be made on the act of operating the mobile phone on behalf of the customer. When the sensitive items are paper or pen, a judgment will be made regarding the act of signing on behalf of a customer.

[0041] In this embodiment, when determining the ownership of operations on sensitive items, a consecutive hit count n can be set. That is, if the item is identified as being operated by the same person more than n times consecutively, then the operation of the sensitive item is considered to belong to that person.

[0042] The beneficial effects of the above design scheme are as follows: By calculating the hand coordinates by combining the elbow and wrist coordinates with a preset extension coefficient, compared with direct visual hand recognition, the actual operation position can be more accurately located, avoiding misjudgment of the operator due to hand occlusion and angle deviation, and providing a reliable basis for the determination of the ownership of sensitive items. By adopting the logic of continuously judging whether the hand is within the rectangle of the sensitive item, rather than making a single judgment, the interference scenario of the hand accidentally approaching the item can be filtered out, which significantly reduces the misjudgment rate of the ownership of sensitive items. By constructing the ownership judgment logic based on the previously obtained personnel identity, posture, and item position information, the data link is coherent, ensuring the consistency between the ownership judgment of sensitive items and the subsequent recognition of customer behavior, laying the foundation for the reliability of the final behavior recognition result.

[0043] Example 6: Based on Example 5, this embodiment of the invention provides a method for identifying suspected proxy operations by bank employees. When the sensitive item is a mobile phone, the method for judging proxy mobile phone operation includes: When the sensitive item is a mobile phone, the mobile phone's ID is determined to be ID0, and the operation ownership information of ID0 is continuously tracked; If the attribution information changes from employee to customer or vice versa, it is determined that a phone operation on behalf of another customer has occurred. If the attribution is determined to be both customer and employee, and the employee is identified as operating the phone near the customer, it is determined that a phone operation on behalf of another customer has occurred.

[0044] The beneficial effects of the above design scheme are as follows: By determining whether the ownership of the mobile phone changes from employee to customer or vice versa, it can directly identify typical customer service scenarios such as employees taking over the customer's phone and then returning it, or customers voluntarily handing over their phones to employees for operation. In these scenarios, the ownership of the mobile phone is clearly transferred, which is a high-risk form of customer service operation. This judgment logic can accurately identify such behaviors and avoid missed detections due to the concealed nature of the operation caused by the transfer of the phone. By addressing the situation where the ownership is simultaneously determined to be that of both customer and employee, it solves the blind spot in identifying atypical scenarios where the phone is not transferred but the employee actually intervenes in the operation. The logic can effectively cover and improve the comprehensiveness of the identification.

[0045] Example 7: Based on Example 5, this embodiment of the invention provides a method for identifying suspected proxy signing behavior by bank employees. When the sensitive items are paper and pen, the method for judging proxy signing behavior includes: When the sensitive items are paper and pen, determine whether there are customers in the image; If it does not exist, no judgment will be made regarding signing on behalf of another person; If such a situation exists, a determination will be made regarding the act of signing on behalf of another person, specifically as follows: The paper is assigned ID1 and the pen is assigned ID2. The operation attribution information of ID1 and ID2 is continuously tracked. When the operation attribution of ID1 and ID2 belongs to the same staff member to be analyzed, it is determined that the staff member to be analyzed is in the signing state. Set the ID of the staff member to be analyzed as ID3, obtain the first position coordinates of the staff member when in the signing state, set the ID of the customer in the screen as ID4, and obtain the second position coordinates of the customer; The distance between the staff member and the customer to be analyzed is calculated using the following formula; ; in, This indicates the distance between the staff and the customer to be analyzed. This represents the x-coordinate of the top-left corner of the staff member to be analyzed. This represents the lower right x-coordinate of the staff member to be analyzed. This represents the top-left ordinate of the staff member to be analyzed. This represents the bottom right ordinate of the staff member to be analyzed. Represents the x-coordinate of the customer's top-left corner. Represents the x-coordinate of the customer's bottom right corner. Represents the top-left vertical coordinate of the customer. Represents the bottom right vertical coordinate of the customer; Based on the coordinate information of the staff and customers to be analyzed, the average body width of the staff and customers to be analyzed is calculated. ; in, This indicates the average body width of the staff and customers being analyzed; Determine the product of the average body width and a preset comparison coefficient, and determine whether the distance is less than the product; If so, it indicates that the staff member being analyzed was in close proximity to the customer and interacted with them; When the staff member being analyzed is in the signing state and interacts with the customer, it is determined that a signing on behalf of the customer has occurred; Otherwise, it is determined that no signing on behalf of another person occurred.

[0046] In this embodiment, the default value of the preset comparison coefficient is 0.9, and the value range can generally be set to [0.4, 1.4]. The value of the comparison coefficient can be adjusted according to the shooting angle of the camera in the branch and the seating arrangement of the financial management room in the branch, so as to improve the accuracy of the judgment of signing on behalf of customers.

[0047] In this embodiment, the aforementioned sensitive item operation attribution judgment logic is used to determine the current attribution of paper and pen operations. The attribution information is then saved. Note that when determining the attribution of paper and pen operations, the coordinates of the skeletal points on the same side of the person must be used. That is, the operation is only ultimately attributable to that person if both the paper and pen are simultaneously identified as belonging to them by the person's left hand or right hand, thus avoiding false alarms caused by a person holding paper in one hand and a pen in the other at a considerable distance.

[0048] The beneficial effects of the above design scheme are as follows: By determining whether a customer is present on the screen beforehand, the system skips the signing-on-behalf judgment when no customer is present, avoiding redundant calculations for scenarios with no potential interaction and improving system efficiency. By requiring that paper and pen operations be performed by the same staff member to be considered a signing state, non-signing scenarios such as staff members organizing paper and pens or picking up stationery can be filtered out, avoiding misjudging normal work behaviors as suspected signing-on-behalf operations. By combining average body width rather than a fixed distance threshold to judge interaction, the system can adapt to staff and customers of different body types, avoiding misjudgments caused by fixed thresholds and enhancing scenario adaptability. By simultaneously ensuring that the staff member is in a signing state and interacts closely with the customer to determine signing-on-behalf, the system avoids false alarms caused by a single condition and improves recognition accuracy.

[0049] Example 8: Based on Example 1, this embodiment of the invention provides a method for identifying suspected proxy operations by bank employees. In step S1, the real-time video data transmitted from the bank branch's camera is processed, including: The system performs preliminary identification on the real-time video data transmitted from the bank branch cameras to determine the personnel density, scene dynamics, and lighting stability. Based on the pre-set weighting of personnel density, scene dynamics, and lighting stability, the system calculates the complexity of the real-time video. The system acquires the range of human movements, the proportion of sensitive items, and the degree of human-item interaction in real-time video data. Based on the pre-set weighted weights of these parameters, it calculates the matching degree between the real-time video and suspected surrogate operation behavior. The overall recognition difficulty of real-time images is determined based on complexity and matching degree; When the overall recognition difficulty is less than the preset recognition difficulty, a lightweight model is matched for the corresponding real-time image; otherwise, a high-precision model is matched for the corresponding real-time image. Once the high-precision model is determined to match the corresponding real-time image, the risk level of the corresponding real-time image is calculated based on the matching degree between the corresponding real-time image and the suspected proxy operation behavior, combined with the dynamic degree of the corresponding real-time image. Based on the rule that the higher the risk level, the higher the task priority, the task processing order for the corresponding real-time images is determined, and further information identification is performed on the corresponding real-time images according to the task processing order.

[0050] In this embodiment, lightweight models are, for example, YOLOV8-nano and MobileNetV4-tiny, while high-precision models are, for example, YOLOV8-x and pose estimation HRNet.

[0051] In this embodiment, the complexity of the real-time image is calculated based on the pre-set weighted weights of personnel density, scene dynamism, and lighting stability. Specifically, the personnel density, scene dynamism, and lighting stability are first standardized, and then multiplied by their respective weights and summed to obtain the complexity of the real-time image. The matching degree is calculated in the same way.

[0052] In this embodiment, the real-time video data transmitted from the bank branch camera is initially identified, and a rough identification analysis is performed based on a preliminary identification model to determine the identity.

[0053] In this embodiment, the sum of the matching degree and the dynamic degree is used as the risk level of the corresponding real-time image.

[0054] In this embodiment, scene dynamism refers to the comprehensive measurement index of the movement status, movement frequency and movement range of all elements, such as personnel, items and equipment, in the real-time screen of the bank branch. It mainly reflects the degree of change of the screen content. The specific quantitative analysis process is as follows: taking a unit time period as the cycle, the number of dynamic elements in the screen, the complexity of the movement trajectory and the movement speed are counted, and finally quantified into a value of 0-100 (0 is completely static, 100 is extreme dynamic).

[0055] In this embodiment, illumination stability refers to the continuous stability of illumination intensity, color temperature, and brightness distribution in the real-time image of the bank branch. It reflects the "reliability of lighting conditions" in the image and directly affects the AI's accuracy in recognizing people's movements and object details. The specific quantitative analysis process is as follows: the average brightness, brightness fluctuation range, and local brightness contrast of the image are monitored on a unit time basis and quantified into a value of 0-100 (0 is extremely unstable, and 100 is completely stable).

[0056] The beneficial effects of the above design scheme are as follows: Bank branches have a large number of cameras, and the complexity of the images varies greatly at different times. Traditional solutions use fixed computing power allocation, which leads to resource waste. This scheme preliminarily calculates the matching degree between the complexity of the image and the suspected proxy behavior, and configures models with different precision according to the actual situation to ensure recognition efficiency and achieve reasonable resource allocation. Based on the matching degree between the corresponding real-time image and the suspected proxy behavior, combined with the dynamic degree of the corresponding real-time image, the risk level of the corresponding real-time image is calculated. The processing order of the corresponding image is determined according to the risk level to ensure the timeliness of suspected proxy behavior and to make reasonable resource allocation for subsequent analysis of suspected proxy behavior.

[0057] Example 9: Based on Example 8, this embodiment of the invention provides a method for identifying suspected proxy operations by bank employees. The method determines the overall recognition difficulty of real-time video based on complexity and matching degree, including: Assign a first weight to complexity and a second weight to the matching degree. Obtain the first product of complexity and first weight, and the second product of matching degree and second weight; Adding the first and second products together gives the overall recognition difficulty of the real-time image.

[0058] The beneficial effects of the above design scheme are: by matching a first weight to the complexity and configuring a second weight to the matching degree; obtaining the first product of complexity and the first weight, and the second product of matching degree and the second weight; The first and second products are added together to obtain the overall recognition difficulty of the real-time image, enabling a preliminary judgment of the real-time image and providing a basis for the reasonable setting of subsequent model configuration and task sequence.

[0059] Example 10: This embodiment of the invention provides a system for identifying suspected proxy transactions by bank employees, such as... Figure 3 As shown, it includes: The deep learning computing module is used to process real-time video data transmitted from bank branch cameras and output personnel location and identity information, human posture information, and location information of sensitive items. The proxy operation analysis module is used to analyze personnel location and identity information, human posture information and sensitive item location information through preset judgment logic to identify proxy mobile phone operation behavior and proxy signature behavior. The alarm module is used to trigger on-site alarms and remote alarms when it detects that the user is operating the phone on behalf of another user or signing on behalf of another user.

[0060] In this embodiment, the real-time video data captured by the bank branch camera is authorized by employees and other personnel.

[0061] In this embodiment, the personnel location and identity information includes the personnel's coordinate information, work uniform information, and facial recognition information.

[0062] In this embodiment, sensitive items include mobile phones, pens, and paper.

[0063] In this embodiment, when the aforementioned operations and logic detect proxy operations on the monitoring screen, including proxy phone operation and proxy signatures, warnings can be issued to on-site personnel and off-site management personnel through various methods. For example: (1) IoT lighting fixtures can be placed on-site. The fixtures will light up when the monitoring screen detects valet operation. Preferably, the fixtures are natural-looking, soft-lighting, or relatively small indicator lights. They only serve to remind on-site staff and avoid disturbing customers when the lights are on. (2) When valet services are detected on the screen, the current screen image should be saved in a timely manner. The branch manager and other management personnel can be notified through WeChat official account, SMS or other channels to intervene and stop the valet services in a timely manner and reduce the financial risk of the branch.

[0064] The beneficial effects of the above design scheme are as follows: By directly connecting to the real-time images of bank branch cameras, the deep learning model processes the image data in seconds, outputting key information about personnel, postures, and sensitive items in real time without data storage delay. Through dynamic analysis based on the real-time output information, the time from determining the ownership of sensitive items to identifying the act of acting on behalf of others is controlled in milliseconds. Problems can be detected during the process of employees acting on behalf of others, rather than after it is completed. An alarm mechanism is triggered synchronously with the identification results, avoiding the waiting time for manual review and gaining a critical window for stopping violations on-site. Ultimately, it achieves real-time, automatic, and accurate identification of bank employees acting on behalf of others by using deep learning image target detection, posture analysis, facial recognition, and other methods, effectively reducing the financial risks caused by such actions.

[0065] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of this application and its equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for identifying suspected customer operation behavior of an employee in a bank, characterized by, include: S1: Processes real-time video data from bank branch cameras, outputting personnel location and identity information, human posture information, and location information of sensitive items, including: The system performs preliminary identification on the real-time video data transmitted from the bank branch cameras to determine the personnel density, scene dynamics, and lighting stability. Based on the pre-set weighting of personnel density, scene dynamics, and lighting stability, the system calculates the complexity of the real-time video. The system acquires the range of human movements, the proportion of sensitive items, and the degree of human-item interaction in real-time video data. Based on the pre-set weighted weights of these parameters, it calculates the matching degree between the real-time video and suspected surrogate operation behavior. The overall recognition difficulty of real-time images is determined based on complexity and matching degree, including: Assign a first weight to complexity and a second weight to the matching degree. Obtain the first product of complexity and first weight, and the second product of matching degree and second weight; Add the first product and the second product together to get the overall recognition difficulty of the real-time image; When the overall recognition difficulty is less than the preset recognition difficulty, a lightweight model is matched for the corresponding real-time image; otherwise, a high-precision model is matched for the corresponding real-time image. Once the high-precision model is determined to match the corresponding real-time image, the risk level of the corresponding real-time image is calculated based on the matching degree between the corresponding real-time image and the suspected proxy operation behavior, combined with the dynamic degree of the corresponding real-time image. Based on the rule that the higher the risk level, the higher the task priority, the task processing order of the corresponding real-time screen is determined, and further information identification is performed on the corresponding real-time screen according to the task processing order. S2: By analyzing the location and identity information of personnel, human posture information and location information of sensitive items through preset judgment logic, the behavior of operating a mobile phone on behalf of a customer and signing on behalf of a customer can be identified. S3: When the system detects that a customer is operating a mobile phone or signing documents on behalf of another customer, it will trigger both on-site and remote alarms.

2. The method for identifying suspected proxy transactions by bank employees according to claim 1, characterized in that, In step S1, the real-time video data transmitted from the bank branch camera is processed to output personnel location and identity information, human posture information, and location information of sensitive items, including: Based on a pre-trained recognition model, personnel detection, personnel tracking, and uniform recognition are performed on real-time video data, and personnel location and identity information are obtained based on the recognition results. Based on a pre-trained pose estimation model, key point coordinates are inferred and calculated from human images in real-time video data. Based on the coordinate integration results, human pose information is obtained. Based on a pre-trained sensitive object recognition model, sensitive objects are identified in real-time video data to determine their location information.

3. The method for identifying suspected proxy transactions by bank employees according to claim 2, characterized in that, Based on a pre-trained recognition model, personnel detection, tracking, and uniform identification are performed on real-time video data. The location and identity information of personnel are obtained based on the recognition results, including: Based on a pre-trained personnel detection model, the location of personnel in real-time video data is detected, and the coordinates of the personnel are determined by the minimum bounding rectangle of the personnel location. Based on the personnel tracking algorithm, the personnel coordinates are tracked in real time, and a unique index ID is assigned to the personnel coordinates of each person; Based on the index ID and a pre-trained uniform recognition model, uniform recognition is performed on the rectangular image corresponding to the personnel coordinates to determine whether the personnel are bank employees. The pre-trained face recognition model identifies the rectangular image corresponding to the coordinates of the person, matches the face recognition result with a pre-established database of bank staff faces, and determines whether the person is a bank employee based on the matching result. Based on personnel coordinates, uniform recognition results, and facial recognition results, the personnel's location and identity information are obtained; Among them, facial recognition has a higher priority. When the face is obscured or unclear, the identity is determined by the recognition result of the work clothes.

4. The method for identifying suspected proxy transactions by bank employees according to claim 2, characterized in that, Based on a pre-trained sensitive object recognition model, sensitive objects are identified in real-time video data to determine their location information, including: Based on publicly available internet data, we collected and labeled image data of mobile phones, paper, and pens, and trained a YOLOV8 object detection model to obtain a sensitive item recognition model. Based on the sensitive item recognition model, sensitive items are identified in real-time video data to determine the location information of sensitive items in the real-time video. Based on the sensitive item tracking algorithm, an item type index ID is assigned to the location information of sensitive items.

5. The method for identifying suspected proxy transactions by bank employees according to claim 1, characterized in that, In step S2, the location and identity information of the person, the posture information of the person, and the location information of sensitive items are analyzed through preset judgment logic to identify the behavior of operating a mobile phone on behalf of another person and the behavior of signing on behalf of another person, including: Based on personnel location and identity information, human posture information, and sensitive item location information, the ownership of sensitive items for operation is determined, specifically as follows: The current person to be identified is determined from their location and identity information. The bounding box of the sensitive item is determined based on the location information of the sensitive item. The coordinates of the elbow and wrist are obtained from the human posture information, and the hand coordinates are calculated according to the following formula. ； ； in, Indicates the x-coordinate point of the hand. Indicates the vertical coordinate point of the hand. Indicates the x-coordinate of the elbow point. Represents the ordinate of the elbow point. Indicates the x-coordinate of the wrist point. Represents the ordinate of the wrist point. This indicates the preset extension factor 'a', which defaults to 0.

4. Continuously determine whether the hand's coordinates are within the bounding box of the item to which it belongs; If so, confirm that the sensitive items belong to the currently identified personnel; Otherwise, determine that the sensitive items do not belong to the currently identified personnel; When the sensitive item is a mobile phone, a judgment will be made on the act of operating the mobile phone on behalf of the customer. When the sensitive items are paper or pen, a judgment will be made regarding the act of signing on behalf of a customer.

6. The method for identifying suspected proxy transactions by bank employees according to claim 5, characterized in that, When the sensitive item is a mobile phone, the judgment of the act of operating the mobile phone on behalf of another person is carried out, including: When the sensitive item is a mobile phone, the mobile phone's ID is determined to be ID0, and the operation ownership information of ID0 is continuously tracked; If the attribution information changes from employee to customer or vice versa, it is determined that a phone operation on behalf of another customer has occurred. If the attribution is determined to be both customer and employee, and the employee is identified as operating the phone near the customer, it is determined that a phone operation on behalf of another customer has occurred.

7. The method for identifying suspected proxy transactions by bank employees according to claim 5, characterized in that, When the sensitive items are paper and pen, the determination of whether someone signed on behalf of another person includes: When the sensitive items are paper and pen, determine whether there are customers in the image; If it does not exist, no judgment will be made regarding signing on behalf of another person; If such a situation exists, a determination will be made regarding the act of signing on behalf of another person, specifically as follows: The paper is assigned ID1 and the pen is assigned ID2. The operation attribution information of ID1 and ID2 is continuously tracked. When the operation attribution of ID1 and ID2 belongs to the same staff member to be analyzed, it is determined that the staff member to be analyzed is in the signing state. Set the ID of the staff member to be analyzed as ID3, obtain the first position coordinates of the staff member when in the signing state, set the ID of the customer in the screen as ID4, and obtain the second position coordinates of the customer; The distance between the staff member and the customer to be analyzed is calculated using the following formula; ； in, This indicates the distance between the staff and the customer to be analyzed. This represents the x-coordinate of the top-left corner of the staff member to be analyzed. This represents the lower right x-coordinate of the staff member to be analyzed. This represents the top-left ordinate of the staff member to be analyzed. This represents the bottom right ordinate of the staff member to be analyzed. Represents the x-coordinate of the customer's top-left corner. Represents the x-coordinate of the customer's bottom right corner. Represents the top-left vertical coordinate of the customer. Represents the bottom right vertical coordinate of the customer; Based on the coordinate information of the staff and customers to be analyzed, the average body width of the staff and customers to be analyzed is calculated. ； in, This indicates the average body width of the staff and customers being analyzed; Determine the product of the average body width and a preset comparison coefficient, and then determine whether the distance is less than the product; If so, it indicates that the staff member being analyzed was in close proximity to the customer and interacted with them; When the staff member being analyzed is in the signing state and interacts with the customer, it is determined that a signing on behalf of the customer has occurred; Otherwise, it is determined that no signing on behalf of another person occurred.

8. A system for identifying suspected proxy transactions by bank employees, specifically used to implement the method for identifying suspected proxy transactions by bank employees as described in claim 1, characterized in that, include: The deep learning computing module is used to process real-time video data transmitted from bank branch cameras and output personnel location and identity information, human posture information, and location information of sensitive items. The proxy operation analysis module is used to analyze personnel location and identity information, human posture information and sensitive item location information through preset judgment logic to identify proxy mobile phone operation behavior and proxy signature behavior. The alarm module is used to trigger on-site alarms and remote alarms when it detects that the user is operating the phone on behalf of another user or signing on behalf of another user.

Images