Bank cash box counting behavior compliance real-time monitoring method based on deep learning

Abstract

The invention discloses a bank cash box counting behavior compliance real-time monitoring method based on deep learning. The method comprises the steps of system deployment and parameter calibration; data acquisition: acquiring image data of the counting area based on a camera, and synchronously acquiring sound data in the counting area through a microphone; multi-module cooperative processing: inputting the image data to a preset algorithm processing layer; full-process compliance judgment: sequentially executing personnel number and dressing compliance monitoring, personnel distance compliance monitoring, cash box state monitoring, hand cash holding state monitoring, cash counting machine use state monitoring and cash box closing state monitoring, and entering the next link when each monitoring step meets a preset compliance condition; and if any link does not meet the preset compliance condition, the decision judgment and alarm layer is triggered to give an alarm. Automatic, standardized and real-time monitoring of the whole bank note box counting process is achieved, and the safety and efficiency of bank operation are effectively improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of behavior monitoring, in particular to a bank note case counting behavior compliance real-time monitoring method based on deep learning. BACKGROUND

[0002] Bank note case counting is a key link in the daily operation of banks, involving a large number of cash handover and accounting, and the compliance of its operation is directly related to the safety of bank funds, the operation order and the trust of customers. In the process of counting the bank note case, the established operation specification needs to be strictly followed, including the number of operating personnel, dressing requirements, operation process sequence, equipment use specification, etc., so as to prevent the occurrence of risks such as cash omission, misoperation, fraud, etc.

[0003] Currently, the existing technologies for monitoring the compliance of bank cash box counting mainly include the following solutions: Manual supervision: This involves assigning professional supervisors to conduct real-time monitoring at the cash box counting site, or manually reviewing surveillance footage frame by frame afterward, checking each step of the staff's operation according to established operating procedures to determine compliance. This solution has significant limitations: on-site supervision is constrained by labor costs and working hours, making it difficult to achieve full-time coverage; post-event review requires significant time and effort, resulting in low efficiency, and prolonged manual observation can lead to visual fatigue and potential missed detections. Furthermore, human judgment standards are subjective; different supervisors may have different compliance assessments of the same operation, affecting the accuracy and consistency of the supervision results. Limitations of traditional image algorithms: Early bank cash box counting monitoring often relied on traditional image algorithms, such as template matching algorithms. These algorithms traverse and search images using pre-defined cash box shape templates to determine the presence and status of cash boxes. However, these algorithms are poorly adaptable to changes in lighting and viewing angle, easily leading to misjudgments; while Hough transform can detect geometric shapes in images (such as the rectangular outline of a cash box), its computational complexity is high and its real-time performance is insufficient. Furthermore, image analysis methods based on threshold segmentation can only simply distinguish between the cash box and the background, failing to identify crucial information such as the serial number on the cash box and details of personnel operations. Simple technical support solutions: Some banks have introduced basic video surveillance systems and single-function algorithms, such as using ordinary personnel detection algorithms to count the number of people on site. However, these algorithms cannot be comprehensively judged in conjunction with the specific business scenario of cash box counting; or they use simple object state detection algorithms, which can only determine whether the cash box is open in isolation, without relating it to the preceding and following operational processes. Such solutions only provide fragmented information and cannot meet the actual needs of cash box counting compliance monitoring. For example, even if the correct number of people on site is counted, it cannot determine whether personnel are active in the designated area or whether they have performed standardized operations; simply detecting that the cash box is open does not reveal whether the timing of the opening conforms to the procedure or whether the correct counting action was performed after opening, making it difficult to achieve effective supervision of the entire cash box counting process. System monitoring is mostly concentrated on discrete single-point verification links (such as quantity statistics verification), lacking coherent modeling and analysis of the entire lifecycle of the counting operation. The lack of a real-time event-driven anomaly detection mechanism results in delayed responses to violations, failing to form an effective risk prevention and control closed loop. Summary of the Invention

[0004] This invention aims to at least partially solve one of the technical problems in the aforementioned technologies. Therefore, the purpose of this invention is to propose a real-time monitoring method for the compliance of bank cash box counting activities based on deep learning. This method addresses the problems of low efficiency, high cost, and strong subjectivity in manual supervision, as well as the limitations of existing technical assistance solutions, such as limited functionality, lack of process correlation, and insufficient algorithm accuracy. The goal is to achieve automated, standardized, and real-time monitoring of the entire bank cash box counting process, effectively improving the security and efficiency of bank operations.

[0005] To achieve the above objectives, this invention proposes a real-time monitoring method for the compliance of bank cash box counting activities based on deep learning, comprising: System Deployment and Parameter Calibration: Cameras and microphones are deployed in the bank's cash box counting area. The intrinsic parameter data of the cameras is obtained through Zhang Zhengyou's calibration method. A world coordinate system is constructed with one corner of the cash box placement table as the origin of the world coordinate system, the long side of the table as the X-axis, the short side as the Y-axis, and the vertical upward of the table as the Z-axis. Images are captured by feature points at known positions in the world coordinate system, and the camera rotation matrix and translation vector are calculated to obtain extrinsic parameter data. Data acquisition: Image data of the inventory area is collected using a camera, and sound data of the inventory area is collected simultaneously using a microphone; Multi-module collaborative processing: Image data is input to a pre-defined algorithm processing layer. This layer includes a target detection module, a coordinate transformation module, a pose estimation module, a banknote counter operating status detection module, and a classification module. Specifically, the target detection module uses an optimized YOLOv8s algorithm to detect human targets, cash boxes, the banknote counter, and cash in the image data; the coordinate transformation module converts human pixel coordinates to world coordinates based on the camera's intrinsic and extrinsic parameters; the pose estimation module uses an optimized RTMPose algorithm to detect key points on the upper part of the human body and hands; the banknote counter operating status detection module combines image recognition and sound data analysis to determine the banknote counter's operating status; and the classification module uses a lightweight deep learning algorithm to determine the person's clothing and the opening / closing status of the cash box. Full-process compliance assessment: In accordance with the standard procedures for counting bank cash boxes, the following are performed in sequence: personnel number and attire compliance monitoring, personnel distance compliance monitoring, cash box status monitoring, cash-holding status monitoring, cash counting machine usage status monitoring, and cash box closed status monitoring. Each monitoring step must meet the preset compliance conditions before proceeding to the next step. If any step fails to meet the preset compliance conditions, the decision-making and alarm layer will issue an alarm.

[0006] According to some embodiments of the present invention, the optimized YOLOv8s algorithm includes an adaptive scale allocation mechanism; feature extraction is performed on image data based on the optimized YOLOv8s algorithm to obtain several feature maps; the several feature maps are divided according to a preset scale to obtain a first-scale feature map, a second-scale feature map, and a third-scale feature map; wherein, the first-scale feature map is only used for anchor point matching with human targets; the second-scale feature map is only used for anchor point matching with cash boxes and banknote counters, and deformable convolution is introduced in the cash box detection branch; the third-scale feature map is only used for anchor point matching with cash targets, and a CBAM attention mechanism is introduced in the small-scale feature extraction layer.

[0007] According to some embodiments of the present invention, converting human pixel coordinates into world coordinates based on intrinsic and extrinsic data from a camera includes: ; in, and These are the focal lengths of the camera along the x-axis and y-axis, respectively. and R represents the principal point coordinates, which are the pixel coordinates of the image data center; R is the rotation matrix in the extrinsic parameter data. (u,v) is the translation vector; (u,v) are the pixel coordinates of the human body. () represents world coordinates; It is the transpose of the zero vector.

[0008] According to some embodiments of the present invention, the optimized RTMPose algorithm includes: in the feature extraction network, the input feature map size is 256*192, and the number of output channels is adjusted to 32; for the detection of the upper part of the human body, the output layer only predicts the key points of the upper limbs, nose and ears, and for the detection of key points of the hand, only the key points of the palm are predicted.

[0009] According to some embodiments of the present invention, the banknote counter operating status detection module combines image recognition and sound data analysis to determine the operating status of the banknote counter, including: Based on the location of the banknote counter determined by the target detection module, the banknote counter area in the image data is cropped; A color recognition algorithm is used to detect the on / off status of the indicator lights on the banknote counter within the banknote counter area; The DBnet text detection algorithm is used to locate the digital area of ​​the banknote counter screen in the banknote counter area. After grayscale, noise reduction and binarization preprocessing of the digital area, the CRNN text recognition algorithm is used to identify the numbers on the screen to determine whether the number of banknotes is displayed. The sound data is converted into a frequency domain signal using Fourier transform, and the presence of noise frequency components from the banknote counting machine is analyzed in the frequency domain signal. If the indicator light is on, the number of banknotes counted is detected, and noise frequency components are detected, the banknote counter is determined to be running; otherwise, it is determined to be not running.

[0010] According to some embodiments of the present invention, the monitoring of personnel numbers and dress code compliance includes: Based on the target detection module, the number of people in the image data is counted. If the number of people is not equal to 2, the decision-making and alarm layer will issue an alarm for "non-compliant number of people". If the number of personnel is equal to 2, the image regions of the two personnel are input into the clothing classification model included in the classification module. If either personnel is not identified as wearing compliant work clothes, an "non-compliant clothing" alarm is issued; otherwise, the personnel distance compliance monitoring stage is entered.

[0011] According to some embodiments of the present invention, personnel distance compliance monitoring includes: The coordinate transformation module converts the human pixel coordinates of the two people into world coordinates, resulting in first world coordinates and second world coordinates. The actual distance between the two people is calculated based on the Euclidean distance formula using first-world and second-world coordinates. The actual distance is then compared with a preset personnel safety distance threshold. If the actual distance is greater than the preset personnel safety distance threshold, an alarm for "personnel distance non-compliance" is issued; otherwise, the cash box status monitoring stage is initiated.

[0012] According to some embodiments of the present invention, cash box status monitoring includes: The image of the cash box detected by the target detection module is input into the cash box opening and closing classification model included in the classification module to determine the current state of the cash box; If the cash box is closed, a timer will start counting. If the timer exceeds the first preset threshold and the cash box is still not detected to be open, an alarm will be issued that the cash box has not been opened on time. If the cash box is detected to be open, the timer stops and the system proceeds to monitor the cash-holding status in the hands.

[0013] According to some embodiments of the present invention, hand-held cash status monitoring includes: Obtain the coordinates of key points output by the pose estimation module, and filter out people with invalid poses based on the relative positional relationship of the coordinates of key points of the upper limbs, nose and ears. For personnel in a valid pose, obtain the coordinates of key points on the palm and the coordinates of the cash area detected by the target detection module, and calculate the ratio of the overlapping area of ​​the two to the area of ​​the cash area. If the proportion is greater than or equal to the proportion threshold, it is determined that the person is holding cash, and the cash counting machine usage status monitoring stage is entered; if the proportion is less than the preset proportion threshold and the duration is greater than the second preset threshold, an alarm "No cash detected" is issued.

[0014] According to some embodiments of the present invention, the monitoring of the usage status of a banknote counter includes: After determining that the person is holding cash, the banknote counter operation status detection module is activated to determine the status of the banknote counter; Start the timer. If the timer exceeds the third preset threshold and the money counter is still not detected, issue a "money counter not in use" alarm. If the banknote counter is detected to be running, the system will proceed to the banknote box closed status monitoring stage.

[0015] This invention proposes a real-time monitoring method for compliance of bank cash box counting activities based on deep learning. Based on Zhang Zhengyou's calibration method and the construction of a world coordinate system, it enables more accurate calculation of personnel distance and target location judgment, avoiding false alarms caused by inaccurate spatial mapping. Multi-module collaboration improves processing efficiency, solving the problems of single function and poor real-time performance. The cash counter status detection improves accuracy through multi-modal fusion of vision and hearing. Following the standard bank cash box counting process, it sequentially performs monitoring of personnel number and attire compliance, personnel distance compliance, cash box status, cash-holding status, cash counter usage status, and cash box closure status, achieving full-process compliance control covering risk points and solving the problems of incomplete and loophole-ridden monitoring. This achieves automated, standardized, and real-time monitoring of the entire bank cash box counting process, effectively improving the security and efficiency of bank operations.

[0016] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description and the accompanying drawings.

[0017] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0018] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of a method for real-time monitoring of compliance of bank cash box counting behavior based on deep learning according to an embodiment of the present invention; Figure 2 This is a flowchart for determining the operating status of a banknote counter according to an embodiment of the present invention; Figure 3 This is a flowchart of a method for monitoring the number of personnel and their dress code compliance according to an embodiment of the present invention. Detailed Implementation

[0019] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0020] like Figure 1 As shown, this embodiment of the invention proposes a real-time monitoring method for the compliance of bank cash box counting behavior based on deep learning, including steps S1-S4: S1. System Deployment and Parameter Calibration: Deploy cameras and microphones in the bank's cash box counting area. Obtain the intrinsic parameter data of the cameras using the Zhang Zhengyou calibration method. Construct a world coordinate system with one corner of the cash box placement table as the origin, the long side of the table as the X-axis, the short side as the Y-axis, and the vertical upward direction of the table as the Z-axis. Capture images using feature points at known locations in the world coordinate system, and calculate the camera rotation matrix and translation vector to obtain extrinsic parameter data. S2. Data Acquisition: Image data of the inventory area is acquired based on the camera, and sound data of the inventory area is acquired simultaneously through the microphone; S3. Multi-module collaborative processing: Image data is input to a preset algorithm processing layer. The algorithm processing layer includes a target detection module, a coordinate transformation module, a pose estimation module, a banknote counter operating status detection module, and a classification module. Among them, the target detection module uses an optimized YOLOv8s algorithm to detect human targets, cash boxes, banknote counters, and cash targets in the image data; the coordinate transformation module converts human pixel coordinates into world coordinates based on the camera's intrinsic and extrinsic parameter data; the pose estimation module uses an optimized RTMPose algorithm to detect key points on the upper part of the human body and hands; the banknote counter operating status detection module combines image recognition and sound data analysis to determine the operating status of the banknote counter; the classification module uses a lightweight deep learning algorithm to determine the clothing of the personnel and the opening and closing status of the cash box. S4. Full-process compliance assessment: In accordance with the standard procedures for counting bank cash boxes, the following are performed in sequence: personnel number and attire compliance monitoring, personnel distance compliance monitoring, cash box status monitoring, cash-holding status monitoring, cash counting machine usage status monitoring, and cash box closed status monitoring. Each monitoring step must meet the preset compliance conditions before proceeding to the next step. If any step fails to meet the preset compliance conditions, the decision-making and alarm layer will issue an alarm.

[0021] The working principle of the above technical solution is as follows: Calibration is performed using the Zhang Zhengyou calibration method. A checkerboard calibration board with known and precise black and white square dimensions is prepared. Images of the checkerboard are taken from different positions and angles, typically 10-20 images. Using relevant functions from computer vision libraries such as OpenCV, the corner points of the checkerboard in the images are detected. Based on the pixel coordinates and actual physical coordinates of the corner points, the camera intrinsic parameter matrix is ​​solved using optimization algorithms such as the least squares method. This matrix includes parameters such as the camera's focal length on the x and y axes and the coordinates of the principal point at the image center. A world coordinate system is constructed with one corner of the cash box placement table as the origin, the long side of the table as the X-axis, the short side as the Y-axis, and the vertical direction upwards from the table as the Z-axis. By setting feature points at known locations in the world coordinate system, images of these feature points are taken. Based on the correspondence between the feature points in the world coordinate system and the image coordinate system, the camera's rotation matrix and translation vector are calculated, thereby determining the camera's position and orientation in the world coordinate system. Image data of the counting area is acquired using the camera, and sound data of the counting area is simultaneously acquired using a microphone. Image data is input into a pre-defined algorithm processing layer for multi-module collaborative processing. The target detection module accurately detects corresponding targets using feature maps at different scales, outputting pixel coordinates and category confidence scores for human targets, cash boxes, cash counting machines, and cash targets. This provides input to the coordinate transformation module (providing human coordinates) and the classification module (providing image regions of cash boxes / personnel). The coordinate transformation module converts pixel coordinates to physical coordinates in the world coordinate system, facilitating the determination of the actual distance between two individuals during compliance monitoring of personnel distance. The pose estimation module detects key points on the upper body and hands of the human body based on an optimized RTMPose algorithm. The cash counting machine operation status detection module combines image recognition and sound data analysis to determine the cash counting machine's operation status, using multimodal fusion to determine whether the machine is running. The classification module uses a lightweight deep learning algorithm—MobileNet-V4—to determine the clothing of personnel and the open / closed state of the cash box. Following the standard procedures for bank cash box counting, the following steps are performed sequentially: monitoring of personnel number and attire compliance, personnel distance compliance, cash box status, hand holding cash status, cash counting machine usage status, and cash box closed status. Each monitoring step must meet preset compliance conditions before proceeding to the next step. If any step fails to meet the preset compliance conditions, an alarm is triggered at the decision-making and alarm layer. Preset compliance conditions include: two personnel, compliant attire, actual distance less than a preset personnel safety distance threshold, cash box open, overlap ratio between key hand points and the cash area of ​​the target detection module greater than a preset ratio threshold, timely use of the cash counting machine, and cash box closed.

[0022] The beneficial effects of the above technical solution are as follows: Based on Zhang Zhengyou's calibration method and the construction of a world coordinate system, it enables more accurate calculation of personnel distance and judgment of target location, avoiding false alarms caused by inaccurate spatial mapping. Multi-module collaboration improves processing efficiency, solving the problems of single function and poor real-time performance. The banknote counter status detection improves accuracy through multi-modal fusion of vision and hearing. Following the standard bank cash box counting process, it sequentially performs monitoring of personnel quantity and attire compliance, personnel distance compliance, cash box status, cash-holding status, banknote counter usage status, and cash box closure status, achieving full-process compliance control covering risk points and solving the problems of incomplete and loophole-ridden monitoring. It achieves automated, standardized, and real-time monitoring of the entire bank cash box counting process, effectively improving the security and efficiency of bank operations.

[0023] According to some embodiments of the present invention, the optimized YOLOv8s algorithm includes an adaptive scale allocation mechanism; feature extraction is performed on image data based on the optimized YOLOv8s algorithm to obtain several feature maps; the several feature maps are divided according to a preset scale to obtain a first-scale feature map, a second-scale feature map, and a third-scale feature map; wherein, the first-scale feature map is only used for anchor point matching with human targets; the second-scale feature map is only used for anchor point matching with cash boxes and banknote counters, and deformable convolution is introduced in the cash box detection branch; the third-scale feature map is only used for anchor point matching with cash targets, and a CBAM attention mechanism is introduced in the small-scale feature extraction layer.

[0024] The working principle of the above technical solution is as follows: the first scale, second scale, and third scale decrease sequentially. The first-scale feature map comes from a deeper layer of the network and is only used for anchor point matching with human targets, effectively isolating interference from medium and small targets. The second-scale feature map comes from a middle layer of the network and is only used for anchor point matching with cash boxes and money counting machines. Deformable convolution is introduced in the cash box detection branch to improve recall. Deformable convolution in the cash box detection branch: the appearance of a cash box changes significantly when it is open or closed (it can undergo deformations such as rotation and stretching). Introducing deformable convolution in the cash box detection branch corresponding to the second-scale feature map allows the network to dynamically adjust the sampling position of the convolution kernel when extracting features, better adapting to various deformations of the cash box, thereby improving the detection accuracy of cash boxes in different states. The third-scale feature map comes from a shallower layer of the network and is only matched with cash. Cash is a small target. To improve recall and accuracy, the CBAM attention mechanism is introduced on the basis of the small-scale feature map to enhance the network's sensitivity to subtle features of cash and improve the model's feature extraction performance for small targets. Introducing the Convolutional Block Attention Module (CBAM) into the cash detection branch corresponding to the third-scale feature map allows the network to automatically learn to focus on key features of the cash region (such as texture and edges) while suppressing interference from irrelevant background information, thereby significantly improving the detection rate of small-scale cash targets. Based on an adaptive scale matching mechanism, feature maps of different scales are only responsible for predicting targets of specific scales, which facilitates the improvement of image recognition efficiency and accuracy.

[0025] The beneficial effects of the above technical solutions are as follows: By forcing scale allocation, it avoids large-scale feature maps learning trivial details of small targets, and also avoids small-scale feature maps learning the overall semantics of large targets, resulting in higher learning efficiency of network parameters and more accurate feature representation of each target category. Introducing deformable convolutions to cash boxes solves the detection problem caused by shape changes; introducing the CBAM attention mechanism to cash solves the problem of missed detections caused by its small size and weak features, facilitating improved detection accuracy and recall. Since each scale's feature map is only responsible for a specific range of targets, it facilitates improved model stability and robustness.

[0026] According to some embodiments of the present invention, converting human pixel coordinates into world coordinates based on intrinsic and extrinsic data from a camera includes: ; in, and These are the focal lengths of the camera along the x-axis and y-axis, respectively. and R represents the principal point coordinates, which are the pixel coordinates of the image data center; R is the rotation matrix in the extrinsic parameter data. (u,v) is the translation vector; (u,v) are the pixel coordinates of the human body. () represents world coordinates; It is the transpose of the zero vector.

[0027] The working principle of the above technical solution: These are pixel homogeneous coordinates; This is the intrinsic parameter matrix used to implement camera-to-pixel conversion; This is an external parameter matrix used to convert between the world and the camera. The coordinates are world homogeneous coordinates; the rotation matrix describes the camera's attitude angle in the world coordinate system; the translation vector describes the camera's absolute position in the world coordinate system. The transpose of the zero vector helps maintain the legality of the matrix dimensions.

[0028] The beneficial effects of the above technical solution are: converting human pixel coordinates into world coordinates based on the intrinsic and extrinsic data of the camera, which facilitates the accurate determination of the human body's position parameters and thus facilitates distance calculation.

[0029] According to some embodiments of the present invention, the optimized RTMPose algorithm includes: in the feature extraction network, the input feature map size is 256*192, and the number of output channels is adjusted to 32; for the detection of the upper part of the human body, the output layer only predicts the key points of the upper limbs, nose and ears, and for the detection of key points of the hand, only the key points of the palm are predicted.

[0030] The working principle of the above technical solution is as follows: The input feature map size is 256*192, which is reduced in size. The number of output channels is adjusted to 32, which is achieved by pruning the output channels. The key points of the output layer are optimized by subtraction. The key points of the human upper limbs, nose and ears are used for posture validity judgment. Upper limb joints (6): left shoulder, elbow and wrist, right shoulder, elbow and wrist, are used to capture the complete motion trajectory of "reaching out to take cash → handing it to the money counting machine" and judge whether the arm movement conforms to the standard process; head and face key points (3): nose (1), left ear (1), right ear (2), invalid postures are filtered by the visibility and relative position of the key points (for example, when the person is facing away from the camera, the nose key point is not visible, or the angle between the nose and the shoulder is >120, which can be directly judged as invalid posture without subsequent hand detection). Hand key point: 1 (palm), used for cash holding judgment.

[0031] The beneficial effects of the above technical solution are: it facilitates the reduction of computational load, improves the model inference speed, and the model parameters are no longer scattered for whole-body feature learning, but are concentrated on the upper human body and palm, achieving accurate filtering of invalid poses and improving detection accuracy.

[0032] likeFigure 2 As shown, according to some embodiments of the present invention, the banknote counter operating status detection module combines image recognition and sound data analysis to determine the operating status of the banknote counter, including steps S11-S15: S11. Based on the location of the banknote counter determined by the target detection module, crop the banknote counter area in the image data; S12. Use a color recognition algorithm to detect the on / off status of the indicator lights in the banknote counter area; S13. The DBnet text detection algorithm is used to locate the digital area of ​​the banknote counter screen in the banknote counter area. After grayscale, noise reduction and binarization preprocessing of the digital area, the CRNN text recognition algorithm is used to identify the screen numbers to determine whether the number of banknotes is displayed. S14. Convert the sound data into a frequency domain signal using Fourier transform, and analyze whether there are noise frequency components from the operation of the banknote counting machine in the frequency domain signal. S15. If the indicator light is on, the number of banknotes counted is recognized, and noise frequency components are detected, the banknote counter is determined to be in operation; otherwise, it is determined to be not in operation.

[0033] The working principle of the above technical solution is as follows: Based on the location of the banknote counter determined by the target detection module, the banknote counter area in the image data is cropped; the standard color parameters of the banknote counter's operating status indicator light are determined using the HSV color space threshold segmentation method. The color parameters of the banknote counter indicator light within the banknote counter area are determined, and compared with the standard color parameters. Based on the comparison result, the on / off state of the indicator light is determined. DBnet (Differentiable Binarization Network) can accurately segment the digital display area on the banknote counter screen. After grayscale conversion, noise reduction, and binarization preprocessing of the digital area, the CRNN text recognition algorithm is used to identify the screen numbers to determine whether the banknote quantity is displayed; the sound data is converted into a frequency domain signal through Fourier transform, and the presence of noise frequency components indicating banknote counter operation is analyzed. For example, the energy proportion of the "500-2000Hz band" in the frequency domain signal is calculated. If the proportion is ≥30%, it is determined that "noise frequency components indicating banknote counter operation have been detected." If the indicator light is on, the banknote quantity is identified, and noise frequency components are detected, the banknote counter is determined to be in operation; otherwise, it is determined to be not in operation.

[0034] The beneficial effects of the above technical solution are as follows: Triple verification significantly reduces the false positive rate, ensuring the accuracy of compliance monitoring. The use of the HSV color space ensures accurate on / off detection even if changes in lighting cause slight color shifts in the indicator lights; cropping the banknote counter area avoids the influence of background elements on indicator light detection; and filtering by characteristic frequency range eliminates low-frequency conversations and high-frequency air conditioning noise, capturing only the unique noise of the banknote counter, making it suitable for the complex environment of banks.

[0035] likeFigure 3 As shown, according to some embodiments of the present invention, the monitoring of personnel numbers and dress code compliance includes steps S21-S22: S21. Based on the target detection module, count the number of people in the image data. If the number of people is not equal to 2, trigger the decision-making and alarm layer to issue an alarm for "non-compliant number of people". S22. If the number of personnel is equal to 2, the image areas of the two personnel are input into the clothing classification model included in the classification module. If either personnel is not identified as wearing compliant work clothes, an "non-compliant clothing" alarm is issued; otherwise, the personnel distance compliance monitoring stage is entered.

[0036] The working principle and beneficial effects of the above technical solution are as follows: Based on the target detection module, the number of people in the statistical image data is realized to supervise the operation of two people. The work clothes of the two people are identified based on the clothing classification model, which helps to eliminate the subjectivity of clothing judgment, ensure the accuracy of identity verification, and thus improve the accuracy of personnel number and clothing compliance monitoring.

[0037] According to some embodiments of the present invention, personnel distance compliance monitoring includes: The coordinate transformation module converts the human pixel coordinates of the two people into world coordinates, resulting in first world coordinates and second world coordinates. The actual distance between the two people is calculated based on the Euclidean distance formula using first-world and second-world coordinates. The actual distance is then compared with a preset personnel safety distance threshold. If the actual distance is greater than the preset personnel safety distance threshold, an alarm for "personnel distance non-compliance" is issued; otherwise, the cash box status monitoring stage is initiated.

[0038] The working principle and beneficial effects of the above technical solution are as follows: The positions of two individuals in real space are obtained, namely, first-world coordinates and second-world coordinates. Based on these coordinates, the actual distance between the two individuals is calculated using the Euclidean distance formula. The actual distance is compared with a preset safe distance threshold. If the actual distance is greater than the preset safe distance threshold, an alarm for "personnel distance non-compliance" is issued; otherwise, the cash box status monitoring stage is initiated. This facilitates accurate calculation of the actual distance between individuals, improves the accuracy of the comparison between the actual distance and the preset safe distance threshold, and thus enhances the monitoring of personnel distance compliance.

[0039] According to some embodiments of the present invention, cash box status monitoring includes: The image of the cash box detected by the target detection module is input into the cash box opening and closing classification model included in the classification module to determine the current state of the cash box; If the cash box is closed, a timer will start counting. If the timer exceeds the first preset threshold and the cash box is still not detected to be open, an alarm will be issued that the cash box has not been opened on time. If the cash box is detected to be open, the timer stops and the system proceeds to monitor the cash-holding status in the hands.

[0040] The working principle and beneficial effects of the above technical solution are as follows: Based on inputting the cash box image detected by the target detection module into the cash box opening / closing classification model included in the classification module, the current state of the cash box is determined; if the timeout exceeds the first preset threshold and the cash box is still not detected to be open, an alarm of "cash box not opened on time" is issued; if the cash box is detected to be open, the timeout is stopped and the hand holding cash status monitoring stage is entered. This facilitates the improvement of the accuracy of cash box status monitoring.

[0041] According to some embodiments of the present invention, hand-held cash status monitoring includes: Obtain the coordinates of key points output by the pose estimation module, and filter out people with invalid poses based on the relative positional relationship of the coordinates of key points of the upper limbs, nose and ears. For personnel in a valid pose, obtain the coordinates of key points on the palm and the coordinates of the cash area detected by the target detection module, and calculate the ratio of the overlapping area of ​​the two to the area of ​​the cash area. If the proportion is greater than or equal to the proportion threshold, it is determined that the person is holding cash, and the cash counting machine usage status monitoring stage is entered; if the proportion is less than the preset proportion threshold and the duration is greater than the second preset threshold, an alarm "No cash detected" is issued.

[0042] The working principle of the above technical solution is as follows: Obtain the coordinates of key points output by the pose estimation module; based on the relative positional relationship of the coordinates of key points on the upper limbs, nose, and ears, filter out invalid poses; set filtering rules based on the relative positional relationship of key points, retaining only valid poses facing the camera without obvious obstruction. Specific rules include: Filtering when facing away from the camera: Calculate the angle between the nose and the line connecting the two shoulders; if the nose is behind the line connecting the two shoulders (e.g., the midpoint of the line (x...)...) m ,y m ), the x-coordinate of the nose < x m If the included angle is greater than 120°, it is judged as "facing away from the camera," and hand movements cannot be observed, thus classifying it as an invalid posture. Severe occlusion filtering: If ≥3 key points of the upper limb (shoulder, elbow, wrist) are missing (e.g., if the arm is completely obscured by the cash box, only 1 elbow joint point can be detected), or the confidence score of the palm key point is <0.7 (key point confidence score output by the posture estimation module), it is judged as "occlusion invalid," and the palm position cannot be located. From the key points of the person in the valid posture, the pixel coordinates of the palm are extracted, and a rectangle of a preset size is generated with the palm as the center. The ratio of the overlapping area of ​​the two to the cash area is calculated; if the ratio is greater than or equal to the ratio threshold, it is determined that the person is holding cash, and the cash counting machine usage status monitoring stage is entered; if the ratio is less than the preset ratio threshold and the duration is greater than the second preset threshold, an alarm of "no cash being detected" is issued.

[0043] The beneficial effects of the above technical solution are: early exclusion of people with their backs to the camera or those whose hands are obstructed, filtering of invalid postures, improving the accuracy of handheld judgment, reducing false and missed judgments, and improving the efficiency and accuracy of monitoring the state of holding cash in the hand.

[0044] According to some embodiments of the present invention, the monitoring of the usage status of a banknote counter includes: After determining that the person is holding cash, the banknote counter operation status detection module is activated to determine the status of the banknote counter; Start the timer. If the timer exceeds the third preset threshold and the money counter is still not detected, issue a "money counter not in use" alarm. If the banknote counter is detected to be running, the system will proceed to the banknote box closed status monitoring stage.

[0045] The working principle and beneficial effects of the above technical solution are as follows: After determining that a person is holding cash, the banknote counter operation status detection module is activated to determine the status of the banknote counter; a timer is started to keep track of the time. If the timer exceeds the third preset threshold and the banknote counter is still not detected to be running, an alarm for "banknote counter not in use" is issued; if the banknote counter is detected to be running, the cash box closed status monitoring stage is entered, which helps to improve the accuracy of banknote counter usage status monitoring.

[0046] In one embodiment, the cash box closure status monitoring module detects the cash box's status after it is determined that a staff member is using a cash counter to count money. If the cash box is not closed within a specified time, an alarm is issued; if the cash box is closed, the entire cash box counting process is monitored and deemed a compliant operation.

[0047] The beneficial effects of this invention are as follows: Real-time and efficient monitoring: By collecting data in real time through multiple cameras, combined with the high-performance YOLOv8s algorithm and optimized positive and negative sample matching logic, every operational step during the cash box counting process can be monitored instantly. Once a violation is detected, an alarm is issued immediately, greatly improving the timeliness of monitoring and effectively avoiding potential risks caused by violations. Comprehensive and accurate coverage: It covers compliance monitoring of the entire process of cash box counting, from personnel configuration and dress requirements to cash box operation, cash handling, and the use of cash counting machines. Each link is interconnected and progressively enhanced, avoiding monitoring loopholes. The optimized YOLOv8s algorithm allows feature maps of different scales to be responsible for the prediction of targets of corresponding sizes, maintaining high detection accuracy even in complex environments and ensuring the accuracy of monitoring results. Reduced operating costs: It eliminates the need for extensive manual supervision, reducing labor costs and improving work efficiency. This allows banks to allocate human resources more rationally to other core businesses, improving overall operational efficiency. Standardized and Unified Mechanisms: Judgments are based on pre-defined algorithms and rules, eliminating the subjectivity of manual supervision and ensuring the consistency and impartiality of cash box counting compliance monitoring standards. This is conducive to the standardization of internal bank management and risk control. Strong Data Traceability: The system automatically records all data and results during the monitoring process, including images, detection results, and alarm information. This provides complete data support for subsequent auditing, querying, and analysis, facilitating banks' retrospective review and experience summarization of cash box counting operations.

[0048] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A deep learning based real-time monitoring method for banknote counting behavior compliance of a banknote cassette, characterized in that, The application relates to a banknote counting area monitoring system and method. System deployment and parameter calibration: a camera and a microphone are arranged in a banknote counting area, internal parameter data of the camera is obtained through Zhang Zhengyou calibration method, a world coordinate system is constructed with a corner of a banknote placing table top as an origin, a long side of the table top as an X-axis, a short side as a Y-axis and a vertical direction of the table top as a Z-axis, a feature point at a known position in the world coordinate system is imaged, a camera rotation matrix and a translation vector are calculated to obtain external parameter data; Data acquisition: image data of the counting area is collected based on the camera, and sound data in the counting area is synchronously collected through the microphone; Multi-module collaborative processing: the image data is input into a preset algorithm processing layer; the algorithm processing layer comprises a target detection module, a coordinate conversion module, a pose estimation module, a point counting machine running state detection module and a classification module; wherein the target detection module detects human targets, banknote boxes, point counting machines and cash targets in the image data based on an optimized YOLOv8s algorithm; the coordinate conversion module converts human pixel coordinates into world coordinates based on the internal parameter data and the external parameter data of the camera; the pose estimation module detects upper body and hand key points of a human body based on an optimized RTMPose algorithm; the point counting machine running state detection module judges the running state of the point counting machine by combining image recognition and sound data analysis; and the classification module judges personnel clothing and banknote box opening and closing states based on a lightweight deep learning algorithm; Full-process compliance judgment: according to a standard process of banknote counting, personnel quantity and clothing compliance monitoring, personnel distance compliance monitoring, banknote box state monitoring, hand cash holding state monitoring, point counting machine use state monitoring and banknote box closing state monitoring are sequentially executed, each monitoring step needs to meet preset compliance conditions to enter the next step, and if any step does not meet the preset compliance conditions, a decision judgment and an alarm layer are triggered to issue an alarm. 2.The deep learning-based banknote case counting behavior compliance real-time monitoring method of claim 1, wherein, The optimized YOLOv8s algorithm comprises an adaptive scale allocation mechanism; feature extraction is performed on the image data based on the optimized YOLOv8s algorithm to obtain a plurality of feature maps; the plurality of feature maps are divided according to a preset scale to obtain a first scale feature map, a second scale feature map and a third scale feature map; wherein the first scale feature map is only used for anchor point matching with a human target; the second scale feature map is only used for anchor point matching with a banknote box and a point counting machine, and a deformable convolution is introduced in a banknote box detection branch; and the third scale feature map is only used for anchor point matching with a cash target, and a CBAM attention mechanism is introduced in a small scale feature extraction layer. 3.The deep learning-based banknote case counting behavior compliance real-time monitoring method of claim 1, wherein, The human pixel coordinates are converted into world coordinates based on the internal parameter data and the external parameter data of the camera, and the conversion comprises the following steps: ； wherein, and are focal lengths of the camera in the x-axis direction and the y-axis direction, respectively; and are principal point coordinates, which are pixel coordinates of the center of the image data; R is a rotation matrix in the extrinsic parameter data; is a translation vector; (u, v) is a human body pixel coordinate, is a world coordinate; is the transpose of the zero vector. 4.The deep learning-based banknote case counting behavior compliance real-time monitoring method of claim 3, wherein, The optimized RTMPose algorithm comprises: in a feature extraction network, the input feature map size is 256*192, and the output channel number is adjusted to 32; for human upper body detection, the output layer only predicts key points of human upper limbs, a nose and ears, and for hand key point detection, only a palm center key point is predicted.

5. The deep learning based banknote case counting behavior compliance real-time monitoring method of claim 4, wherein, The point counting machine running state detection module judges the running state of the point counting machine by combining image recognition and sound data analysis, and the judgment comprises the following steps: Based on the position of the point counting machine determined by the target detection module, the point counting machine region in the image data is cropped. A color recognition algorithm is used to detect the on-off state of the indicator light of the currency counter in the currency counter region; A DBnet text detection algorithm is used to locate the digital area of the currency counter screen in the currency counter region. After grayscale, noise reduction, and binarization preprocessing of the digital area, a CRNN text recognition algorithm is used to recognize the screen numbers to determine whether there is a currency count display; The sound data is converted into a frequency domain signal through Fourier transform, and it is analyzed whether there is a noise frequency component of the currency counter operation in the frequency domain signal; If it is determined that the indicator light is on, the currency count is recognized, and the noise frequency component is detected, it is determined that the currency counter is in operation, otherwise it is determined that it is not in operation.

6. The deep learning-based banknote case counting behavior compliance real-time monitoring method of claim 5, wherein, Personnel quantity and dress compliance monitoring, including: Based on the target detection module, the number of personnel in the image data is counted. If the number of personnel is not equal to 2, a "personnel quantity not compliant" alarm is triggered by the decision-making and alarm layer; If the number of personnel is equal to 2, the image area of the two personnel is input into the dress classification model included in the classification module. If any personnel is not identified as wearing a compliant uniform, an "incompliant dress" alarm is issued; otherwise, proceed to personnel distance compliance monitoring.

7. The deep learning-based banknote case counting behavior compliance real-time monitoring method of claim 6, wherein, Personnel distance compliance monitoring, including: Based on the coordinate conversion module, the body pixel coordinates of the two personnel are converted into world coordinates to obtain first and second world coordinates; According to the first and second world coordinates, the actual distance between the two people is calculated based on the Euclidean distance formula, and the actual distance is compared with the preset personnel safety distance threshold. When it is determined that the actual distance is greater than the preset personnel safety distance threshold, an "incompliant personnel distance" alarm is issued; otherwise, proceed to banknote case state monitoring.

8. The deep learning-based banknote case counting behavior compliance real-time monitoring method of claim 7, wherein, Banknote case state monitoring, including: The banknote case image detected by the target detection module is input into the banknote case opening and closing classification model included in the classification module to determine the current state of the banknote case; If the banknote case is in the closed state, start the timer to begin timing. If the timing duration is greater than the first preset threshold and the banknote case has not been detected to be opened, an "unopened banknote case on time" alarm is issued; If the banknote case is detected to be opened, stop the timer and proceed to hand-held cash state monitoring. 9.The deep learning-based banknote case counting behavior compliance real-time monitoring method of claim 8, wherein, Hand-held cash state monitoring, including: Obtain the coordinates of the key points output by the pose estimation module. Based on the relative positional relationship of the key points of the human upper limbs, nose, and ears, filter out invalid postures of personnel; For personnel with valid postures, obtain the key point coordinates of the palm center and the cash region coordinates detected by the target detection module, and calculate the proportion of the overlapping area of the two to the cash region area; If the proportion is greater than or equal to the proportion threshold, it is determined that the personnel are holding cash, and proceed to currency counter usage state monitoring; if the proportion is less than the preset proportion threshold and the duration is greater than the second preset threshold, an "undetected hand-held cash" alarm is issued.

10. The deep learning-based banknote case counting behavior compliance real-time monitoring method of claim 9, wherein, Currency counter usage state monitoring, including: After determining that the personnel are holding cash, start the currency counter operation state detection module to determine the currency counter state; Start the timer to time. If the timing duration is greater than the third preset threshold and the currency counter has not been detected to be in operation, an "unused currency counter" alarm is issued; If the currency counter is detected to be in operation, proceed to banknote case closed state monitoring.

Images