Bank passenger flow monitoring method and related device
By adopting the improved two-way feature pyramid structure and the object detection model of the multi-receptive field adaptive fusion module in bank customer flow monitoring, the problem that traditional methods cannot accurately capture complex customer behavior is solved, and higher monitoring accuracy and timeliness are achieved.
Patent Information
- Application Number
- CN202411743743.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-30
- Publication Date
- 2025-03-11
AI Technical Summary
Traditional bank customer flow monitoring methods are based on simple rules or statistical methods, and cannot accurately capture complex customer behavior and flow patterns, resulting in low accuracy of detection results.
The improved object detection model of the two-way feature pyramid structure is adopted, combined with the multi-receptive field adaptive fusion module, multi-level feature information fusion is carried out, and a high-resolution feature map is output, which is used to provide a fine tracking foundation for the target tracking model.
It improves the accuracy of bank customer flow monitoring, can capture customer behavior and flow patterns more accurately, and enhances the accuracy and timeliness of detection results.
Smart Images
Figure CN119600544A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer vision technology, and in particular to a bank customer flow monitoring method and related devices. Background Art
[0002] As an important part of the financial industry, banks play a role in capital circulation, storage, financing and other aspects. Banks need to accurately understand the customer flow, customer categories, flow trends, peak hours and abnormal behavior of bank branches, so as to help banks make business decisions, staff deployment, service optimization and security monitoring. Traditional customer flow monitoring methods are usually based on simple rules or statistical methods, which are time-consuming and labor-intensive and cannot accurately capture complex customer behaviors and flow patterns, and the accuracy of the detection results is low. Summary of the invention
[0003] In view of the above problems, the present application provides a bank passenger flow monitoring method and related devices to achieve the purpose of improving the accuracy of bank passenger flow monitoring. The specific scheme is as follows:
[0004] The first aspect of the present application provides a bank customer flow monitoring method, comprising:
[0005] Obtain video data of target bank branches;
[0006] The video data is input into the target detection model to obtain the target information; the target detection model is configured as follows: the improved bidirectional feature pyramid structure is used to realize multi-level feature information fusion, feature maps of different scales are output, the bounding box, confidence and classification loss of the feature maps of different scales are calculated, and the bounding box, confidence and classification loss are used as the target information; the improved bidirectional feature pyramid structure is integrated with the multi-receptive field adaptive fusion module; the feature maps of each scale in the feature maps of different scales are fused feature maps output by the convolutional layers in each feature pyramid that match the feature maps of each scale;
[0007] The target information is input into the target tracking model to obtain the customer flow monitoring results of the target bank branch.
[0008] In a possible implementation, the target detection model is a simplified model based on a channel pruning algorithm;
[0009] The process of simplifying the target detection model includes:
[0010] Get the initial target detection model;
[0011] Determine the scaling factor of each channel in the current channel set of the initial object detection model; the scaling factor is used to characterize the importance of the channel;
[0012] Prune channels whose scaling factors do not reach the preset pruning threshold of the channel itself; take each channel after the current round of pruning as the current channel set;
[0013] Iterative execution determines the scaling factor of each channel in the current channel set of the initial target detection model, prunes the channels whose scaling factors do not reach the preset pruning threshold of the channel itself, and uses each channel after the current round of pruning as the current channel set until the iteration stop condition is reached to obtain the target detection model.
[0014] In a possible implementation, the target detection model includes a feature extraction model and a feature fusion model;
[0015] The ghost module and efficient channel attention module are embedded in the feature extraction model;
[0016] The feature fusion network includes an improved bidirectional feature pyramid structure.
[0017] In a possible implementation, the feature fusion model includes an input stage, an intermediate state stage, and an output stage; the processing flow of the feature fusion model includes:
[0018] Inputting the feature extraction information output by the feature extraction model into the first bidirectional feature pyramid structure in the input stage to obtain a first fusion feature;
[0019] Inputting the first fused feature into the second bidirectional feature pyramid structure in the intermediate state stage to obtain a second fused feature;
[0020] The second fused features are input into the output stage, and the fused features of the first bidirectional feature pyramid structure and the corresponding level of the output stage, the fused features of the second bidirectional feature pyramid structure and the corresponding level of the output stage are fused with the convolution layer of the corresponding level of the output stage and then output to obtain feature maps of different scales.
[0021] In a possible implementation, each convolutional layer in the second bidirectional feature pyramid structure includes a multi-receptive field adaptive fusion module;
[0022] The multi-receptive field adaptive fusion module is used to process the input feature map before feature fusion in the corresponding convolutional layer.
[0023] In one possible implementation, the processing flow of the target tracking model includes:
[0024] Use target tracking algorithm to process video data to obtain target tracking information;
[0025] The associated target tracking information is matched with the target information received from the target detection model to obtain the passenger flow monitoring result.
[0026] A second aspect of the present application provides a bank customer flow monitoring device, comprising:
[0027] Acquisition unit, target detection unit and target tracking unit; wherein:
[0028] An acquisition unit, used for acquiring video data of a target bank branch;
[0029] The target detection unit is used to input the video data into the target detection model to obtain the target information; the target detection model is configured to: use the improved bidirectional feature pyramid structure to realize multi-level feature information fusion, output feature maps of different scales, calculate the bounding box, confidence and classification loss of the feature maps of different scales, and use the bounding box, confidence and classification loss as the target information; the improved bidirectional feature pyramid structure integrates the multi-receptive field adaptive fusion module; the feature maps of each scale in the feature maps of different scales are fused feature maps output by the convolution layers in each feature pyramid that match the feature maps of each scale;
[0030] The target tracking unit is used to input the target information into the target tracking model to obtain the customer flow monitoring result of the target bank branch.
[0031] The third aspect of the present application provides a bank customer flow monitoring device, comprising at least one processor and a memory connected to the processor, wherein:
[0032] The memory is used to store computer programs;
[0033] The processor is used to execute a computer program so that the bank customer flow monitoring device can implement the bank customer flow monitoring method as described in any one of the above.
[0034] The fourth aspect of the present application provides a computer program product, including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements the bank customer flow monitoring method as described in any one of the above.
[0035] The fifth aspect of the present application provides a computer storage medium, which carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement a bank customer flow monitoring method as described in any one of the above.
[0036] By means of the above technical solution, the bank passenger flow monitoring method and related equipment provided by the present application obtain the video data of the target bank branch, input it into the target detection model for processing, and obtain the target information; then input the target information into the target tracking model to obtain the passenger flow monitoring result. The target detection model in the present application adopts an improved bidirectional pyramid structure to realize multi-level feature information fusion, and the improved bidirectional pyramid structure integrates a multi-receptive field adaptive fusion module, which can output a high-resolution feature map that integrates the outputs of each level in each stage, realize richer feature fusion, and provide a fine tracking basis for the target tracking model, thereby improving the accuracy of the detection results. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.
[0038] Figure 1 A flow chart of a bank passenger flow monitoring method provided in this application;
[0039] Figure 2 This is a structural example diagram of the improved bidirectional pyramid structure provided by this application;
[0040] Figure 3 The working principle diagram of the multi-receptive field adaptive fusion module provided in this application;
[0041] Figure 4 This is an example diagram of the simplified process of obtaining the target detection model provided in this application;
[0042] Figure 5 An example diagram of the bank customer flow monitoring method provided in this application;
[0043] Figure 6 A schematic diagram of the structure of the bank passenger flow monitoring device provided for this application;
[0044] Figure 7 A schematic diagram of the structure of the bank passenger flow monitoring equipment provided for this application. DETAILED DESCRIPTION
[0045] The following describes the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. The terms used in the implementation method section of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.
[0046] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.
[0047] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.
[0048] Existing passenger flow monitoring methods are mainly based on simple rules or statistical methods, which are time-consuming and labor-intensive but cannot accurately capture complex customer behaviors and flow patterns. The accuracy and timeliness of the detection results are difficult to guarantee.
[0049] In order to solve the above problems, known technologies use deep learning-based target detection and target tracking algorithms to monitor passenger flow. However, the accuracy of deep learning-based target detection and target tracking algorithms in known technologies is obviously insufficient, and it is difficult to cope with the complex passenger flow situation of bank outlets, which may cause false detection, missed detection, etc., thereby affecting subsequent data analysis results.
[0050] In order to solve the above problems, the present application provides a bank customer flow monitoring method and related devices.
[0051] Optional, see Figure 1 , a flow chart of a bank customer flow monitoring method provided in this application.
[0052] like Figure 1 As shown, the bank customer flow monitoring method includes the following steps:
[0053] Step 101: Acquire video data of a target bank branch.
[0054] It should be noted that the video data of the target bank branch can be obtained from the local monitoring system of the target bank branch, network video server, intelligent customer flow counting equipment, Internet of Things platform, etc.
[0055] Optionally, video data is acquired in real time from a local monitoring system of a target bank branch.
[0056] Step 102: Input the video data into the target detection model to obtain target information; the target detection model is configured as follows: adopt an improved bidirectional feature pyramid structure to realize multi-level feature information fusion, output feature maps of different scales, calculate the bounding box, confidence and classification loss of the feature maps of different scales, and use the bounding box, confidence and classification loss as target information; the improved bidirectional feature pyramid structure integrates a multi-receptive field adaptive fusion module; the feature maps of each scale in the feature maps of different scales are fused feature maps output by the convolutional layers in each feature pyramid that match the feature maps of each scale.
[0057] Object detection is a computer vision task that aims to identify and locate different target objects in an image or video. It can not only identify the category of the target, but also indicate the precise location of the target in the image or video. Deep convolutional neural networks can automatically learn various features of images from training data samples, generate hierarchical features from raw pixels to high-level semantic information, and can show better accuracy and robustness in complex scenes and multi-category detection tasks. Therefore, using deep learning technology to complete object detection tasks has become mainstream. Object detection methods based on deep learning can be divided into two-stage object detection algorithms and single-stage object detection algorithms. The two-stage object detection algorithm splits the object detection into two steps. First, candidate regions are generated for the entire image, and then each candidate region is classified and adjusted in position and the final result is output. Typical models include R-CNN, Fast R-CNN, and Faster R-CNN. The single-stage object detection algorithm converts the object detection problem into a regression problem. It does not calculate the candidate regions, but directly uses the extracted image features to predict the category and position of the target, thereby effectively improving the detection speed. Typical models include YOLO, SSD, and CornerNet. In the early stage, the single-stage target detection algorithm has an advantage in speed, and the two-stage target detection algorithm has an advantage in accuracy. With the update and development of technology, both types of algorithms have better performance in speed and accuracy.
[0058] It should be noted that the target detection model provided in this application is an improved target detection model, which improves the accuracy and speed of the existing target detection model based on deep convolutional neural network. The improved target detection model is called BF-YOLO detection model.
[0059] The BF-YOLO detection model uses an improved bidirectional feature pyramid structure to achieve the fusion of hierarchical feature information, and can output feature maps of different scales. Then, it calculates the bounding box, confidence and classification loss of feature maps of different scales, and finally uses the bounding box, confidence and classification loss as target information.
[0060] Specifically, the BF-YOLO detection model includes a feature extraction model and a feature fusion model.
[0061] For the feature extraction model, the ghost module and efficient channel attention module are embedded in the model. Specifically, the ghost module refers to the lightweight convolutional neural network Ghost module. The module is used as the backbone network of the detection algorithm in the feature extraction model to simplify the model. The color, texture and shape features of the input image are extracted through structured combination of convolution and pooling operations to obtain multi-level semantic information; the efficient channel attention module is embedded, which can also be called the ECA (Efficient Channel Attention) channel attention module to improve the feature extraction ability of the feature extraction model. The ECA attention mechanism calculates the importance weights of each channel and then applies these weights to each channel of the input data to produce weighted outputs.
[0062] For the feature fusion model, an improved bidirectional pyramid structure is used to introduce shallow high-resolution feature maps to make full use of pixel details and positioning information, improve the positioning accuracy of the model and its sensitivity to similar features. Multi-level feature information is fused to obtain a target feature map containing multiple scales.
[0063] Specifically, the bidirectional feature pyramid structure can be called the BiFPN structure. The improved bidirectional feature pyramid structure adds an additional cross-layer path between the input nodes and output nodes at the same level in the network, achieving richer feature fusion at a small cost. In addition, the element-by-element addition operation during feature map fusion is replaced with channel-by-channel concatenation. In this way, the features on each channel are learned independently, so that the feature information of different channels can be better preserved, and information confusion and conflict can be avoided to a certain extent.
[0064] For example, see Figure 2 , a structural example diagram of the improved bidirectional pyramid structure provided in this application.
[0065] like Figure 2 As shown, , and Respectively represent The input, intermediate state and output feature maps of the layer. Then the feature fusion process can be described as the following formula, where represents the convolution operation, Represents resizing.
[0066] The formula for the output of the bidirectional pyramid structure can be:
[0067]
[0068] Optionally, the processing of the feature fusion model includes:
[0069] The feature extraction information output by the feature extraction model is input into the first bidirectional feature pyramid structure in the input stage to obtain the first fusion feature. The first bidirectional feature pyramid structure is: Figure 2 The pyramid structure of the input stage on the left, the first fusion feature from C 5 Layer output.
[0070] The first fused feature is input into the second bidirectional feature pyramid structure in the intermediate state stage to obtain the second fused feature; specifically, the second bidirectional feature pyramid structure is the pyramid structure in the intermediate state stage, and the second fused feature is M 2 The fused features of the layer output.
[0071] The second fused feature is input to the output stage, and the fused features of the first bidirectional feature pyramid structure and the corresponding level of the output stage, the fused features of the second bidirectional feature pyramid structure and the corresponding level of the output stage, and the convolution layer of the corresponding level of the output stage are fused and output to obtain feature maps of different scales. Figure 2 As shown in , the feature maps of different scales are P 3 , P 4 and P 5 , P 3 The feature map output by the layer combines the C 3 The fusion features of the layer output and the intermediate state stage M 3 The fusion features of the layer output, P 4 The feature map output by the layer combines the C 4 The fusion features of the layer output and the intermediate state stage M 4 The fusion features of the layer output, P 5 The feature map output by the layer combines the C 5 The fusion features of the layer output and the intermediate state stage M 5 The fused features of the layer output.
[0072] In addition, each convolutional layer in the second bidirectional feature pyramid structure in the intermediate state stage includes a multi-receptive field adaptive fusion module, which processes the input feature map before feature fusion in the corresponding convolutional layer.
[0073] Specifically, this module uses a residual connection similar to the convolution kernel to obtain feature maps of different receptive fields, and enhances the multi-scale expression of features at a finer granularity to obtain feature information of receptive fields of different scales.
[0074] For example, see Figure 3, the working principle diagram of the multi-receptive field adaptive fusion module provided in this application. The specific operation is as follows:
[0075] Feature Map In the channel dimension, it is evenly divided into subset, using Indicates that . For elements Do equivalent transformation; for elements arrive , each After a convolution Operation Output .element and The outputs of the operations are concatenated and then input to . Output feature maps of different groups , are receptive fields of different sizes. The formula is:
[0076]
[0077] Then, the channel is shuffled, and a channel shuffling strategy similar to ShuffleNet is used to convert the calculation of the receptive field weight into the calculation of the channel weight to achieve information exchange between receptive fields. Finally, dynamic weight assignment is performed, and global information is embedded through global average pooling to generate channel weight values. The specific operation is as follows, where represents global average pooling, Represents the height and width of the feature map.
[0078]
[0079] Finally, group convolution is used to perform convolution operations on each subgroup to independently learn the information exchange within each group, further correct the channel weights, and use the Sigmoid activation function to introduce nonlinear relationships. The specific operations are as follows, where represents the Sigmoid activation function, and GC represents grouped convolution.
[0080]
[0081] Finally, the bounding boxes of the three feature maps of different scales are constructed in the detection head of the BF-YOLO detection model ( ), confidence level ( ) and classification loss ( ) to achieve customer target detection, and the loss function calculation is shown in the following formula:
[0082]
[0083] In summary, this application adopts a multi-receptive field adaptive fusion module, the purpose of which is to automatically strengthen useful features and suppress useless features by modeling the dependencies between different receptive fields to enhance key information. First, the layered receptive fields are aggregated into one layer through the residual-like structure of the convolution kernel, and then the receptive fields of different sizes are adaptively weighted to enrich the multi-receptive field information and strengthen the expression of key features.
[0084] It should also be noted that the target detection model is a simplified model using the channel pruning algorithm. Simplifying the model using the channel pruning algorithm can also be called model compression. Model compression refers to reducing the size of the deep learning model by reducing the storage requirements and computational overhead of the model. This is a technology developed to deploy models in resource-constrained environments or run models on mobile devices. Model compression methods include quantization (quantizing parameters to lower precision), pruning (removing redundant neurons or connections), and distillation (learning a large model with a small model).
[0085] In this application, channel pruning technology is used for model compression, which can reduce the number of channels (or feature maps) in convolutional neural networks (CNNs). By automatically or manually identifying unimportant channels, the amount of computation, storage requirements, and model complexity can be reduced while retaining key feature extraction capabilities.
[0086] Model pruning specifically refers to pruning convolutional neural networks to remove unimportant weights, convolution kernels, channels, or layers, thereby making the network sparse to reduce storage space and computational effort.
[0087] Optionally, the process of simplifying the object detection model includes:
[0088] Step 1: Obtain an initial target detection model. Specifically, the initial target detection model is an initial model obtained by training a neural network on the original data set, which is a model with good accuracy and performance.
[0089] Step 2: Determine the scaling factor of each channel in the current channel set of the initial target detection model; the scaling factor is used to characterize the importance of the channel.
[0090] Assume that the scaling factor is BN, and evaluate the importance of the channels of each convolutional layer by analyzing the scaling factor of each channel. Add L1 regularization term to the loss function to encourage channel weights to approach zero.
[0091] Step 3: Prune the channels whose scaling factors do not reach the preset pruning threshold of the channel itself; and use each channel after the current round of pruning as the current channel set.
[0092] The channels that have not reached their own preset thresholds in each channel are taken as channels to be pruned. The preset thresholds themselves may also be referred to as preset pruning thresholds. Specifically, a pruning threshold may be set for each convolutional layer.
[0093] Step 4: Iterate and determine the scaling factor of each channel in the current channel set of the initial target detection model, prune the channels whose scaling factor does not reach the preset pruning threshold of the channel itself, and use each channel after this round of pruning as the current channel set until the iteration stop condition is reached to obtain the target detection model.
[0094] Prune the channels whose scaling factors do not reach the preset pruning threshold, obtain the pruned channels, re-evaluate the importance of the remaining channels, and then perform further pruning. Prune multiple times until the iteration stop condition is reached. The iterative pruning method can gradually optimize the pruning effect, and then fine-tune or retrain to obtain the final target detection model.
[0095] For example, see Figure 4 , the present application provides a simplified process example diagram for obtaining a target detection model.
[0096] like Figure 4 As shown, firstly, an initial network model is obtained, sparse training is performed on the initial network model, channels with small scaling factors in the model are pruned, and then parameters of the pruned network model are fine-tuned. The process of sparse training, pruning and parameter fine-tuning is iterated multiple times until a pruned model is obtained.
[0097] It should be noted that the channel pruning algorithm is used to prune the convolutional layers in the network model to build a lighter and more efficient detector. First, the scaling factors of some weights are pushed to zero through sparse training, and then the importance of the channel is evaluated through the scaling factor, and the channel with the smallest scaling factor is pruned according to a certain ratio. Finally, the parameters are fine-tuned to improve the accuracy of the pruned model. This solution can introduce sparsity to the convolutional neural network and achieve redundant parameter pruning.
[0098] The channel pruning model compression method is used to process the initial target detection model to obtain the target detection model, which can remove redundant parameters and connections in the network of the model, thereby reducing the number of parameters and calculations of the model and improving the detection efficiency of the model.
[0099] Step 103: Input the target information into the target tracking model to obtain the customer flow monitoring result of the target bank branch.
[0100] The target tracking model can be called the DeepSORT model, which is mainly used for target tracking, which refers to the continuous tracking of one or more moving target objects in a video sequence. It focuses on maintaining the continuity of the target in the time series, usually by updating the position of the target between frames.
[0101] It should be noted that target tracking aims to accurately track the position and trajectory of moving targets in continuous images or video sequences. SORT and DeepSORT are two commonly used target tracking methods. SORT is a simple and efficient target tracking algorithm, mainly based on Kalman filtering and Hungarian algorithm. Its core idea is to match the detection results (bounding boxes) in each frame with the tracking results in the previous frame. DeepSORT adds a re-identification network on the basis of SORT. By comparing the distance between the input and output vectors, it is determined whether the two input images are the same pedestrian. In addition, appearance information is added to achieve target tracking that is occluded for a long time.
[0102] Specifically, the DeepSORT model,DeepSORT algorithm, aims to associate objects from one frame to another by tracking their motion in video frames.,To achieve this, DeepSORT needs to learn the unique appearance features of the,object, which helps to accurately match objects between different frames.
[0103] Optionally, a target tracking algorithm is used to process the video data to obtain target tracking information; the target tracking information is specifically target trajectory information obtained by processing the video data using the target tracking algorithm DeepSORT.
[0104] The target tracking information is matched with the target information received from the target detection model to obtain the passenger flow monitoring result. Specifically, the target information detected by the BF-YOLO detection model in the current frame is matched with the tracked target trajectory information in the previous frame obtained by processing the video data using the target tracking algorithm, so as to ensure that the targets can be correctly associated between different frames. The cascade matching can be performed using a Kalman filter to ensure the stability and continuity of the tracking.
[0105] In summary, the bank passenger flow monitoring method provided by the present application obtains the video data of the target bank branch, inputs it into the target detection model for processing, and obtains the target information; then the target information is input into the target tracking model to obtain the passenger flow monitoring result. The target detection model in the present application adopts an improved bidirectional pyramid structure to realize multi-level feature information fusion, and the improved bidirectional pyramid structure integrates a multi-receptive field adaptive fusion module, which can output a high-resolution feature map that integrates the outputs of each level in each stage, realizes richer feature fusion, and can provide a fine tracking basis for the target tracking model, thereby improving the accuracy of the detection results.
[0106] Optional, see Figure 5 , an example diagram of the bank customer flow monitoring method provided in this application.
[0107] like Figure 5 As shown in the figure, the bank passenger flow monitoring method first obtains the video stream, then performs passenger flow analysis based on the bank passenger flow monitoring system of BF-YOLO and DeepSORT, and finally generates a passenger flow monitoring report including business decision-making, staffing, service optimization and security monitoring.
[0108] A bank customer flow monitoring method provided by an embodiment of the present application is introduced above, and a device for executing the above bank customer flow monitoring method will be introduced below.
[0109] See also Figure 6 , Figure 6 This is a schematic diagram of the structure of the bank passenger flow monitoring device provided in this application. Figure 6 As shown, the device comprises:
[0110] Acquisition unit 10, target detection unit 20 and target tracking unit 30; wherein:
[0111] An acquisition unit 10 is used to acquire video data of a target bank branch;
[0112] The target detection unit 20 is used to input the video data into the target detection model to obtain the target information; the target detection model is configured to: use the improved bidirectional feature pyramid structure to realize multi-level feature information fusion, output feature maps of different scales, calculate the bounding box, confidence and classification loss of the feature maps of different scales, and use the bounding box, confidence and classification loss as the target information; the improved bidirectional feature pyramid structure integrates the multi-receptive field adaptive fusion module; the feature maps of each scale in the feature maps of different scales are fused feature maps output by the convolution layers in each feature pyramid that match the feature maps of each scale;
[0113] The target tracking unit 30 is used to input the target information into the target tracking model to obtain the customer flow monitoring result of the target bank branch.
[0114] In one embodiment, the target detection model in the target detection unit 20 is a simplified model based on a channel pruning algorithm;
[0115] The process of simplifying the target detection model includes:
[0116] Get the initial target detection model;
[0117] Determine the scaling factor of each channel in the current channel set of the initial object detection model; the scaling factor is used to characterize the importance of the channel;
[0118] Prune channels whose scaling factors do not reach the preset pruning threshold of the channel itself; take each channel after the current round of pruning as the current channel set;
[0119] Iterative execution determines the scaling factor of each channel in the current channel set of the initial target detection model, prunes the channels whose scaling factors do not reach the preset pruning threshold of the channel itself, and uses each channel after the current round of pruning as the current channel set until the iteration stop condition is reached to obtain the target detection model.
[0120] In one embodiment, the target detection model in the target detection unit 20 includes a feature extraction model and a feature fusion model;
[0121] The ghost module and efficient channel attention module are embedded in the feature extraction model;
[0122] The feature fusion network includes an improved bidirectional feature pyramid structure.
[0123] In one embodiment, the feature fusion model in the target detection unit 20 includes an input stage, an intermediate state stage, and an output stage; the processing flow of the feature fusion model includes:
[0124] Inputting the feature extraction information output by the feature extraction model into the first bidirectional feature pyramid structure in the input stage to obtain a first fusion feature;
[0125] Inputting the first fused feature into the second bidirectional feature pyramid structure in the intermediate state stage to obtain a second fused feature;
[0126] The second fused features are input into the output stage, and the fused features of the first bidirectional feature pyramid structure and the corresponding level of the output stage, the fused features of the second bidirectional feature pyramid structure and the corresponding level of the output stage are fused with the convolution layer of the corresponding level of the output stage and then output to obtain feature maps of different scales.
[0127] In one embodiment, each convolutional layer in the second bidirectional feature pyramid structure in the target detection unit 20 includes a multi-receptive field adaptive fusion module;
[0128] The multi-receptive field adaptive fusion module is used to process the input feature map before feature fusion in the corresponding convolutional layer.
[0129] In one embodiment, the processing flow of the target tracking model in the target tracking unit 30 includes:
[0130] Use target tracking algorithm to process video data to obtain target tracking information;
[0131] The associated target tracking information is matched with the target information received from the target detection model to obtain the passenger flow monitoring result.
[0132] The present application also provides a bank customer flow monitoring device. Figure 7 As shown, it shows a schematic diagram of the structure of the bank passenger flow monitoring device provided by the present application. The bank passenger flow monitoring device in the embodiment of the present application may include but is not limited to fixed terminals such as mobile phones, laptops, PDAs (personal digital assistants), PADs (tablet computers), desktop computers, etc. Figure 7 The bank customer flow monitoring device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.
[0133] like Figure 7 As shown, the bank passenger flow monitoring device may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 to a random access memory (RAM) 603. When the bank passenger flow monitoring device is powered on, various programs and data required for the operation of the bank passenger flow monitoring device are also stored in the RAM 603. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.
[0134] Typically, the following devices may be connected to the I / O interface 605: input devices 606 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 608 including, for example, a memory card, a hard disk, etc.; and communication devices 609. The communication device 609 may allow the bank customer flow monitoring device to communicate with other devices wirelessly or by wire to exchange data. Although Figure 7 The bank customer flow monitoring device with various devices is shown, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed alternatively.
[0135] Also provided in an embodiment of the present application is a computer program product including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements any one of the bank customer flow monitoring methods provided in the embodiments of the present application.
[0136] A computer-readable storage medium is also provided in an embodiment of the present application. The storage medium carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any bank customer flow monitoring method provided in the embodiment of the present application.
[0137] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.
[0138] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a magnetic disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.
[0139] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.
[0140] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, a computer, a training device or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device or data center. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integration. Available media can be magnetic media, (e.g., floppy disk, hard disk, tape), optical media (e.g., DVD), or semiconductor media (e.g., solid-state hard disk (SSD)), etc.
Claims
1. A bank customer flow monitoring method, characterized in that: include: Obtain video data of target bank branches; Inputting the video data into a target detection model to obtain target information; The target detection model is configured to: use an improved bidirectional feature pyramid structure to implement multi-level feature information fusion, output feature maps of different scales, calculate bounding boxes, confidences, and classification losses of the feature maps of different scales, and use the bounding boxes, confidences, and classification losses as the target information; The improved bidirectional feature pyramid structure integrates a multi-receptive field adaptive fusion module; the feature maps of each scale in the feature maps of different scales are fused feature maps output by convolution layers in each feature pyramid that match the feature maps of each scale; The target information is input into a target tracking model to obtain a customer flow monitoring result of the target bank branch.
2. The bank passenger flow monitoring method according to claim 1, characterized in that: The target detection model is a simplified model based on the channel pruning algorithm; The process of simplifying the target detection model includes: Get the initial target detection model; Determining a scaling factor for each channel in a current channel set of the initial target detection model; the scaling factor is used to characterize the importance of the channel; Prune the channels whose scaling factors do not reach the preset pruning threshold of the channels themselves; and use the channels after the current round of pruning as the current channel set; Iterative execution determines the scaling factor of each channel in the current channel set of the initial target detection model, prunes the channels whose scaling factors do not reach the preset pruning threshold of the channel itself, and uses each channel after the current round of pruning as the current channel set until the iteration stop condition is reached to obtain the target detection model.
3. The bank passenger flow monitoring method according to claim 1, characterized in that: The target detection model includes a feature extraction model and a feature fusion model; A ghost module and an efficient channel attention module are embedded in the feature extraction model; The feature fusion model includes the improved bidirectional feature pyramid structure.
4. The bank passenger flow monitoring method according to claim 3, characterized in that: The feature fusion model includes an input stage, an intermediate state stage and an output stage; The processing flow of the feature fusion model includes: Inputting the feature extraction information output by the feature extraction model into the first bidirectional feature pyramid structure in the input stage to obtain a first fusion feature; Inputting the first fused feature into the second bidirectional feature pyramid structure in the intermediate state stage to obtain a second fused feature; The second fused features are input into the output stage, and the first bidirectional feature pyramid structure and each fused feature of the corresponding level of the output stage, the second bidirectional feature pyramid structure and each fused feature of the corresponding level of the output stage are fused with the convolution layer of the corresponding level of the output stage and then output to obtain feature maps of different scales.
5. The bank passenger flow monitoring method according to claim 4, characterized in that: Each convolutional layer in the second bidirectional feature pyramid structure includes a multi-receptive field adaptive fusion module; The multi-receptive field adaptive fusion module is used to process the input feature map before feature fusion in the corresponding convolutional layer.
6. The bank customer flow monitoring method according to claim 1, characterized in that: The processing flow of the target tracking model includes: Processing the video data using a target tracking algorithm to obtain target tracking information; The target tracking information is matched and associated with the target information received from the target detection model to obtain the passenger flow monitoring result.
7. A bank passenger flow monitoring device, characterized in that: include: Acquisition unit, target detection unit and target tracking unit; wherein: The acquisition unit is used to acquire video data of a target bank branch; The target detection unit is used to input the video data into the target detection model to obtain target information; the target detection model is configured to: use the improved bidirectional feature pyramid structure to realize multi-level feature information fusion, output feature maps of different scales, calculate the bounding box, confidence and classification loss of the feature maps of different scales, and use the bounding box, confidence and classification loss as the target information; the improved bidirectional feature pyramid structure integrates a multi-receptive field adaptive fusion module; the feature maps of each scale in the feature maps of different scales are fused feature maps output by convolution layers in each feature pyramid that match the feature maps of each scale; The target tracking unit is used to input the target information into a target tracking model to obtain the customer flow monitoring result of the target bank branch.
8. A bank passenger flow monitoring device, characterized in that: The method comprises at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program so that the bank customer flow monitoring device can implement the bank customer flow monitoring method as described in any one of claims 1 to 6.
9. A computer program product, characterized in that It includes computer-readable instructions, and when the computer-readable instructions are executed on an electronic device, the electronic device implements the bank customer flow monitoring method as described in any one of claims 1 to 6.
10. A computer storage medium, characterized in that: The storage medium carries one or more computer programs, and when the one or more computer programs are executed by an electronic device, the electronic device can implement the bank customer flow monitoring method as described in any one of claims 1 to 6.