Financial escort process abnormal behavior identification method and system

Abstract

The invention discloses a financial escorting process abnormal behavior identification method and system, and relates to the field of data processing. The method comprises the following steps: acquiring personnel characteristic data, and establishing a multi-modal biological characteristic database; matching features extracted from the real-time personnel feature data with features stored in a multi-modal biological feature database by using a convolutional neural network to obtain a matching result; when the matching result is that the matching is successful, continuing to obtain the video stream of the monitoring area in real time and preprocessing the video stream; performing target detection on the preprocessed video stream by using a deep learning algorithm, and identifying a tracking target; performing continuous frame tracking on the detected tracking target by using a target tracking algorithm to obtain a motion track of the tracking target; extracting attitude information of the target by using an attitude estimation algorithm, and constructing a multi-dimensional behavior feature vector in combination with a motion track and spatial-temporal features of the target; establishing a preset abnormal behavior rule base, wherein the abnormal behavior rule base comprises a known abnormal behavior set; and based on the output of the multi-dimensional behavior feature vector, in combination with a preset abnormal behavior rule base, performing real-time analysis on the identified behavior, and identifying whether the target is performing an abnormal behavior or not. The technical problems that in the prior art, risks in the escort process cannot be recognized at the same time through a single personnel recognition technology, and the requirement for timely finding out the escort risk is difficult to meet are solved, real-time comparative analysis of the movement track and the preset track is achieved through verification of personnel multi-mode biological characteristics and tracking of the movement track of the target, and the safety of the escort risk is improved. Therefore, the real-time monitoring efficiency is improved, and the technical effects of reducing the escort risk are achieved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and specifically to a method and system for identifying abnormal behavior in a financial escort process. Background Art

[0002] With the rapid development of financial services, financial escort work has become increasingly important and complex. Traditional identity authentication methods such as passwords and certificates have security risks such as easy leakage and easy forgery, and can no longer meet the security needs of modern financial escort.

[0003] The existing facial recognition technology or iris recognition technology, although a single personnel recognition technology, has improved the convenience and accuracy of identity authentication to a certain extent, there is still a risk that the escort process cannot be identified at the same time after personnel recognition. Financial institutions will find it difficult to grasp the escort dynamics in real time, leading to increased security risks and poor escort processes.

[0004] In summary, due to the single personnel identification technology in the prior art, it is impossible to simultaneously identify the risks of the escort process, and it is difficult to meet the technical problem of timely discovering the escort risks. Summary of the invention

[0005] Based on this, it is necessary to provide a method and system for identifying abnormal behaviors in the financial escort process in response to the above-mentioned technical problems. This can solve the technical problems in the prior art that a single personnel identification technology cannot simultaneously identify the risks of the escort process and is difficult to meet the requirements for timely discovery of escort risks. By verifying the multimodal biometric features of personnel and tracking the motion trajectory of the target, the real-time motion trajectory can be compared and analyzed with the preset trajectory to identify abnormal behaviors, thereby improving the efficiency of real-time monitoring and reducing the technical effect of escort risks.

[0006] In a first aspect, a method for identifying abnormal behavior in a financial escort process is provided, the method comprising: collecting personnel feature data, the feature data comprising fingerprint data, facial data, and iris data; extracting features from the personnel feature data and establishing a multimodal biometric database; acquiring real-time personnel feature data, and matching the features extracted from the real-time personnel feature data with the features stored in the multimodal biometric database using a convolutional neural network to obtain a matching result; when the matching result is a successful match, continuing to acquire a video stream of a monitoring area in real time and performing preprocessing; performing target detection on the preprocessed video stream using a deep learning algorithm to identify a tracking target, the tracking target comprising a person and a vehicle; performing continuous frame tracking of the detected tracking target using a target tracking algorithm to obtain a motion trajectory of the tracking target; extracting the target's posture information using a posture estimation algorithm, and constructing a multidimensional behavior feature vector in combination with the target's motion trajectory and spatiotemporal features; establishing a preset abnormal behavior rule library, the abnormal behavior rule library comprising a known abnormal behavior set; based on the output of the multidimensional behavior feature vector, in combination with the preset abnormal behavior rule library, performing real-time analysis on the identified behavior to identify whether the target is performing abnormal behavior.

[0007] In the second aspect, a system for identifying abnormal behavior in a financial escort process is provided, the system comprising: a data acquisition module, the data acquisition module is used to collect personnel feature data, the feature data includes fingerprint data, facial data, and iris data; a feature extraction module, the feature extraction module is used to extract features from the personnel feature data and establish a multimodal biometric database; a feature matching module, the feature matching module is used to obtain real-time personnel feature data, use a convolutional neural network to match the features extracted from the real-time personnel feature data with the features stored in the multimodal biometric database to obtain a matching result; a video stream acquisition module, the video stream acquisition module is used to continue to acquire the video stream of the monitoring area in real time and perform preprocessing when the matching result is a successful match; a target recognition module, the target recognition module is used to use a deep learning algorithm to perform target detection on the preprocessed video stream The invention also provides a method for detecting and identifying tracking targets, wherein the tracking targets include persons and vehicles; a motion trajectory acquisition module, wherein the motion trajectory acquisition module is used to track the detected tracking targets in continuous frames by using a target tracking algorithm to obtain the motion trajectory of the tracking targets; a multi-dimensional behavior feature vector construction module, wherein the multi-dimensional behavior feature vector construction module is used to extract the posture information of the target by using a posture estimation algorithm, and to construct a multi-dimensional behavior feature vector in combination with the motion trajectory and spatiotemporal characteristics of the target; an abnormal behavior rule library establishment module, wherein the abnormal behavior rule library establishment module is used to establish a preset abnormal behavior rule library, wherein the abnormal behavior rule library contains a known abnormal behavior set; and an abnormal behavior recognition module, wherein the abnormal behavior recognition module is used to analyze the identified behaviors in real time based on the output of the multi-dimensional behavior feature vector and in combination with a preset abnormal behavior rule library, and to recognize whether the target is performing abnormal behaviors.

[0008] The above-mentioned abnormal behavior identification method and system in the financial escort process solves the technical problems in the prior art that a single personnel identification technology cannot simultaneously identify the risks of the escort process and is difficult to meet the requirements for timely discovery of escort risks. By verifying the multimodal biometric features of personnel and tracking the motion trajectory of the target, the real-time motion trajectory is compared and analyzed with the preset trajectory, and abnormal behavior is identified, thereby improving the efficiency of real-time monitoring and reducing the technical effect of escort risk.

[0009] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 It is a flowchart of a method for identifying abnormal behavior in a financial escort process in one embodiment; Figure 2 A flowchart of a method for identifying abnormal behavior in a financial escort process in an embodiment of the present invention is to use a posture estimation algorithm to extract posture information of a target, and to construct a multi-dimensional behavior feature vector by combining the motion trajectory and spatiotemporal characteristics of the target; Figure 3 The structure block diagram of the abnormal behavior identification system in the financial escort process in one embodiment.

[0011] Explanation of the reference numerals: data acquisition module 11, feature extraction module 12, feature matching module 13, video stream acquisition module 14, target recognition module 15, motion trajectory acquisition module 16, multi-dimensional behavior feature vector construction module 17, abnormal behavior rule base establishment module 18, abnormal behavior recognition module 19. DETAILED DESCRIPTION

[0012] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0013] like Figure 1 As shown, the present application provides a method for identifying abnormal behavior in a financial escort process, the method comprising: Collecting personnel characteristic data, including fingerprint data, facial data, and iris data; Fingerprint data, facial data, and iris data are all unique biometric features with high uniqueness and stability. In the process of financial escort, the fingerprint data, facial data, and iris data of the escort personnel are collected to identify and verify the identity of the personnel. Among them, because iris features are difficult to forge or imitate, iris recognition technology is used to provide a higher level of identity authentication, which is used to enhance security measures and prevent unauthorized persons from approaching the escorted items.

[0014] Collecting personal characteristic data, including fingerprint data, facial data, and iris data, including: Use professional fingerprint collection equipment to collect fingerprints and obtain fingerprint data; Use a high-resolution camera to capture a person's face to obtain facial data; Use the iris acquisition device to perform multiple scans to obtain iris data under different angles and lighting conditions.

[0015] Optical fingerprint collection equipment is used to capture the subtle texture and feature points of fingerprints, ensuring that the collected fingerprint data has high resolution and clarity. When the finger is pressed on the collection surface during the fingerprint data collection process, the lines and valleys on the finger surface reflect light differently. The device captures the difference in these reflected light through the sensor to form a fingerprint image. There is a light source inside the optical fingerprint collection device to illuminate the finger surface, and the sensor detects the intensity and pattern of the reflected light to obtain the fingerprint line information; when collecting facial data, a high-resolution camera is used to capture the details of the person's face and generate facial data, including the shape and contour of the eyes, nose, mouth and other parts, as well as features such as skin texture. Dynamic capture is also performed to capture the facial expressions and movements of the escort personnel in real time to identify potential abnormal behaviors, such as tension and anxiety. In order to adapt to different lighting environments, the camera is equipped with autofocus and autoexposure functions. In a dimly lit environment, the camera can automatically increase the exposure, and in an overly bright environment, it can automatically reduce the exposure to ensure that the collected facial image has appropriate contrast and brightness; the iris acquisition device is a high-precision biometric device that uses image processing and optical technology to accurately capture the subtle texture and feature points of the iris and achieve high-precision iris recognition. The iris acquisition device has a special optical system inside that can adjust the incident angle and intensity of light within a certain range, focus on the iris area, and accurately capture the texture information of the iris.

[0016] Extracting features from the personnel feature data and establishing a multimodal biometric database; First, feature extraction is performed from the personnel feature data obtained from the acquisition device, including fingerprint features, facial features, and iris features. Fingerprint feature extraction determines the pattern of the fingerprint image by analyzing the overall texture direction and pattern, including bucket-shaped, dustpan-shaped, and bow-shaped. The texture of the bucket-shaped fingerprint presents a circular or spiral trend; the dustpan-shaped fingerprint has a dustpan-like shape; and the texture of the bow-shaped fingerprint is relatively smooth and arched. The fingerprint image contains many unique detail points, such as endpoints and bifurcation points. The fingerprint texture is refined into lines with a single pixel width using the image thinning method, and the coordinate position (x, y) and direction (expressed in angles) and type (endpoint or bifurcation point) of each detail point are marked. Facial feature extraction locates the position of facial key points, such as the corners of the eyes, the tip of the nose, the corners of the mouth, etc., through the active appearance model. The distribution pattern of facial key points is learned on the training set through the facial feature point detection algorithm, and the key point positions that match them are searched on the input facial image. Next, the positions of the facial key points of the person are determined, and the relative distance, angle, and other relationships between them are calculated. Including calculating the distance between the two eyes, the distance from the nose tip to the corner of the mouth, etc. Finally, extract the texture information of the facial skin. Iris feature extraction first locates the collected iris image, determines the inner and outer boundaries of the iris, then normalizes the iris area to a fixed size, and finally performs filtering on the normalized iris area.

[0017] Extracting features from the personnel feature data and establishing a multimodal biometric feature database includes: Use fingerprint feature extraction algorithm to extract fingerprint texture, direction, frequency and other feature information from the collected fingerprint data; Using a facial feature extraction algorithm, feature information such as facial key points, texture and shape are extracted from the collected facial data, wherein the facial key points include the corners of the eyes and the corners of the mouth; Use iris feature extraction algorithm to extract iris texture, spots, lines and other feature information from the collected iris data; The extracted fingerprint features, facial features and iris features are integrated to form each person's multimodal biometric data and establish a multimodal biometric database.

[0018] Feature extraction is performed through a convolutional neural network (CNN). First, a convolution operation is performed to output a feature map, then the size of the feature map generated after the convolution operation is calculated, then a pooling operation is performed on the feature map, and finally a fully connected layer operation is performed.

[0019] Convolution operation expression:

[0020] Among them, Y is the output feature map; W is the convolution kernel; X is the input image; valid is the convolution type (no zero padding); b is the bias term; conv2 is the convolution operation.

[0021] The formula for calculating the size of the feature map generated after the convolution operation is:

[0022] Where w is the size of the input image; p is the number of filling layers; k is the size of the convolution kernel; s is the stride; Constraint: Division operations are rounded down.

[0023] The maximum pooling operation expression is:

[0024] Among them, S is the output feature map; max_pooling represents the maximum pooling operation; X is the input feature map; k is the window size value.

[0025] The output Y of the fully connected layer is expressed as:

[0026] Among them, W represents the weight of the fully connected layer; A represents the output of the previous layer; b represents the bias of the fully connected layer.

[0027] Acquire real-time personnel feature data, and use a convolutional neural network to match features extracted from the real-time personnel feature data with features stored in a multimodal biometric database to obtain a matching result; First, a convolutional neural network is used for feature matching. Then, the verification and matching process of cosine similarity is adopted. A convolutional neural network is defined and trained to extract features from the input data, and these features are used to calculate the similarity matrix between samples. In the feature extraction stage, the input data is forward propagated through CNN to obtain feature vectors; in the similarity measurement stage, the similarity between feature vectors is calculated using measurement methods such as cosine similarity, thereby completing the verification and matching tasks. Finally, the verification results of multiple modalities are combined to obtain the final verification conclusion. Based on the verification results, feedback is provided to the user (such as "verification successful" or "verification failed").

[0028] The output feature map expression is:

[0029]

[0030] Among them, X is an m×n input image matrix; W is a k×k (k < m, k < n) convolution kernel matrix; Y is a output feature map matrix.

[0031] When the matching result is a successful match, continue to obtain the video stream of the monitoring area in real time and perform preprocessing; After a successful match, continue to obtain the video stream of the monitoring area in real time. To improve the visibility of the video image, it is necessary to preprocess the captured video frames for subsequent analysis. First, perform a size adjustment operation on the video stream image. Due to the complexity of the monitoring environment, the video stream may contain various noises, so Gaussian filtering is performed for denoising. Enhance the contrast of the license plate of the escort vehicle and the faces of the escort personnel during the financial escort process.

[0032] Use a deep learning algorithm to perform object detection on the preprocessed video stream to identify the tracking targets, where the tracking targets include personnel and vehicles; Distinguish escort personnel from other personnel according to the color and style characteristics of the escort uniforms. By analyzing multiple frames of images in the video stream, identify whether the personnel are walking normally, running, or making abnormal movements (such as suddenly squatting down, waving their arms, etc.). Vehicles have obvious shape characteristics, such as the rectangular outline of the vehicle body and the circular shape of the wheels. At the same time, the color and license plate information of the vehicle are also important identification features. The driving state of the vehicle, such as speed, driving direction, etc., can also be obtained by analyzing the position changes of the vehicle in consecutive frames in the video stream. In this way, monitor whether the escort vehicle is driving along the predetermined route and whether there are abnormal accelerations or decelerations, etc.

[0033] Use a deep learning algorithm to perform object detection on the preprocessed video stream to identify the tracking targets, where the tracking targets include personnel and vehicles, including: Extract images frame by frame from the video stream for frame-by-frame analysis; Enhance the extracted images and adjust the images to a size suitable for input to the deep learning model; Normalize the images so that the pixel values are within a specific range; Use a convolutional neural network for feature extraction and classification; Use the labeled dataset to train the selected deep learning model; Apply the trained model to each frame of the video stream for object detection to identify the positions and categories of personnel and vehicles; Apply a tracking algorithm between consecutive frames to track the movement trajectory of the target in the video.

[0034] Extract images frame by frame from the video stream, perform contrast enhancement and noise reduction on the extracted images, and adjust the images to a size suitable for the input of the deep learning model. Normalize the images and normalize the pixel values ​​to a specific range. Use convolutional neural networks for feature extraction and classification. First, perform convolution operations, then pooling operations, and then fully connected layer operations. Use the labeled dataset to train the selected deep learning model, apply the trained model to each frame in the video stream for target detection, slide the convolution kernel on the image, extract features, and determine whether there are human or vehicle targets based on the output of the fully connected layer. If there are targets, the model will determine the location and category of the person and vehicle based on the feature map and related algorithms. Finally, apply the tracking algorithm between consecutive frames to track the movement trajectory of the target in the video.

[0035] The target tracking algorithm is used to track the detected tracking target in continuous frames to obtain the motion trajectory of the tracking target; The algorithm is used to generate areas that may contain targets, and then these areas are classified and regressed to determine the exact location and category of the target. Then, the optical flow method is used to assume that the pixel intensity in the image remains unchanged for a short period of time, and the grayscale changes of pixels between adjacent frames are analyzed to determine the direction and speed of pixel movement to achieve target tracking. During the target tracking process, the target's features are continuously extracted from the current frame as the target moves in the video. The sliding window method is used to search for the most similar area in subsequent frames. A window similar to the initial size of the target is slid in the subsequent frames, and the similarity between the area in the window and the target features is calculated. When the most similar area is found (based on the set similarity threshold), it is considered that the position of the target in the frame has been found. Once the most similar area is found in the subsequent frames, the target's location information is updated in a timely manner to accurately identify the target.

[0036] The target tracking algorithm is used to track the detected target in consecutive frames to obtain the motion trajectory of the target, including: Use the target detection algorithm to detect the target in each frame of the video stream; Initializing the detected target, wherein the initialization includes the initial position, size, and possible appearance features of the target; Use the optical flow method to calculate the movement of pixels in the image, thereby tracking the movement trajectory of the target; Extract the features of the target from the current frame; According to the features of the extracted target, searching for the area most similar to the target in subsequent frames; When the most similar area is found, the location information of the target is updated; According to the position information of the target in the continuous frames, the motion state of the target is estimated and the motion trajectory of the target is obtained.

[0037] The posture estimation algorithm is used to extract the posture information of the target, and the multi-dimensional behavior feature vector is constructed by combining the target's motion trajectory and spatiotemporal characteristics. The posture estimation algorithm is used to determine the posture of the escort personnel and vehicles in the video frame, such as the position of the joint points of the human body, the angle of the limbs, etc. The position coordinate points at different times are used to represent the motion trajectory of the escort personnel and vehicles. For the time characteristics such as the appearance time, duration and time rhythm of the escort personnel and vehicles in the video, the spatial characteristics include whether the vehicle is in different spatial areas such as the lane, intersection or parking lot. The posture information, motion trajectory characteristics and spatiotemporal characteristics are combined into a multi-dimensional behavior feature vector.

[0038] The posture estimation algorithm is used to extract the posture information of the target, and combined with the target's motion trajectory and spatiotemporal characteristics, a multi-dimensional behavior feature vector is constructed, including: Use the posture estimation algorithm to estimate the posture of the target in the video stream and obtain the key points of the target and the position information of the key points; The output of the posture estimation algorithm is used to extract the posture information of the target, wherein the posture information includes the coordinates of key points, the relative position relationship between key points, and the motion trajectory of key points; Using a target tracking algorithm to obtain a motion trajectory of the target, the motion trajectory is a sequence of position information of the target in consecutive frames; Extracting the spatiotemporal features of the target, wherein the spatiotemporal features refer to the change information of the target in time and space, and the change information includes the acceleration, speed change, posture change, etc. of the target; The extracted posture information, motion trajectory and spatiotemporal features are combined to construct a multi-dimensional behavior feature vector.

[0039] Establishing a preset abnormal behavior rule library, wherein the abnormal behavior rule library includes a known abnormal behavior set; Establish an abnormal behavior rule library, which is designed to identify irregular behaviors of financial escort personnel in the process of performing tasks. By clarifying the rules of these behaviors, these abnormal behaviors are automatically detected using monitoring technology to ensure the safety of financial escort on the spot.

[0040] Establish a preset abnormal behavior rule library, which contains a known abnormal behavior set, including: Collect sample data containing various abnormal behaviors and establish a preset abnormal behavior rule library; Extracting behavioral feature information that can describe abnormal behavior samples, the feature information includes the target's posture, motion trajectory, speed, acceleration, interaction with other targets, etc.; Based on the extracted feature information, an abnormal behavior rule library is constructed.

[0041] Establish a preset abnormal behavior rule library, collect abnormal behavior sample data of escort personnel and vehicles, among which the abnormal behaviors of escort personnel include irregular sitting posture, dozing, making phone calls, eating, smoking, whispering, etc. Abnormal behaviors of vehicles include not driving according to the set trajectory, speeding or slow speed, collision between the target vehicle and other vehicles or people, abnormal vehicle parts, etc. Build an abnormal behavior rule library, and extract behavioral feature information from abnormal behavior samples, including the posture of escort personnel, the movement trajectory of escort vehicles, vehicle speed, acceleration, and interaction with other targets.

[0042] Based on the output of the multi-dimensional behavior feature vector and in combination with a preset abnormal behavior rule library, the identified behavior is analyzed in real time to identify whether the target is performing abnormal behavior.

[0043] First, the behaviors of the escort personnel and vehicles in the video frames output by the multi-dimensional behavior feature vector are compared and analyzed with the preset abnormal behavior rule library to identify whether the escort personnel and vehicles are performing abnormal behaviors.

[0044] Based on the output of the multi-dimensional behavior feature vector and in combination with a preset abnormal behavior rule library, the identified behavior is analyzed in real time to identify whether the target is performing abnormal behavior, including: Using posture estimation algorithm, target tracking algorithm and spatiotemporal feature extraction method, the multi-dimensional behavior feature vector of the target is extracted from the video stream in real time; Matching the extracted multi-dimensional behavior feature vector with the preset abnormal behavior rule library; Based on the matching results between the feature vector and the rule base, determine in real time whether the target is performing abnormal behavior; When abnormal behavior is identified, the system immediately triggers the alarm mechanism.

[0045] Using posture estimation algorithm, target tracking algorithm and spatiotemporal feature extraction method, the position of human joints and target motion trajectory information are identified, including the coordinates of the escort personnel's head, shoulder, elbow, wrist, hip, knee and ankle joints and the running trajectory of the escort vehicle. The position relationship of the personnel joints is used to describe the posture of the human body, such as standing, bending, raising hands, and the trajectory of the vehicle's motion trajectory, and further analysis is performed to obtain the target's motion direction, speed change and other characteristics. For example, by calculating the vector between adjacent points on the trajectory, the target's motion direction vector is obtained, and by calculating the rate of change of the distance between the trajectory points over time, the target's speed information is obtained; the target's multi-dimensional behavior feature vector is extracted. After the target's multi-dimensional behavior feature vector is extracted, it is matched one by one with the rules in the abnormal behavior rule library. If an abnormal behavior is matched, the system will trigger an alarm and send the alarm information to the remote monitoring center through the network.

[0046] like Figure 3 As shown, the embodiment of the present application includes a system for identifying abnormal behavior in a financial escort process, and the system includes: A data collection module 11, which is used to collect personnel feature data, including fingerprint data, facial data, and iris data; A feature extraction module 12, the feature extraction module is used to extract features from the personnel feature data and establish a multimodal biometric database; A feature matching module 13, which is used to obtain real-time personnel feature data, and use a convolutional neural network to match features extracted from the real-time personnel feature data with features stored in a multimodal biometric database to obtain a matching result; A video stream acquisition module 14, wherein when the matching result is a successful match, the video stream acquisition module continues to acquire the video stream of the monitoring area in real time and performs preprocessing; A target recognition module 15, which is used to perform target detection on the pre-processed video stream using a deep learning algorithm to identify tracking targets, including people and vehicles; A motion trajectory acquisition module 16, which is used to track the detected tracking target in consecutive frames using a target tracking algorithm to acquire the motion trajectory of the tracking target; A multi-dimensional behavior feature vector construction module 17, which is used to extract the target's posture information using a posture estimation algorithm, and to construct a multi-dimensional behavior feature vector in combination with the target's motion trajectory and spatiotemporal characteristics; An abnormal behavior rule base establishing module 18, the abnormal behavior rule base establishing module is used to establish a preset abnormal behavior rule base, the abnormal behavior rule base contains a known abnormal behavior set; The abnormal behavior identification module 19 is used to analyze the identified behavior in real time based on the output of the multi-dimensional behavior feature vector and in combination with a preset abnormal behavior rule library to identify whether the target is performing abnormal behavior.

[0047] Furthermore, the data collection module 11 also includes: Use professional fingerprint collection equipment to collect fingerprints and obtain fingerprint data; Use a high-resolution camera to capture a person's face to obtain facial data; Use the iris acquisition device to perform multiple scans to obtain iris data under different angles and lighting conditions.

[0048] Furthermore, the feature extraction module 12 also includes: Use fingerprint feature extraction algorithm to extract fingerprint texture, direction, frequency and other feature information from the collected fingerprint data; Using a facial feature extraction algorithm, feature information such as facial key points, texture and shape are extracted from the collected facial data, wherein the facial key points include the corners of the eyes and the corners of the mouth; Use iris feature extraction algorithm to extract iris texture, spots, lines and other feature information from the collected iris data; The extracted fingerprint features, facial features and iris features are integrated to form each person's multimodal biometric data and establish a multimodal biometric database.

[0049] Furthermore, the target recognition module 15 also includes: Extract images frame by frame from the video stream for frame-by-frame analysis; Enhance the extracted images and resize them to a size suitable for deep learning model input; Normalize the image so that the pixel values ​​are within a specific range; Use convolutional neural networks for feature extraction and classification; Use the labeled dataset to train the selected deep learning model; Apply the trained model to each frame in the video stream for object detection, identifying the location and category of people and vehicles; Apply tracking algorithms between consecutive frames to track the movement trajectory of the target in the video.

[0050] Furthermore, the motion trajectory acquisition module 16 also includes: Use the target detection algorithm to detect the target in each frame of the video stream; Initializing the detected target, wherein the initialization includes the initial position, size, and possible appearance features of the target; Use the optical flow method to calculate the movement of pixels in the image, thereby tracking the movement trajectory of the target; Extract the features of the target from the current frame; According to the features of the extracted target, searching for the area most similar to the target in subsequent frames; When the most similar area is found, the location information of the target is updated; According to the position information of the target in the continuous frames, the motion state of the target is estimated and the motion trajectory of the target is obtained.

[0051] Furthermore, the multi-dimensional behavior feature vector construction module 17 also includes: Use the posture estimation algorithm to estimate the posture of the target in the video stream and obtain the key points of the target and the position information of the key points; The output of the posture estimation algorithm is used to extract the posture information of the target, wherein the posture information includes the coordinates of key points, the relative position relationship between key points, and the motion trajectory of key points; Using a target tracking algorithm to obtain a motion trajectory of the target, the motion trajectory is a sequence of position information of the target in consecutive frames; Extracting the spatiotemporal features of the target, wherein the spatiotemporal features refer to the change information of the target in time and space, and the change information includes the acceleration, speed change, posture change, etc. of the target; The extracted posture information, motion trajectory and spatiotemporal features are combined to construct a multi-dimensional behavior feature vector.

[0052] Furthermore, the abnormal behavior rule base building module 18 also includes: Collect sample data containing various abnormal behaviors and establish a preset abnormal behavior rule library; Extracting behavioral feature information that can describe abnormal behavior samples, the feature information includes the target's posture, motion trajectory, speed, acceleration, interaction with other targets, etc.; Based on the extracted feature information, an abnormal behavior rule library is constructed.

[0053] Furthermore, the abnormal behavior identification module 19 also includes: Using posture estimation algorithm, target tracking algorithm and spatiotemporal feature extraction method, the multi-dimensional behavior feature vector of the target is extracted from the video stream in real time; Matching the extracted multi-dimensional behavior feature vector with the preset abnormal behavior rule library; Based on the matching results between the feature vector and the rule base, determine in real time whether the target is performing abnormal behavior; When abnormal behavior is identified, the system immediately triggers the alarm mechanism.

[0054] For the specific embodiment of the abnormal behavior identification system in the financial escort process, please refer to the embodiment of the abnormal behavior identification method in the financial escort process mentioned above, which will not be repeated here. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0055] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0056] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for identifying abnormal behavior in a financial escort process, characterized in that: The method comprises: Collecting personnel characteristic data, including fingerprint data, facial data, and iris data; Extracting features from the personnel feature data and establishing a multimodal biometric database; Acquire real-time personnel feature data, and use a convolutional neural network to match features extracted from the real-time personnel feature data with features stored in a multimodal biometric database to obtain a matching result; When the matching result is successful, continue to obtain the video stream of the monitored area in real time and perform preprocessing; Using a deep learning algorithm to perform target detection on the preprocessed video stream and identify tracking targets, which include people and vehicles; The target tracking algorithm is used to track the detected tracking target in continuous frames to obtain the motion trajectory of the tracking target; The posture estimation algorithm is used to extract the posture information of the target, and the multi-dimensional behavior feature vector is constructed by combining the target's motion trajectory and spatiotemporal characteristics. Establishing a preset abnormal behavior rule library, wherein the abnormal behavior rule library includes a known abnormal behavior set; Based on the output of the multi-dimensional behavior feature vector and in combination with a preset abnormal behavior rule library, the identified behavior is analyzed in real time to identify whether the target is performing abnormal behavior.

2. The method according to claim 1, characterized in that Collecting personal characteristic data, including fingerprint data, facial data, and iris data, including: Use professional fingerprint collection equipment to collect fingerprints and obtain fingerprint data; Use a high-resolution camera to capture a person's face to obtain facial data; Use the iris acquisition device to perform multiple scans to obtain iris data under different angles and lighting conditions.

3. The method according to claim 1, characterized in that Extracting features from the personnel feature data and establishing a multimodal biometric database includes: Use fingerprint feature extraction algorithm to extract fingerprint texture, direction, frequency and other feature information from the collected fingerprint data; Using a facial feature extraction algorithm, feature information such as facial key points, texture and shape are extracted from the collected facial data, wherein the facial key points include the corners of the eyes and the corners of the mouth; Use iris feature extraction algorithm to extract iris texture, spots, lines and other feature information from the collected iris data; The extracted fingerprint features, facial features and iris features are integrated to form each person's multimodal biometric data and establish a multimodal biometric database.

4. The method according to claim 1, characterized in that The deep learning algorithm is used to detect the target of the preprocessed video stream and identify the tracking target, which includes people and vehicles, including: Extract images frame by frame from the video stream for frame-by-frame analysis; Enhance the extracted images and resize them to a size suitable for deep learning model input; Normalize the image so that the pixel values ​​are within a specific range; Use convolutional neural networks for feature extraction and classification; Use the labeled dataset to train the selected deep learning model; Apply the trained model to each frame in the video stream for object detection, identifying the location and category of people and vehicles; Apply tracking algorithms between consecutive frames to track the movement trajectory of the target in the video.

5. The method according to claim 1, characterized in that The target tracking algorithm is used to track the detected target in consecutive frames to obtain the motion trajectory of the target, including: Use the target detection algorithm to detect the target in each frame of the video stream; Initializing the detected target, wherein the initialization includes the initial position, size, and possible appearance features of the target; Use the optical flow method to calculate the movement of pixels in the image, thereby tracking the movement trajectory of the target; Extract the features of the target from the current frame; According to the features of the extracted target, searching for the area most similar to the target in subsequent frames; When the most similar area is found, the location information of the target is updated; According to the position information of the target in the continuous frames, the motion state of the target is estimated and the motion trajectory of the target is obtained.

6. The method according to claim 1, characterized in that The posture estimation algorithm is used to extract the posture information of the target, and combined with the target's motion trajectory and spatiotemporal characteristics, a multi-dimensional behavior feature vector is constructed, including: Use the posture estimation algorithm to estimate the posture of the target in the video stream and obtain the key points of the target and the position information of the key points; The output of the posture estimation algorithm is used to extract the posture information of the target, wherein the posture information includes the coordinates of key points, the relative position relationship between key points, and the motion trajectory of key points; Using a target tracking algorithm to obtain a motion trajectory of the target, the motion trajectory is a sequence of position information of the target in consecutive frames; Extracting the spatiotemporal features of the target, wherein the spatiotemporal features refer to the change information of the target in time and space, and the change information includes the acceleration, speed change, posture change, etc. of the target; The extracted posture information, motion trajectory and spatiotemporal features are combined to construct a multi-dimensional behavior feature vector.

7. The method according to claim 1, characterized in that Establish a preset abnormal behavior rule library, which contains a known abnormal behavior set, including: Collect sample data containing various abnormal behaviors and establish a preset abnormal behavior rule library; Extracting behavioral feature information that can describe abnormal behavior samples, the feature information includes the target's posture, motion trajectory, speed, acceleration, interaction with other targets, etc.; Based on the extracted feature information, an abnormal behavior rule library is constructed.

8. The method according to claim 1, characterized in that Based on the output of the multi-dimensional behavior feature vector and in combination with a preset abnormal behavior rule library, the identified behavior is analyzed in real time to identify whether the target is performing abnormal behavior, including: Using posture estimation algorithm, target tracking algorithm and spatiotemporal feature extraction method, the multi-dimensional behavior feature vector of the target is extracted from the video stream in real time; Matching the extracted multi-dimensional behavior feature vector with the preset abnormal behavior rule library; Based on the matching results between the feature vector and the rule base, determine in real time whether the target is performing abnormal behavior; When abnormal behavior is identified, the system immediately triggers the alarm mechanism.

9. A system for identifying abnormal behavior during financial escort, characterized in that: The system comprises: A data collection module, wherein the data collection module is used to collect personnel characteristic data, wherein the characteristic data includes fingerprint data, facial data, and iris data; A feature extraction module, the feature extraction module is used to extract features from the personnel feature data and establish a multimodal biometric database; A feature matching module, the feature matching module is used to obtain real-time personnel feature data, and use a convolutional neural network to match features extracted from the real-time personnel feature data with features stored in a multimodal biometric database to obtain a matching result; A video stream acquisition module, wherein when the matching result is a successful match, the video stream acquisition module continues to acquire the video stream of the monitoring area in real time and performs preprocessing; A target recognition module, which is used to perform target detection on the preprocessed video stream using a deep learning algorithm to identify tracking targets, including people and vehicles; A motion trajectory acquisition module, wherein the motion trajectory acquisition module is used to track the detected tracking target in consecutive frames using a target tracking algorithm to acquire the motion trajectory of the tracking target; A multi-dimensional behavior feature vector construction module, wherein the multi-dimensional behavior feature vector construction module is used to extract the target's posture information by using a posture estimation algorithm, and to construct a multi-dimensional behavior feature vector by combining the target's motion trajectory and spatiotemporal characteristics; An abnormal behavior rule base establishment module, the abnormal behavior rule base establishment module is used to establish a preset abnormal behavior rule base, the abnormal behavior rule base contains a known abnormal behavior set; The abnormal behavior recognition module is used to analyze the recognized behavior in real time based on the output of the multi-dimensional behavior feature vector and in combination with a preset abnormal behavior rule library to identify whether the target is performing abnormal behavior.

Images