Method and apparatus for identifying risk behavior

Abstract

The application provides a risk behavior recognition method and device, relates to the technical field of artificial intelligence, and can be applied to the technical field of finance or other technical fields. The risk behavior recognition method comprises the following steps: acquiring a video image, extracting a skeleton sequence from the video image; extracting skeleton key point information features and skeleton connection relationship features from the skeleton sequence; inputting the skeleton key point information features and the skeleton connection relationship features into a first behavior prediction model and a second behavior prediction model respectively, and obtaining a first behavior and a second behavior respectively; and obtaining a risk behavior recognition result according to the first behavior and the second behavior.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and more specifically, to a method and apparatus for identifying risky behaviors. Background Technology

[0002] To strengthen internal control risk prevention and personnel management in data security management areas of bank branches, existing surveillance video footage should be utilized to detect abnormal behaviors within the areas, thereby improving the overall security and reliability of bank services. With numerous bank branches and a dense network of surveillance equipment, a large amount of surveillance data remains underutilized. Currently, risky behaviors within the service areas are primarily identified visually by internal control personnel, resulting in wasted human resources and missed detections. There is an urgent need to improve detection efficiency through technological means and develop a fully automated and intelligent risk behavior detection method. Summary of the Invention

[0003] The main objective of this invention is to provide a method and apparatus for identifying risky behaviors, thereby reducing labor costs and improving work efficiency.

[0004] To achieve the above objectives, embodiments of the present invention provide a risk behavior identification method, comprising:

[0005] Acquire employee video images and extract skeleton sequences from the employee video images;

[0006] Extract skeleton key point information features and skeleton connection relationship features from the skeleton sequence;

[0007] The skeleton key point information features and skeleton connection relationship features are respectively input into the first behavior prediction model and the second behavior prediction model to obtain the first employee behavior and the second employee behavior, respectively.

[0008] Risk behavior identification results are obtained based on the first employee's behavior and the second employee's behavior.

[0009] In one embodiment, the step of creating a first behavior prediction model includes:

[0010] Perform the following iterative processing:

[0011] The training features of the skeleton keypoints are input into the convolutional network of the first image to obtain the spatial features of the skeleton keypoints.

[0012] The spatial features of the skeleton key points are processed according to the first compression excitation network architecture to obtain the spatiotemporal features of the skeleton key points.

[0013] A first behavior prediction result is obtained based on the spatiotemporal features of the skeleton key points, and a first loss function is determined based on the first behavior prediction result and the corresponding actual behavior.

[0014] When the first loss function converges, the first behavior prediction model is determined based on the first graph convolutional network and the first compressed excitation network architecture; otherwise, the first compressed excitation network architecture is updated based on the first loss function.

[0015] In one embodiment, processing the spatial features of the skeleton key points according to the first compression excitation network architecture to obtain the spatiotemporal features of the skeleton key points includes:

[0016] The spatial features of the skeleton key points are processed sequentially according to the first convolutional layer, the first compression layer and the first pooling layer in the first compression excitation network architecture to obtain intermediate data of the first skeleton key points.

[0017] The intermediate data of the second skeleton key points are obtained by sequentially processing the intermediate data of the first skeleton key points using multiple first convolutional compression layers.

[0018] The spatiotemporal features of the skeleton key points are obtained based on the intermediate data of the second skeleton key points and the spatial features of the skeleton key points.

[0019] In one embodiment, the step of creating a second behavior prediction model includes:

[0020] Perform the following iterative processing:

[0021] The skeleton connection training features are input into the second graph convolutional network to obtain the skeleton connection spatial features;

[0022] The spatial features of the skeleton connection relationship are processed according to the second compression excitation network architecture to obtain the spatiotemporal features of the skeleton connection relationship.

[0023] The second behavior prediction result is obtained based on the spatiotemporal features of the skeleton connection relationship, and the second loss function is determined based on the second behavior prediction result and the corresponding actual behavior.

[0024] When the second loss function converges, the second behavior prediction model is determined based on the second graph convolutional network and the second compressed activation network architecture; otherwise, the second compressed activation network architecture is updated based on the second loss function.

[0025] In one embodiment, processing the spatial features of the skeleton connection relationships according to the second compression excitation network architecture to obtain the spatiotemporal features of the skeleton connection relationships includes:

[0026] The skeleton connection spatial features are processed sequentially according to the second convolutional layer, the second compression layer and the first pooling layer in the second compressed excitation network architecture to obtain the first skeleton connection intermediate data.

[0027] The intermediate data of the second skeleton connection relationship is obtained by sequentially processing the intermediate data of the first skeleton connection relationship using multiple second convolutional compression layers;

[0028] The spatiotemporal features of the skeleton connection relationship are obtained based on the intermediate data of the second skeleton connection relationship and the spatial features of the skeleton connection relationship.

[0029] In one embodiment, acquiring employee video images includes:

[0030] Perform face correction on faces in video images;

[0031] The face-corrected video image is input into the face recognition model to obtain the employee recognition result;

[0032] The employee video image is obtained based on the employee identification result and the video image.

[0033] This invention also provides a risk behavior identification device, comprising:

[0034] The skeleton sequence extraction module is used to acquire employee video images and extract skeleton sequences from the employee video images;

[0035] The skeleton feature relationship extraction module is used to extract skeleton key point information features and skeleton connection relationship features from the skeleton sequence;

[0036] The behavior prediction module is used to input the skeleton key point information features and skeleton connection relationship features into the first behavior prediction model and the second behavior prediction model respectively, so as to obtain the first employee behavior and the second employee behavior respectively.

[0037] The identification result module is used to obtain risk behavior identification results based on the first employee behavior and the second employee behavior.

[0038] In one embodiment, it further includes:

[0039] The skeleton key point spatiotemporal feature module is used to process the spatial features of the skeleton key points according to the first compression excitation network architecture to obtain the spatiotemporal features of the skeleton key points.

[0040] The first loss function module is used to obtain a first behavior prediction result based on the spatiotemporal features of the skeleton key points, and to determine a first loss function based on the first behavior prediction result and the corresponding actual behavior.

[0041] The first behavior prediction model module is used to determine the first behavior prediction model based on the first graph convolutional network and the first compressed excitation network architecture when the first loss function converges; otherwise, it updates the first compressed excitation network architecture based on the first loss function.

[0042] In one embodiment, the spatiotemporal feature module of the skeleton key points includes:

[0043] The first skeleton key point intermediate data unit is used to process the spatial features of the skeleton key points in sequence according to the first convolutional layer, the first compression layer and the first pooling layer in the first compressed excitation network architecture to obtain the first skeleton key point intermediate data.

[0044] The second skeleton key point intermediate data unit is used to process the first skeleton key point intermediate data sequentially according to multiple first convolutional compression layers to obtain the second skeleton key point intermediate data.

[0045] The skeleton key point spatiotemporal feature unit is used to obtain the skeleton key point spatiotemporal features based on the second skeleton key point intermediate data and the skeleton key point spatial features.

[0046] In one embodiment, it further includes:

[0047] The skeleton connection spatial feature module is used to input the skeleton connection training features into the second graph convolutional network to obtain the skeleton connection spatial features;

[0048] The skeleton connection spatiotemporal feature module is used to process the skeleton connection spatial features according to the second compression excitation network architecture to obtain the skeleton connection spatiotemporal features.

[0049] The second loss function module is used to obtain the second behavior prediction result based on the spatiotemporal features of the skeleton connection relationship, and to determine the second loss function based on the second behavior prediction result and the corresponding actual behavior.

[0050] The second behavior prediction model module is used to determine the second behavior prediction model based on the second graph convolutional network and the second compressed activation network architecture when the second loss function converges; otherwise, it updates the second compressed activation network architecture based on the second loss function.

[0051] In one embodiment, the spatiotemporal feature module of skeleton connection relationship includes:

[0052] The first skeleton connection relationship intermediate data unit is used to process the skeleton connection relationship spatial features sequentially according to the second convolutional layer, the second compression layer and the first pooling layer in the second compressed excitation network architecture to obtain the first skeleton connection relationship intermediate data.

[0053] The second skeleton connection relationship intermediate data unit is used to process the first skeleton connection relationship intermediate data sequentially according to multiple second convolutional compression layers to obtain the second skeleton connection relationship intermediate data.

[0054] The skeleton connection spatiotemporal feature unit is used to obtain the skeleton connection spatiotemporal features based on the intermediate data of the second skeleton connection and the spatial features of the skeleton connection.

[0055] In one embodiment, the skeleton sequence extraction module includes:

[0056] The face correction unit is used to correct faces in video images;

[0057] The employee identification result unit is used to input the face-corrected video image into the face recognition model to obtain the employee identification result;

[0058] An employee video image unit is used to acquire the employee video image based on the employee identification result and the video image.

[0059] This invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the steps of the risk behavior identification method.

[0060] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the risk behavior identification method.

[0061] This invention also provides a computer program product, including a computer program / instructions, wherein the computer program / instructions, when executed by a processor, implement the steps of the risk behavior identification method.

[0062] The risk behavior identification method and apparatus of this invention first extracts skeleton sequences from employee video images to obtain skeleton key point information features and skeleton connection relationship features. Then, the skeleton key point information features and skeleton connection relationship features are respectively input into the behavior prediction model to obtain the corresponding employee behavior and obtain the risk behavior identification result. This can reduce labor costs and improve work efficiency. Attached Figure Description

[0063] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0064] Figure 1 This is a flowchart of the risk behavior identification method in an embodiment of the present invention;

[0065] Figure 2This is a flowchart of acquiring employee video images in an embodiment of the present invention;

[0066] Figure 3 This is a flowchart of creating a first behavior prediction model in an embodiment of the present invention;

[0067] Figure 4 This is a flowchart of S302 in an embodiment of the present invention;

[0068] Figure 5 This is a flowchart of creating a second behavior prediction model in an embodiment of the present invention;

[0069] Figure 6 This is a flowchart of S502 in an embodiment of the present invention;

[0070] Figure 7 This is a schematic diagram of the face alignment rotation angle in an embodiment of the present invention;

[0071] Figure 8 This is a schematic diagram of the corrected state in an embodiment of the present invention;

[0072] Figure 9 This is a flowchart of face recognition in an embodiment of the present invention;

[0073] Figure 10 This is a schematic diagram of the OPENPOSE convolutional neural network in an embodiment of the present invention;

[0074] Figure 11 This is a schematic diagram of 13 key detection points based on sitting posture;

[0075] Figure 12 This is a schematic diagram of 20 key detection points based on standing posture;

[0076] Figure 13 This is a schematic diagram of a dynamic skeleton sequence extracted based on sitting posture in an embodiment of the present invention;

[0077] Figure 14 This is a schematic diagram of a dynamic skeleton sequence extracted based on standing posture in an embodiment of the present invention;

[0078] Figure 15 This is a schematic diagram of the compression excitation network architecture in an embodiment of the present invention;

[0079] Figure 16 This is a structural block diagram of the risk behavior recognition device in an embodiment of the present invention;

[0080] Figure 17 This is a structural block diagram of a risk behavior recognition device according to another embodiment of the present invention;

[0081] Figure 18 This is a functional diagram of the employee identification module in an embodiment of the present invention;

[0082] Figure 19 This is a functional diagram of the abnormal risk behavior identification module in an embodiment of the present invention;

[0083] Figure 20 This is a schematic block diagram illustrating the system configuration of an electronic device 9600 according to an embodiment of this application. Detailed Implementation

[0084] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0085] Those skilled in the art will recognize that embodiments of the present invention can be implemented as a system, apparatus, device, method, or computer program product. Therefore, this disclosure can be specifically implemented in the following forms: entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0086] The acquisition, storage, use, and processing of data in this application comply with relevant laws and regulations.

[0087] Given the low efficiency of existing technologies, this invention provides a risk behavior identification method and apparatus. It detects camera footage in both the business service area and the office area, identifying bank employees and intelligently detecting different categories of risk behaviors according to the requirements of the bank employee handbook for different areas. Personal information and risk video clips are stored locally. An automated script is written to read risk video clip information daily, automatically summarizing all results into a table and generating statistical charts for each institution, achieving fully automated and intelligent detection and recording. Furthermore, this invention requires high accuracy in all stages of task processing to reduce false positives and enhance the security of customer transactions and information management. Therefore, each stage utilizes the latest models and deep learning algorithms to improve the overall detection effect. The invention will be described in detail below with reference to the accompanying drawings.

[0088] Figure 1 This is a flowchart of the risk behavior identification method in an embodiment of the present invention. Figure 1 As shown, risk behavior identification methods include:

[0089] S101: Acquire employee video images and extract skeleton sequences from the employee video images.

[0090] Figure 2This is a flowchart illustrating the acquisition of employee video images in an embodiment of the present invention. Figure 2 As shown, acquiring employee video images includes:

[0091] S201: Perform face correction on faces in video images.

[0092] In practice, the employees to be detected are first identified, and a facial database is constructed. Folders are named after the employees, and each folder stores at least three facial photos of that employee. Then, a Retinaface model is trained to build a lightweight, camera-based real-time face detection platform. This platform detects all face locations in the video content, the locations of five key facial points, and the 3D position and correspondence of each facial pixel, thus representing facial feature information.

[0093] Figure 7 This is a schematic diagram of the face alignment rotation angle in an embodiment of the present invention. Figure 8 This is a schematic diagram after correction in an embodiment of the present invention. Figures 7-8 As shown, a face alignment algorithm based on binocular coordinates is used to correct the face, ensuring that the obtained face is as upright as possible and facing forward, thus improving the accuracy of subsequent detection. Specifically, the method involves connecting the eyes in the image and calculating the angle between this line and the horizontal line, while simultaneously finding the center of the image to determine the center of rotation.

[0094] S202: Input the face-corrected video image into the face recognition model to obtain the employee recognition result.

[0095] Figure 9 This is a flowchart of face recognition in an embodiment of the present invention. For example... Figure 9 As shown, the aligned face photo can be input into the Facenet face recognition model to encode the face and obtain the feature vector of each face.

[0096] S203: Obtain the employee video image based on the employee identification result and the video image.

[0097] like Figure 9 As shown, the facial feature vector detected in the video surveillance is compared with the feature vector in the pre-stored employee photo to determine whether it is an employee and to identify the employee's identity.

[0098] This invention utilizes the latest machine learning algorithms to monitor employee behavior in real-time via surveillance videos. It determines model detection tasks according to a risk behavior manual, identifies problematic risk behaviors, and incorporates them into the performance evaluation system, thus changing the bank's existing reliance on manual visual monitoring of surveillance videos. The risk monitoring behaviors identified include employees spending extended periods at self-service terminals on behalf of customers, behavioral conflicts with customers, risky behaviors in secure computer rooms, and unauthorized visitors tailgating into secure areas. Furthermore, the types of risky behaviors to be detected can be expanded subsequently.

[0099] Since cameras in different scenarios have different acquisition angles, this invention sets different feature extraction tasks and parameter settings for camera detection tasks in business service areas and office areas. The overall abnormal risk behavior recognition includes two main parts: feature extraction based on human pose and behavior recognition based on graph convolutional networks.

[0100] Based on the feature extraction of human posture, the video is processed in combination with the results of employee identification. The OPENPOSE algorithm is used to extract the human skeleton sequence of the employee in each frame to represent behavioral characteristics.

[0101] In practice, employee identity is identified from a specific frame, with that frame serving as the start time of the video and the first frame in which the employee disappears from the monitor serving as the end time. Video segments are captured every minute to ensure real-time detection. 20 frames of equal length are captured per second to maximize detection density and accuracy. If a captured video segment is less than one minute long, the last frame of that segment is used as padding to maintain a fixed video length.

[0102] Figure 10 This is a schematic diagram of the OPENPOSE convolutional neural network in an embodiment of the present invention. Figure 11 This is a schematic diagram of 13 key detection points based on sitting posture. Figure 12 This is a schematic diagram of 20 key detection points based on standing posture. (Example) Figures 10-12 As shown, based on the behavioral characteristics and target extraction models within the banking scene, different key points are extracted. Video frames from two cameras are fed into convolutional neural networks with different OPENPOSE configurations to identify the skeletal sequences of specified key nodes and adjacent nodes of all figures in the image. The Hungarian algorithm is then used to connect these sequences to obtain the skeletal sequence of each individual. Figure 11 and Figure 12 The single-frame skeleton rendering of a person. For videos with a seated posture, 13 key points are extracted from the face and upper body of the person in the image. For a standing posture-based detection task, 20 key points are extracted from the entire body of the person.

[0103] Figure 13 This is a schematic diagram of a dynamic skeleton sequence extracted based on sitting posture in an embodiment of the present invention. Figure 14 This is a schematic diagram of a dynamic skeleton sequence extracted based on standing posture in an embodiment of the present invention. For example... Figures 13-14 As shown, connecting the same joint of all adjacent frames in the same video segment can yield a dynamic skeleton sequence containing temporal and spatial attributes, which can be used to represent the dynamic behavioral characteristics of a person and the continuous action information of a person in a video.

[0104] S102: Extract skeleton key point information features and skeleton connection relationship features from the skeleton sequence.

[0105] In practice, the joint position information (skeleton key point information) of the dynamic skeleton sequence can be extracted as first-order information, and the direction information and length information of the limbs (skeleton connection relationship features) can be extracted as second-order information for recognition tasks.

[0106] S103: Input the skeleton key point information features and skeleton connection relationship features into the first behavior prediction model and the second behavior prediction model respectively to obtain the first employee behavior and the second employee behavior respectively.

[0107] The behavior prediction model of this invention is a graph convolutional network. Traditional graph convolutional networks used for behavior recognition are 3D convolutional neural networks and RNNs. These models mine the continuity information of actions by designing the network structure. In contrast, graph convolutional neural networks mine high-level features in the graph topology, which can effectively improve the recognition accuracy of the model.

[0108] In the behavior recognition part based on graph convolutional networks, the two-stream adaptive graph convolutional network (2s-AGCN) currently achieves good classification performance. To match and mine the latent features of human skeletal sequences, the 2s-AGCN is designed to fully learn the features of the human body and key joints, improving the accuracy and computational power of human behavior recognition. To improve skeleton-based behavior classification results, this invention optimizes the 2s-AGCN network structure by introducing an attention mechanism to quantify the interdependencies and importance between feature channels. Considering the model's computational power and the number of operational parameters, the attention mechanism module is embedded in the network to merge feature channel information, rather than introducing a new spatial dimension attention mechanism, thus reducing the model size. A comparative analysis of various embedded attention mechanism model structures is conducted, drawing on the compression and activation block (SE) structure, the ACTIONNet structure, and the principle of asymmetric convolution, to design a new embedded attention network model. This model recalibrates the features obtained in the previous stage through compression, activation, and weight adjustment, assigning different weights to different features to improve the classification performance during training. The overall idea of ​​the model is to connect the human skeleton sequence according to the order in which the actions occur, resulting in a dynamic skeleton sequence. Two types of high-level features are extracted and trained separately to obtain a fused, high-precision behavior recognition result. Through continuous training and parameter experimentation of the network, an attention mechanism model was designed, which was embedded and replaced with the Temporal Convolutional Network (TCN) block in the original 2s-AGCN network structure to improve the model's accuracy in recognizing graph-structured behaviors.

[0109] Figure 3 This is a flowchart illustrating the creation of the first behavior prediction model in an embodiment of the present invention. For example... Figure 3 As shown, the steps to create a first-behavior prediction model include:

[0110] Perform the following iterative processing:

[0111] S301: Input the training features of the skeleton key points into the first convolutional network to obtain the spatial features of the skeleton key points.

[0112] S302: Process the spatial features of the skeleton key points according to the first compression excitation network architecture to obtain the spatiotemporal features of the skeleton key points.

[0113] The compressed activation network architecture of this invention implements an attention mechanism through three operations: compression, activation, and weight readjustment. First, compression involves compressing features along the spatial dimension, transforming each two-dimensional feature channel into a real number. To some extent, this real number possesses a global receptive field, and the output dimension corresponds to the number of input feature channels. It describes the global distribution of feature channels and allows layers closer to the input to obtain a global receptive field, which is very useful in many tasks. The second operation is activation, which is similar to the gating mechanism in RNNs. Weights are generated for each feature channel using parameters, and these parameters are learned to explicitly model the correlations between feature channels. Finally, the weight readjustment process treats the weights output from the activation operation as the importance of each feature channel after feature selection, and then multiplies these weights channel-by-channel to recalibrate the original features along the channel dimension.

[0114] Figure 4 This is a flowchart of S302 in an embodiment of the present invention. For example... Figure 4 As shown, S302 includes:

[0115] S401: The spatial features of the skeleton key points are processed sequentially according to the first convolutional layer, the first compression layer and the first pooling layer in the first compression excitation network architecture to obtain the intermediate data of the first skeleton key points.

[0116] Figure 15 This is a schematic diagram of the compression excitation network architecture in an embodiment of the present invention. Figure 15 As shown, to maintain the temporal order of the sequence, this invention adds one-dimensional convolutions to the two fully connected (FC) layers and uses spatial pyramid pooling as the pooling strategy. Its advantage lies in converting inputs of arbitrary size into fixed-size outputs. When training the network structure to obtain better mechanism parameters, it was found that defining the first layer convolution kernel size as 9*1 achieved better feature integration. The parameters and structure of the network are combined with the feature map structure of the detection task, and after multiple comparative experiments, it was determined that this effectively improves the overall learning ability of the attention mechanism. Finally, this compressed excitation network architecture is used to replace the TCN block in the original 2s-AGCN network structure. A basic block in the original 2s-AGCN network consists of GCN, TCN, and ReLU; the replaced basic block consists of GCN, SE, and ReLU to improve the recognition performance in the classification stage.

[0117] In practice, the spatial features of the skeleton key points are processed sequentially according to 2D convolution, the first Reshape operation, and the first spatial pyramid pooling layer to obtain the intermediate data of the first skeleton key points.

[0118] S402: The intermediate data of the first skeleton key points are obtained by sequentially processing the intermediate data of the first skeleton key points according to multiple first convolutional compression layers.

[0119] In practice, the first convolutional compression layer includes 1D convolution and reshape operations. Figure 13 The first convolutional compression layer in the model has three layers.

[0120] S403: Obtain the spatiotemporal features of the skeleton key points based on the intermediate data of the second skeleton key points and the spatial features of the skeleton key points.

[0121] like Figure 15 As shown, after processing the intermediate data of the second skeleton keypoints using the Sigmoid function, the spatiotemporal features of the skeleton keypoints can be obtained as output based on the spatial features of the original input skeleton keypoints and the processed intermediate data of the second skeleton keypoints.

[0122] S303: Obtain the first behavior prediction result based on the spatiotemporal features of the skeleton key points, and determine the first loss function based on the first behavior prediction result and the corresponding actual behavior.

[0123] S304: Determine whether the first loss function has converged.

[0124] S305: When the first loss function converges, the first behavior prediction model is determined based on the architecture of the first graph convolutional network and the first compressed excitation network.

[0125] S306: When the first loss function does not converge, update the first compressed excitation network architecture according to the first loss function.

[0126] Figure 5 This is a flowchart illustrating the creation of a second behavior prediction model in an embodiment of the present invention. For example... Figure 5 As shown, the steps to create a second behavior prediction model include:

[0127] Perform the following iterative processing:

[0128] S501: Input the skeleton connection training features into the second graph convolutional network to obtain the skeleton connection spatial features.

[0129] S502: Process the spatial features of the skeleton connection relationship according to the second compression excitation network architecture to obtain the spatiotemporal features of the skeleton connection relationship.

[0130] Figure 6 This is a flowchart of S502 in an embodiment of the present invention. For example... Figure 6 As shown, S502 includes:

[0131] S601: The spatial features of the skeleton connection relationship are processed sequentially according to the second convolutional layer, the second compression layer and the first pooling layer in the second compressed excitation network architecture to obtain the intermediate data of the first skeleton connection relationship.

[0132] S602: The intermediate data of the first skeleton connection relationship is obtained by sequentially processing the intermediate data of the first skeleton connection relationship according to multiple second convolutional compression layers.

[0133] S603: Obtain the spatiotemporal features of the skeleton connection relationship based on the intermediate data of the second skeleton connection relationship and the spatial features of the skeleton connection relationship.

[0134] S503: Obtain the second behavior prediction result based on the spatiotemporal features of the skeleton connection relationship, and determine the second loss function based on the second behavior prediction result and the corresponding actual behavior.

[0135] S504: Determine whether the second loss function has converged.

[0136] S505: When the second loss function converges, the second behavior prediction model is determined based on the architecture of the second graph convolutional network and the second compressed excitation network.

[0137] S506: When the second loss function does not converge, update the second compressed excitation network architecture according to the second loss function.

[0138] S104: Obtain risk behavior identification results based on the first employee behavior and the second employee behavior.

[0139] In practical implementation, this invention first iterates through and reads the name of each local folder, saving it as a specific data storage object. Then, it stores the folder names in an Excel spreadsheet. Sub-tables in the Excel spreadsheet are named "Branch," storing records from all branches within the same branch in the same sub-table. The table fields are set according to the folder names, including "Employee Name," "Employee ID," "Risk Behavior Category," and "Voucher Number." For each sub-table, the number of risk occurrences for the same person is counted, and a bar chart is plotted according to behavior category, generating statistical reports for each branch. Using RPA's email processing and sending functions, the Excel file and statistical reports are automatically sent to the designated staff's email addresses.

[0140] Figure 1 The risk behavior identification method shown can be implemented by a computer. Figure 1As shown in the process, the risk behavior identification method of this embodiment first extracts skeleton sequences from employee video images to obtain skeleton key point information features and skeleton connection relationship features. Then, the skeleton key point information features and skeleton connection relationship features are respectively input into the behavior prediction model to obtain the corresponding employee behavior and obtain the risk behavior identification result. This can reduce labor costs and improve work efficiency.

[0141] The specific process of the risk behavior identification method of the present invention is as follows:

[0142] 1. Input the training features of the skeleton key points into the convolutional network of the first image to obtain the spatial features of the skeleton key points.

[0143] 2. The spatial features of the skeleton key points are processed sequentially according to the first convolutional layer, the first compression layer and the first pooling layer in the first compression excitation network architecture to obtain the intermediate data of the first skeleton key points.

[0144] 3. The intermediate data of the first skeleton key points are obtained by sequentially processing the intermediate data of the first skeleton key points according to multiple first convolutional compression layers.

[0145] 4. Obtain the spatiotemporal features of the skeleton key points based on the intermediate data of the second skeleton key points and the spatial features of the skeleton key points.

[0146] 5. Obtain the first behavior prediction result based on the spatiotemporal features of the skeleton key points, and determine the first loss function based on the first behavior prediction result and the corresponding actual behavior.

[0147] 6. When the first loss function converges, determine the first behavior prediction model based on the first graph convolutional network and the first compressed excitation network architecture; otherwise, update the first compressed excitation network architecture based on the first loss function and return to step 1.

[0148] 7. Input the skeleton connection training features into the second convolutional network to obtain the skeleton connection spatial features.

[0149] 8. Process the spatial features of the skeleton connection relationship in sequence according to the second convolutional layer, the second compression layer and the first pooling layer in the second compressed excitation network architecture to obtain the intermediate data of the first skeleton connection relationship.

[0150] 9. The intermediate data of the first skeleton connection relationship is obtained by sequentially processing the intermediate data of the first skeleton connection relationship according to multiple second convolutional compression layers.

[0151] 10. Obtain the spatiotemporal features of the skeleton connection relationship based on the intermediate data of the second skeleton connection relationship and the spatial features of the skeleton connection relationship.

[0152] 11. Obtain the second behavior prediction result based on the spatiotemporal features of the skeleton connection relationship, and determine the second loss function based on the second behavior prediction result and the corresponding actual behavior.

[0153] 12. When the second loss function converges, determine the second behavior prediction model based on the second graph convolutional network and the second compressed activation network architecture; otherwise, update the second compressed activation network architecture based on the second loss function and return to step 7.

[0154] 13. Perform face correction on the faces in the video image, and input the face-corrected video image into the face recognition model to obtain the employee recognition results.

[0155] 14. Obtain the employee video image based on the employee identification result and the video image.

[0156] 15. Extract the skeleton sequence from the employee video images.

[0157] 16. Extract skeleton key point information features and skeleton connection relationship features from the skeleton sequence.

[0158] 17. Input the skeleton key point information features and skeleton connection relationship features into the first behavior prediction model and the second behavior prediction model respectively to obtain the first employee behavior and the second employee behavior respectively.

[0159] 18. Obtain risk behavior identification results based on the first employee's behavior and the second employee's behavior.

[0160] In summary, the risk behavior identification method provided by the embodiments of the present invention has the following beneficial effects:

[0161] (1) This invention integrates and innovates monitoring data and the latest deep learning methods for use in a risk behavior system for bank employees. It can detect camera scenes in the business service area and the office area respectively. It can identify in real time various risk behaviors such as employees operating on behalf of customers for a long time in front of self-service terminals, behavioral conflicts with customers, risk behaviors in the confidential computer room area, or unauthorized visitors tailgating into the confidential area. It can also increase the categories of actions to be detected according to actual applications to improve the accuracy and real-time performance of detection.

[0162] (2) By using this invention, the problem of insufficient development and utilization of monitoring data and the need for dedicated personnel to visually inspect the work mode is solved. The big data + deep learning algorithm reduces the omissions and drawbacks of manual monitoring, realizes a comprehensive, intelligent and efficient monitoring and detection system, and improves employees' awareness of behavior management.

[0163] (3) In constructing the model, this invention selects a relatively new and accurate algorithm from existing research. Considering practical application issues, it uses a new lightweight and embeddable face recognition model to build an employee identity recognition model, replacing the traditional industrial face recognition model algorithm and improving the overall operating efficiency. Replacing the image-based recognition algorithm in existing industrial recognition with a human pose estimation algorithm can avoid interference from factors such as lighting and background, and protect the privacy of customers when they are detected. At the same time, the graph convolutional network 2s-AGCN is improved, and a new attention mechanism network model is designed to embed and replace the TCN module in the 2s-AGCN network to improve the overall recognition effect of the classification model.

[0164] (4) This invention considers the entire process of intelligent assessment. After detecting risky behavior, the video evidence is saved locally. Through automated deployment, risk information is automatically summarized daily according to the hierarchical relationship of branches and outlets, and various related reports are automatically generated. The types of reports can also be increased according to actual applications. An intelligent monitoring and detection interface is designed to present the various types of information generated in summary, completely freeing up manual costs and strengthening the security of customer transactions and information management.

[0165] Based on the same inventive concept, this invention also provides a risk behavior identification device. Since the principle of this device in solving the problem is similar to that of the risk behavior identification method, the implementation of this device can refer to the implementation of the method, and the repeated parts will not be described again.

[0166] Figure 16 This is a structural block diagram of the risk behavior identification device in an embodiment of the present invention. Figure 16 As shown, the risk behavior identification device includes:

[0167] The skeleton sequence extraction module is used to acquire employee video images and extract skeleton sequences from the employee video images;

[0168] The skeleton feature relationship extraction module is used to extract skeleton key point information features and skeleton connection relationship features from the skeleton sequence;

[0169] The behavior prediction module is used to input the skeleton key point information features and skeleton connection relationship features into the first behavior prediction model and the second behavior prediction model respectively, so as to obtain the first employee behavior and the second employee behavior respectively.

[0170] The identification result module is used to obtain risk behavior identification results based on the first employee behavior and the second employee behavior.

[0171] In one embodiment, it further includes:

[0172] The skeleton key point spatiotemporal feature module is used to process the spatial features of the skeleton key points according to the first compression excitation network architecture to obtain the spatiotemporal features of the skeleton key points.

[0173] The first loss function module is used to obtain a first behavior prediction result based on the spatiotemporal features of the skeleton key points, and to determine a first loss function based on the first behavior prediction result and the corresponding actual behavior.

[0174] The first behavior prediction model module is used to determine the first behavior prediction model based on the first graph convolutional network and the first compressed excitation network architecture when the first loss function converges; otherwise, it updates the first compressed excitation network architecture based on the first loss function.

[0175] In one embodiment, the spatiotemporal feature module of the skeleton key points includes:

[0176] The first skeleton key point intermediate data unit is used to process the spatial features of the skeleton key points in sequence according to the first convolutional layer, the first compression layer and the first pooling layer in the first compressed excitation network architecture to obtain the first skeleton key point intermediate data.

[0177] The second skeleton key point intermediate data unit is used to process the first skeleton key point intermediate data sequentially according to multiple first convolutional compression layers to obtain the second skeleton key point intermediate data.

[0178] The skeleton key point spatiotemporal feature unit is used to obtain the skeleton key point spatiotemporal features based on the second skeleton key point intermediate data and the skeleton key point spatial features.

[0179] In one embodiment, it further includes:

[0180] The skeleton connection spatial feature module is used to input the skeleton connection training features into the second graph convolutional network to obtain the skeleton connection spatial features;

[0181] The skeleton connection spatiotemporal feature module is used to process the skeleton connection spatial features according to the second compression excitation network architecture to obtain the skeleton connection spatiotemporal features.

[0182] The second loss function module is used to obtain the second behavior prediction result based on the spatiotemporal features of the skeleton connection relationship, and to determine the second loss function based on the second behavior prediction result and the corresponding actual behavior.

[0183] The second behavior prediction model module is used to determine the second behavior prediction model based on the second graph convolutional network and the second compressed activation network architecture when the second loss function converges; otherwise, it updates the second compressed activation network architecture based on the second loss function.

[0184] In one embodiment, the spatiotemporal feature module of skeleton connection relationship includes:

[0185] The first skeleton connection relationship intermediate data unit is used to process the skeleton connection relationship spatial features sequentially according to the second convolutional layer, the second compression layer and the first pooling layer in the second compressed excitation network architecture to obtain the first skeleton connection relationship intermediate data.

[0186] The second skeleton connection relationship intermediate data unit is used to process the first skeleton connection relationship intermediate data sequentially according to multiple second convolutional compression layers to obtain the second skeleton connection relationship intermediate data.

[0187] The skeleton connection spatiotemporal feature unit is used to obtain the skeleton connection spatiotemporal features based on the intermediate data of the second skeleton connection and the spatial features of the skeleton connection.

[0188] In one embodiment, the skeleton sequence extraction module includes:

[0189] The face correction unit is used to correct faces in video images;

[0190] The employee identification result unit is used to input the face-corrected video image into the face recognition model to obtain the employee identification result;

[0191] An employee video image unit is used to acquire the employee video image based on the employee identification result and the video image.

[0192] Figure 17 This is a structural block diagram of a risk behavior recognition device according to another embodiment of the present invention. Figure 17 As shown, in practical applications, the risk behavior identification device includes an employee identification module, a risk behavior identification module, and a risk information automatic generation and summary module. These modules perform real-time monitoring of surveillance footage captured by cameras in the office area and the business service area of ​​the bank branch. They can identify various risky actions such as employees spending extended periods operating on behalf of customers at self-service terminals, conflicts with customers, risky behaviors in secure computer rooms, or unauthorized visitors tailgating into secure areas. Furthermore, other risk behavior categories can be added as needed. The results are automatically summarized daily to generate various reports, forming a comprehensive, intelligent risk behavior assessment model that replaces manual monitoring with big data and deep learning algorithms, improving work efficiency and preventing problems before they arise.

[0193] The risk behavior recognition device first uses a lightweight facial recognition model, applicable to industrial settings, to detect the identities of individuals appearing in surveillance footage. If the individual is an employee, further detection is performed. Facial alignment algorithms and the latest deep learning models are incorporated into the facial recognition process to improve accuracy in identifying multiple individuals. Secondly, an abnormal risk behavior recognition module identifies surveillance footage from cameras in both the business service area and the office area. To enhance the accuracy of behavior recognition, a human pose estimation algorithm is used to extract the employee's skeletal sequence, avoiding interference from background and lighting factors. A newer dual-stream adaptive graph convolutional network is used for deep mining of the skeletal graph topology, extracting high-level features to improve the overall recognition performance. After identifying risk behaviors, all saved risk behavior video evidence is automatically retrieved daily through automated deployment. All information is stored according to a multi-level relationship between branches and outlets, and various reports are automatically generated and displayed on the intelligent monitoring platform interface. This achieves fully automated, high-precision, and intelligent employee behavior detection and performance recording, reducing manual labor costs and improving work efficiency.

[0194] Figure 18 This is a functional diagram of the employee identification module in an embodiment of the present invention. Figure 18 As shown, the employee identification module includes a skeleton sequence extraction module. If a non-employee is identified in a confidential area of ​​the office, it is considered an unauthorized intruder; otherwise, it is used for subsequent identification. In the business service area, only employee identities are identified for subsequent risk behavior detection. Considering the model's scale, running speed, and detection accuracy, the lightweight Retina Face face detection model is selected to construct the face detection module, and FaceNet is used to construct the face recognition module. Real-time detection of images captured by ordinary cameras is performed to determine whether the person is an employee of the bank and to obtain the employee's identity information. The employee identification module can be divided into three processing tasks: video-based face detection, face alignment, and employee identity comparison.

[0195] Figure 19 This is a functional diagram of the abnormal risk behavior identification module in an embodiment of the present invention. Figure 19 As shown, the abnormal risk behavior recognition module includes a skeleton feature relationship extraction module, a behavior prediction module, a recognition result module, a skeleton key point spatiotemporal feature module, a first loss function module, a first behavior prediction model module, a skeleton connection relationship spatial feature module, a skeleton connection relationship spatiotemporal feature module, a second loss function module, and a second behavior prediction model module. Since the camera acquisition angles differ in different scenarios, this invention sets different feature extraction tasks and parameter settings for camera detection tasks in business service areas and office areas. The overall abnormal risk behavior recognition comprises two main parts: feature extraction based on human pose and behavior recognition based on graph convolutional networks.

[0196] If abnormal risk behavior is detected, the automatic risk information generation and aggregation module can save a one-minute video frame file of the detected data locally, naming the file as "Serial Number-Branch Institution-Employee Name-Employee ID-Risk Behavior Category". By setting up scheduled tasks and deploying automated scripts, the module can automatically summarize and organize the abnormal risk behaviors of the day every evening, generating relevant statistical reports. This achieves full automation of the risk performance evaluation process. Furthermore, using Robotic Process Automation (RPA), the module automatically sends the Excel file and statistical reports to the designated staff's email address, achieving low coding and high operational efficiency.

[0197] In summary, the risk behavior recognition device of this invention first extracts skeleton sequences from employee video images to obtain skeleton key point information features and skeleton connection relationship features. Then, it inputs the skeleton key point information features and skeleton connection relationship features into the behavior prediction model to obtain the corresponding employee behavior and obtain the risk behavior recognition result. This can reduce labor costs and improve work efficiency.

[0198] Figure 20 This is a schematic block diagram illustrating the system configuration of the electronic device 9600 according to an embodiment of this application. Figure 20 As shown, the electronic device 9600 may include a central processing unit 9100 and a memory 9140; the memory 9140 is coupled to the central processing unit 9100. It is worth noting that... Figure 20 This is an example; other types of structures can also be used to supplement or replace this structure to achieve telecommunications functions or other functions.

[0199] In one embodiment, the risk behavior identification method functionality can be integrated into the central processing unit 9100. The central processing unit 9100 can be configured to perform the following controls:

[0200] Acquire employee video images and extract skeleton sequences from the employee video images;

[0201] Extract skeleton key point information features and skeleton connection relationship features from the skeleton sequence;

[0202] The skeleton key point information features and skeleton connection relationship features are respectively input into the first behavior prediction model and the second behavior prediction model to obtain the first employee behavior and the second employee behavior, respectively.

[0203] Risk behavior identification results are obtained based on the first employee's behavior and the second employee's behavior.

[0204] As can be seen from the above description, the risk behavior identification method provided in this application first extracts skeleton sequences from employee video images to obtain skeleton key point information features and skeleton connection relationship features. Then, the skeleton key point information features and skeleton connection relationship features are respectively input into the behavior prediction model to obtain the corresponding employee behavior and obtain the risk behavior identification results. This can reduce labor costs and improve work efficiency.

[0205] In another embodiment, the risk behavior recognition device can be configured separately from the central processing unit 9100. For example, the risk behavior recognition device can be configured as a chip connected to the central processing unit 9100, and the function of the risk behavior recognition method can be realized through the control of the central processing unit.

[0206] like Figure 20 As shown, the electronic device 9600 may further include: a communication module 9110, an input unit 9120, an audio processor 9130, a display 9160, and a power supply 9170. It is worth noting that the electronic device 9600 does not necessarily need to include these components. Figure 20 All components shown; in addition, the electronic device 9600 may also include Figure 20 For components not shown, please refer to existing technologies.

[0207] like Figure 20 As shown, the central processing unit 9100, sometimes also referred to as a controller or operating control, may include a microprocessor or other processor device and / or logic device, which receives inputs and controls the operation of various components of the electronic device 9600.

[0208] The memory 9140 may be, for example, one or more of a cache, flash memory, hard drive, removable media, volatile memory, non-volatile memory, or other suitable devices. It may store the aforementioned failure-related information, and also store a program for executing that information. The central processing unit 9100 may execute the program stored in the memory 9140 to perform information storage or processing, etc.

[0209] Input unit 9120 provides input to central processing unit 9100. Input unit 9120 may be, for example, a keypad or touch input device. Power supply 9170 provides power to electronic device 9600. Display 9160 displays images and text. Display may be, for example, an LCD display, but is not limited thereto.

[0210] The memory 9140 can be a solid-state memory, such as a read-only memory (ROM), random access memory (RAM), a SIM card, etc. It can also be a memory that retains information even when power is off, can be selectively erased, and contains more data; examples of this type of memory are sometimes referred to as EPROMs. The memory 9140 can also be some other type of device. The memory 9140 includes a buffer 9141 (sometimes referred to as a buffer memory). The memory 9140 may include an application / function storage unit 9142 for storing application programs and function programs or processes for executing the operation of the electronic device 9600 via the central processing unit 9100.

[0211] The memory 9140 may also include a data storage unit 9143 for storing data, such as contacts, digital data, pictures, sounds, and / or any other data used by the electronic device. The driver storage unit 9144 of the memory 9140 may include various drivers for the electronic device's communication functions and / or for performing other functions of the electronic device (such as messaging applications, address book applications, etc.).

[0212] The communication module 9110 is a transmitter / receiver 9110 that transmits and receives signals via the antenna 9111. The communication module (transmitter / receiver) 9110 is coupled to the central processing unit 9100 to provide input signals and receive output signals, which can be the same as in a conventional mobile communication terminal.

[0213] Based on different communication technologies, multiple communication modules 9110 can be configured in the same electronic device, such as cellular network modules, Bluetooth modules, and / or wireless LAN modules. The communication module (transmitter / receiver) 9110 is also coupled to a speaker 9131 and a microphone 9132 via an audio processor 9130 to provide audio output via the speaker 9131 and receive audio input from the microphone 9132, thereby realizing typical telecommunications functions. The audio processor 9130 may include any suitable buffer, decoder, amplifier, etc. Additionally, the audio processor 9130 is coupled to a central processing unit 9100, enabling on-device recording via the microphone 9132 and on-device playback of stored sound via the speaker 9131.

[0214] This invention also provides a computer-readable storage medium capable of implementing all steps of the risk behavior identification method in the above embodiments, wherein the execution subject is a server or a client. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements all steps of the risk behavior identification method in the above embodiments. For example, when the processor executes the computer program, it implements the following steps:

[0215] Acquire employee video images and extract skeleton sequences from the employee video images;

[0216] Extract skeleton key point information features and skeleton connection relationship features from the skeleton sequence;

[0217] The skeleton key point information features and skeleton connection relationship features are respectively input into the first behavior prediction model and the second behavior prediction model to obtain the first employee behavior and the second employee behavior, respectively.

[0218] Risk behavior identification results are obtained based on the first employee's behavior and the second employee's behavior.

[0219] In summary, the computer-readable storage medium of this invention first extracts skeleton sequences from employee video images to obtain skeleton key point information features and skeleton connection relationship features. Then, it inputs the skeleton key point information features and skeleton connection relationship features into the behavior prediction model to obtain the corresponding employee behavior and obtain risk behavior identification results. This can reduce labor costs and improve work efficiency.

[0220] This invention also provides a computer program product capable of implementing all steps of the risk behavior identification method in the above embodiments, where the execution subject is a server or a client. The computer program product includes a computer program / instruction, which, when executed by a processor, implements all steps of the risk behavior identification method in the above embodiments. For example, when the processor executes the computer program, it implements the following steps:

[0221] Acquire employee video images and extract skeleton sequences from the employee video images;

[0222] Extract skeleton key point information features and skeleton connection relationship features from the skeleton sequence;

[0223] The skeleton key point information features and skeleton connection relationship features are respectively input into the first behavior prediction model and the second behavior prediction model to obtain the first employee behavior and the second employee behavior, respectively.

[0224] Risk behavior identification results are obtained based on the first employee's behavior and the second employee's behavior.

[0225] In summary, the computer program product of this invention first extracts skeleton sequences from employee video images to obtain skeleton key point information features and skeleton connection relationship features. Then, it inputs the skeleton key point information features and skeleton connection relationship features into the behavior prediction model to obtain the corresponding employee behavior and obtain risk behavior identification results. This can reduce labor costs and improve work efficiency.

[0226] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. In particular, hardware + program embodiments are relatively simple in description because they are fundamentally similar to method embodiments; relevant parts can be referred to the descriptions in the method embodiments.

[0227] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0228] While this application provides the method operation steps as described in the embodiments or flowcharts, more or fewer operation steps may be included based on conventional or non-inventive labor. The order of steps listed in the embodiments is merely one possible execution order among many and does not represent the only execution order. In actual device or client product execution, the methods shown in the embodiments or drawings can be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment).

[0229] While this specification provides method operation steps as described in the embodiments or flowcharts, more or fewer operation steps may be included based on conventional or non-inventive means. The order of steps listed in the embodiments is merely one possible execution order among many and does not represent the only execution order. In actual device or end product execution, the methods shown in the embodiments or drawings may be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment, or even a distributed data processing environment). The terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, product, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, product, or apparatus. Without further limitations, the presence of other identical or equivalent elements in the process, method, product, or apparatus that includes said elements is not excluded.

[0230] For ease of description, the above devices are described in terms of function, divided into various modules. Of course, in implementing the embodiments of this specification, the functions of each module can be implemented in one or more software and / or hardware components, or a module that performs the same function can be implemented by a combination of multiple sub-modules or sub-units. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between devices or units, and may be electrical, mechanical, or other forms.

[0231] Those skilled in the art will also know that, besides implementing the controller using purely computer-readable program code, the same functions can be achieved by logically programming the method steps, making the controller function as logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers (PLCs), and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the devices within it used to implement various functions can also be considered structures within that hardware component. Alternatively, the devices used to implement various functions can be considered as both software modules implementing the method and structures within a hardware component.

[0232] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0233] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0234] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0235] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0236] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0237] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0238] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, the embodiments of this specification can take the form of computer program products implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0239] The embodiments described in this specification can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. The embodiments of this specification can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0240] The embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on its differences from other embodiments. In particular, system embodiments are basically similar to method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. In the description of this specification, the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the embodiments in this specification. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described can be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0241] The above description is merely an embodiment of the present specification and is not intended to limit the embodiments of the present specification. For those skilled in the art, various modifications and variations can be made to the embodiments of the present specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of the embodiments of the present specification should be included within the scope of the claims of the embodiments of the present specification.

Claims

1. A method for identifying risky behaviors, characterized in that, include: Acquire employee video images and extract skeleton sequences from the employee video images; Extract skeleton key point information features and skeleton connection relationship features from the skeleton sequence; The skeleton key point information features and skeleton connection relationship features are respectively input into the first behavior prediction model and the second behavior prediction model to obtain the first employee behavior and the second employee behavior, respectively. Risk behavior identification results are obtained based on the first employee's behavior and the second employee's behavior; The steps to create a second behavior prediction model include: Perform the following iterative processing: The skeleton connection training features are input into the second graph convolutional network to obtain the skeleton connection spatial features; The spatial features of the skeleton connection relationship are processed according to the second compression excitation network architecture to obtain the spatiotemporal features of the skeleton connection relationship. The second behavior prediction result is obtained based on the spatiotemporal features of the skeleton connection relationship, and the second loss function is determined based on the second behavior prediction result and the corresponding actual behavior. When the second loss function converges, the second behavior prediction model is determined based on the second graph convolutional network and the second compressed activation network architecture; otherwise, the second compressed activation network architecture is updated based on the second loss function.

2. The risk behavior identification method according to claim 1, characterized in that, The steps to create a first-behavior prediction model include: Perform the following iterative processing: The training features of the skeleton keypoints are input into the convolutional network of the first image to obtain the spatial features of the skeleton keypoints. The spatial features of the skeleton key points are processed according to the first compression excitation network architecture to obtain the spatiotemporal features of the skeleton key points. A first behavior prediction result is obtained based on the spatiotemporal features of the skeleton key points, and a first loss function is determined based on the first behavior prediction result and the corresponding actual behavior. When the first loss function converges, the first behavior prediction model is determined based on the first graph convolutional network and the first compressed excitation network architecture; otherwise, the first compressed excitation network architecture is updated based on the first loss function.

3. The risk behavior identification method according to claim 2, characterized in that, The spatial features of the skeleton key points are processed according to the first compression excitation network architecture to obtain the spatiotemporal features of the skeleton key points, including: The spatial features of the skeleton key points are processed sequentially according to the first convolutional layer, the first compression layer and the first pooling layer in the first compression excitation network architecture to obtain intermediate data of the first skeleton key points. The intermediate data of the second skeleton key points are obtained by sequentially processing the intermediate data of the first skeleton key points using multiple first convolutional compression layers. The spatiotemporal features of the skeleton key points are obtained based on the intermediate data of the second skeleton key points and the spatial features of the skeleton key points.

4. The risk behavior identification method according to claim 1, characterized in that, The spatial features of the skeleton connection relationships are processed according to the second compression excitation network architecture to obtain the spatiotemporal features of the skeleton connection relationships, including: The skeleton connection spatial features are processed sequentially according to the second convolutional layer, the second compression layer and the first pooling layer in the second compressed excitation network architecture to obtain the first skeleton connection intermediate data. The intermediate data of the second skeleton connection relationship is obtained by sequentially processing the intermediate data of the first skeleton connection relationship using multiple second convolutional compression layers; The spatiotemporal features of the skeleton connection relationship are obtained based on the intermediate data of the second skeleton connection relationship and the spatial features of the skeleton connection relationship.

5. The risk behavior identification method according to claim 1, characterized in that, The acquisition of employee video images includes: Perform face correction on faces in video images; The video image after face correction is input into the face recognition model to obtain the employee recognition result; The employee video image is obtained based on the employee identification result and the video image.

6. A risk behavior identification device, characterized in that, include: The skeleton sequence extraction module is used to acquire employee video images and extract skeleton sequences from the employee video images; The skeleton feature relationship extraction module is used to extract skeleton key point information features and skeleton connection relationship features from the skeleton sequence. The behavior prediction module is used to input the skeleton key point information features and skeleton connection relationship features into the first behavior prediction model and the second behavior prediction model respectively, so as to obtain the first employee behavior and the second employee behavior respectively. The identification result module is used to obtain risk behavior identification results based on the first employee behavior and the second employee behavior. Also includes: The skeleton connection spatial feature module is used to input the skeleton connection training features into the second graph convolutional network to obtain the skeleton connection spatial features; The skeleton connection spatiotemporal feature module is used to process the skeleton connection spatial features according to the second compression excitation network architecture to obtain the skeleton connection spatiotemporal features. The second loss function module is used to obtain the second behavior prediction result based on the spatiotemporal features of the skeleton connection relationship, and to determine the second loss function based on the second behavior prediction result and the corresponding actual behavior. The second behavior prediction model module is used to determine the second behavior prediction model based on the second graph convolutional network and the second compressed activation network architecture when the second loss function converges; otherwise, it updates the second compressed activation network architecture based on the second loss function.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the risk behavior identification method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the risk behavior identification method according to any one of claims 1 to 5.

9. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the risk behavior identification method according to any one of claims 1 to 5.

Images