Network point employee risk identification method, device, equipment, medium and program product

Abstract

The embodiment of the application provides a kind of network point staff risk identification method, device, equipment, medium and program product, it is related to artificial intelligence field.The method comprises: obtaining the target video data of network point staff that target camera of network point deployment collects, and obtaining the target audio data that target microphone of network point deployment collects;Target video data and target audio data are input into pre-trained risk identification model, and first identification result is obtained;If first identification result does not satisfy preset abnormal condition, then first identification result, target video data and target audio data are uploaded to cloud node, to make cloud node according to first identification result, target video data and target audio data, generate second identification result;Risk early warning is carried out based on second identification result.Through cloud edge collaborative technology, multi-level risk identification to staff behavior is realized, improve the identification efficiency, also guarantee the accuracy of identification result.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and in particular to a method, apparatus, equipment, medium, and program product for identifying risks of branch employees. Background Technology

[0002] As banks deepen their digital transformation, their branches are generating massive amounts of audio and video data. This data covers interactions between employees and customers, as well as employee actions during financial transactions. This audio and video data allows for the review of employee behavior within branches, preventing risks to the bank from unusual activity.

[0003] Currently, employee behavior reviews primarily rely on manual review or simple rule-based filtering. However, for manual review, each branch may generate a large amount of audio and video data daily, making manual review extremely inefficient and prone to overlooking key risk points, resulting in low accuracy. Simple rule-based filtering, on the other hand, struggles to comprehensively cover abnormal behavior, easily leading to misjudgments or omissions. Summary of the Invention

[0004] This application provides a method, device, equipment, medium, and program product for identifying risks of employees at business outlets. It acquires multimodal video and audio data from business outlets and then performs hierarchical identification through cloud-edge collaboration technology. This enables low-latency identification at edge nodes, ensuring that high-risk behaviors are detected in real time and improving the immediacy of risk identification. At the same time, secondary analysis is performed at cloud nodes to further ensure the accuracy of identification and improve the comprehensiveness and reliability of risk identification.

[0005] Firstly, this application provides a method for identifying risks of branch employees, applied to edge nodes deployed in various branches within a cloud-edge collaborative system. The cloud-edge collaborative system also includes cloud nodes. The method includes:

[0006] Acquire target video data of branch employees collected by target cameras deployed at the branch, and acquire target audio data collected by target microphones deployed at the branch;

[0007] The target video data and target audio data are input into a pre-trained risk identification model to obtain the first identification result;

[0008] If the first recognition result does not meet the preset abnormal conditions, the first recognition result, the target video data, and the target audio data are uploaded to the cloud node so that the cloud node can generate a second recognition result based on the first recognition result, the target video data, and the target audio data.

[0009] Risk warnings are issued based on the second identification result.

[0010] Secondly, this application provides a method for identifying risks of branch employees, applied to cloud nodes in a cloud-edge collaborative system. The cloud-edge collaborative system also includes edge nodes deployed at each branch. The method includes:

[0011] In response to the first identification result, target video data, and target audio data sent by the edge node, the first identification result, target video data, and target audio data are input into a pre-trained cross-node identification model, so that the cross-node identification model determines the second identification result based on the features of video data and audio data from multiple pre-learned edge nodes;

[0012] The second identification result is sent to the edge node.

[0013] Thirdly, this application provides a risk identification device for branch employees, applied to edge nodes deployed in various branches within a cloud-edge collaborative system. The cloud-edge collaborative system also includes cloud nodes, and the device further includes:

[0014] The acquisition module is used to acquire target video data of employees at the network outlets collected by target cameras deployed at the network outlets, and target audio data collected by target microphones deployed at the network outlets.

[0015] The first identification result determination module is used to input the target video data and target audio data into a pre-trained risk identification model to obtain the first identification result;

[0016] The second identification result determination module is used to upload the first identification result, target video data and target audio data to the cloud node if the first identification result does not meet the preset abnormal conditions, so that the cloud node can generate a second identification result based on the first identification result, target video data and target audio data.

[0017] The early warning module is used to issue risk warnings based on the second identification result.

[0018] Fourthly, this application provides a risk identification device for branch employees, applied to a cloud node in a cloud-edge collaborative system. The cloud-edge collaborative system also includes edge nodes deployed at each branch. The device includes:

[0019] The second identification result determination module is used to respond to the first identification result, target video data and target audio data sent by the edge node, and input the first identification result, target video data and target audio data into the pre-trained cross-network node identification model, so that the cross-network node identification model determines the second identification result based on the features of the video data and audio data of multiple edge nodes that have been pre-learned;

[0020] The communication module is used to send the second identification result to the edge node.

[0021] Fifthly, this application provides an electronic device, including: a processor, and a memory communicatively connected to the processor;

[0022] The memory stores the instructions that the computer executes;

[0023] The processor executes computer-executable instructions stored in memory to implement the methods provided in the first and second aspects above.

[0024] In a sixth aspect, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the methods provided in the first and second aspects above.

[0025] In a seventh aspect, this application provides a computer program product, including a computer program that, when executed by a processor, implements the methods provided in the first and second aspects above.

[0026] The branch employee risk identification method, device, equipment, medium, and program products provided in this application acquire video and audio data from branch locations through edge nodes in a cloud-edge collaborative system. Machine learning models are used to mine hidden risk behaviors within the video and audio data, outputting a first identification result. By combining video and audio data, cross-modal hidden risks that cannot be detected by a single dimension can be discovered, enhancing the comprehensiveness of risk identification. Then, when the edge node's identification result indicates that no risk behavior has been detected, the identification result, video data, and audio data are uploaded to the cloud node of the cloud-edge collaborative system. The cloud node performs secondary identification to obtain a second identification result, and an early warning is issued based on the risk situation indicated by the second identification result. This application achieves multi-level risk identification based on cloud-edge collaborative technology. Edge nodes quickly complete the initial risk identification, reducing response time and dependence on cloud nodes, resulting in higher identification efficiency. The cloud node is used to achieve more accurate risk identification, supplementing and verifying the identification results of the edge nodes, effectively avoiding missed detections of abnormal behaviors, making the overall risk identification more accurate and reliable. Attached Figure Description

[0027] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0028] Figure 1 A flowchart illustrating a method for identifying risks among branch employees, provided as an embodiment of this application;

[0029] Figure 2 This is a schematic diagram of the structure of a cloud-edge collaborative system provided in an embodiment of this application;

[0030] Figure 3A schematic diagram of the hardware devices installed at each branch office as provided in the embodiments of this application;

[0031] Figure 4 A flowchart illustrating another method for identifying risks among branch employees provided in this application embodiment;

[0032] Figure 5 A flowchart illustrating another method for identifying risks among branch employees provided in this application embodiment;

[0033] Figure 6 A schematic diagram of the overall architecture of the system provided in the embodiments of this application;

[0034] Figure 7 A schematic diagram of the structure of a branch employee risk identification device provided in this application embodiment;

[0035] Figure 8 A schematic diagram of another branch employee risk identification device provided in this application embodiment;

[0036] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0037] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0038] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0039] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of the relevant data all comply with the relevant laws, regulations, and standards of the relevant countries and regions, have taken necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation access points for users to choose to authorize or refuse.

[0040] Furthermore, the technical solution involved in this application, which involves big data analysis of user information (including but not limited to personal biometrics, identity data, consumption data, asset data, electronic terminal operation data, etc.) and the use of artificial intelligence technology for automated decision-making, and makes decisions that have a significant impact on personal rights based on the results of automated decision-making, provides users with corresponding operation entry points for users to choose to agree to or reject the results of automated decision-making; if the user chooses to reject, the process will proceed to the expert decision-making process.

[0041] It should be noted that the branch employee risk identification method, device, equipment, medium and program products provided in this application can be used in the field of artificial intelligence, or in any field other than artificial intelligence. The application field of the branch employee risk identification method, device, equipment, medium and program products in this application is not limited.

[0042] As banks and other financial institutions deepen their digital transformation, data collection devices, such as cameras and microphones, are typically deployed in their branches. These devices collect audio and video data from each branch, which can then be used to monitor behavior within the branch and ensure branch security.

[0043] This application is applied to scenarios involving the review of employee behavior in banks and other financial institutions. Specifically, it involves collecting audio and video data from branch offices using data collection devices, identifying employee behavior within the data, including interactions between employees and customers and employee actions during business transactions. When abnormal behavior is detected, an alert is issued to prevent such behavior from posing a risk to the branch office.

[0044] Currently, employee behavior review mainly relies on manual review or expert rule screening. This involves monitoring data collected by relevant staff through monitoring equipment, and manually triggering an alarm when relevant staff discover abnormal behavior. Alternatively, it involves identifying the behavior of people in video data, and if it matches the set static abnormal behavior rules, such as destroying documents or privately transferring large amounts of cash, an alert is triggered.

[0045] However, manual review requires staff to monitor and collect data for extended periods, resulting in high labor costs, poor timeliness, and susceptibility to factors such as attention deficit, leading to missed risky behaviors and low accuracy, thus failing to meet review requirements. As for expert rule-based screening, the rules are insufficient to comprehensively cover abnormal behaviors, easily leading to missed or misjudged cases, especially novel abnormal behaviors, making it difficult to guarantee review accuracy.

[0046] The branch employee risk identification method provided in this application aims to solve the above-mentioned technical problems. Specifically, through a distributed architecture that combines edge computing and cloud collaboration, and by combining multimodal video and audio data, the powerful learning and reasoning capabilities of machine learning models are utilized at the edge nodes to perform preliminary risk identification, breaking through the limitations of a single data dimension and improving the comprehensiveness of risk identification. When no abnormal behavior is identified, the identification results, video data, and audio data are uploaded to the cloud nodes for secondary analysis, supplementing and verifying the identification results from the edge nodes. This reduces the risk of missed or false positives, enabling multi-level branch employee risk identification and making risk identification more accurate.

[0047] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0048] Figure 1 This is a flowchart illustrating a method for identifying risks among branch employees, provided as an embodiment of this application. Figure 1 As shown, the method provided in this embodiment includes the following steps:

[0049] Step S101: Obtain target video data of branch employees collected by target cameras deployed at the branch, and obtain target audio data collected by target microphones deployed at the branch.

[0050] The method provided in this application is applied to edge nodes deployed at various network points in a cloud-edge collaborative system, which also includes cloud nodes.

[0051] Figure 2 This is a schematic diagram of the structure of a cloud-edge collaborative system provided in an embodiment of this application. Figure 2 As shown, the cloud-edge collaborative system provided in this embodiment consists of a distributed architecture composed of cloud nodes and multiple edge nodes. Each edge node is deployed based on proximity, located in the branches of various banks or financial institutions. Cloud nodes can be flexibly deployed in core hub locations, such as the data center of the main branch. Cloud nodes connect to all edge nodes via a high-speed network, enabling high-speed data transmission and command delivery.

[0052] Edge nodes are computing devices deployed locally at network sites to perform risk identification on audio and video data locally, thereby reducing response time, improving identification efficiency, and alleviating the computing pressure on cloud nodes. Edge nodes include integrated smart terminals (also known as edge AI (Artificial Intelligence) all-in-one machines) and network video recorders (NVRs).

[0053] For example, a network video recorder is used to receive and store audio and video data captured by a target microphone and a target camera. The audio and video data are then transmitted to an integrated smart terminal, which processes and analyzes the data.

[0054] Cloud nodes are computing platforms deployed in the cloud. They can aggregate data from various edge nodes using a federated learning framework and identify employee risks based on this data. Cloud nodes consist of GPU (Graphics Processing Unit) server clusters and NPU (Neural Processing Unit) server clusters. These GPU and NPU server clusters enable cloud nodes to process multimodal data.

[0055] A branch is a branch or service point established by banks and other financial institutions to facilitate business operations for customers in various regions. A target camera is a camera deployed at the branch to collect video data of employees within the branch. A target microphone is a microphone deployed at the branch to collect audio data of employees within the branch.

[0056] It should be noted that the outlets in this employee risk identification method must meet the following conditions: they must have a network of digital video surveillance equipment, that is, support the deployment of cameras and microphones; they must have basic network communication infrastructure, that is, support the deployment of cloud-edge collaborative systems; and they must have compliant video and audio data processing permissions, that is, employees and users are allowed to take photos or record audio, and are allowed to perform subsequent processing on the collected data.

[0057] In one example, both the target camera and the target microphone are used to collect video and audio data from branch employees. The target camera and target microphone can be placed in the areas where branch employees are active, such as the counter office area, the back office area, or the business processing area.

[0058] In this embodiment, the positions and angles of target cameras and target microphones can be deployed for specific areas to be monitored, taking into account the corresponding business needs of the area.

[0059] The target camera and target microphone can communicate with the edge node of the network point and send the collected target video data and target audio data to the edge node through the Internet of Things protocol.

[0060] In this step, the edge node of the branch communicates with the target camera to acquire the target video data of the branch employees captured by the target camera. The edge node also communicates with the target microphone to acquire the target audio data of the branch employees captured by the target microphone.

[0061] In some embodiments, the edge node can acquire video stream data of branch employees captured by the target camera, and extract the video stream data within a preset time window to obtain target video data. The edge node can also acquire audio stream data of branch employees captured by the target microphone, and extract the audio stream data within a preset time window to obtain target audio data.

[0062] The preset time window can be a pre-set configurable parameter, such as 1 minute, 1 hour, etc.

[0063] The preset time window can also be dynamically changed based on the video stream data and audio stream data.

[0064] For example, a preset time window is determined based on the duration of an employee's appearance in the video stream data. If employee 1 appears in the video stream data for 15 seconds, the preset time window is determined to be 15 seconds, and target video data within 15 seconds from the moment employee 1 appears in the video stream data, as well as target audio data within the corresponding time window in the audio stream data, are extracted.

[0065] For example, a preset time window is determined based on the recorded conversation time of branch employees in the audio stream data. If employee 2's conversation in the audio stream data lasts for 20 seconds, then the preset time window is determined to be 20 seconds. Target audio data within 20 seconds from the start of employee 2's conversation in the audio stream data is extracted, as well as target video data within the corresponding time window in the video stream data.

[0066] In some embodiments, after acquiring target video data of branch employees collected by target cameras deployed at the branch and target audio data collected by target microphones deployed at the branch, the video data and audio data can be processed, including but not limited to encrypting the target video data and target audio data to ensure data privacy and security; cleaning, encoding standardization, noise suppression, or frame processing optimization of the target video data and target audio data to improve the quality of the video data and audio data.

[0067] For example, after acquiring target video data of employees at the branch office collected by target cameras deployed at the branch office, and target audio data collected by target microphones deployed at the branch office, the quality of the video data and audio data can be assessed, such as determining the resolution and frame rate, and removing low-quality video data or audio data.

[0068] Step S102: Input the target video data and target audio data into the pre-trained risk identification model to obtain the first identification result.

[0069] The risk identification model is a type of machine learning model. This application's risk identification model integrates multiple seed machine learning models, enabling the identification of cross-modal data.

[0070] For example, a risk identification model can integrate a target identification model, a behavior identification model, and a speech recognition model. The target identification model is used to identify branch employees and electronic devices (such as mobile phones, computers, etc.). The behavior identification model is used to identify the behaviors of branch employees (such as tearing up receipts, screen recording, taking photos, etc.). The speech recognition model is used to identify keywords in the conversations of branch employees.

[0071] By training a risk identification model, the model can output identification results based on input video and audio data. The training method involves acquiring historically stored video and audio data and labeling them. This includes labeling targets in the video data (e.g., whether employees or electronic devices are present), actions in the video data, and keywords in the audio data. The historically stored video and audio data are then divided into training samples, validation samples, and test samples for training the risk identification model.

[0072] In some embodiments, the sub-models in the risk identification model can be trained separately. Specifically, a target identification model is trained using targets in video data stored over a historical period and labeled video data. A behavior identification model is trained using behaviors in video data stored over a historical period and labeled video data. A speech recognition model is trained using keywords in audio data stored over a historical period and labeled audio data.

[0073] After the risk identification model is trained, it is deployed on edge nodes. Specifically, model inference optimization tools and containerization tools can be used to optimize the performance of the risk identification model and package it into an edge container image. Through the IoT intelligent edge computing system, the edge container image is deployed to the edge nodes of the network points, thus realizing the deployment of the risk identification model.

[0074] In some embodiments, to achieve low latency, the risk identification model can be trimmed and quantized to generate a lightweight risk identification model, which can then be deployed on edge nodes.

[0075] When deploying lightweight risk identification models at edge nodes, the model structure can be dynamically adjusted based on real-time computing load (such as pruning the number of neural network layers or channels) to balance processing efficiency and model accuracy.

[0076] Specifically, the number of parameters in the risk identification model is reduced through model quantization and channel pruning (removing low-weight channels). Furthermore, during the risk identification model's operation, the CPU utilization of edge nodes is monitored in real time. When the load exceeds a preset threshold, the risk identification model is automatically pruned; when the load decreases, the complete risk identification model is restored.

[0077] In addition, dynamic switching of risk identification model versions can be achieved through model inference optimization tools or machine learning inference engines to ensure that the risk identification model can still work effectively after being pruned.

[0078] In this embodiment, dynamic pruning reduces the computational load of the risk identification model, lowers the response latency of edge nodes, and ensures real-time risk identification even under high load scenarios. Furthermore, by removing low-weight channels, the pruned risk identification model can still accurately and effectively identify abnormal behavior. Simultaneously, dynamically adjusting the risk identification model avoids wasting computational resources at edge nodes.

[0079] By training the risk identification model, it can learn the features of targets, behaviors, and keywords. When target video and audio data are input into the risk identification model, the model matches the learned features with the target video and audio data, outputting a first identification result.

[0080] Specifically, target video and audio data are input into the risk identification model. The target identification model, based on the target video data, identifies branch staff and electronic devices within the video. The behavior identification model, based on the target video data and the identified branch staff and electronic devices, identifies the behavior of branch staff, including interactions between branch staff and electronic devices, as well as the branch staff's own actions. The speech recognition model, based on the audio data, identifies keywords in the conversation. The output of each model is then used as the primary identification result.

[0081] Step S103: If the first recognition result does not meet the preset abnormal conditions, the first recognition result, the target video data, and the target audio data are uploaded to the cloud node so that the cloud node can generate a second recognition result based on the first recognition result, the target video data, and the target audio data.

[0082] The preset abnormal conditions are pre-set criteria for judging abnormal behavior, including the first identification result indicating that the behavior of the branch employee is abnormal (such as not wearing a name tag, tearing up receipts, private cash transactions, and recording the screen of office equipment, etc.), and the first identification result indicating that the identified keywords include sensitive words (such as forgery).

[0083] In this step, if the first recognition result does not meet the preset anomaly conditions, it means that the edge node's recognition result shows no abnormal behavior. To avoid missed detections, the first recognition result, target video data, and target audio data can be uploaded to the cloud node. The cloud node performs secondary recognition to generate a second recognition result.

[0084] Optionally, if the first identification result meets the preset abnormal conditions, a risk warning will be issued.

[0085] Step S104: Based on the second identification result, a risk warning is issued.

[0086] In this step, if the second identification result meets the preset abnormal conditions, it indicates that there is abnormal behavior, and a risk warning is issued.

[0087] Specifically, if the second identification result meets the preset abnormal conditions, risk warnings can be issued through branch voice prompts, linkage with access control and alarm lights.

[0088] In some embodiments, if the second identification result meets the preset abnormal conditions, the warning method can be determined according to the risk level corresponding to the abnormal behavior, and then the risk warning can be carried out through the warning method.

[0089] Risk level is a quantitative classification of the degree of risk of abnormal behavior, used to distinguish the extent of harm that different abnormal behaviors cause to the network.

[0090] The risk levels corresponding to various abnormal behaviors can be preset, and the warning methods corresponding to each risk level can be set.

[0091] For example, the risk level of not wearing an employee badge is set to low risk, and the corresponding warning method for low risk is simply to record the abnormal behavior in the edge node's log. The risk level of recording the screen of office equipment is set to medium risk, and the corresponding warning method for medium risk is to provide on-site voice prompts and record the abnormal behavior in the edge node's log. The abnormal behaviors of tearing up receipts and engaging in private cash transactions are set to high risk, and the corresponding warning method for high risk is to restrict their access control privileges and trigger an alarm, and record the abnormal behavior in the edge node's log.

[0092] If the second identification result does not meet the preset abnormal conditions, it indicates that there is no abnormal behavior, and no risk warning is required. The second identification result, target video data, and target audio data can be saved to the edge node and destroyed after a preset time (e.g., 1 week).

[0093] In some embodiments, after issuing a risk warning based on the second identification result, the method further includes sending the identification result to internal control and compliance personnel so that they can verify the identification result, promptly detect and handle abnormal behavior, and reduce branch risks.

[0094] The branch employee risk identification method provided in this application acquires video and audio data from branch locations through edge nodes in a cloud-edge collaborative system. It then uses a machine learning model to mine hidden risk behaviors within the video and audio data, outputting a first identification result. By combining video and audio data, it can discover cross-modal hidden risks that cannot be detected by a single dimension, enhancing the comprehensiveness of risk identification. Then, when the edge node's identification result indicates that no risk behavior has been detected, the identification result, video data, and audio data are uploaded to the cloud node of the cloud-edge collaborative system. The cloud node performs secondary identification to obtain a second identification result, and an early warning is issued based on the risk situation indicated by the second identification result. This application achieves multi-level risk identification based on cloud-edge collaborative technology. Edge nodes quickly complete the initial risk identification, reducing response time and dependence on cloud nodes, resulting in higher identification efficiency. The cloud node is used to achieve more accurate risk identification, supplementing and verifying the identification results of the edge nodes, effectively avoiding missed detections of abnormal behaviors, making the overall risk identification more accurate and reliable.

[0095] Figure 3 A schematic diagram of the deployed hardware device provided in an embodiment of this application. For example... Figure 3 As shown, this scenario includes the head office side, the branch side, and the branch branch's sub-branch side. The sub-branch side includes sub-branch 1 and sub-branch 2.

[0096] The branch office side is equipped with target cameras, target microphones, and edge nodes. The sub-branch side is equipped with cloud nodes, which consist of a server cluster.

[0097] Specifically, the target camera at the service point collects target video data from the service point employees, and the target microphone collects target audio data. The target camera and target microphone are connected to the edge node to transmit the collected target video data and target audio data to the edge node.

[0098] The edge node inputs the target video data and target audio data into a pre-trained risk identification model to obtain the first identification result; if the first identification result does not meet the preset abnormal conditions, the first identification result, the target video data, and the target audio data are uploaded to the cloud node.

[0099] The cloud node generates a second recognition result based on the first recognition result, target video data, and target audio data uploaded by the edge node, and sends the second recognition result to the edge node.

[0100] The edge node receives the second identification result and performs risk warning based on the second identification result.

[0101] In some embodiments, the hardware devices installed at each branch also include an application front-end set up on the head office side. The application front-end is used to display the identification results (including the second identification result and the first identification result). Internal control and compliance personnel obtain the first identification result and the second identification result from the application front-end, verify the identification results, promptly detect and handle abnormal behavior, and reduce branch risks.

[0102] In some embodiments, the parameters of the risk identification model may be adjusted based on the verification results of internal control and compliance personnel.

[0103] Specifically, the edge node receives feedback data from compliance personnel on the initial identification results and generates a feedback training set; based on the feedback training set, the risk identification model is fine-tuned to generate optimized model parameters.

[0104] In one possible implementation, maintaining the risk identification model includes training the risk identification model, evaluating the risk identification model, and retraining the risk identification model.

[0105] The training method for the risk identification model is as follows: First, training samples are collected, specifically historical video and audio data, which are then used to construct a training set. Next, video and audio annotation tools are used to annotate the training samples, labeling their behavior and semantics. Then, a machine learning platform training container is launched using training tools, and the training samples from the training set are read to train the risk identification model. Finally, an evaluation benchmark for the risk identification model is constructed, and the model is scored. If the score is greater than or equal to a preset score, the risk identification model training is considered complete.

[0106] The system receives feedback data from compliance personnel regarding the initial identification results. If the feedback data indicates that the accuracy of the risk identification model is lower than the preset accuracy, then for scenarios where the accuracy indicated by the feedback data is low, video and audio data are acquired using event tracking technology to form a retraining dataset. Samples in the retraining dataset are labeled, and a machine learning platform training container is launched using training tools. Samples from the retraining dataset are then read to retrain the risk identification model.

[0107] Figure 4 This is a flowchart illustrating another method for identifying branch employee risks provided in this application embodiment. The method provided in this embodiment is practical... Figure 1Based on the illustrated embodiment, steps S101 and S102 are further refined, and a step of updating the model parameters is added. For example... Figure 4 As shown, the method provided in this embodiment includes:

[0108] Step S401: Obtain the initial video data collected by the target cameras deployed at the site.

[0109] The initial video data is unprocessed video data captured by the target camera.

[0110] In this step, the edge node reads the initial video data collected by the target cameras deployed at the site.

[0111] Step S402: If a branch employee identifier is identified in the initial video data, and the duration of the branch employee identifier in the initial video data is greater than or equal to a preset time threshold, then the initial video data is segmented according to the branch employee identifier to obtain the target video data.

[0112] Employee identification at a branch office refers to identification information used to identify individuals as employees of that branch office, including but not limited to work permits and uniforms. By using work permits and uniforms as employee identification, biometric identification (such as facial recognition) is avoided, thus ensuring information security to a certain extent.

[0113] In this step, if a branch employee identifier is identified in the initial video data, it indicates that the initial video data records the behavior of a branch employee at that moment, and this segment of video data is valid data. Simultaneously, if the duration of the branch employee identifier in the initial video data is greater than or equal to a preset time threshold, such as 2 seconds, it indicates that the video data records valid branch employee behavior. Therefore, based on the branch employee identifier, the footage containing the branch employee can be segmented from the video data to obtain the target video data.

[0114] For example, feature comparison can be used to determine whether a branch employee identifier exists in the initial video data. The duration of the identifier can then be determined based on the number of frames in the initial video data.

[0115] If no branch employees are identified in the initial video data, the initial video data is invalid and will not be processed further. If the duration of the branch employee identification in the initial video data is less than a preset time threshold, and the short duration may prevent the subsequent risk identification model from identifying them, the initial video data is also invalid and will not be processed further.

[0116] In this embodiment, the target video data mainly includes footage of branch employees, eliminating other irrelevant environmental factors. This reduces the network and hardware resources required for subsequent data transmission and storage, improving processing efficiency. Simultaneously, it avoids including irrelevant users in the target video data, further protecting user privacy and security.

[0117] Step S403: Obtain the target audio data collected by the target microphones deployed at the network point.

[0118] In step S404, the target video data and target audio data are input into the pre-trained risk identification model. The risk identification model executes steps S4041 to S4044 to obtain the first identification result.

[0119] Step S4041: Target detection is performed on the target video data to identify the behavioral characteristics of the branch employees in the target video data and generate behavioral information.

[0120] The first identification result includes behavioral and semantic information of the branch employees.

[0121] The behavioral information of the branch employees refers to the structured data corresponding to their actions in the video data. Semantic information is also structured data, which includes keywords in the audio data, including but not limited to terms such as transfer, transaction, and forgery.

[0122] In this step, the risk identification model performs target detection based on the input target video data, detecting branch employees and electronic devices (especially those used by branch employees) in the target video data, and locating the branch employees. Based on the location results, it identifies the behavioral characteristics of the branch employees in the target video data, and determines the behavior information of the branch employees by matching these behavioral characteristics.

[0123] In this step, the risk identification model can also perform target detection based on the input target video data, identify the behavioral characteristics of the target in the video data, track branch employees, and identify the behavioral characteristics of multiple consecutive frames. The behavioral characteristics of these multiple consecutive frames are then classified to determine the corresponding behavior type, thereby identifying the behavioral information.

[0124] Step S4042: Perform speech recognition on the target audio data, detect keywords in the target audio data, and generate semantic information.

[0125] Speech recognition is the process of converting audio data into text data.

[0126] Specifically, the risk identification model performs speech recognition on the target audio data, converting it into text data. It then detects whether the text data contains keywords through keyword matching. Based on the contained keywords, semantic information is generated.

[0127] In some embodiments, the risk identification model can perform speech recognition on the target audio data to obtain text data. Then, natural language processing techniques are used to identify keywords in the text data. Based on the keywords, semantic information is generated.

[0128] Step S4043: Align the time windows of behavioral information and semantic information to generate spatiotemporally aligned data.

[0129] In this step, time window alignment is achieved based on the timestamps of behavioral information and semantic information. The aligned behavioral information and semantic information are then concatenated to obtain spatiotemporally aligned data.

[0130] In this step, deep learning algorithms can be used to perform spatiotemporal alignment of audio and video data.

[0131] Specifically, the Transformer architecture or attention mechanism can be used to align the timestamps of behavioral and semantic information.

[0132] Step S4044: Determine the risk level based on the spatiotemporal alignment data.

[0133] In this embodiment, the first identification result also includes a risk level. The risk level is a quantified value representing the degree of risk associated with the behavior of branch employees. Risk levels can include no risk, low risk, medium risk, and high risk.

[0134] In this step, the risk level can be determined based on the spatiotemporal alignment data according to the rule matching method.

[0135] In some embodiments, the risk identification model further includes a joint identification model as a sub-model. The joint identification model is trained using spatiotemporally aligned historical data as training samples, with the labels of the training samples representing the labeled risk levels. The trained joint identification model can learn the correspondence between spatiotemporally aligned data and risk levels.

[0136] In this step, spatiotemporal alignment data can also be input into a pre-trained joint recognition model, which determines the risk level based on the learned correspondence between spatiotemporal alignment data and risk levels.

[0137] Optionally, the risk level can be determined based on spatiotemporal alignment data, including: obtaining transaction operation data corresponding to the time window; and determining the risk level based on the spatiotemporal alignment data and the transaction operation data.

[0138] In this embodiment, transaction operation data can also be introduced for further joint modeling to improve the accuracy of identification.

[0139] Transaction operation data refers to the data recorded by branch employees when handling business, such as records of money transfers.

[0140] In this step, based on the time window of the spatiotemporal alignment data, the transaction operation data generated by branch employees within that time window is obtained. The risk level is then determined based on the spatiotemporal alignment data and the transaction operation data, using a rule-based matching method.

[0141] For example, a rule can be set to tear up receipts after a transaction is completed. If spatiotemporal alignment data indicates that the behavior is to tear up receipts, and the branch employee completed the transaction within the corresponding time window, then the risk level is determined to be the risk level corresponding to this rule, such as a high-risk level.

[0142] In some embodiments, the risk identification model further includes a joint identification model as a sub-model. The joint identification model is trained using historical spatiotemporally aligned data and transaction data as training samples, with the labels of the training samples representing the labeled risk levels. The trained joint identification model can learn the correspondence between spatiotemporally aligned data, transaction data, and risk levels.

[0143] In this step, spatiotemporal alignment data and transaction operation data can also be input into a pre-trained joint recognition model, which determines the risk level based on the learned correspondence.

[0144] In step S405, if the first recognition result does not meet the preset abnormal conditions, the first recognition result, the target video data, and the target audio data are uploaded to the cloud node so that the cloud node can generate a second recognition result based on the first recognition result, the target video data, and the target audio data.

[0145] Step S406: Based on the second identification result, a risk warning is issued.

[0146] Step S407: Based on the second identification result, update the parameters of the risk identification model using a federated learning framework.

[0147] Federated learning is a machine learning framework that uses encrypted parameter aggregation technology to share model parameters across network points while protecting privacy.

[0148] In this embodiment, a risk identification model is also deployed on the cloud node. Based on the second identification result, the cloud node pushes updated model parameters to the edge node through a federated learning framework to optimize the risk identification model of the edge node.

[0149] By using a federated learning framework, model parameters can be shared across network points while protecting data privacy. This ensures the continuous optimization of the risk identification model at the edge nodes, making it more adaptable to new abnormal behaviors, improving the generalization ability of the risk identification model, and ensuring that the identification results can adapt to constantly changing risk scenarios.

[0150] Step S408: If the first identification result meets the preset abnormal conditions, then a risk warning is issued based on the first identification result.

[0151] In this embodiment, a risk identification model identifies behavioral and semantic information. By integrating these two types of information, joint analysis of multimodal data is achieved. This enables the identification of abnormal behaviors in complex scenarios and uncovers hidden risks that cannot be identified by single-modal information. For example, when a branch employee uses voice commands to guide a customer's actions, the identification results are more accurate. Furthermore, by aligning time windows, behavioral and semantic information form a precise temporal correlation, resulting in spatiotemporally aligned data. This allows for efficient collaboration between behavioral and semantic information, ultimately quantifying it into a risk level. This risk level provides a more intuitive understanding of the degree of risk associated with abnormal behavior.

[0152] Figure 5 This is a flowchart illustrating another method for identifying risks among branch employees provided in an embodiment of this application. Figure 5 As shown, the method provided in this embodiment includes:

[0153] Step S501: In response to the first identification result, target video data and target audio data sent by the edge node, the first identification result, target video data and target audio data are input into the pre-trained cross-node identification model, so that the cross-node identification model determines the second identification result based on the features of the video data and audio data of multiple edge nodes that have been pre-learned.

[0154] The method provided in this application is applied to cloud nodes in a cloud-edge collaborative system, which also includes edge nodes deployed at various network points.

[0155] The cross-site identification model is a type of risk identification model. It can use the same machine learning model architecture as the risk identification model deployed at edge nodes.

[0156] The training method for the cross-site identification model can be as follows: Using a federated learning architecture, historical identification results, video data, and audio data from each site are obtained. The video and audio data are then labeled, specifically, targets in the video data (e.g., whether employees or electronic devices are present), behaviors in the video data, and keywords in the audio data. The historical identification results, video data, and audio data from each site are then divided into training samples, validation samples, and test samples to train the cross-site identification model.

[0157] After the cross-site identification model is trained, it is deployed on cloud nodes. Specifically, based on a high-performance machine learning service framework used for model deployment, the trained risk identification model can be encapsulated as a standard API or DDS cloud service. This cloud service is deployed on workstations, providing inference capabilities to the outside world in a unified manner.

[0158] By training a cross-node recognition model, the model can learn the features of video and audio data from multiple edge nodes. The first recognition result, target video data, and target audio data are input into the cross-node recognition model, which then outputs a second recognition result based on the features of the video and audio data from multiple edge nodes.

[0159] In this step, the cloud node responds to the first identification result, target video data, and target audio data sent by the edge node, and inputs the first identification result, target video data, and target audio data into the cross-node identification model, so that the cross-node identification model determines the second identification result based on the features of video data and audio data from multiple edge nodes that have been pre-learned.

[0160] In some embodiments, the parameters of the cross-site identification model can be adjusted based on the verification results of internal control and compliance personnel.

[0161] Specifically, the edge node receives feedback data from compliance personnel on the second identification result and generates a feedback training set; based on the feedback training set, the cross-site identification model is fine-tuned to generate optimized model parameters.

[0162] Step S502: Send the second recognition result to the edge node.

[0163] In this step, the cloud node communicates with the edge node and sends the obtained second recognition result to the corresponding edge node.

[0164] The risk identification method for branch employees provided in this embodiment uses a model to identify risks again at cloud nodes, realizing secondary identification and analysis, supplementing the first identification result, making the final identification result more accurate and reliable, and improving the generalization ability of the model by learning the features of multiple edge nodes and integrating global data.

[0165] Optionally, step S501, in response to the first identification result, target video data, and target audio data sent by the edge nodes, inputs the first identification result, target video data, and target audio data into a pre-trained cross-node identification model, so that the cross-node identification model determines the second identification result based on the features of video data and audio data of multiple edge nodes that have been pre-learned, including: in response to the first identification result, target video data, and target audio data sent by the edge nodes, inputting the first identification result, target video data, and target audio data into a pre-trained cross-node identification model, so that the cross-node identification model obtains the transaction operation data of multiple edge nodes corresponding to the time window of the first identification result, and determines the second identification result based on the features of video data, audio data, and transaction operation data of multiple edge nodes that have been pre-learned.

[0166] In this embodiment, the training samples for the cross-site identification model are the historical identification results, video data, audio data, and transaction operation data of each site within the corresponding time window. By training the cross-site identification model, it can learn the features of video data, audio data, and transaction operation data from multiple edge nodes.

[0167] After inputting the first recognition result, target video data, and target audio data into the pre-trained cross-network point recognition model, the cross-network point recognition model first obtains the transaction operation data of multiple edge nodes in the time window corresponding to the first recognition result; then, it calculates the matching degree between the features of the video data, audio data, and transaction operation data of the multiple edge nodes and the input data, and finally determines the second recognition result.

[0168] By introducing transaction operation data, the data perspective is further enriched, and the identification results are more accurate based on more comprehensive data.

[0169] Figure 6 This is a schematic diagram of the overall architecture of the system provided in the embodiments of this application. Figure 6 As shown, the system provided in this application includes an application layer, a platform layer, an algorithm layer, and a computing power layer. This system is used to implement the method provided in this application.

[0170] The computing layer comprises cloud nodes and edge nodes, which are the hardware devices supporting the operation of this method. Edge nodes include integrated smart terminals (also known as edge AI (Artificial Intelligence) all-in-one machines) and network video recorders (NVRs). Cloud nodes include GPU (Graphics Processing Unit) server clusters and NPU (Neural Processing Unit) server clusters.

[0171] The algorithm layer consists of models deployed on hardware devices at the computing power layer, including risk identification models and cross-site identification models. These risk identification and cross-site identification models can integrate target identification models, behavior identification models, and speech recognition models.

[0172] The platform layer is used to implement the entire process of risk identification methods for branch employees.

[0173] Specifically, the platform layer is used for data collection, that is, to collect target video data of branch employees through target cameras and target audio data through target microphones. The platform layer is also used to execute the branch employee risk identification method provided in this application to achieve branch employee risk identification.

[0174] In some embodiments, the platform layer can also perform computing power allocation, task management, system parameter configuration, and model configuration. For example, adjusting the runtime cycle and allocating hardware resources for edge nodes and cloud nodes.

[0175] The application layer is used to visualize the collected or identified data so that relevant personnel can verify the data.

[0176] Figure 7 This is a schematic diagram of a risk identification device for branch employees provided in an embodiment of this application. The branch employee risk identification device provided in this embodiment is deployed at edge nodes in various branch locations within a cloud-edge collaborative system, which also includes cloud nodes. Figure 7 As shown, the branch employee risk identification device provided in this embodiment includes an acquisition module 701, a first identification result determination module 702, a second identification result determination module 703, and an early warning module 704.

[0177] The acquisition module 701 is used to acquire target video data of branch employees collected by target cameras deployed at the branch, and target audio data collected by target microphones deployed at the branch; the first identification result determination module 702 is used to input the target video data and target audio data into a pre-trained risk identification model to obtain a first identification result; the second identification result determination module 703 is used to upload the first identification result, target video data and target audio data to a cloud node if the first identification result does not meet the preset abnormal conditions, so that the cloud node can generate a second identification result based on the first identification result, target video data and target audio data; the early warning module 704 is used to issue a risk warning based on the second identification result.

[0178] Optionally, the first identification result includes behavioral information and semantic information of the branch employees; the first identification result determination module 702 is specifically used for:

[0179] The target video data and target audio data are input into a pre-trained risk identification model. The risk identification model performs the following steps to obtain the first identification result: target detection is performed on the target video data to identify the behavioral characteristics of the branch employees in the target video data and generate behavioral information; speech recognition is performed on the target audio data to detect keywords in the target audio data and generate semantic information.

[0180] Optionally, the first identification result may also include a risk level; the first identification result determination module 702 is further used for:

[0181] Align the time windows of behavioral and semantic information to generate spatiotemporally aligned data; determine the risk level based on the spatiotemporally aligned data.

[0182] Optionally, the first recognition result determination module 702 is also used for:

[0183] Align the time windows of behavioral and semantic information to generate spatiotemporal aligned data; obtain the transaction operation data corresponding to the time window; and determine the risk level based on the spatiotemporal aligned data and the transaction operation data.

[0184] Optionally, the branch employee risk identification device also includes a model update module, which is used for:

[0185] After issuing a risk warning based on the second identification result, the parameters of the risk identification model are updated using a federated learning framework based on the second identification result.

[0186] Optionally, module 701 is used for:

[0187] Acquire initial video data collected by the target cameras deployed at the branch; if the branch employee identifier is identified in the initial video data, and the duration of the branch employee identifier in the initial video data is greater than or equal to a preset time threshold, then the initial video data is segmented according to the branch employee identifier to obtain the target video data; acquire target audio data collected by the target microphones deployed at the branch.

[0188] The branch employee risk identification device provided in this application embodiment can be used to execute the technical solution of the branch employee risk identification method provided in any of the above embodiments of this application. Its implementation principle and technical effect are similar, and will not be described again in this embodiment.

[0189] Figure 8 This is a schematic diagram of another branch employee risk identification device provided in an embodiment of this application. The branch employee risk identification device provided in this embodiment is applied to a cloud node in a cloud-edge collaborative system, which also includes edge nodes deployed at each branch. Figure 8 As shown, the branch employee risk identification device includes a second identification result determination module 801 and a communication module 802.

[0190] The second identification result determination module 801 is used to respond to the first identification result, target video data and target audio data sent by the edge node, input the first identification result, target video data and target audio data into the pre-trained cross-network node identification model, so that the cross-network node identification model determines the second identification result based on the features of the video data and audio data of multiple edge nodes that have been pre-learned; the communication module 802 is used to send the second identification result to the edge node.

[0191] Optionally, the second recognition result determination module 801 is specifically used for:

[0192] In response to the first identification result, target video data, and target audio data sent by the edge nodes, the first identification result, target video data, and target audio data are input into a pre-trained cross-network point identification model, so that the cross-network point identification model can obtain the transaction operation data of multiple edge nodes corresponding to the time window of the first identification result, and determine the second identification result based on the features of the video data, audio data, and transaction operation data of multiple edge nodes that have been pre-learned.

[0193] The branch employee risk identification device provided in this application embodiment can be used to execute the technical solution of the branch employee risk identification method provided in any of the above embodiments of this application. Its implementation principle and technical effect are similar, and will not be described again in this embodiment.

[0194] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 9As shown, the electronic device of this embodiment may include: at least one processor 901; and a memory 902 communicatively connected to at least one processor; wherein the memory 902 stores instructions that can be executed by at least one processor 901, and the instructions are executed by at least one processor 901 to cause the electronic device to perform the method as described in any of the above embodiments.

[0195] Optionally, the memory 902 can be either standalone or integrated with the processor 901. When the memory 902 is configured independently, the device also includes a bus for connecting the memory 902 and the processor 901.

[0196] The implementation principle and technical effects of the electronic device provided in this embodiment can be found in the foregoing embodiments, and will not be repeated here.

[0197] This application also provides a computer-readable storage medium storing computer-executable instructions. When the computer-executable instructions are executed by a processor, the methods provided in any of the foregoing embodiments can be implemented.

[0198] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the method provided in any of the foregoing embodiments.

[0199] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this application.

[0200] It should be further noted that although the steps in the flowchart are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowchart may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0201] It should be understood that the above-described device embodiments are merely illustrative, and the device of this application can also be implemented in other ways. For example, the division of units / modules in the above embodiments is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units, modules, or components may be combined, or integrated into another system, or some features may be ignored or not executed.

[0202] Furthermore, unless otherwise specified, the functional units / modules in the various embodiments of this application can be integrated into one unit / module, or each unit / module can exist physically separately, or two or more units / modules can be integrated together. The integrated units / modules described above can be implemented in hardware or as software program modules.

[0203] When integrated units / modules are implemented in hardware, the hardware can be digital circuits, analog circuits, etc. The physical implementation of the hardware structure includes, but is not limited to, transistors, memristors, etc. Unless otherwise specified, the processor can be any suitable hardware processor, such as a CPU, GPU, FPGA, DSP, and ASIC, etc. Unless otherwise specified, the storage unit can be any suitable magnetic or magneto-optical storage medium, such as Resistive Random Access Memory (RRAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Enhanced Dynamic Random Access Memory (EDRAM), High-Bandwidth Memory (HBM), Hybrid Memory Cube (HMC), etc.

[0204] If the integrated unit / module is implemented as a software program module and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0205] In the above embodiments, the descriptions of each embodiment have their own emphasis. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.

[0206] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0207] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method for identifying risks among branch employees, characterized in that, The method is applied to edge nodes deployed at various network points in a cloud-edge collaborative system, which also includes cloud nodes. Acquire target video data of branch employees collected by target cameras deployed at the branch, and acquire target audio data collected by target microphones deployed at the branch; The target video data and the target audio data are input into a pre-trained risk identification model to obtain a first identification result; If the first recognition result does not meet the preset abnormal conditions, the first recognition result, the target video data, and the target audio data are uploaded to the cloud node so that the cloud node generates a second recognition result based on the first recognition result, the target video data, and the target audio data. Based on the second identification result, a risk warning is issued.

2. The method according to claim 1, characterized in that, The first identification result includes behavioral and semantic information of the branch employees; the step of inputting the target video data and the target audio data into a pre-trained risk identification model to obtain the first identification result includes: The target video data and the target audio data are input into a pre-trained risk identification model, which performs the following steps to obtain a first identification result: Target detection is performed on the target video data to identify the behavioral characteristics of the branch employees in the target video data and generate the behavioral information. The target audio data is subjected to speech recognition to detect keywords in the target audio data and generate the semantic information.

3. The method according to claim 2, characterized in that, The first identification result also includes a risk level, and the method further includes: Align the time windows of the behavioral information and the semantic information to generate spatiotemporal aligned data; The risk level is determined based on the spatiotemporal alignment data.

4. The method according to claim 3, characterized in that, The step of determining the risk level based on the spatiotemporal alignment data includes: Obtain the transaction operation data corresponding to the time window; The risk level is determined based on the spatiotemporal alignment data and the transaction operation data.

5. The method according to any one of claims 1-4, characterized in that, After issuing a risk warning based on the second identification result, the method further includes: Based on the second identification result, the parameters of the risk identification model are updated using a federated learning framework.

6. The method according to any one of claims 1-4, characterized in that, The acquisition of target video data of branch employees collected by target cameras deployed at the branch includes: Obtain initial video data collected by the target cameras deployed at the network points; If a branch employee identifier is identified in the initial video data, and the duration of the branch employee identifier in the initial video data is greater than or equal to a preset time threshold, then the initial video data is segmented according to the branch employee identifier to obtain the target video data.

7. A method for identifying risks among branch employees, characterized in that, The method includes: cloud nodes applied in a cloud-edge collaborative system, wherein the cloud-edge collaborative system also includes edge nodes deployed at various network points; and the method comprises: In response to the first identification result, target video data, and target audio data sent by the edge node, the first identification result, the target video data, and the target audio data are input into a pre-trained cross-node identification model, so that the cross-node identification model determines a second identification result based on the features of video data and audio data from multiple pre-learned edge nodes; The second identification result is sent to the edge node.

8. The method according to claim 7, characterized in that, In response to the first identification result, target video data, and target audio data sent by the edge node, the first identification result, the target video data, and the target audio data are input into a pre-trained cross-node identification model, so that the cross-node identification model determines a second identification result based on the pre-learned features of video data and audio data from multiple edge nodes, including: In response to the first identification result, target video data, and target audio data sent by the edge node, the first identification result, the target video data, and the target audio data are input into a pre-trained cross-node identification model, so that the cross-node identification model obtains the transaction operation data of multiple edge nodes corresponding to the time window of the first identification result, and determines the second identification result based on the features of the pre-learned video data, audio data, and transaction operation data of multiple edge nodes.

9. A risk identification device for branch employees, characterized in that, The device is used in edge nodes deployed at various network points in a cloud-edge collaborative system, which also includes cloud nodes. The device includes: The acquisition module is used to acquire target video data of branch employees collected by target cameras deployed at the branch, and to acquire target audio data collected by target microphones deployed at the branch. The first identification result determination module is used to input the target video data and the target audio data into a pre-trained risk identification model to obtain a first identification result; The second identification result determination module is used to upload the first identification result, the target video data, and the target audio data to a cloud node if the first identification result does not meet the preset abnormal conditions, so that the cloud node generates a second identification result based on the first identification result, the target video data, and the target audio data. The early warning module is used to issue risk warnings based on the second identification result.

10. A risk identification device for branch employees, characterized in that, A cloud node used in a cloud-edge collaboration system, the cloud-edge collaboration system also including edge nodes deployed at various network points, the device comprising: The second identification result determination module is used to respond to the first identification result, target video data and target audio data sent by the edge node, and input the first identification result, the target video data and the target audio data into a pre-trained cross-network point identification model, so that the cross-network point identification model determines the second identification result based on the features of video data and audio data of multiple edge nodes that have been pre-learned; A communication module is used to send the second identification result to the edge node.

11. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1 to 8.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 8.

13. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method of any one of claims 1 to 8.

Images