Data processing method and device, equipment, medium and program product

Abstract

The invention provides a data processing method which can be applied to the technical field of artificial intelligence, and particularly relates to the technical field of biological recognition. The method comprises the following steps: under the condition that a customer agrees or authorizes, acquiring behavior data of the customer in a monitoring area; performing feature extraction processing on the behavior data to obtain a customer behavior feature set; based on the customer behavior feature set and a pre-trained local model, the customer behavior state is determined, and the pre-trained local model is obtained based on aggregation training processing of local statistical feature parameters and global model parameters. The invention further provides a data processing device and equipment, a storage medium and a program product.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to the technical field of biometric identification, and more particularly to a data processing method, device, equipment, medium and program product. BACKGROUND

[0002] With the development of financial technology and the intelligent transformation of bank outlets, bank outlets not only need to ensure the security of financial transactions, but also increasingly value improving the service experience of customers. In actual operation, customers may encounter situations such as unfamiliar processes, difficult device operation, or the need for immediate consultation when conducting business. These potential breakpoints or service needs in the service process have traditionally relied on the experience and patrol of the hall manager to discover and respond. However, the manual service mode inevitably has problems such as response delay, incomplete coverage, or inconsistent standards during high customer flow periods.

[0003] Currently, there have been some attempts in the prior art to analyze customer behavior at bank outlets based on computer vision, audio processing and other technologies in order to timely discover customer needs. However, these schemes generally have two major bottlenecks. First, the computing performance and real-time bottleneck: high-precision behavior recognition algorithms are usually complex and computationally intensive, making it difficult to deploy on edge devices with limited computing power (such as outlet cameras and local servers). If raw data such as video streams are all transmitted to a cloud data center for processing, significant network delays will be introduced, resulting in excessively long system response times that cannot meet the requirements for real-time early warning and response to risk events. Second, the data privacy and compliance bottleneck: customers' images, audio and other biometric data are highly sensitive personal information. Transmitting raw data out of the local outlet for cloud server processing poses a significant risk of leakage and compliance pressure, and there is a serious risk of damaging customers' rights and interests and the bank's reputation. In addition, the customer groups and device environments of different outlets differ, resulting in heterogeneous data distribution. If each outlet's data is simply centrally learned, the trained model will not be able to adapt to the unique environment of each outlet, ultimately leading to decreased recognition accuracy and low training efficiency. SUMMARY

[0004] In view of the above problems, the present application provides a data processing method, device, equipment, medium and program product.

[0005] According to a first aspect of the present application, a data processing method is provided, applied to an edge device end, the method comprising: obtaining behavior data of a customer in a monitoring area under the condition that the customer agrees or authorizes; performing feature extraction processing on the behavior data to obtain a customer behavior feature set; determining a customer behavior state based on the customer behavior feature set and a pre-trained local model, wherein the pre-trained local model is obtained based on local statistical characteristic parameters and global model parameter aggregation training processing.

[0006] According to an embodiment of the present application, the local model is obtained based on aggregation training of the local statistical characteristic parameters and the global model parameters, comprising: training an initial local model based on a historical behavior feature set of local customers to obtain local statistical characteristic parameters and initial weight parameters; sending the initial weight parameters to a server end; receiving global model parameters issued by the server end, the global model parameters being obtained based on aggregation of initial weight parameters uploaded by a plurality of edge device ends; and generating the local model based on the global model parameters and the local statistical characteristic parameters.

[0007] According to an embodiment of the present application, the initial local model is trained based on a historical behavior feature set of local customers to obtain local statistical characteristic parameters and initial weight parameters, comprising: performing an iterative training process based on the historical behavior feature set of local customers and the initial local model to update the weight of the initial local model to obtain the initial weight parameters and the local statistical characteristic parameters, wherein the local statistical characteristic parameters are obtained based on statistical distribution of feature data in the historical behavior feature set flowing through the initial local model.

[0008] According to an embodiment of the present application, the local statistical characteristic parameters are obtained based on statistical distribution of feature data in the historical behavior feature set flowing through the initial local model, comprising: sequentially performing statistical distribution processing on feature data in the historical behavior feature set flowing through the initial local model in batches to obtain statistical distribution parameters; and generating the local statistical characteristic parameters by an iterative updating method based on a moving average value of the statistical distribution parameters determined in multiple batches.

[0009] According to an embodiment of the present application, the local model is generated based on the global model parameters and the local statistical characteristic parameters, comprising: loading the global model parameters to a weight part of the initial local model, and loading the local statistical characteristic parameters to a standardization part of the initial local model to generate the local model.

[0010] According to an embodiment of the present application, the behavior data of customers in the monitoring area is obtained under the condition that the customers agree or authorize, comprising: obtaining original behavior data of customers in the monitoring area under the condition that the customers agree or authorize; and performing data desensitization processing on the original behavior data to obtain the behavior data of customers in the monitoring area.

[0011] According to an embodiment of the present application, the original behavior data includes image data, audio data and text data, and the data desensitization processing performed on the original behavior data includes at least one of: performing body blurring processing on the image data of the customer; performing voiceprint desensitization processing on the audio data of the customer; and performing sensitive information filtering processing on the text data of the customer.

[0012] The second aspect of the present application provides a global model parameter acquisition method, applied to a server side, the method comprising: receiving initial weight parameters from a plurality of different edge device sides, wherein the initial weight parameters are obtained after training an initial local model based on a historical behavior feature set of a local customer of each edge device side; performing weighted average aggregation processing based on a plurality of the initial weight parameters to obtain the global model parameter; and sending the global model parameter to each of the edge device sides respectively.

[0013] The third aspect of the present application provides a data processing apparatus, applied to an edge device side, the apparatus comprising: a first acquisition unit configured to acquire behavior data of a customer in a monitoring area under the condition that the customer agrees or authorizes; a first processing unit configured to perform feature extraction processing on the behavior data to obtain a customer behavior feature set; and a second processing unit configured to determine a customer behavior state based on the customer behavior feature set and a pre-trained local model, wherein the pre-trained local model is obtained based on local statistical characteristic parameters and global model parameter aggregation training processing.

[0014] The fourth aspect of the present application provides an electronic device, comprising: one or more processors; a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0015] The fifth aspect of the present application further provides a computer-readable storage medium having a computer program or instructions stored thereon, wherein the computer program or instructions are executed by a processor to implement the steps of the above method.

[0016] The sixth aspect of the present application further provides a computer program product comprising a computer program or instructions, wherein the computer program or instructions are executed by a processor to implement the steps of the above method. BRIEF DESCRIPTION OF DRAWINGS

[0017] The above and other objects, features and advantages of the present application will become more apparent from the following description of embodiments of the present application taken in conjunction with the accompanying drawings, in which:

[0018] Figure 1 The application scenario of the data processing method, apparatus, device, medium and program product according to the embodiments of the present application is schematically shown.

[0019] Figure 2 A flowchart of a data processing method applied to an edge device end according to an embodiment of the present application is schematically shown;

[0020] Figure 3 A specific embodiment flowchart framework of a data processing method applied to an edge device end according to an embodiment of the present application is schematically shown;

[0021] Figure 4 A flowchart of step S300 of a data processing method applied to an edge device end according to an embodiment of the present application is schematically shown;

[0022] Figure 5 A flowchart of step S311 of a data processing method applied to an edge device end according to an embodiment of the present application is schematically shown;

[0023] Figure 6 A flowchart of step S100 of a data processing method applied to an edge device end according to an embodiment of the present application is schematically shown;

[0024] Figure 7 A flowchart of a global model parameter acquisition method applied to a server end according to an embodiment of the present application is schematically shown;

[0025] Figure 8 A structural block diagram of a data processing apparatus applied to an edge device end according to an embodiment of the present application is schematically shown; and

[0026] Figure 9 A block diagram of an electronic device suitable for implementing a data processing method according to an embodiment of the present application is schematically shown. DETAILED DESCRIPTION

[0027] Hereinafter, embodiments of the present application will be described with reference to the accompanying drawings. It is to be understood, however, that the description is merely exemplary and is not intended to limit the scope of the present application. In the following detailed description of the embodiments of the present application, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the present application. However, it would be apparent to those skilled in the art that the embodiments of the present application can be practiced without these specific details. In other instances, well-known structures and functions have been omitted to avoid unnecessarily complicating the present application with details that would be apparent to those skilled in the art.

[0028] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the present application. As used herein, the term "includes" and "comprising" and the like means the inclusion of the stated features, steps, operations, and / or components, but not the exclusion of one or more other features, steps, operations, or components.

[0029] All terms used herein, including technical and scientific terms, have the meanings commonly understood by one of ordinary skill in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning that is consistent with the context of the specification, and should not be interpreted in an idealized or overly formal way.

[0030] In the case where expressions such as "at least one of A, B, and C, etc." are used, it generally should be interpreted that the meaning is the same as "at least one of A or B; at least one of A or C; at least one of B or C; at least one of A, B, and C; and the like." In the case where expressions such as "at least one of A, B, or C, etc." are used, it generally should be interpreted that the meaning is the same as "at least one of A or B; at least one of A or C; at least one of B or C; at least one of A, B, and C; and the like."

[0031] The data processing method, device, equipment, medium, and program product of the present application can be used in the field of artificial intelligence and the field of biometric identification, and can also be used in any field other than the field of artificial intelligence. The application field of the data processing method, device, equipment, medium, and program product of the present application is not limited

[0032] In the technical solutions of the present application, the user information (including but not limited to user personal information, user image information, user device information such as location information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved are information and data authorized by the user or authorized by all parties, and the collection, storage, use, processing, transmission, provision, application, and application of related data comply with relevant laws, regulations, and standards, take necessary security measures, do not violate public order and good customs, and provide corresponding operation portals for users to choose authorization or refusal.

[0033] In the scenario of using personal information for automated decision-making, the method, device, and system provided by the embodiments of the present application all provide corresponding operation portals for users to choose to agree or refuse the automated decision-making result; if the user chooses to refuse, the expert decision-making process is entered.

[0034] Figure 1 The application scenario diagram of the data processing method, device, equipment, medium, and program product according to the embodiments of the present application is schematically shown.

[0035] As Figure 1As shown, the application scenario 100 according to the embodiment can include a data processing method, device, equipment, medium and program product. The network 104 is a medium for providing a communication link between the first terminal equipment 101, the second terminal equipment 102, the third terminal equipment 103 and the server 105. The network 104 can include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0036] The user can use the first terminal equipment 101, the second terminal equipment 102, the third terminal equipment 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal equipment 101, the second terminal equipment 102, the third terminal equipment 103, such as financial service applications, web browser applications, search applications, email clients, social platform software, etc. (only as an example).

[0037] The first terminal equipment 101, the second terminal equipment 102, the third terminal equipment 103 can be various electronic devices with display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers and desktop computers, etc.

[0038] The server 105 can be a server providing various services, such as a background management server supporting the information browsed by the user using the first terminal equipment 101, the second terminal equipment 102, the third terminal equipment 103 (only as an example). The background management server can analyze and process the received user request data, etc., and feed back the processing result (such as a web page, information or data, etc. obtained or generated according to the user request) to the terminal equipment.

[0039] It should be noted that, in the first aspect, the data processing method provided by the embodiment of the present application can be executed by the server 105, and in the second aspect, the data processing method provided by the embodiment of the present application can be executed by the first terminal equipment 101, the second terminal equipment 102 or the third terminal equipment 103. Accordingly, in the third aspect, the data processing device provided by the embodiment of the present application can be arranged in the server 105, and in the fourth aspect, the data processing device provided by the embodiment of the present application can be arranged in the first terminal equipment 101, the second terminal equipment 102 or the third terminal equipment 103. The data processing method provided by the embodiment of the present application can also be executed by a server or a server cluster different from the server 105 and capable of communicating with the first terminal equipment 101, the second terminal equipment 102, the third terminal equipment 103 and / or the server 105. Accordingly, the data processing device provided by the embodiment of the present application can also be arranged in a server or a server cluster different from the server 105 and capable of communicating with the first terminal equipment 101, the second terminal equipment 102, the third terminal equipment 103 and / or the server 105.

[0040] It should be understood that Figure 1 the number of terminal devices, networks and servers in the above-mentioned scenarios is only illustrative. Any number of terminal devices, networks and servers can be provided according to implementation needs.

[0041] The following will be based on Figure 1 the scenario described, and the data processing method of the application embodiment will be described in detail. Figure 2 , Figures 4-6

[0042] Figure 2 The flowchart of the data processing method applied to the edge device end according to the embodiments of the application is schematically shown.

[0043] As Figure 2 shown, the application provides a data processing method applied to the edge device end, including operations S100-S300.

[0044] Specifically, in operation S100, the behavior data of the customer in the monitoring area is acquired under the condition that the customer agrees or authorizes. In some specific embodiments, the behavior data of the customer in the monitoring area can be collected by various sensing devices deployed in the key area of the bank business site. In the present embodiment, the monitoring area is the area where the customer often handles business in the site, such as the operation area of the unattended counter machine, etc.; in the link of customer agreement or authorization, the unattended counter machine can push the unattended counter machine intelligent auxiliary confirmation to the customer before starting the card reading, and only after the customer agrees or authorizes, the operation process of the unattended counter machine is started and the acquisition process of the behavior data of the customer in the monitoring area is started; while for the operation of the customer's disagreement or non-authorization, the customer can be suggested by the unattended counter machine to go to the manual service area for handling, or the information of the customer's disagreement and authorization can be pushed to the site manager, so that the site manager goes to the monitoring area for guidance service or stays. As for the behavior data of the customer collected in the monitoring area, only the visual behavior data of the individual in the monitoring area, such as body posture, gesture and moving track, etc. which does not involve the customer's identity information is collected in some embodiments; a microphone array is used to collect the audio behavior data of the individual in the conversation process which does not involve the customer's identity information; in addition, transaction data recorded by the business system can also be combined, such as the operation behavior, operation time, transaction type and other data of the customer on the self-service terminal. These multi-modal behavior data jointly constitute the basis for comprehensive analysis of the customer's state. The collection process strictly follows the minimization principle, only the necessary data related to behavior perception is acquired, and preliminary processing is performed at the device end to ensure that the original sensitive information is not leaked, and at the same time, all data collection is performed under the premise that the customer has explicitly authorized or conforms to the relevant laws, regulations and policies to ensure business compliance.

[0045] ​Specifically in operation S200, feature extraction processing is performed on the behavior data to obtain a customer behavior feature set. In some embodiments, the "feature extraction processing" refers to a process of converting original, unstructured behavior data (such as video, audio) into machine-understandable, structured numerical feature vectors, that is, a process of dimensionality reduction and abstraction on original, unstructured behavior data, converting it into a set of structured numerical vectors that can efficiently represent the nature of behavior. Through this series of processing, the original behavior data is converted into a multi-dimensional behavior feature set, which can quantitatively describe the behavior performance at a specific time point, providing accurate input for subsequent model judgment.

[0046] Specifically in operation S300, a customer behavior state is determined based on the customer behavior feature set and a pre-trained local model, wherein the pre-trained local model is obtained based on local statistical characteristic parameters and global model parameter aggregation training processing. In some embodiments, determining the customer behavior state based on the customer behavior feature set and the pre-trained local model means that the real-time extracted customer behavior feature set is loaded into the pre-trained local model already deployed locally as input. The local model internally contains a series of calculation layers and network structures, which perform forward propagation calculation on the input feature vectors. This calculation process is essentially a deep analysis and pattern matching of customer behavior features, and finally outputs one or more probability values or classification results representing the current state of the customer, such as "need to ask questions", "need to assist operation", "need to pay attention to", etc. The edge device can determine the current behavior state of the customer according to the highest probability value output by the model, and the staff of the network point can set a threshold (such as a need for help probability greater than 0.7) to automatically trigger a reminder mechanism, such as notifying the concierge to pay attention.

[0047] For reference Figure 3As shown, the pre-trained local model is not simply issued after being uniformly trained by a central server, but adopts a collaborative training framework. The parameters of the pre-trained local model are composed of two parts: one part is the global model parameters, which combine the common knowledge from many other edge devices (network points), form the general behavior patterns learned on a wide range of data, and are formed by aggregating the model parameters uploaded by each device on the server side; the other part is the "local statistical characteristic parameters", which are unique to the local edge device, and are obtained according to the unique historical data distribution statistics flowing through the device in the local training process, and its role is to make the model better adapt to the unique customer group characteristics (such as regional cultural differences) and environmental conditions (such as device model, light, background) of the local network point. The local model skillfully combines the two, uses the globally shared "general knowledge" (global model parameters) as its core judgment ability, and uses the "local characteristics" (local statistical characteristic parameters) to calibrate and adapt to the specific environment of the local. This aggregated training method makes the local model of each network point not only has the macro wisdom of the whole network, but also retains the micro adaptability to the local environment, so as to realize the great improvement of model performance and accuracy without sharing the original private data. The three steps of operation S100~operation S300 solve the privacy risk, network delay and data island problem brought by the traditional centralized processing method. By completing data acquisition, feature extraction and model reasoning on the edge device end, real-time and rapid response to customer behavior is realized, and through a model aggregation training mechanism, the model on the edge device end can continuously learn and evolve, protecting data privacy while ensuring the accuracy of identification and adaptability to the local environment.

[0048] As shown in FIG. 1, in some example embodiments, the method comprises operations S100~operation S300. Figure 4 As shown, in some example embodiments, the obtaining the local model based on the aggregation training processing of the local statistical characteristic parameters and the global model parameters comprises: operation S310~operation S340.

[0049] Specifically in operation S310, an initial local model is trained based on the historical behavior feature set of local customers to obtain local statistical characteristic parameters and initial weight parameters. In some specific embodiments, the "local statistical characteristic parameters" and the "initial weight parameters" are two different types of parameters generated simultaneously in the local training process of the initial local model. The "initial weight parameters" are core adjustable parameters responsible for learning and decision-making in the model, and the optimization goal is to minimize the error between the model prediction and the real situation. The initial weight parameters reflect the knowledge learned by the model from local data about which combination of behavior features may correspond to which customer state, and this part of knowledge is universal and can be shared with other outlets. The "local statistical characteristic parameters" are different. The local statistical characteristic parameters are a set of parameters specially used to describe and standardize the distribution characteristics of the historical behavior feature data in the local outlet. The local statistical characteristic parameters are not parameters about judgment logic, but a quantitative record of the statistical characteristics of the local data itself that flows through the model being trained. Since the data distribution (such as customer average speech speed, ambient light level) of each outlet is different, this part of the parameter is highly localized, reflecting the unique data "background" or "environmental fingerprint" of the local outlet. Therefore, through a complete local training, the training process learns an initialized initial local model using the "historical behavior feature set" accumulated by the local outlet, and through iterative optimization, two sets of results are obtained at the same time. One is the "initial weight parameter" after optimization convergence, which learns how to make judgments based on input features; the other is the "local statistical characteristic parameter" determined at the same time, which records the characteristics of the data in the local outlet to prepare for subsequent aggregation and localized reconstruction.

[0050] In some example embodiments, the training of the initial local model based on the historical behavior feature set of local customers to obtain local statistical characteristic parameters and initial weight parameters includes operation S311.

[0051] Specifically, at operation S311, based on the set of historical behavior features of the local customers and the initial local model, an iterative training process is performed to update the weights of the initial local model, to obtain the initial weight parameters and the local statistical characteristic parameters, wherein the local statistical characteristic parameters are obtained based on the statistical distribution of the feature data in the set of historical behavior features flowing through the initial local model. In some embodiments, the "iterative training process" refers to inputting the set of historical behavior features of all local customers into the initial local model for learning multiple rounds. In each round of training, the initial local model makes a prediction according to the input feature data, compares the prediction result with the real customer state label, calculates the prediction error, and adjusts the weight parameters inside the initial local model according to the error, aiming to make the error of the next round of prediction smaller. This "prediction-comparison-weight adjustment" process is repeated until the prediction accuracy of the initial local model reaches the preset standard, at which time the obtained weight is the "initial weight parameter".

[0052] Meanwhile, the "feature data in the set of historical behavior features flowing through the initial local model" refers to that in the above iterative training process, data is sent into the model for processing batch by batch. The model contains some special standardization layers (Standardization Layers), which are used to normalize the data before it enters the core computing unit, in order to speed up the training process and improve stability. The "statistical distribution of feature data" is generated in this normalization process. Whenever a batch of data flows through the standardization layer, the layer calculates the statistical features of this batch of data, such as mean and variance. These statistical features objectively describe the distribution of local data. Therefore, while updating the weights in the iterative training, the standardization layer is also continuously observing and recording the statistical distribution characteristics of the local data, and finally forming the "local statistical characteristic parameters" which are stable and can represent the overall picture of the local data.

[0053] As shown in FIG. 3B, in some example embodiments, the method further includes operations S312-S313. Figure 5 As shown in FIG. 3B, in some example embodiments, the method further includes operations S312-S313.

[0054] Specifically, in operation S3111, the statistical distribution processing is performed on the feature data in the historical behavior feature set in the initial local model in batches to obtain statistical distribution parameters. In some embodiments, since the locally accumulated historical behavior feature set usually has a large amount of data, it cannot be loaded into the limited memory for processing at one time, and therefore the training process adopts a batch processing strategy. This step is the core calculation performed independently for each input training data batch. The "flowing through" means that each batch of sample data is sent into the initial local model for forward propagation, and the statistical distribution processing is performed on all data in the current batch to dynamically calculate the key indicators that can characterize the internal data distribution characteristics of the current batch. These indicators are the statistical distribution parameters, which mainly include but are not limited to: the mean of the batch data in each feature dimension (Mean), which is used to describe the central tendency or general level of the data distribution; and the variance of the batch data in each feature dimension (Variance), which is used to describe the volatility or dispersion of the data around the mean; and the context information provided for the subsequent calculation of the initial local model, that is, the distribution form of the data being processed, so that the initial local model can adaptively adjust and standardize the unique distribution of the batch data, lay an accurate foundation for the current round of parameter optimization, and ensure the stability of the training process. The statistical distribution parameters of each batch are a momentary "snapshot" of the mini data set. In some embodiments, assuming that the local historical behavior feature set contains 10,000 samples. During training, the 10,000 samples are not sent into the model at one time, but are divided into multiple small batches, for example, 100 samples per batch, and 100 batches in total. When the 100 sample data in the first batch flows through the normalization layer in the initial local model, the normalization layer in the initial local model will specially calculate a mean (Mean) and a variance (Variance) for the 100 samples. These two values are the "statistical distribution parameters" of this batch of data. Then, the 100 sample data in the second batch flows through the layer, and a new mean and variance belonging to the second batch are also calculated. This process is repeated until all 100 batches of data are processed. During the entire training period, this batch-by-batch calculation process is repeated for multiple rounds to provide basic and dynamic original statistical measurements for subsequent generation of global and stable local statistical characteristic parameters.

[0055] Specifically, in operation S3112, the local statistical characteristic parameters are generated by iteratively updating based on the moving average of the statistical distribution parameters determined in multiple batches. In some embodiments, operation S3112 fuses the instantaneous batch statistical information generated in operation S3111, which is independent of each other and may be full of noise, into a set of solidified parameters that can stably and reliably represent the global distribution characteristics of the entire local historical behavior characteristic set. The iterative updating mechanism of the "moving average" maintains a set of continuously updated global state variables for tracking the long-term trend of historical statistics, i.e., for generating a stable and reliable long-term trend value according to a series of time-sequentially generated data points. It is not simply the arithmetic mean of all batch parameters, but gives higher weight to the parameters of recent batches, i.e., after processing each new data batch at the beginning of training, the old global statistical distribution variable is not simply discarded, nor is the old data directly replaced by the statistical parameters of the new batch, but the statistical distribution parameters calculated from the new batch are weighted and fused with the currently maintained statistical distribution variable. For example, after the first batch of data is processed, a first set of statistical distribution parameters (mean M1, variance V1) is obtained, and the moving average at this time is M1 and V1. When the second batch of data is processed, a second set of parameters (M2, V2) is obtained, and the new moving average is not directly taken as (M1+M2) / 2, but is calculated by a formula, for example, "new moving average = (1-α)*old moving average + α*current batch value", where α is a small learning rate. This process occurs when each batch of data flows through the model, and the moving average is continuously and smoothly iteratively updated. After a complete round of training, i.e., after all batches are processed, the moving average updated by multiple batches is finally determined as the "local statistical characteristic parameter" that can represent the statistical distribution of the entire local data set. The purpose of this step is to avoid the random fluctuations of single batch data from interfering with the overall statistical characteristic evaluation, and to ensure that the finally generated local statistical characteristic parameters have high stability and representativeness. The effect is to obtain a core parameter that accurately reflects the long-term and overall data distribution law of the local network, providing a solid foundation for the localization of the model.

[0056] Specifically in operation S320, the initial weight parameters are sent to the server side. In some specific embodiments, when the training of the initial model is completed locally and the initial weight parameters and the local statistical characteristic parameters are successfully separated, an uploading program is started. This program will only package the "initial weight parameters". In order to ensure the security of data transmission, encryption processing is performed on this part of the parameters before transmission, for example, using an asymmetric encryption algorithm to ensure that only the server can decrypt. Subsequently, the encrypted initial weight parameter data packet is sent to the pre-configured server address through a secure network channel (such as HTTPS or a proprietary encryption channel). Throughout the process, the "local statistical characteristic parameters" are strictly retained locally, and the local initial weight parameters are contributed to the server-side global knowledge base while strictly avoiding any leakage of original privacy data, all under the premise of absolutely ensuring data security.

[0057] Specifically in operation S330, the global model parameters issued by the server side are received, which are aggregated based on the initial weight parameters uploaded by multiple edge device sides. In some specific embodiments, after sending its own initial weight parameters, it enters a waiting and listening state. After the time required for the server side to complete aggregation, the edge device side receives the data packet sent from the server side, which contains the updated "global model parameters"; the global model parameters collect the initial weight parameters uploaded by multiple other edge devices and are obtained through aggregation calculation. In some embodiments, the other edge devices are bank branches distributed in different cities across the country, serving different customer groups, uploading initial weight parameters, and calculating through an advanced aggregation algorithm. This enables each edge device side to regularly obtain a model parameter basis that is more optimized and robust, which is based on the common knowledge of the entire network, thereby realizing the collaborative evolution of model capabilities.

[0058] Specifically in operation S340, the local model is generated based on the global model parameters and the local statistical characteristic parameters. In some specific embodiments, after receiving the new global model parameters, it does not simply overwrite the entire local model with it, but combines the global model parameters issued from the server with the local statistical characteristic parameters that are generated during the local training process and are always retained locally on the device. The local statistical characteristic parameters encapsulate the unique distribution characteristics of the customer behavior data of the local branch, such as the common behavior patterns of local customers, the imaging characteristics of local cameras, etc. By combining the global parameters representing general knowledge with the statistical parameters representing local adaptability, the final generated local model is a highly customized local model. The local model can both utilize the knowledge shared by all branches to make more accurate judgments, breaking down the data silo problem, and utilize the statistical parameters tailored for itself to better understand and process locally unique data, solving the data heterogeneity problem, ultimately improving the accuracy and reliability of behavior state recognition on the edge device side.

[0059] In some exemplary embodiments, generating the local model based on the global model parameters and the local statistical characteristic parameters includes: operation S341.

[0060] Specifically, in operation S341, the global model parameters are loaded into the weight portion of the initial local model, and the local statistical characteristic parameters are loaded into the normalization portion of the initial local model to generate the local model. In some specific embodiments, at the software level, the model structure is predefined, and its parameter storage area can be logically divided into two main functional parts: the weight portion contains all trainable connection weights in the model, mainly responsible for pattern recognition and decision-making logic; the normalization portion contains parameters specifically used for feature scaling and normalization of input data, i.e., local statistical characteristic parameters. When generating the final model, the global model parameters received from the server are precisely filled into all corresponding positions in the model weight portion. Then, the local statistical characteristic parameters, which have been trained and stored locally but never uploaded, are refilled into the corresponding positions in the model's normalization portion. This achieves highly efficient updating and localization of the local model, ensuring that the core decision-making capabilities of the local model keep pace with global updates, while also ensuring that the local model has optimal understanding and processing capabilities for local input data based on local normalized parameters. This enables accurate and efficient behavior recognition and reasoning at the edge device.

[0061] like Figure 6 As shown, in some exemplary embodiments, obtaining customer behavior data within the monitoring area with the customer's consent or authorization includes: operations S110 to S120.

[0062] Specifically in operation S110, the original behavior data of the customer in the monitoring area is obtained under the condition that the customer agrees or authorizes. In some specific embodiments, the original behavior data refers to the original data stream directly collected by various types of sensing devices deployed in the bank branch without any modification or processing. This usually includes: video stream data continuously captured by a camera, for example, the video stream data is collected by a camera in the branch that meets the national data security standard, which can specifically include the reasonable moving track of the customer in the business handling area, waiting area and other areas in the branch, and the relevant gestures in the business handling process; audio stream data collected by a microphone array, wherein the audio stream data can contain the tone characteristics, volume characteristics and environmental background sound characteristics of the customer; in addition, it can also include the interaction data generated by the customer when handling business through the touch screen device in the branch, or the data containing the business to be handled submitted through the appointment system. The collection of all these data is authorized after the customer is explicitly informed, signs an electronic agreement or agrees to the privacy policy framework that meets the national laws and regulations and industry regulations when starting to handle the business, to ensure the legality and compliance of the business.

[0063] Specifically in operation S120, data desensitization processing is performed on the original behavior data to obtain the behavior data of the customer in the monitoring area. In some specific embodiments, data desensitization processing refers to performing irreversible conversion or masking operation on the original behavior data containing personal sensitive information immediately at the source of data generation, aiming to completely eliminate or greatly reduce the risk of identifying and restoring personal privacy, while retaining as much value as possible for subsequent behavior analysis tasks.

[0064] Specifically, in various embodiments of this application, the original behavioral data includes image data, audio data, and text data. The data anonymization processing of the original behavioral data includes at least one of the following: performing shape blurring processing on the customer's image data; performing voiceprint anonymization processing on the customer's audio data; and performing sensitive information filtering processing on the customer's text data. In some specific embodiments, shape blurring processing on image data includes: after the video stream is input, accurately locating the overall outline (including face and body) of each customer in the image, applying high-intensity Gaussian blur or pixelated mosaic processing to each detected human body area, making the individual completely unrecognizable, retaining only key macroscopic information for behavioral analysis such as their overall posture, movement amplitude, direction of movement, and speed. In other specific embodiments, voiceprint anonymization processing on audio data includes: the acquired original audio is first distinguished from different speakers using voiceprint separation technology, and voiceprint feature annihilation processing is performed on the audio segments of each speaker, for example, by maintaining the content but changing its pitch, timbre, and other acoustic features (voice changing processing), or by using specific audio filtering techniques to filter out its unique biometric frequency components. After processing, the audio still clearly reflects the speech rate and semantics, but it can no longer be traced back to a specific individual's identity using voiceprint recognition technology. In some specific embodiments, the sensitive information filtering process for text data includes: replacing and masking sensitive fields entirely with meaningless general symbols such as "*", or randomly generated pseudo-tokens, to ensure that no real personal information exists in plaintext form in any storage space on the server. Data processed by any or a combination of the above methods becomes privacy-protected "behavioral data" that can be used for subsequent feature extraction.

[0065] like Figure 7 As shown, this application also provides a method for obtaining global model parameters, applied to the server side, including operations S400 to S600.

[0066] Specifically, in operation S400, initial weight parameters are received from multiple different edge devices. These initial weight parameters are obtained after training an initial local model based on the historical behavioral feature sets of local customers on each edge device. In some specific embodiments, these multiple different edge devices are typically widely deployed in bank branches in different cities. Each edge device operates independently in its local environment, based on its unique customer base and historical data. The server collects, stores, and manages these initial weight parameters from different regions, customer groups, and hardware environments. This results in the server aggregating highly diverse and heterogeneous data features, providing a data foundation for subsequently aggregating a global model with generalization capabilities.

[0067] Specifically, in operation S500, a weighted average aggregation process is performed based on the plurality of initial weight parameters to obtain the global model parameters. After receiving the initial weight parameters from the plurality of edge devices, the server can assign a weight to each initial weight parameter according to the metadata (such as the number of samples used for training, the loss function value of the model, etc.) attached to the initial weight parameter when each edge device uploads the initial weight parameter. An edge device that has processed a large amount of data may be given a higher weight for the initial weight parameter it uploads; conversely, an edge device with a small amount of data may have a lower weight for the initial weight parameter. In a specific process, the server multiplies each device's initial weight parameter by its corresponding weight value, then adds all these products together, and finally divides by the sum of all weight values. Through this aggregation, the features that are common and consistent among all sites in the global model parameters are strengthened, while abnormal parameters that may be introduced due to local data bias or noise at a site are weakened and smoothed.

[0068] Specifically, in operation S600, the global model parameters are sent to each of the edge devices. In some embodiments, after successfully generating new global model parameters, the server will start a distribution process. It will actively push or respond to requests to send a copy of the global model parameters to each edge device through a secure encrypted communication protocol. The content sent is the global model parameters without any localized information. After the distribution is complete, each edge device will update its local model based on the global model parameters.

[0069] In this embodiment, the operations S100-S300, S400, S500, S600, S310, S320, S330, S340, S311, S3111, S3112, S341, S110, and S120 described above can also be performed at the system end, and will not be described again here.

[0070] Based on the above-described data processing method applied to the edge device end, the application also provides a data processing apparatus applied to the edge device end, which will be described below in combination with Figure 9 The apparatus will be described in detail.

[0071] Figure 8 A structural block diagram of the data processing apparatus applied to the edge device end according to an embodiment of the application is schematically shown.

[0072] As Figure 8 shown, the data processing apparatus 200 applied to the edge device end of this embodiment includes a first acquisition unit 210, a first processing unit 220, and a second processing unit 230.

[0073] The first obtaining unit 210 is configured to obtain behavior data of a customer in a monitoring area under the condition that the customer agrees or authorizes. In some embodiments, the first obtaining unit 210 is specifically configured to control and schedule various types of sensor hardware at the edge device end, such as a camera and a microphone. It starts a data collection program according to a preset rule, for example, when the customer confirms authorization to perform security monitoring behavior in a specific business handling area, and buffers the collected original data stream after preliminary privacy blurring processing, to ensure the integrity and timeliness of the data, and to ensure that the data does not involve the privacy of the behavior individual, thereby providing a stable and reliable data source for subsequent processing units. In an embodiment, the first obtaining unit 210 can be configured to perform the operation S100 described above, and the operation S110 and the operation S120, which will not be described herein again.

[0074] The first processing unit 220 is configured to perform feature extraction processing on the behavior data to obtain a customer behavior feature set. In some embodiments, after the first processing unit 220 receives the original behavior data from the first obtaining unit 210, it analyzes the video stream to extract the coordinates of the key nodes, processes the audio stream to extract the voiceprint features, integrates, aligns and normalizes all the extracted multi-modal features, and finally forms a standard format customer behavior feature set, which is transmitted to the second processing unit 230. The first processing unit 220 can be configured to perform the operation S200 described above, which will not be described herein again.

[0075] The second processing unit 230 is configured to determine a customer behavior state based on the customer behavior feature set and a pre-trained local model, wherein the pre-trained local model is obtained based on local statistical characteristic parameters and global model parameter aggregation training processing. In some embodiments, after the second processing unit 230 receives the customer behavior feature set generated by the first processing unit 220, it performs forward inference calculation of the model, calculates and analyzes according to the input features, and finally outputs one or more quantitative judgment results (such as classification labels, risk probability scores, etc.) about the customer behavior state. In addition, by receiving and executing instructions from the server end, it starts the local training process, manages and separates the initial weight parameters and the local statistical characteristic parameters, and after receiving new global model parameters, performs the reconstruction and update tasks of the local model. In an embodiment, the second processing unit 230 can be configured to perform the operation S300 described above, and the operation S310, the operation S320, the operation S330, the operation S340, the operation S3111, the operation S3112, the operation S311, and the operation S341, which will not be described herein again.

[0076] According to an embodiment of the present application, any of the plurality of modules in the first obtaining unit 210, the first processing unit 220 and the second processing unit 230 can be combined in one module, or any of the plurality of modules can be split into a plurality of modules. Alternatively, at least part of the function of one or more of the plurality of modules can be combined with at least part of the function of another module, and implemented in one module. According to an embodiment of the present application, at least one of the first obtaining unit 210, the first processing unit 220 and the second processing unit 230 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on chip, a system on board, a system in package, an application specific integrated circuit (ASIC), or any other reasonable manner of integrating or packaging a circuit, etc. or implemented by hardware or firmware, or implemented in any one of software, hardware and firmware or in a proper combination of any of these. Alternatively, at least one of the first obtaining unit 210, the first processing unit 220 and the second processing unit 230 can be at least partially implemented as a computer program module which, when executed, can perform the corresponding function.

[0077] Figure 9 A block diagram of an electronic device suitable for implementing the data processing method according to an embodiment of the present application is schematically shown.

[0078] As shown in Figure 9 , the electronic device 900 according to an embodiment of the present application includes a processor 901 which can perform various appropriate actions and processes according to a program stored in a read only memory (ROM) 902 or a program loaded from a storage portion 908 into a random access memory (RAM) 903. The processor 901 can include, for example, a general purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special purpose microprocessor (such as an application specific integrated circuit (ASIC)), etc. The processor 901 can also include an on-board memory for cache use. The processor 901 can include a single processing unit or a plurality of processing units for performing different actions of the method processes according to an embodiment of the present application.

[0079] In the RAM 903, various programs and data required for the operation of the electronic device 900 are stored. The processor 901, the ROM 902, and the RAM 903 are connected to each other via the bus 904. The processor 901 performs various operations of the method flow according to the embodiments of the present application by executing the programs in the ROM 902 and / or the RAM 903. It should be noted that the programs can also be stored in one or more memories other than the ROM 902 and the RAM 903. The processor 901 can also perform various operations of the method flow according to the embodiments of the present application by executing the programs stored in the one or more memories.

[0080] According to the embodiments of the present application, the electronic device 900 can further include an input / output (I / O) interface 905, which is also connected to the bus 904. The electronic device 900 can further include one or more of the following components connected to the input / output (I / O) interface 905: an input part 906 including a keyboard, a mouse, and the like; an output part 907 including a cathode ray tube (CRT), a liquid crystal display (LCD), and the like, and a speaker, and the like; a storage part 908 including a hard disk, and the like; and a communication part 909 including a network interface card such as a LAN card, a modem, and the like. The communication part 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the input / output (I / O) interface 905 as necessary. A removable medium 911 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, and the like is mounted on the drive 910 as necessary, so that a computer program read therefrom is installed in the storage part 908 as necessary.

[0081] The present application also provides a computer readable storage medium, which can be included in the apparatus described in the above embodiments; or can exist separately without being assembled into the apparatus. The above computer readable storage medium carries one or more programs, when the one or more programs are executed, the method according to the embodiments of the present application is implemented.

[0082] According to an embodiment of the present application, the computer readable storage medium can be a non-transitory computer readable storage medium, which can include, but is not limited to, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), portable compact disk read only memory (CD-ROM), optical storage, magnetic storage, or any suitable combination of the foregoing. In this application, a computer readable storage medium can be any tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. For example, according to an embodiment of the present application, a computer readable storage medium can include one or more of the above-described ROM 902 and / or RAM 903 and / or one or more memory other than the ROM 902 and the RAM 903.

[0083] Embodiments of the present application also include a computer program product, which includes a computer program containing program codes for executing the methods shown in the flowcharts. When the computer program product is run in a computer system, the program codes are used to make the computer system implement the data processing method provided by the embodiments of the present application.

[0084] The above-described functions defined in the apparatus of the embodiments of the present application are performed when the computer program is executed by the processor 901. According to an embodiment of the present application, the above-described apparatus and the like can be implemented by computer program modules.

[0085] In one embodiment, the computer program can rely on tangible storage media such as optical storage media, magnetic storage media, and the like. In another embodiment, the computer program can also be transmitted, distributed, and downloaded in the form of signals on network media. The computer program is downloaded and installed through the communication part 909 and / or installed from the detachable medium 911. The program codes contained in the computer program can be transmitted by any appropriate network media, including but not limited to wireless, wired, and the like, or any suitable combination of the foregoing.

[0086] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 909 and / or installed from the detachable medium 911. When the computer program is executed by the processor 901, the above-described functions defined in the system of the embodiments of the present application are performed. According to an embodiment of the present application, the above-described apparatus and the like can be implemented by computer program modules.

[0087] According to embodiments of the present application, program code for implementing the computer programs provided by embodiments of the present application can be written in any combination of one or more programming languages, and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. The programming language includes, but is not limited to, such languages as Java, C++, python, "C" language, or the like. The program code can execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the remote computing device, or entirely on the remote computing device or server. In the latter scenario, the remote computing device can be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computing device, such as through the Internet using an Internet Service Provider.

[0088] The flow diagrams and the block diagrams in the drawings are illustrations of the possible architectures, functional and operational, of the methods and computer program products according to various embodiments of the present application. In this regard, each block in the flow diagrams or block diagrams can represent a module, a segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flow diagrams, and combinations thereof, can be implemented by special purpose hardware-based systems that perform the specified functions or operations, or combinations of special purpose hardware and computer instructions.

[0089] Those skilled in the art will understand that features recited in the various embodiments of the present application can be combined and / or integrated in various ways, even if such combinations or integrations are not expressly noted in the present application. In particular, features recited in the various embodiments of the present application can be combined and / or integrated in various ways without departing from the spirit and teachings of the present application. All such combinations and / or integrations are within the scope of the present application.

[0090] The embodiments of the present application have been described above. However, these embodiments are merely for the purpose of illustration and are not intended to limit the scope of the present application. Although the respective embodiments are described above separately, this does not mean that the measures in the respective embodiments cannot be used advantageously in combination. Without departing from the scope of the present application, those skilled in the art can make various substitutions and modifications, and these substitutions and modifications shall fall within the scope of the present application.

Claims

1. A data processing method, characterized by, The method applied to an edge device end comprises: obtaining behavior data of a customer in a monitoring area under the condition that the customer agrees or authorizes; performing feature extraction processing on the behavior data to obtain a customer behavior feature set; determining a customer behavior state based on the customer behavior feature set and a pre-trained local model, wherein the pre-trained local model is obtained based on local statistical characteristic parameters and global model parameter aggregation training processing.

2. The method of claim 1, wherein, The local model obtained based on the local statistical characteristic parameters and the global model parameter aggregation training processing comprises: training an initial local model based on a historical behavior feature set of a local customer to obtain local statistical characteristic parameters and initial weight parameters; sending the initial weight parameters to a server end; receiving global model parameters issued by the server end, wherein the global model parameters are obtained based on initial weight parameters uploaded by multiple edge device ends; generating the local model based on the global model parameters and the local statistical characteristic parameters.

3. The method of claim 2, wherein, The initial local model trained based on the historical behavior feature set of the local customer to obtain the local statistical characteristic parameters and the initial weight parameters comprises: performing iterative training processing based on the historical behavior feature set of the local customer and the initial local model to update the weight of the initial local model to obtain the initial weight parameters and the local statistical characteristic parameters, wherein the local statistical characteristic parameters are obtained based on the statistical distribution of feature data in the historical behavior feature set flowing through the initial local model.

4. The method of claim 3, wherein, The local statistical characteristic parameters obtained based on the statistical distribution of feature data in the historical behavior feature set flowing through the initial local model comprise: sequentially performing statistical distribution processing on feature data in the historical behavior feature set flowing through the initial local model in batches to obtain statistical distribution parameters; generating the local statistical characteristic parameters by iterative updating based on the moving average of the statistical distribution parameters determined in multiple batches.

5. The method of claim 2, wherein, The local model generated based on the global model parameters and the local statistical characteristic parameters comprises: loading the global model parameters to the weight part of the initial local model and loading the local statistical characteristic parameters to the standardization part of the initial local model to generate the local model.

6. The method of claim 2, wherein, The behavior data of the customer in the monitoring area obtained under the condition that the customer agrees or authorizes comprises: obtaining original behavior data of the customer in the monitoring area under the condition that the customer agrees or authorizes; performing data desensitization processing on the original behavior data to obtain the behavior data of the customer in the monitoring area.

7. The method of claim 6, wherein, The original behavior data comprises image data, audio data and text data, and the data desensitization processing performed on the original behavior data comprises at least one of the following: performing shape blurring processing on the image data of the customer; performing voiceprint desensitization processing on the audio data of the customer; performing sensitive information filtering processing on the text data of the customer.

8. A global model parameter obtaining method, characterized in that, The method applied to a server end comprises: receive initial weight parameters from a plurality of different edge device terminals, wherein the initial weight parameters are obtained after training an initial local model based on a historical behavior feature set of a local customer of each edge device terminal; perform weighted average aggregation processing based on a plurality of the initial weight parameters to obtain the global model parameters; send the global model parameters to each of the edge device terminals respectively.

9. A data processing apparatus, characterized by, An apparatus applied to an edge device terminal, the apparatus comprising: a first obtaining unit configured to obtain behavior data of a customer in a monitoring area under the condition that the customer agrees or authorizes; a first processing unit configured to perform feature extraction processing on the behavior data to obtain a customer behavior feature set; a second processing unit configured to determine a customer behavior state based on the customer behavior feature set and a pre-trained local model, wherein the pre-trained local model is obtained based on local statistical characteristic parameters and global model parameter aggregation training processing. 10.An electronic device comprising: one or more processors; a memory for storing one or more computer programs, characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1-8.

11. A computer readable storage medium having stored thereon a computer program or instructions, characterized in that, The computer program or instructions are executed by the processor to implement the steps of the method according to any one of claims 1-8.

12. A computer program product comprising computer programs or instructions, characterized in that, The computer program or instructions are executed by the processor to implement the steps of the method according to any one of claims 1-8. The computer program or instructions are executed by the processor to implement the steps of the method according to any one of claims 1-8.

Images