Abnormal behavior dynamic identification method and system based on multi-view decoupling representation learning

Abstract

The invention provides an abnormal behavior dynamic identification method and system based on multi-view decoupling representation learning, and relates to the field of multi-view data modeling. According to the orthogonal constraint-based mask vector quantization method designed by the invention, the codebook vectors can be promoted to be pairwise orthogonal, the decoupling property and generalization ability of vector features are improved, and a discrete representation-oriented adaptive mask mechanism constructed on the basis can effectively remove redundant information in multi-view representation and reserve complementary information, so that the method has the advantages of high robustness and high robustness. According to the method, the robustness and generalization ability of the model are improved, and more effective and non-redundant representation can be obtained, so that the abnormal behavior of the individual is dynamically and accurately identified, and a more reliable feature basis is provided for downstream tasks. Besides, according to the designed time-sample monotonicity joint perception multi-period survival probability prediction method, cross-time monotonicity is realized by constructing a risk accumulation function, and cross-sample sorting monotonicity is realized by constructing focus survival loss and combining the focus survival loss with multi-period BCE loss.

Description

Technical Field

[0001] The present invention relates to the field of multi-view data modeling and abnormal behavior dynamic identification, and in particular to a method and system for abnormal behavior dynamic identification based on multi-view decoupled representation learning. Background Art

[0002] Dynamic abnormal behavior recognition is an important branch of machine learning and computer vision. It aims to detect and identify behaviors or events that deviate significantly from normal patterns by analyzing data. It has been widely used in intelligent security, medical monitoring, and traffic control, providing technical support for preventing accidents, improving efficiency, and ensuring safety. However, most of the existing abnormal behavior recognition methods are static, that is, predicting whether abnormal behavior will occur at a certain time point or period in the future. It is difficult to effectively characterize the dynamics of abnormal behavior, such as the time when abnormal behavior occurs. There are two main types of existing methods that can be used for dynamic abnormal behavior recognition: (1) traditional survival analysis methods, which are often highly dependent on assumptions and have difficulty capturing complex relationships, especially complex nonlinear relationships in multiple views; (2) dynamic deep survival methods, which have advantages in adaptively fitting complex relationships without assuming survival time distribution or other factors, but it is difficult to ensure monotonicity at the time and sample levels.

[0003] In recent years, as all walks of life have increasingly adopted digital, networked and intelligent technologies, various data information has been recorded in real time in various information systems, forming multi-view data. Multi-view data can complement, coordinate and verify each other from multiple perspectives, and provide a more accurate description of the same object.

[0004] In this context, the demand for dynamic identification of abnormal behaviors has become increasingly prominent. Multi-view data provides a more comprehensive data analysis perspective for dynamic identification of abnormal behaviors, capturing the changes in behavior over time, thereby improving the accuracy and efficiency of abnormal behavior identification. However, existing solutions for multi-view modeling cannot effectively remove redundant information. There are two main methods for multi-view modeling: (1) direct concatenation, which obtains view features and embeddings from different views and directly concatenates them, but ignores information redundancy; (2) attention mechanism, which can adaptively identify and capture key information of multi-view data for downstream tasks, but while reducing redundant information, it also leads to a reduction in the importance weight of complementary information, and cannot accurately retain complementary information or remove redundant information. Summary of the invention

[0005] 1. Technical issues to be resolved In view of the deficiencies in the prior art, the present invention provides a method and system for dynamic identification of abnormal behaviors based on multi-view decoupled representation learning, which solves the technical problem that the prior methods cannot accurately retain complementary information and remove redundant information.

[0006] (II) Technical solution To achieve the above objectives, the present invention is implemented through the following technical solutions: A method for dynamic recognition of abnormal behavior based on multi-view decoupled representation learning, based on a pre-trained model, the model includes multiple specific view encoders, a Transformer encoder, a discrete multi-view encoder, a view fusion decoder based on an adaptive mask mechanism, and a prediction layer; the method includes: Collect and preprocess various view data related to the task and obtain the feature vector corresponding to each view; Taking each of the feature vectors as input of the corresponding specific view encoder to obtain a corresponding feature embedding vector; Using each of the feature embedding vectors as an input of the Transformer encoder to obtain a corresponding enhanced feature embedding vector, and concatenating each of the enhanced feature embedding vectors to obtain a multi-view embedding vector; Using the multi-view embedding vector as input of the discrete multi-view encoder to obtain a matching representation with a preset codebook; Taking the matching representation as the input of the view fusion decoder, finding redundant features in discrete latent variables through pairwise redundant probes, dynamically obtaining a mask vector to extract unique elements from the matching representation and set repeated elements to zero, and obtaining a mask representation; The mask representation is used as the input of the prediction layer to obtain the original output of multiple future moments, and the risk probability for predicting the occurrence of abnormal behavior is generated through a risk accumulation function.

[0007] Preferably, the step of using the multi-view embedding vector as an input of the discrete multi-view encoder to obtain a matching representation with a preset codebook comprises: Using a nearest neighbor search method, obtaining a set of codebook indexes corresponding to the multi-view embedding vector and the potential vector of the codebook; The codebook index is used as a discrete latent variable to map each embedding of the multi-view embedding vector to the closest latent vector in the codebook to obtain a matching representation.

[0008] Preferably, the method uses the mask representation as the input of the prediction layer, obtains the original output at multiple moments in the future, and generates the risk probability for predicting the occurrence of abnormal behavior through a risk accumulation function; including: The mask represents As input to the prediction layer, a set of predicted future m The original output at that moment , where o j Indicates time j The original output of The risk accumulation function is used to generate the risk probability for predicting the occurrence of abnormal behavior. , calculated at any time u The risk probability formula is as follows: Where exp is an exponential function.

[0009] Preferably, in the pre-training stage of the model, a codebook loss, a consistency loss and an orthogonal loss are constructed for the discrete multi-view encoder; wherein: The codebook loss is expressed as: Among them, sg[·] means stopping the gradient operation, Represents the square of the L2 norm, where the enhanced feature embedding vector is calculated after the gradient stop operation Matching expression The loss of L2 distance between them; The consistency loss is expressed as: The enhanced feature embedding vector is calculated here The matching representation after the gradient stop operation The loss of L2 distance between them; The orthogonal loss is expressed as: in, represents the L2 norm; triu(∙) means returning the upper triangular part of the matrix and setting other elements to zero; Indicates that the codebook The vectors in are normalized; Indicates that the codebook The vectors in are normalized and transposed.

[0010] Preferably, in the pre-training stage of the model, a multi-period BCE loss is also constructed, expressed as: Among them, y t Indicates time t The true label is 0 or 1; p t Indicates the time used to predict t The risk probability of abnormal behavior occurring; log represents the logarithmic function.

[0011] Preferably, in the pre-training stage of the model, a multi-period BCE loss is used to determine whether the individual as a sample has a risk during the observation period based on the true label; Define if the sample i If no risk event has occurred, mark δ i =0, otherwise δ i =1; in two comparable samples i and samples j If the sample i Risk time T i Earlier than the sample j Risk time T j , and the sample i The risk probability Higher than sample j The risk probability , then the sorting is correct, otherwise the sorting is wrong; the function clamp(∙) means limiting the input to [0,∞), which is used to exclude comparable sample pairs with correct sorting; Construct the focal survival loss for pairs of comparable samples with ordering errors, expressed as: Among them, the use of represents comparable sample pairs, represents the indicator function; Representation sample i In time A =min( T , C ), A =min( T , C ) indicates the actual time when the event occurred T and observation end time C The smaller value of ; σ Represents the Sigmoid function, and since in the formula The operation is not differentiable. σ (x) is approximately: in, γ is a tuning parameter used to control the sensitivity of comparable changes. σ The range of (x) is [0, 1).

[0012] Preferably, in the pre-training stage of the model, the multi-period BCE loss is used as the main target, the focal survival loss is used as the auxiliary target, and an adaptive gradient balancing method is designed; specifically comprising: make and They represent the gradient vectors of multi-period BCE loss and focal survival loss at the τth iteration respectively; calculate and The cosine similarity between them is used to determine whether there is a conflict between the two, and the Gram-Schmidt orthogonalization is used to transform Updated to , the formula is: Among them, proj u (v) represents a vector v In vector u The projection on Projection to the gradient vector Above; cosine represents cosine similarity.

[0013] A dynamic abnormal behavior recognition system based on multi-view decoupled representation learning, based on a pre-trained model, the model includes multiple specific view encoders, a Transformer encoder, a discrete multi-view encoder, a view fusion decoder based on an adaptive mask mechanism, and a prediction layer; the system includes: The preprocessing module is used to collect and preprocess various view data related to the task and obtain the feature vector corresponding to each view; An embedding module, configured to use each of the feature vectors as an input of the corresponding specific view encoder to obtain a corresponding feature embedding vector; An enhancement module, used to take each of the feature embedding vectors as an input of the Transformer encoder, obtain a corresponding enhanced feature embedding vector, and concatenate each of the enhanced feature embedding vectors to obtain a multi-view embedding vector; A matching module, configured to use the multi-view embedding vector as an input of the discrete multi-view encoder to obtain a matching representation with a preset codebook; A mask module, configured to use the matching representation as an input of the view fusion decoder, find redundant features in discrete latent variables through pairwise redundant probes, dynamically obtain a mask vector to extract unique elements from the matching representation and set repeated elements to zero, and obtain a mask representation; The prediction module is used to use the mask representation as the input of the prediction layer, obtain the original output of multiple moments in the future, and generate the risk probability for predicting the occurrence of abnormal behavior through the risk accumulation function.

[0014] A storage medium stores a computer program for dynamic identification of abnormal behavior based on multi-view decoupled representation learning, wherein the computer program enables a computer to execute the above-mentioned method for dynamic identification of abnormal behavior.

[0015] An electronic device, comprising: One or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the programs include a method for executing the abnormal behavior dynamic identification method as described above.

[0016] (III) Beneficial effects The present invention provides a method and system for dynamic recognition of abnormal behavior based on multi-view decoupled representation learning. Compared with the prior art, it has the following beneficial effects: The present invention uses the multi-view embedding vector as the input of the discrete multi-view encoder to obtain the matching representation with the preset codebook, improves the decoupling and generalization ability of the vector features, and constructs an adaptive masking mechanism for discrete representation on this basis, which can effectively remove redundant information in multi-view data and retain complementary information, improves the robustness and generalization ability of the model, and can obtain more effective and non-redundant representation, thereby dynamically and accurately identifying individual abnormal behaviors. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 A block diagram of a method for dynamic identification of abnormal behaviors based on multi-view decoupled representation learning provided by an embodiment of the present invention; Figure 2 A flowchart of a method for dynamic identification of abnormal behaviors based on multi-view decoupled representation learning provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention are clearly and completely described. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] The embodiments of the present application provide a method and system for dynamic identification of abnormal behaviors based on multi-view decoupled representation learning, which solves the technical problem that existing methods are unable to accurately retain complementary information and remove redundant information, and further solves the technical problem of how to simultaneously adapt to time monotonicity and sample monotonicity, thereby achieving the purpose of assisting individuals and organizations to obtain non-redundant information and make accurate decisions.

[0021] The technical solution in the embodiment of the present application is to solve the above technical problems, and the overall idea is as follows: The orthogonal constraint-based mask vector quantization method designed in the embodiment of the present invention can make the codebook vectors orthogonal to each other, improve the decoupling and generalization ability of vector features, and the adaptive mask mechanism for discrete representation constructed on this basis can effectively remove redundant information in multi-view representation and retain complementary information, thereby improving the robustness and generalization ability of the model, and can obtain more effective and non-redundant representation, thereby dynamically and accurately identifying individual abnormal behaviors, providing a more reliable feature basis for downstream tasks.

[0022] Furthermore, the multi-period survival probability prediction method with joint perception of time-sample monotonicity designed in the embodiment of the present invention realizes cross-time monotonicity by constructing a risk accumulation function, that is, the predicted survival probability decreases monotonically with time; at the same time, a focal survival loss is constructed in combination with a multi-period BCE loss to realize cross-sample ranking monotonicity. The combination of the two can accurately capture the risk ranking of samples between different time points, thereby more accurately predicting the time point of occurrence of abnormal behavior. However, there is a potential conflict between the two. Therefore, an adaptive gradient balancing method is designed to resolve this conflict, thereby ensuring that the model can effectively balance the two objectives during parameter optimization.

[0023] In addition, it is understandable that the samples in the embodiments of the present invention may be, but are not limited to, individuals. Abnormal behaviors reflect abnormal changes in individuals' recent behaviors. Detecting these abnormal changes is of great significance for reducing the occurrence of high-risk events. At the same time, studying individual abnormal behaviors also has important application value for various tasks such as product recommendations, community predictions, and abnormal warnings. Since the occurrence of behaviors has an obvious temporal relationship, the present invention effectively identifies whether individual abnormal behaviors occur and their specific occurrence time by constructing a risk accumulation function and a focal survival loss function.

[0024] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0025] Embodiment 1: like Figure 1As shown, an embodiment of the present invention provides a method for dynamic recognition of abnormal behavior based on multi-view decoupled representation learning, based on a pre-trained model, the model includes multiple specific view encoders, a Transformer encoder, a discrete multi-view encoder, a view fusion decoder based on an adaptive mask mechanism, and a prediction layer; the method includes: S1. Collect and preprocess various view data related to the task and obtain the feature vector corresponding to each view; S2. Taking each of the feature vectors as the input of the corresponding specific view encoder to obtain a corresponding feature embedding vector; S3, taking each of the feature embedding vectors as the input of the Transformer encoder, obtaining a corresponding enhanced feature embedding vector, and concatenating each of the enhanced feature embedding vectors to obtain a multi-view embedding vector; S4, using the multi-view embedding vector as an input of the discrete multi-view encoder to obtain a matching representation with a preset codebook; S5, using the matching representation as the input of the view fusion decoder, finding redundant features in discrete latent variables through pairwise redundant probes, dynamically obtaining a mask vector to extract unique elements from the matching representation and set repeated elements to zero, and obtaining a mask representation; S6. Using the mask representation as the input of the prediction layer, obtaining the original outputs at multiple moments in the future, and generating the risk probability for predicting the occurrence of abnormal behavior through a risk accumulation function.

[0026] The embodiment of the present invention uses the multi-view embedding vector as the input of the discrete multi-view encoder to obtain a matching representation with a preset codebook, thereby improving the decoupling and generalization capabilities of the vector features. The adaptive masking mechanism for discrete representation constructed on this basis can effectively remove redundant information in multi-view data and retain complementary information, thereby improving the robustness and generalization capabilities of the model, and can obtain a more effective and non-redundant representation, thereby dynamically and accurately identifying abnormal behaviors of individuals.

[0027] like Figure 2 As shown, Figure 2 The complete process of the abnormal behavior dynamic recognition method based on multi-view decoupled representation learning is given. Figure 2 The following are the steps of the above solution: In step S1, various view data related to the task are collected and preprocessed to obtain a feature vector corresponding to each view.

[0028] In this step, collect task-related kThe collected multi-view data sets are processed by using data processing methods such as missing filling and normalization, which can improve the quality of the multi-view data sets and ensure the integrity of the data. After data processing, each view obtains the corresponding feature vector, and the feature vector is defined. ,in num Represents the index of the view, R represents the set of real numbers, q Represents the number of features of the corresponding view.

[0029] Exemplarily, the task of the embodiment of the present invention is set to be dynamic identification of abnormal behavior, and the multi-view data here can be information such as numerical values, texts, and relationship networks related to individual behaviors.

[0030] In step S2, each of the feature vectors is used as the input of the corresponding specific view encoder to obtain a corresponding feature embedding vector.

[0031] In this step, based on the specific view encoder pre-built for each view, each view feature is mapped to the corresponding feature embedding vector, thereby enhancing the features of the vector; specifically, it includes: First, the feature vector Perform numerical embedding. The process of numerical embedding is as follows: Among them, v i ∈R d represents the trainable weight vector, b i Representation characteristics The bias term, d Indicates the dimension.

[0032] Secondly, in q After numerical embedding of eigenvalues, we get the feature embedding vector .

[0033] In step S3, each of the feature embedding vectors is used as the input of the Transformer encoder to obtain a corresponding enhanced feature embedding vector, and each of the enhanced feature embedding vectors is concatenated to obtain a multi-view embedding vector.

[0034] In this step, the feature embedding vector is input into the Transformer encoder to form a new enhanced feature embedding matrix , taking into full account the global dependencies between features, each enhanced feature embedding vector is concatenated, and the concatenated multi-view feature embedding vector is , that is is obtained from each view Spliced ​​together, frepresents the sum of the number of features in all views, .

[0035] In step S4, the multi-view embedding vector is used as an input of the discrete multi-view encoder to obtain a matching representation with a preset codebook.

[0036] In an optional embodiment, this step includes: S41. Using a nearest neighbor search method, obtain a set of codebook indexes corresponding to the multi-view embedding vector and the potential vector of the codebook.

[0037] A codebook based on discrete latent space is designed to map continuous feature vectors into discrete representations. The discrete multi-view encoder maps multi-view embedding vectors into a set of codebook indices. Each codebook index corresponds to a latent vector in the codebook. Let the codebook of latent embedding be ,in l Expressed as the number of features in the codebook, d Expressed as the dimension, the embedding vector is and Make a comparison.

[0038] Use the nearest neighbor search method to find the closest discrete embedding in the codebook to replace the continuous representation, producing a set of discrete latent variables (i.e., codebook indices) .

[0039] Computing discrete latent variables Z The formula is as follows: in, Indicates to minimize the value in the expression, here it means to minimize and The Euclidean distance between them is the smallest.

[0040] S42: Use the codebook index as a discrete latent variable to map each embedding of the multi-view embedding vector to a closest latent vector in the codebook to obtain a matching representation.

[0041] Based on discrete latent variables Z , by Each embedding in is mapped to The closest latent vector is used to generate the matching representation .

[0042] Understandably, since the matching representation There are repeated elements in it, so it has redundant information within and across data views.

[0043] In step S5, the matching representation is used as the input of the view fusion decoder, redundant features in discrete latent variables are found through pairwise redundant probes, and a mask vector is dynamically obtained to extract unique elements from the matching representation and set repeated elements to zero to obtain a mask representation.

[0044] In this step, a view fusion decoder based on an adaptive mask mechanism is designed to remove redundant information within and across data views, including: First, As input to the decoder, and through pairwise redundant probes (i.e., through pairwise comparison and analysis), the discrete latent variables are found Z Redundant features in, dynamically obtain mask vector to ensure that Extract unique elements and set repeated elements to zero; the mask vector formula is as follows: Among them, m i is the mask vector m The i elements.

[0045] Secondly, based on the given mask vector, the mask representation Can be represented by matching The Hadamard product of the mask vector is generated as follows: in, Represents element-wise multiplication.

[0046] It should be noted that the embodiments of the present invention have successfully solved the problem of how to decouple to extract complementary information while discarding redundant information by adopting the above method, thereby obtaining a more efficient and non-redundant representation. This method not only enables the model to learn a more comprehensive feature representation, but also improves the overall quality of the data. It effectively integrates complementary information from different views, thereby providing a solid foundation for in-depth analysis and accurate modeling.

[0047] In step S6, the mask representation is used as the input of the prediction layer to obtain the original output of multiple future moments, and the risk probability for predicting the occurrence of abnormal behavior is generated through a risk accumulation function.

[0048] In this step, after obtaining the extracted representation After that, the risk accumulation function is constructed to extract the representation Mapping to risk probabilities at different moments in the future enables accurate identification and dynamic tracking of abnormal behaviors while achieving monotonicity across time.

[0049] In an optional embodiment, this step includes: S61, the mask represents As input to the prediction layer, a set of predicted future m The original output at that moment (output without sigmoid function), where o j Indicates time j The original output of .

[0050] S62: Generate a risk probability for predicting the occurrence of abnormal behavior through the risk accumulation function , calculated at any time u The risk probability formula is as follows: Where exp is an exponential function.

[0051] It should be noted that in the above risk probability formula, the j The exponential function is non-negative and preserves the risk ranking. The sum function gives time monotonicity. The external exponential function returns a value that aligns with the range of survival probabilities while preserving time monotonicity and ensuring that the final output increases monotonically over time (i.e., 0 <p1≤p2≤⋯≤p m <1).

[0052] And, j Subscript in j The corresponding time period is (t j-1 ,t j ), but because the calculation is the cumulative probability of risk, p u Subscript in u The corresponding time period (0, t u ), there are differences between the two.

[0053] The risk accumulation function introduced in the embodiment of the present invention can accurately predict the possibility of abnormal behavior by accumulating the risk probability of each time period, thereby dynamically capturing the evolution of abnormal behavior in time series data. In addition, the function can adapt to a variety of data structures and application scenarios. By adjusting the prediction layer and the risk accumulation function, it can be flexibly applied to different prediction tasks, so that the model can more accurately capture the risk trend that changes over time, thereby improving the accuracy and reliability of the prediction.

[0054] In particular, in an optional embodiment, during the pre-training phase of the model: The embodiment of the present invention constructs codebook loss, consistency loss and orthogonal loss for the discrete multi-view encoder; wherein: (1) Codebook loss Since the argmin operation is not differentiable, back propagation is not possible. The gradient cannot be transmitted to To solve this problem, the straight-through estimator method is adopted. Gradient Copy to , which means that the codebook itself cannot be trained directly through gradient updates, so by calculating and The codebook loss between is used to optimize the codebook. The calculation formula of the codebook loss is as follows: Among them, sg[·] indicates stopping the gradient operation, Represents the square of the L2 norm, where the enhanced feature embedding vector is calculated after the gradient stop operation Matching expression The loss of L2 distance between them.

[0055] Need to explain, such as Figure 2 As shown, the codebook loss is mainly used to update the loss term of the codebook to minimize the Euclidean distance between the codebook vector and the encoder output vector, thereby ensuring that the codebook vector can be as close to the encoder output as possible.

[0056] (2) Consistency loss Since the codebook has no dimension, in order to prevent the codebook from growing arbitrarily, the encoder is helped to learn to generate a potential representation that is closer to the codebook vector by affecting the encoder's parameter update. The formula for calculating the consistency loss is as follows: The enhanced feature embedding vector is calculated here The matching representation after the gradient stop operation The loss of L2 distance between them.

[0057] (3) Orthogonal loss Orthogonal loss is used to train the codebook. By adding vector orthogonal constraints to the codebook, the codebook vectors are forced to be orthogonal to each other, making each element in the codebook as independent of each other as possible, thereby improving the decoupling and generalization ability of vector features. The formula for calculating orthogonal loss is as follows: in, represents the L2 norm; triu(∙) means returning the upper triangular part of the matrix and setting other elements to zero; Indicates that the codebook The vectors in are normalized; Indicates that the codebook The vectors in are normalized and transposed.

[0058] In addition, the embodiment of the present invention also constructs a multi-period BCE loss, which focuses on the risk prediction at a single moment. By independently evaluating the prediction at each moment, it can accurately reflect the risk status of the sample at a specific moment and achieve efficient risk ranking. However, since the prediction at each time point is independent, the multi-period BCE loss fails to capture the dependency between time points, and therefore cannot directly reflect the dynamic trend of risk changes in the time series; specifically as follows: (4) Multi-period BCE losses The core of multi-period BCE loss is to calculate the error between the predicted risk probability and the true label at each moment. The calculation formula is as follows: Among them, y t Indicates time t The true label is 0 or 1; p t Indicates the time used to predict t The risk probability of abnormal behavior occurring; log represents the logarithmic function.

[0059] Need to explain, such as Figure 2 As shown, the multi-period BCE loss measures the prediction probability p of the model at each moment by accumulating the losses at all moments. t and the true label y t The matching degree can realize the independent assessment of multi-period risks.

[0060] Furthermore, in order to adapt to the monotonicity of samples, an embodiment of the present invention constructs a focal survival loss function to impose penalties on samples with incorrect rankings, thereby achieving ranking monotonicity across samples, thereby improving the accuracy of the model in predicting risk probability.

[0061] This sample takes an individual as an example and uses multi-period BCE loss based on the true label to determine whether the sample has a risk during the observation period. i If no risk event has occurred, mark δ i =0, otherwise δ i =1; in two comparable samples i and samples j If the sample i Risk time T i Earlier than the sample j Risk time T j , and the sample i The risk probability Higher than sample j The risk probability , then the sorting is correct, otherwise the sorting is wrong; the function clamp(∙) means limiting the input to [0,∞), which is used to exclude comparable sample pairs that are sorted correctly. The details are as follows: (5) Focus survival loss The focal survival loss is mainly used to make up for the shortcomings of multi-period BCE. This loss can accurately capture the dependencies between time points and ensure that the model output satisfies the monotonicity between samples by penalizing incorrectly sorted sample pairs. The calculation formula is as follows: Among them, the use of represents comparable sample pairs, represents the indicator function; Representation sample i In time A =min( T , C ), A =min( T , C ) indicates the actual time when the event occurred T and observation end time C The smaller value of ; σ Represents the Sigmoid function, and since in the formula The operation is not differentiable. σ (x) is approximately: in, γ is a tuning parameter used to control the sensitivity of comparable changes. σ The range of (x) is [0, 1).

[0062] It should be noted that the focal survival loss can avoid the interference of non-risk and correctly ranked sample pairs on the model, making the model more focused on incorrectly ranked sample pairs. By penalizing incorrectly ranked sample pairs, the model's ability to capture changes in abnormal behavior over time is significantly enhanced, thereby more accurately predicting the time when abnormal behavior occurs.

[0063] It is necessary to point out that after constructing the multi-period BCE loss and focal survival loss, the two are trained together, and there will actually be conflicts in the parameter optimization process.

[0064] In order to solve the above problems, Figure 2 As shown, in the pre-training stage of the model, the embodiment of the present invention takes the multi-period BCE loss as the main target and the focal survival loss as the auxiliary target, designs an adaptive gradient balancing method, removes the conflicting components by orthogonalization, and retains the vertical component of the auxiliary loss gradient, thereby solving the potential conflict between the two losses; specifically, it includes: make and They represent the gradient vectors of multi-period BCE loss and focal survival loss at the τth iteration respectively; calculate and The cosine similarity between them is used to determine whether there is a conflict between the two, and the Gram-Schmidt orthogonalization is used to transform Updated to , the formula is: Among them, proj u (v) represents a vector v In vector u The projection on Projection to the gradient vector Above; cosine represents cosine similarity. When the cosine similarity of the two vectors is less than 0, it means that the directions of the two vectors are opposite or unrelated, and a projection operation is performed; in other cases, the directions of the two vectors are the same or related, and no projection operation is performed.

[0065] At this point, the total loss of the pre-trained model in the embodiment of the present invention can be obtained: The model is trained by minimizing the above total loss until the model converges. Based on the converged model, the entire process of the above abnormal behavior dynamic identification method can be executed.

[0066] Embodiment 2: The embodiment of the present invention provides an abnormal behavior dynamic recognition system based on multi-view decoupled representation learning, based on a pre-trained model, the model includes multiple specific view encoders, a Transformer encoder, a discrete multi-view encoder, a view fusion decoder based on an adaptive mask mechanism, and a prediction layer; the system includes: The preprocessing module is used to collect and preprocess various view data related to the task and obtain the feature vector corresponding to each view; An embedding module, configured to use each of the feature vectors as an input of the corresponding specific view encoder to obtain a corresponding feature embedding vector; An enhancement module, used to take each of the feature embedding vectors as an input of the Transformer encoder, obtain a corresponding enhanced feature embedding vector, and concatenate each of the enhanced feature embedding vectors to obtain a multi-view embedding vector; A matching module, configured to use the multi-view embedding vector as an input of the discrete multi-view encoder to obtain a matching representation with a preset codebook; A mask module, configured to use the matching representation as an input of the view fusion decoder, find redundant features in discrete latent variables through pairwise redundant probes, dynamically obtain a mask vector to extract unique elements from the matching representation and set repeated elements to zero, and obtain a mask representation; The prediction module is used to use the mask representation as the input of the prediction layer, obtain the original output of multiple moments in the future, and generate the risk probability for predicting the occurrence of abnormal behavior through the risk accumulation function.

[0067] Embodiment 3: An embodiment of the present invention provides a storage medium storing a computer program for dynamic identification of abnormal behavior based on multi-view decoupled representation learning, wherein the computer program enables a computer to execute the method for dynamic identification of abnormal behavior as described in Example 1.

[0068] Embodiment 4: An embodiment of the present invention provides an electronic device, including: One or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the programs include a method for executing the abnormal behavior dynamic identification method as described in Example 1.

[0069] It can be understood that the abnormal behavior dynamic identification system based on multi-view decoupling representation learning, storage medium and electronic device provided in the embodiments of the present invention correspond to the abnormal behavior dynamic identification method based on multi-view decoupling representation learning provided in the embodiments of the present invention. The explanations, examples and beneficial effects of the relevant contents can refer to the corresponding parts in the abnormal behavior dynamic identification method, and will not be repeated here.

[0070] In summary, compared with the prior art, the present invention has the following beneficial effects: 1. The mask vector quantization method based on orthogonal constraints designed in the embodiment of the present invention can make the codebook vectors orthogonal to each other, improve the decoupling and generalization ability of the vector features, and the adaptive mask mechanism for discrete representation constructed on this basis can effectively remove redundant information in the multi-view representation and retain complementary information, thereby improving the robustness and generalization ability of the model, and can obtain more effective and non-redundant representation, thereby dynamically and accurately identifying individual abnormal behaviors, providing a more reliable feature basis for downstream tasks.

[0071] 2. The multi-period survival probability prediction method with joint perception of time-sample monotonicity designed in the embodiment of the present invention realizes cross-time monotonicity by constructing a risk accumulation function, that is, the predicted survival probability decreases monotonically over time; at the same time, a focal survival loss is constructed in combination with a multi-period BCE loss to realize cross-sample ranking monotonicity. The combination of the two can accurately capture the risk ranking of samples between different time points, thereby more accurately predicting the time point of occurrence of abnormal behavior. However, there is a potential conflict between the two. Therefore, an adaptive gradient balancing method is designed to resolve this conflict, thereby ensuring that the model can effectively balance the two objectives during parameter optimization.

[0072] 3. The samples in the embodiments of the present invention may be, but are not limited to, individuals. Abnormal behaviors reflect abnormal changes in individuals' recent behaviors. Detecting these abnormal changes is of great significance for reducing the occurrence of high-risk events. At the same time, studying individual abnormal behaviors also has important application value for various tasks such as product recommendations, community predictions, and abnormal warnings. Since the occurrence of behaviors has an obvious temporal relationship, the present invention effectively identifies whether individual abnormal behaviors occur and their specific occurrence time by constructing a risk accumulation function and a focal survival loss function.

[0073] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0074] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for dynamic recognition of abnormal behavior based on multi-view decoupled representation learning, characterized in that: Based on a pre-trained model, the model includes multiple specific view encoders, a Transformer encoder, a discrete multi-view encoder, a view fusion decoder based on an adaptive mask mechanism, and a prediction layer; the method includes: Collect and preprocess various view data related to the task and obtain the feature vector corresponding to each view; Taking each of the feature vectors as input of the corresponding specific view encoder to obtain a corresponding feature embedding vector; Using each of the feature embedding vectors as an input of the Transformer encoder to obtain a corresponding enhanced feature embedding vector, and concatenating each of the enhanced feature embedding vectors to obtain a multi-view embedding vector; Using the multi-view embedding vector as input of the discrete multi-view encoder to obtain a matching representation with a preset codebook; Taking the matching representation as the input of the view fusion decoder, finding redundant features in discrete latent variables through pairwise redundant probes, dynamically obtaining a mask vector to extract unique elements from the matching representation and set repeated elements to zero, and obtaining a mask representation; The mask representation is used as the input of the prediction layer to obtain the original output of multiple future moments, and the risk probability for predicting the occurrence of abnormal behavior is generated through a risk accumulation function.

2. The abnormal behavior dynamic identification method according to claim 1, characterized in that: The step of using the multi-view embedding vector as an input of the discrete multi-view encoder to obtain a matching representation with a preset codebook comprises: Using a nearest neighbor search method, obtaining a set of codebook indexes corresponding to the multi-view embedding vector and the potential vector of the codebook; The codebook index is used as a discrete latent variable to map each embedding of the multi-view embedding vector to the closest latent vector in the codebook to obtain a matching representation.

3. The abnormal behavior dynamic identification method according to claim 2, characterized in that: The method uses the mask representation as the input of the prediction layer, obtains the original output at multiple moments in the future, and generates a risk probability for predicting the occurrence of abnormal behavior through a risk accumulation function; including: The mask represents As input to the prediction layer, a set of predicted future m The original output at that moment , where o j Indicates time j The original output of The risk accumulation function is used to generate the risk probability for predicting the occurrence of abnormal behavior. , calculated at any time u The risk probability formula is as follows: Where exp is an exponential function.

4. The abnormal behavior dynamic identification method according to claim 3, characterized in that: In the pre-training stage of the model, a codebook loss, a consistency loss and an orthogonal loss are constructed for the discrete multi-view encoder; wherein: The codebook loss is expressed as: Among them, sg[·] indicates stopping the gradient operation, Represents the square of the L2 norm, where the enhanced feature embedding vector is calculated after the gradient stop operation Matching expression The loss of L2 distance between them; The consistency loss is expressed as: The enhanced feature embedding vector is calculated here The matching representation after the gradient stop operation The loss of L2 distance between them; The orthogonal loss is expressed as: in, represents the L2 norm; triu(∙) means returning the upper triangular part of the matrix and setting the other elements to zero; Indicates that the codebook The vectors in are normalized; Indicates that the codebook The vectors in are normalized and transposed.

5. The abnormal behavior dynamic identification method according to claim 4, characterized in that: In the pre-training stage of the model, a multi-period BCE loss is also constructed, expressed as: Among them, y t Indicates time t The true label is 0 or 1; p t Indicates the time used to predict t The risk probability of abnormal behavior occurring; log represents the logarithmic function.

6. The abnormal behavior dynamic identification method according to claim 5, characterized in that: In the pre-training stage of the model, a multi-period BCE loss is used to determine whether the individual as a sample has a risk during the observation period based on the true label; Define if the sample i If no risk event has occurred, mark δ i =0, otherwise δ i =1; in two comparable samples i and samples j If the sample i Risk time T i Earlier than the sample j Risk time T j , and the sample i The risk probability Higher than sample j The risk probability , then the sorting is correct, otherwise the sorting is wrong; the function clamp(∙) means limiting the input to [0,∞), which is used to exclude comparable sample pairs with correct sorting; Construct the focal survival loss for pairs of comparable samples with ordering errors, expressed as: Among them, the use of represents comparable sample pairs, represents the indicator function; Representation sample i In time A =min( T , C ), A =min( T , C ) indicates the actual time when the event occurred T and observation end time C The smaller value of ; σ Represents the Sigmoid function, and since in the formula The operation is not differentiable. σ (x) is approximately: in, γ is a tuning parameter used to control the sensitivity of comparable changes. σ The range of (x) is [0, 1).

7. The abnormal behavior dynamic identification method according to claim 6, characterized in that: In the pre-training stage of the model, the multi-period BCE loss is used as the main target, the focal survival loss is used as the auxiliary target, and an adaptive gradient balancing method is designed; specifically, it includes: make and They represent the gradient vectors of multi-period BCE loss and focal survival loss at the τth iteration respectively; calculate and The cosine similarity between them is used to determine whether there is a conflict between the two, and the Gram-Schmidt orthogonalization is used to transform Updated to , the formula is: Among them, proj u (v) represents a vector v In vector u The projection on Projection to the gradient vector Above; cosine represents cosine similarity.

8. An abnormal behavior dynamic recognition system based on multi-view decoupled representation learning, characterized in that: Based on a pre-trained model, the model includes multiple specific view encoders, a Transformer encoder, a discrete multi-view encoder, a view fusion decoder based on an adaptive mask mechanism, and a prediction layer; the system includes: The preprocessing module is used to collect and preprocess various view data related to the task and obtain the feature vector corresponding to each view; An embedding module, configured to use each of the feature vectors as an input of the corresponding specific view encoder to obtain a corresponding feature embedding vector; An enhancement module, used to take each of the feature embedding vectors as an input of the Transformer encoder, obtain a corresponding enhanced feature embedding vector, and concatenate each of the enhanced feature embedding vectors to obtain a multi-view embedding vector; A matching module, configured to use the multi-view embedding vector as an input of the discrete multi-view encoder to obtain a matching representation with a preset codebook; A mask module, configured to use the matching representation as an input of the view fusion decoder, find redundant features in discrete latent variables through pairwise redundant probes, dynamically obtain a mask vector to extract unique elements from the matching representation and set repeated elements to zero, and obtain a mask representation; The prediction module is used to use the mask representation as the input of the prediction layer, obtain the original output of multiple future moments, and generate the risk probability for predicting the occurrence of abnormal behavior through the risk accumulation function.

9. A storage medium, characterized in that: It stores a computer program for dynamic identification of abnormal behavior based on multi-view decoupled representation learning, wherein the computer program enables a computer to execute the method for dynamic identification of abnormal behavior as described in any one of claims 1 to 7.

10. An electronic device, characterized in that: include: one or more processors; Memory; and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the programs include a program for executing the abnormal behavior dynamic identification method according to any one of claims 1 to 7.

Images