Abnormal scene recognition model training method, abnormal scene recognition method and device

Abstract

The present application discloses a training method, an abnormal scene recognition method and an abnormal scene recognition device for an abnormal scene, and relates to the field of model fine-tuning technology. A sample text data set is obtained, and then a pre-trained language model is trained using multiple sample texts in the sample text data set and a sample abnormal scene label corresponding to each sample text data. The pre-trained language model is fine-tuned with the help of a dynamic rank and control gate mechanism. The dynamic rank mechanism can adaptively adjust the weight matrix decomposition dimension according to the semantic complexity of the text, increase the rank value for complex scenes to capture more details, and reduce the rank value for simple scenes to reduce redundancy, thereby balancing performance and efficiency. The control gate mechanism can screen the sub-weight matrix dimensions, allowing only the key dimensions of the abnormal scene to participate in the update, thereby strengthening abnormal features and suppressing irrelevant interference.

Description

Technical Field

[0001] The present application relates to the field of model fine-tuning technology, and in particular to a training method for an abnormal scene recognition model, an abnormal scene recognition method, and a device. Background Art

[0002] At present, in order to monitor certain abnormal behaviors, corresponding audio collection equipment will be set up in different areas, and the collected audio will be identified through text recognition technology to determine whether abnormal behavior occurs.

[0003] Traditional model training relies on single-language scenario data, making it difficult to gain a deep understanding of specific scenarios. This results in a high error rate in identifying abnormal behaviors in specific environments.

[0004] Therefore, how to improve the accuracy of abnormal scene recognition has become an urgent problem to be solved. Summary of the Invention

[0005] The present application provides a training method for an abnormal scene recognition model, an abnormal scene recognition method and an abnormal scene recognition device, so as to at least solve the problem of low accuracy of text recognition in related technologies.

[0006] In a first aspect, the present application provides a method for training an abnormal scene recognition model, comprising:

[0007] Obtain a sample text data set; the sample text data set includes multiple sample texts and a sample abnormal scene label corresponding to each sample text data;

[0008] Inputting the sample text dataset into a pre-trained language model, and fine-tuning the pre-trained language model based on a dynamic rank and control gate mechanism to obtain a target abnormal scene recognition model;

[0009] The pre-trained language model includes an embedding layer fine-tuning module, a multi-head attention layer fine-tuning module, and a feedforward network layer fine-tuning module.

[0010] In a second aspect, the present application provides a method for identifying abnormal scenes, comprising:

[0011] When the text corresponding to the target audio collected by the microphone contains the target keyword, multiple audio data are separated from the target audio according to the timbre;

[0012] Performing text recognition on the multiple audio data to determine target audio data containing the target keyword;

[0013] Obtaining a target text corresponding to the target audio data;

[0014] The target text is input into a target abnormal scene recognition model to obtain an abnormal scene label corresponding to the target text; the target abnormal scene recognition model is obtained based on a training method of the abnormal scene recognition model.

[0015] In a third aspect, the present application also provides a training device for an abnormal scene recognition model, comprising:

[0016] An acquisition unit is configured to acquire a sample text data set; the sample text data set includes a plurality of sample texts and a sample abnormal scene label corresponding to each sample text data;

[0017] A training unit, configured to input the sample text dataset into a pre-trained language model, and fine-tune the pre-trained language model based on a dynamic rank and control gate mechanism to obtain a target abnormal scene recognition model;

[0018] The pre-trained language model includes an embedding layer fine-tuning module, a multi-head attention layer fine-tuning module, and a feedforward network layer fine-tuning module.

[0019] In a fourth aspect, the present application further provides an abnormal scene recognition device, comprising:

[0020] a separation unit, configured to separate a plurality of audio data from the target audio collected by the microphone according to timbre when the text corresponding to the target audio contains a target keyword;

[0021] a determining unit, configured to perform text recognition on the plurality of audio data to determine target audio data containing the target keyword;

[0022] A text acquisition unit, configured to acquire a target text corresponding to the target audio data;

[0023] A label acquisition unit is used to input the target text into a target abnormal scene recognition model to obtain an abnormal scene label corresponding to the target text; the target abnormal scene recognition model is obtained based on the training method of the abnormal scene recognition model described in the first aspect.

[0024] The present application also provides a computer-readable storage medium, which stores a computer program, wherein when the computer program is executed by a processor, it implements the training method of any of the above-mentioned abnormal scene recognition models and the steps of any of the abnormal scene recognition methods.

[0025] The present application also provides a computer program product, including a computer program, which, when executed by a processor, implements any of the above-mentioned training methods for abnormal scene recognition models and the steps of any of the abnormal scene recognition methods.

[0026] This application obtains a sample text dataset, and then uses multiple sample texts in the sample text dataset and the sample abnormal scene labels corresponding to each sample text data to train the pre-trained language model. The pre-trained language model is fine-tuned with the help of a dynamic rank and control gate mechanism. The dynamic rank mechanism can adaptively adjust the weight matrix decomposition dimension according to the semantic complexity of the text, increase the rank value for complex scenes to capture more details, and reduce the rank value for simple scenes to reduce redundancy, balancing performance and efficiency; the control gate mechanism can filter the sub-weight matrix dimensions, allowing only the key dimensions of the abnormal scene to participate in the update, strengthening abnormal features and suppressing irrelevant interference. At the same time, fine-tuning modules are set for the embedding layer, multi-head attention layer, and feedforward network layer respectively, which can achieve targeted optimization of each layer, greatly improving the recognition accuracy and generalization ability of the target abnormal scene recognition model for abnormal scenes, so that it can be efficiently applied to abnormal behavior monitoring in abnormal scenes, allowing the model to better adapt to the semantic understanding of abnormal scenes, improve the recognition ability of abnormal scenes, and thus reduce the recognition error rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0028] Figure 1 A flowchart of a method for training an abnormal scene recognition model provided in an embodiment of the present application;

[0029] Figure 2 A schematic diagram of the structure of a pre-trained language model provided in an embodiment of the present application;

[0030] Figure 3 A flowchart of an abnormal scene recognition method provided in an embodiment of the present application:

[0031] Figure 4 A schematic diagram of the structure of a training device for an abnormal scene recognition model provided in an embodiment of the present application;

[0032] Figure 5 A schematic diagram of the structure of an abnormal scene recognition device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0033] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0034] It should be noted that, in the description of this application, the terms "comprises," "includes," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. The terms "first," "second," etc., in this application are used to distinguish similar objects, and are not used to describe a particular order or sequence.

[0035] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0036] The embodiment of the present application provides a training method for an abnormal scene recognition model. The execution process of the method is as follows: Figure 1 As shown, the specific steps include:

[0037] S101. Obtain a sample text dataset.

[0038] The sample text dataset includes multiple sample texts and a sample abnormal scene label corresponding to each sample text data.

[0039] In some embodiments, in order to promptly obtain abnormal conversation situations under abnormal behavior, microphones can be deployed in areas with high incidence of abnormal behavior, such as blind spots in monitoring such as restrooms, stairwells, and storage rooms, where abnormal behavior is prone to occur; or areas with dense traffic, such as areas with large personnel flow, to facilitate capturing audio data generated by multi-person interactions; a large amount of audio data corresponding to the target area is collected by microphones to facilitate the subsequent generation of sample data.

[0040] Specifically, the method for obtaining the sample text dataset may be to obtain a text dataset corresponding to the collected audio dataset, and annotate the text dataset with abnormal scenes to generate the sample text dataset.

[0041] In an embodiment of the present application, when generating a sample text data set, each audio data in the audio data set is first converted into corresponding text data through speech recognition technology to achieve conversion from speech signals to semantic information.

[0042] Furthermore, we can manually annotate the text content with abnormal scenarios to determine whether it belongs to an abnormal scenario, generate corresponding abnormal scenario labels, and then manually annotate each text data with abnormal scenario labels to generate a sample text dataset.

[0043] S102: Input the sample text dataset into the pre-trained language model, and fine-tune the pre-trained language model based on the dynamic rank and control gate mechanism to obtain a target abnormal scene recognition model.

[0044] Among them, the pre-trained language model includes the embedding layer fine-tuning module, the multi-head attention layer fine-tuning module and the feedforward network layer fine-tuning module.

[0045] In the embodiment of the present application, the pre-trained language model refers to designing a language model training task based on a large-scale corpus (including language training materials such as sentences and paragraphs), and training a large-scale neural network algorithm structure to learn and implement it. The final large-scale neural network algorithm structure and parameters are the pre-trained language model. Subsequent tasks can perform feature extraction or task fine-tuning based on this model to achieve specific task objectives.

[0046] Furthermore, the neural network algorithm structure of the pre-trained language model is built based on the multi-head attention network mechanism. By introducing multiple attention heads, the multi-head attention network mechanism enables the model to focus on different aspects of the input data at the same time, thereby improving its ability to capture information. At the same time, it allows the model to extract features from multiple subspaces, enhancing the model's learning ability and expressiveness, so that the model ultimately obtains more accurate output.

[0047] In an embodiment of the present application, a pre-trained language model is targetedly trained using a sample data set to obtain a target abnormal scene recognition model; specifically, during training, a large number of sample texts and corresponding sample abnormal scene labels are used, and the feature vectors of the sample texts are obtained through the internal module of the model. The predicted abnormal scene labels are then obtained through forward propagation, and the predicted abnormal scene labels corresponding to the sample texts are combined with the sample abnormal scene labels. The model loss is calculated through a loss function and the model parameters are updated through backpropagation. Finally, the model is converged through iterative training to generate a target abnormal scene recognition model.

[0048] It should be noted that the pre-trained language model includes an embedding layer fine-tuning module, a multi-head attention layer fine-tuning module, and a feedforward network layer fine-tuning module. These modules will fine-tune the pre-trained language model based on the dynamic rank and control gate mechanism, making the target feature vector more suitable for abnormal scenarios.

[0049] Fine-tuning involves further training a pre-trained language model on a task-specific dataset to adjust model parameters and better adapt it to the target task. During fine-tuning, most layers of the pre-trained model are typically frozen, with only newly added layers trained or a small number of key layers adjusted. This preserves the useful features learned by the pre-trained model while enabling rapid adaptation to the specific requirements of the new task. Choosing the appropriate learning rate and number of training rounds is also crucial for successful fine-tuning.

[0050] Furthermore, the dynamic rank mechanism is used to decompose the model's original weight matrix to obtain a sub-weight matrix model. By presetting a rank value range, the rank value of the sub-weight matrix is ​​adaptively adjusted based on the semantic complexity of the input text. For example, when processing descriptive text containing complex expressions such as metaphors about abnormal behavior, the dynamic rank mechanism increases the rank value, increasing the dimensionality of the sub-weight matrix, thereby enhancing the model's ability to capture complex semantics and capturing more dimensional semantic features, facilitating the identification of potential abnormal behavior characteristics. When processing text with relatively simple and direct semantics for everyday communication, the dynamic rank mechanism decreases the rank value and reduces the size of the sub-weight matrix. While ensuring normal semantic understanding, it effectively reduces computational redundancy and improves the overall operational efficiency of the model. This adaptive adjustment method allows the model to detect abnormal behavior with the optimal parameter configuration in scenarios of varying semantic complexity, improving detection accuracy while also taking into account the rational use of computing resources.

[0051] The control gate mechanism uses a diagonal matrix composed of binary numbers 0 and 1 to accurately screen the different dimensions of each sub-weight matrix. Its core function is to only allow key dimensions related to abnormal scenarios to participate in model parameter updates. Specifically, when keywords pointing to abnormal behavior appear in the text, the control gate will switch the update channel of the corresponding weight component to the open state (i.e., assign a value of 1). By strengthening the feature expression of such semantics, the model is more sensitive to abnormal signals. For the semantic features corresponding to regular keywords in daily conversations, the control gate will close their update channel (i.e., assign a value of 0), thereby blocking the interference of irrelevant information, ensuring that the model can focus on the core features that are truly related to abnormal behavior, and improving the accuracy of detection and judgment.

[0052] Specifically, the above-mentioned dynamic rank and control gate mechanism can be set in the embedding layer, multi-head attention layer, and feedforward network layer of the pre-trained language model, and then generate the embedding layer fine-tuning module, the multi-head attention layer fine-tuning module, and the feedforward network layer fine-tuning module, and adjust the original weight matrix of the corresponding layer respectively. Taking the embedding layer fine-tuning module as an example, the fine-tuning process of other multi-head attention layer fine-tuning modules and the feedforward network layer fine-tuning module is similar. The specific fine-tuning process is as follows (taking the first training process of the pre-trained language model using sample text and its corresponding samples as an example):

[0053] Get the original weight matrix W (dimension [m, n]) corresponding to the embedding layer of the pre-trained language model, and split the original weight matrix into two sub-weight matrices along dimension p through the embedding layer fine-tuning module 、 , there is no limit on the number of sub-weight matrices in the embodiments of the present application.

[0054] Then according to the preset rank value range set by the dynamic rank mechanism: ,like( ); Initialize a rank value from the preset rank value range, and further limit the dimension of the sub-weight matrix according to the rank value; at the same time, based on the control gate mechanism, initialize a control gate vector, expressed as: G=diag(g1, g2, ..., g r ), {0, 1} (i=1…r), where diag(g1, g2,…, g r ) represents (g1, g2, …, g r ), that is, the control gate vector is a diagonal matrix composed of 0 and 1.

[0055] When g i =1, indicating activation of the sub-weight matrix component of the corresponding dimension; g i = 0, it means shielding the sub-weight matrix vector of irrelevant dimensions to shield the weight update of irrelevant dimensions; then, according to the sub-weight matrix 、 , as well as the initialized rank value, the initialized control gate vector, and the weight matrix increment are calculated according to the following formula:

[0056]

[0057] in, for The r-th column sub-weight matrix of for The r-th row sub-weight matrix; since G is a diagonal matrix, the result after multiplication can be understood as: pick out The i-th row of The i-th column of i (0 or 1), and then add up these r items.

[0058] Then, based on the above, the weight matrix increment of the original weight matrix W corresponding to the embedding layer is obtained, and then the fine-tuned weight matrix is ​​obtained based on the weight matrix increment and the original weight matrix, and then the fine-tuned weight matrix is ​​fused and superimposed with the original weight matrix to obtain the first weight matrix corresponding to the embedding layer; the fine-tuned weight matrix and the original weight matrix are fused and superimposed in the above process because the original weight matrix carries the basic semantic representation (such as word vector association, syntactic structure, etc.) learned by the pre-trained language model in large-scale general corpus. Directly discarding it will cause the model to lose basic understanding ability. Then, after obtaining the fine-tuned weight matrix, the original weight matrix and the fine-tuned weight matrix are fused, so that the model can maintain a general understanding of natural language while accurately capturing semantic deviations in abnormal scenarios, and ultimately improve the recognition accuracy and generalization ability in complex scenarios.

[0059] Furthermore, the pre-trained language model uses the first weight matrix to obtain the feature vector corresponding to the embedding layer, and then uses the subsequent multiple layers of the model to predict the predicted abnormal scene label corresponding to the current sample text. The model's loss function is calculated by comparing the predicted abnormal scene label with the sample abnormal scene label corresponding to the sample text. Based on the loss function, the model parameters are updated through the backpropagation algorithm to gradually reduce the loss value. It should be noted that when updating the model parameters, it is also necessary to continuously adjust the size of the dynamic rank and the control gate vector to gradually explore the optimal rank value corresponding to the current sample text and the most suitable control gate vector, forming a closed-loop optimization link of feature fine-tuning, loss feedback, and parameter update.

[0060] In an embodiment of the present application, the pre-trained model is fine-tuned and trained using a large amount of sample data, so that the trained target abnormal scene recognition model has the ability to adaptively determine the rank value and control the gate vector, and directionally strengthen the feature expression of key semantics of abnormal scenes (such as words related to abnormal phenomena), thereby improving the robustness and accuracy of abnormal scene recognition.

[0061] For example, when the sample text includes three different semantic dimensions: abnormal text 1, daily conversation, and abnormal text 2, the dynamic rank r of the embedding layer fine-tuning module of the pre-trained language model can be selected as 3 during fine-tuning, which means focusing on these three semantic dimensions. The control gate vector G has g1=1, which activates the "abnormal text 1" dimension, g2=0 blocks the "daily conversation" dimension, and g3=1 activates the "abnormal text 2" dimension. When calculating, it will be used Column 1, The first line (corresponding to "Exception Text 1") is multiplied by g1=1; skip The second column of The second row, because g2=0; then use The third column of The third row of (corresponding to "abnormal text 2") is multiplied by g3=1. Finally, these three items are added together to obtain (weight increment), which is then fused and superimposed with the original weight matrix of the embedding layer to generate the first weight matrix of the embedding layer, making the model more sensitive to abnormal text corresponding to abnormal behavior.

[0062] This application uses a sample text dataset, and then uses multiple sample texts in the sample text dataset and the sample abnormal scene labels corresponding to each sample text data to train the pre-trained language model. The pre-trained language model is fine-tuned with the help of a dynamic rank and control gate mechanism. The dynamic rank mechanism can adaptively adjust the weight matrix decomposition dimension according to the semantic complexity of the text, increase the rank of complex scenes to capture details, and reduce the rank of simple scenes to reduce redundancy, balancing performance and efficiency; the control gate mechanism can filter the sub-weight matrix dimensions, allowing only the key dimensions of abnormal scenes to participate in the update, strengthening abnormal features and suppressing irrelevant interference. At the same time, fine-tuning modules are set for the embedding layer, multi-head attention layer, and feedforward network layer respectively, which can achieve targeted optimization of each layer, greatly improving the recognition accuracy and generalization ability of the target abnormal scene recognition model for abnormal scenes, so that it can be efficiently applied to abnormal behavior monitoring in abnormal scenes, allowing the model to better adapt to the semantic understanding of abnormal scenes, improve the recognition ability of abnormal scenes, and thus reduce the recognition error rate.

[0063] As an extension and refinement of the above embodiment, refer to Figure 2 As shown, the pre-trained language model includes: an embedding layer 21, an embedding layer fine-tuning module 211, a multi-head attention layer 22, a multi-head attention layer fine-tuning module 221, a feedforward network layer 23, a feedforward network layer fine-tuning module 231 and a decoding output layer 24:

[0064] The embedding layer 21 is used to map the sample text into an initial feature vector.

[0065] Specifically, after receiving the sample text, the embedding layer 21 immediately converts the sample text into a numerical embedding vector and establishes a mapping between the text and the semantic space.

[0066] The embedding layer fine-tuning module 221 is used to fine-tune the original weight matrix of the embedding layer based on the dynamic rank and control gate mechanism to obtain a first weight matrix, so as to adjust the initial eigenvector according to the first weight matrix to obtain a first eigenvector.

[0067] In an embodiment of the present application, the embedding layer fine-tuning module 221 adopts the above-mentioned weight increment calculation formula, combined with the dynamic rank and control gate mechanism to perform scenario optimization on the original weight matrix of the embedding layer, so that the model focuses on abnormal text and suppresses interference from daily conversations, and finally generates the first weight matrix of the embedding layer that is adapted to the current abnormal scenario, and then adjusts the initial feature vector to obtain the first feature vector.

[0068] Specifically, referring to the description in step S103 above, a first weight matrix is ​​obtained after fine-tuning the original weight matrix of the embedding layer, and then the initial eigenvector is fine-tuned based on the first weight matrix to obtain the first eigenvector, and then the reasoning analysis of the next layer is performed.

[0069] The multi-head attention layer 22 is used to perform a linear transformation on the first feature vector to generate a query vector, a key vector, and a value vector; and obtain a second feature vector based on the query vector, the key vector, and the value vector.

[0070] Specifically, after receiving the first feature vector, the multi-head attention layer 22 generates a query vector Q through three sets of independent linear transformations, which is used to actively query semantic associations; a key vector K, which is used to respond to queries and build associations; and a value vector V, which is used to carry the semantic information after association.

[0071] Furthermore, the attention mechanism is executed in parallel for each set of query vectors, key vectors, and value vectors, and the similarity score between the query vector and the value vector is calculated and mapped to the attention weight. The value vectors are weighted and summed using the weights to capture the semantic associations between words in the text. The outputs of multiple groups of attention heads are concatenated and fused to obtain a second feature vector containing multi-dimensional semantic associations, providing enhanced features for the subsequent feedforward network layer.

[0072] Specifically, after obtaining the query vector, key vector, and value vector, the multi-head parallel attention calculation divides the query vector, key vector, and value vector into h heads (for example, h = 8) and calculates the independent attention score of each head.

[0073] It should be noted that when calculating the independent attention score of each head, the following mask expression can be used to control the range of attention interaction, capture semantic relationships through multi-head parallel calculation, and provide a more accurate feature expression for the model:

[0074]

[0075] Among them, mask=0 means that the attention interaction between the current position and other positions is allowed (such as the association between "abnormal text 1" and "abnormal text 2" in the text); mask=-∞ means that the interference of irrelevant semantics at certain positions is blocked.

[0076] Furthermore, when calculating the attention score corresponding to each head in combination with the mask, the following formula can be used to calculate the attention score corresponding to each head:

[0077]

[0078] in, represents the feature dimension of K. Q, K, and V are the query vector, key vector, and value vector. By multiplying them by the mask matrix, the scores of locations where interactions are not allowed are masked out. Softmax converts the scores into attention weights (to highlight semantic focus). The weighted sum of V (the value vector) is used to obtain the output of the single-head attention. For example, head1 corresponds to "abnormal behavior corresponding to abnormal text," and head2 corresponds to "scene and location corresponding to abnormal text."

[0079] Since each attention head focuses on different semantic dimensions of the text, the outputs of all heads are spliced ​​together by dimension to obtain high-dimensional features (which integrate multi-dimensional semantic information), and thus a more accurate and comprehensive second feature vector is obtained.

[0080] The multi-head attention layer fine-tuning module 221 is used to fine-tune the original weight matrix in the multi-head attention layer based on the dynamic rank and control gate mechanism to obtain the second weight matrix, so as to adjust the second eigenvector according to the second weight matrix to obtain the third eigenvector.

[0081] In an embodiment of the present application, the multi-head attention layer fine-tuning module 221 also uses the above-mentioned weight increment calculation formula, combined with the dynamic rank and control gate mechanism to optimize the original weight matrix of the multi-head attention layer in the scenario, so that the model focuses on abnormal semantics and suppresses interference from daily conversations, and finally generates a second weight matrix of the multi-head attention layer that is adapted to abnormal scenarios, and then adjusts the second eigenvector to obtain the third eigenvector.

[0082] Specifically, referring to the description in step S103 above, obtain the second weight matrix after fine-tuning the original weight matrix of the multi-head attention layer, then fine-tune the second eigenvector based on the second weight matrix, obtain the third eigenvector, and perform reasoning analysis on the next layer.

[0083] Furthermore, the multi-head attention layer fine-tuning module 221 is specifically used for:

[0084] Based on the dynamic rank and control gate mechanism, the original weight matrix corresponding to the query vector in the multi-head attention layer is fine-tuned to obtain the first sub-weight matrix corresponding to the query vector, so as to adjust the query vector according to the first sub-weight matrix.

[0085] Similarly, the above-mentioned weight increment calculation formula is used in combination with the dynamic rank and control gate mechanism to optimize the original weight matrix corresponding to the query vector, generate the first sub-weight matrix corresponding to the query vector, and then adjust the query vector.

[0086] Specifically, referring to the description in the above step S103, a first sub-weight matrix is ​​obtained after fine-tuning the original weight matrix of the query vector, and then the query vector is fine-tuned based on the first sub-weight matrix to obtain the adjusted query vector, thereby enhancing the semantic association query capability of abnormal text and scenes.

[0087] Based on the dynamic rank and control gate mechanism, the original weight matrix corresponding to the value vector in the multi-head attention layer is fine-tuned to obtain the second sub-weight matrix corresponding to the value vector, so as to adjust the value vector according to the second sub-weight matrix.

[0088] Similarly, the above-mentioned weight increment calculation formula is used in combination with the dynamic rank and control gate mechanism to optimize the original weight matrix corresponding to the query vector, generate the second sub-weight matrix corresponding to the value vector, and then adjust the query vector.

[0089] Specifically, referring to the description in the above step S103, a second sub-weight matrix is ​​obtained after fine-tuning the original weight matrix of the value vector, and then the value vector is fine-tuned based on the second sub-weight matrix to obtain the adjusted value vector, thereby strongly enhancing the value feature expression of the abnormal semantic feature.

[0090] The feedforward network layer 23 is used to fuse the first eigenvector and the third eigenvector to obtain a fourth eigenvector, and perform a linear transformation on the fourth eigenvector to obtain a fifth eigenvector.

[0091] Specifically, the first eigenvector is a semantically enhanced eigenvector fine-tuned by the embedding layer and the embedding layer fine-tuning module. The third eigenvector is the initial eigenvector after fine-tuning processed by the multi-head attention layer, which contains semantic association and interaction information. In order to make the basic semantics and the associated semantics captured by attention complement each other and enrich the feature expression, the first eigenvector and the third eigenvector are directly added element by element through residual connection to retain the original information. The added features are then standardized based on layer normalization (LayerNorm) to make the feature distribution more stable, thereby obtaining the fourth eigenvector.

[0092] After fusing the fourth eigenvector, the feedforward network layer will continue to perform a linear transformation on the fourth eigenvector to generate the fifth eigenvector. Specifically, the linear transformation is performed according to the following formula:

[0093]

[0094] Among them, SwiGLU is an activation function used for weighted feature screening, so as to increase the weights corresponding to important features and reduce the weights of irrelevant features through weighted adjustment; dropout is a regularization method (randomly discarding some neurons to prevent overfitting); w1, b1, w2, b2 are learnable weights and bias parameters, which are randomly initialized during model training and then iteratively updated through backpropagation optimization, allowing the feedforward network layer to adapt to task requirements and gradually determine the optimal w1, b1, w2, b2; thereby minimizing the error between the model prediction results and the true label.

[0095] The feedforward network layer fine-tuning module 231 is used to fine-tune the original weight matrix corresponding to the feedforward network layer based on the dynamic rank and control gate mechanism to obtain a third weight matrix, so as to adjust the fifth eigenvector according to the third weight matrix to obtain a target eigenvector.

[0096] In an embodiment of the present application, the feedforward network layer fine-tuning module 231 also uses the above-mentioned weight increment calculation formula, combined with the dynamic rank and control gate mechanism to optimize the original weight matrix of the multi-head attention layer in the scenario, so that the model focuses on abnormal semantics and suppresses interference from daily conversations, and finally generates a second weight matrix of the multi-head attention layer that is adapted to abnormal scenarios, and then adjusts the fifth eigenvector to obtain the target eigenvector.

[0097] Specifically, referring to the description in the above step S103, the third weight matrix after fine-tuning the original weight matrix of the feedforward network layer is obtained, and then the fifth eigenvector is fine-tuned based on the third weight matrix to obtain the target eigenvector, and then subsequent decoding output is performed based on the target eigenvector.

[0098] The decoding output layer 24 is used to output the predicted abnormal scene label based on the target feature vector.

[0099] In the decoding output layer 24, through feature projection, multi-label classification, and combining the domain knowledge of abnormal scenarios, the aforementioned adjusted target feature vector is converted into an interpretable abnormal scenario label, thus achieving real-time and accurate early warning of abnormal behavior.

[0100] The pre-trained language model in the embodiment of the present application includes an embedding layer, a multi-head attention layer, and a feedforward network layer. Each layer introduces a corresponding fine-tuning module. Through the dynamic rank mechanism (adaptive adjustment of the matrix decomposition dimension) and the control gate mechanism (binary matrix screening of key weight dimensions), the original weight matrix of each layer is targetedly optimized to generate the corresponding fine-tuned weight matrix of each layer, thereby enhancing the extraction and association of semantic features related to abnormal scenarios.

[0101] In an embodiment of the present application, the pre-trained language model further includes: a loss optimization module for calculating the model loss value of the sample abnormal scene label and the predicted abnormal scene label corresponding to the sample text based on the target loss function calculation formula; and optimizing the pre-trained language model based on the model loss value to generate a target abnormal scene recognition model. The target loss function calculation formula is:

[0102]

[0103] Among them, L represents the model loss value, n represents the number of sample texts involved in the calculation, is the sample abnormal scene label corresponding to the sample text i, and x represents the initial feature vector corresponding to the sample text; is the second eigenvector obtained after processing x through the multi-head attention unit; is a feedforward neural network, α

[0104] is a hyperparameter, is the gradient direction value of the weight reduction.

[0105] Specifically, Used to calculate the score between the predicted label and the true label

[0106] Class error, Used to calculate the gradient direction of the model weight adjustment, based on this part, the model knows

[0107] Dao determines which direction to adjust the model to improve when adjusting parameters, and is used to constrain the adjustment direction of the model; α is used to control the "strength" of this constraint (for example, the larger α is, the more it restricts the model from randomly changing weights); it makes the model more cautious when adjusting parameters, prioritizes strengthening features related to abnormal scenarios, and prevents the model from being biased by irrelevant information.

[0108] This embodiment of the application provides a method for comparing the actual sample labels (sample abnormal scene labels) with the model's predicted labels (predicted abnormal scene labels), calculating the model loss using a custom loss function, and then optimizing the model based on the loss value to ultimately generate a target text feature vector recognition model. This loss function measures the gap between the model's predictions and the actual situation, guiding the model to optimize parameters and improve its ability to recognize abnormal scene labels.

[0109] It should also be noted that the embedding layer fine-tuning module 211, the multi-head attention layer fine-tuning module 221, and the feedforward network layer fine-tuning module 231 can all be refined into the following modules:

[0110] The weight matrix block module divides the original weight matrix into blocks to obtain weight matrix sub-blocks.

[0111] Specifically, the weight matrix block module first splits the original weight matrix W (dimension is [m, n]) into two sub-weight matrices along dimension p 、 ; The middle dimension p corresponds to the number of layers of semantic features.

[0112] The dynamic rank adjustment and control gate module is used to calculate the weight increment based on the preset rank value range, the control gate parameter vector and the weight matrix sub-block, and fine-tune the original weight matrix based on the weight increment to obtain the fine-tuned weight matrix.

[0113] Furthermore, in the dynamic rank adjustment and control gate module, dynamic ranks are used to determine which layers to adjust (for example, for complex conversations, the rank value is increased to adjust more layers; for simple texts, the rank value is decreased to adjust only the key layers). And the control gate is used to determine which layers to activate.

[0114] In an embodiment of the present application, after obtaining the target abnormal scene recognition model through the above-mentioned training method, the model can be used to monitor abnormal behavior. Since there are dialect differences in each region, multi-model cross-multiplication and fusion can be performed on the basis of the target abnormal scene recognition model with other dialect recognition models and other semantic recognition models to allow the advantages of different models to be superimposed.

[0115] Specifically, the weight matrix of the target abnormal scene recognition model can be divided into multiple weight matrix sub-blocks W=[W1, W2, ..., Wi] according to different semantic categories. For example, the weight matrix can be divided into semantic weight matrix sub-blocks such as abnormal behavior class 1, abnormal behavior class 2, and daily conversation. Then, multiple weight matrix sub-blocks A=[A1, A2, ..., Ai] of the same semantic category corresponding to other dialect recognition models and multiple weight matrix sub-blocks B=[B1, B2, ..., Bi] of the same semantic category corresponding to other semantic recognition models are obtained. Furthermore, by cross-multiplying the corresponding weight matrix sub-blocks, the fused weight sub-blocks are obtained, and the fused weight sub-blocks are spliced ​​to generate an enhanced weight matrix, so that the model can learn the recognition capabilities of other dialect recognition models and other semantic recognition models, thereby improving the applicability of the abnormal scene recognition model.

[0116] As an extension and application of the above embodiment, the embodiment of the present application provides an abnormal scene recognition method. The execution process of the abnormal scene recognition method is as follows: Figure 3 As shown, the specific steps include:

[0117] S301. When the text corresponding to the target audio collected by the microphone contains a target keyword, multiple audio data are separated from the target audio according to the timbre.

[0118] In some embodiments, microphones are set in areas where abnormal behavior is prone to occur, such as restrooms, stairwells, storage rooms, etc. These areas are prone to hidden abnormal behavior and video surveillance may have blind spots; or areas with dense traffic, such as areas with large personnel flow, to facilitate capturing abnormal voice signals in multi-person interactions.

[0119] The target keywords may be pre-set target keywords related to abnormal behaviors, forming a keyword vocabulary.

[0120] Specifically, after obtaining the target audio collected by the microphone, it can be converted into corresponding text content through automatic speech recognition technology (ASR) to determine whether the text content corresponding to the target audio includes the target keyword. If the target keyword is included, it means that the target audio may be audio related to an abnormal scene. Then, multiple audio data are separated from the target audio according to the timbre, and further subsequent reasoning and analysis are performed. The above steps are used to preliminarily filter the audio related to the abnormal scene, avoid invalid processing of a large amount of normal audio, and greatly improve system efficiency.

[0121] Furthermore, a method for separating multiple audio data from the target audio according to timbre may be: based on classic feature extraction methods in the field of speaker recognition, such as Mel-frequency cepstral coefficients, after framing and windowing the target audio, the target audio also needs to be converted from time domain data to frequency domain data, so as to facilitate the subsequent separation of multiple audio data from the target audio according to timbre.

[0122] Specifically, the time domain signal can be converted into a frequency domain spectrum through short-time Fourier transform. Then, the timbre features of different speakers can be extracted based on the frequency domain spectrum. The features can be grouped using a clustering algorithm (such as K-means or hierarchical clustering), and the speech frames corresponding to similar features can be attributed to the same speaker, ultimately separating multiple audio data.

[0123] In an embodiment of the present application, before obtaining the initial audio collected by the target microphone, it is also necessary to determine the target audio from a large amount of initial audio. The specific method is: obtain the initial audio collected by the target microphone, perform text recognition on the initial audio, and obtain the initial text corresponding to the initial audio; determine whether the initial text contains the target keyword; if so, use the initial audio as the target audio.

[0124] Then, through text recognition, the corresponding text of the initial audio is extracted. Using target keywords, target audio that may exhibit abnormal behavior is accurately screened. By eliminating audio without abnormal behavior, subsequent model training and anomaly detection focus on critical data, improving data utilization efficiency. Furthermore, by reducing irrelevant audio input and the amount of data the model processes, model training and inference are more efficient, saving computing resources and improving the overall speed of the abnormal scene identification process.

[0125] S302: Perform text recognition on multiple audio data to determine target audio data containing target keywords.

[0126] In some embodiments, after separating multiple audio data from the target audio, text recognition is performed again on the multiple audio data to further determine the target audio data in which the target keyword appears.

[0127] Specifically, the automatic speech recognition technology is used again to perform text recognition on the multiple separated audio data to determine the target audio data in which the target keyword appears; in this case, the number of the target audio data is at least one.

[0128] S303: Obtain target text corresponding to the target audio data.

[0129] Specifically, after the target audio data containing the target keyword is determined, the target text corresponding to the target audio data is obtained according to the text recognition result in the above step S302, so as to perform further in-depth analysis later.

[0130] The specific implementation steps of S303 (obtaining the target text corresponding to the target audio data) in the embodiment of the present application can be refined into the following steps 1 and 2:

[0131] Step 1: Obtain the text recognition result corresponding to the target audio data, and determine the position of the target keyword in the target audio data based on the text recognition result.

[0132] Specifically, in addition to the target keywords, the target audio data also includes a lot of other meaningless content. In order to focus the semantic feature optimization on the target keywords, we can locate the target keyword position in the target audio data and intercept the key part of the audio data based on the position, thereby reducing the amount of data analysis while allowing subsequent recognition analysis to focus more on the target keywords.

[0133] Forced alignment technology can be used to time-match the target keywords in the text recognition results with the audio signal, and then calculate the precise position of each keyword pronunciation in the audio, providing a basis for subsequent segmentation and interception.

[0134] Step 2: intercept the audio data within a preset time range before and after the position of the target keyword in the target audio data to generate a target text corresponding to the target audio data.

[0135] Specifically, the preset time range can be set to 10 seconds. After determining the position of the target keyword in the target audio data, the audio data of 10 seconds before and after the position of the target keyword is intercepted, and then the corresponding target text is obtained based on the intercepted audio data, that is, the target text corresponding to the target audio data is generated.

[0136] S304: Input the target text into the target abnormal scene recognition model to obtain the abnormal scene label corresponding to the target text.

[0137] Among them, the target abnormal scene recognition model is obtained based on the aforementioned training of the pre-trained language model.

[0138] Furthermore, the target text is subjected to inference analysis based on the target recognition model obtained by training the pre-trained language model, and the abnormal scene label corresponding to the target text is obtained.

[0139] By enhancing abnormal semantics and suppressing irrelevant semantics through the target recognition model, the final abnormal scene label is more accurate; this label can be used to determine whether the scene is an abnormal scene, and then in the abnormal scene, the alarm process is triggered in time to notify the relevant responsible personnel of the abnormal situation in time and handle it in time.

[0140] Specifically, after performing abnormal scene identification on the target feature vector and obtaining the abnormal scene label, the embodiment of the present application further includes the following steps 1 and 2:

[0141] Step 1: Obtain the collection information of the target audio when the microphone collects the target audio.

[0142] The acquisition information includes the identification of the microphone and the acquisition time of the target audio.

[0143] As an audio capture device, the terminal audio pickup simultaneously records its unique identifier (such as IP address, device ID, etc.) when capturing target audio. This identifier is used to locate the physical location of the abnormal behavior. Furthermore, the audio pickup automatically records the time the target audio was captured, which can be used to trace the time of the abnormal event.

[0144] Step 2: Generate abnormal warning prompts based on the collected information and abnormal scene labels.

[0145] The audio collected by the microphone undergoes time-domain-frequency domain conversion, timbre separation, and speech recognition, and is then analyzed by the target abnormal scene recognition model. If it is determined to be abnormal behavior, a corresponding abnormal scene label is generated. The microphone identifier (such as "microphone IP: 192.168.1.100"), collection time (such as "collection time: 2025-06-3015:30:22") and abnormal scene label are structured and combined to generate an abnormal warning prompt to inform relevant personnel of the above abnormal phenomenon.

[0146] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method.

[0147] The embodiments of the present application also provide a training device for an abnormal scene recognition model, which corresponds one-to-one to the method claims. Figure 4 A structural diagram of a training device 400 for an abnormal scene recognition model provided by the present disclosure is shown in FIG. Figure 4 As shown, the apparatus 400 of this embodiment includes:

[0148] An acquisition unit 401 is configured to acquire a sample text dataset; the sample text dataset includes a plurality of sample texts and a sample abnormal scenario label corresponding to each sample text data;

[0149] A training unit 402 is configured to input the sample text dataset into a pre-trained language model and fine-tune the pre-trained language model based on a dynamic rank sum control gate mechanism to obtain a target abnormal scene recognition model;

[0150] The pre-trained language model includes an embedding layer fine-tuning module, a multi-head attention layer fine-tuning module, and a feedforward network layer fine-tuning module.

[0151] As an optional implementation of an embodiment of the present application, the pre-trained language model includes an embedding layer, the embedding layer fine-tuning module, a multi-head attention layer, the multi-head attention layer fine-tuning module, a feedforward network layer, the feedforward network layer fine-tuning module and a decoding output layer;

[0152] The embedding layer is used to map the sample text into an initial feature vector;

[0153] The embedding layer fine-tuning module is used to fine-tune the original weight matrix of the embedding layer based on the dynamic rank and control gate mechanism to obtain a first weight matrix, so as to adjust the initial eigenvector according to the first weight matrix;

[0154] The multi-head attention layer is used to perform a linear transformation on the adjusted initial feature vector to generate a query vector, a key vector, and a value vector; and obtain a first feature vector based on the query vector, the key vector, and the value vector;

[0155] The multi-head attention layer fine-tuning module is used to fine-tune the original weight matrix in the multi-head attention layer based on the dynamic rank and control gate mechanism to obtain a second weight matrix, so as to adjust the first eigenvector according to the second weight matrix;

[0156] The feedforward network layer is used to fuse the fine-tuned initial eigenvector with the first eigenvector to obtain a second eigenvector, and perform a linear transformation on the second eigenvector to obtain a target eigenvector;

[0157] The feedforward network layer fine-tuning module is used to fine-tune the original weight matrix corresponding to the feedforward network layer based on the dynamic rank and control gate mechanism to obtain a third weight matrix, so as to adjust the target feature vector according to the third weight matrix;

[0158] The decoding output layer is used to output a predicted abnormal scene label based on the target feature vector.

[0159] As an optional implementation of the embodiment of the present application, the multi-head attention layer fine-tuning module is specifically used to:

[0160] Fine-tune the original weight matrix corresponding to the query vector in the multi-head attention layer based on the dynamic rank sum control gate mechanism, obtain a first sub-weight matrix corresponding to the query vector, and adjust the query vector according to the first sub-weight matrix;

[0161] Based on the dynamic rank and control gate mechanism, the original weight matrix corresponding to the value vector in the multi-head attention layer is fine-tuned to obtain the second sub-weight matrix corresponding to the value vector, so as to adjust the value vector according to the second sub-weight matrix.

[0162] As an optional implementation of the embodiment of the present application, the embedding layer fine-tuning module, the multi-head attention layer fine-tuning module, and the feedforward network layer fine-tuning module all include:

[0163] A weight matrix block module is used to perform block processing on the original weight matrix to obtain weight matrix sub-blocks;

[0164] The dynamic rank adjustment and control gate module is used to calculate the weight increment based on the preset rank value range, the control gate parameter vector and the weight matrix sub-block, and fine-tune the original weight matrix based on the weight increment to obtain the fine-tuned weight matrix.

[0165] As an optional implementation of the embodiment of the present application, the pre-trained language model further includes: a loss optimization module for calculating the model loss value of the sample abnormal scene label and the predicted abnormal scene label corresponding to the sample text based on the target loss function calculation formula; and optimizing the pre-trained language model based on the model loss value to generate a target abnormal scene recognition model;

[0166] The objective loss function calculation formula is:

[0167]

[0168] Among them, L represents the model loss value, n represents the number of sample texts involved in the calculation, is the sample abnormal scene label corresponding to the sample text i, and x represents the initial feature vector corresponding to the sample text; is the second eigenvector obtained after processing x through the multi-head attention unit; is a feedforward neural network, α

[0169] is a hyperparameter, is the gradient direction value of the weight reduction.

[0170] The embodiments of the present application also provide an abnormal scene recognition device, which corresponds one-to-one with the method claims. Figure 5 This is a structural diagram of an abnormal scene recognition device 500 provided by the present disclosure, such as Figure 5 As shown, the apparatus 500 of this embodiment includes:

[0171] The separation unit 501 is configured to separate multiple audio data from the target audio collected by the microphone according to the timbre when the text corresponding to the target audio contains a target keyword;

[0172] A determining unit 502 is configured to perform text recognition on the plurality of audio data to determine target audio data containing the target keyword;

[0173] A text acquisition unit 503 is configured to acquire a target text corresponding to the target audio data;

[0174] The label acquisition unit 504 is used to input the target text into the target abnormal scene recognition model to obtain the abnormal scene label corresponding to the target text; the target abnormal scene recognition model is obtained based on any embodiment of the abnormal scene recognition model training method provided in the above method embodiment.

[0175] As an optional implementation of the embodiment of the present application, the separation unit 501 is also used to obtain the initial audio collected by the target microphone; perform text recognition on the initial audio to obtain the initial text corresponding to the initial audio; determine whether the initial text contains the target keyword; if so, use the initial audio as the target audio.

[0176] As an optional implementation of an embodiment of the present application, the text acquisition unit 503 is specifically used to obtain a text recognition result corresponding to the target audio data, and determine the position of the target keyword in the target audio data based on the text recognition result; intercept the audio data within a preset time range before and after the position of the target keyword in the target audio data to generate a target text corresponding to the target audio data.

[0177] As an optional implementation of the embodiment of the present application, the label acquisition unit 504 is used to obtain the collection information of the microphone when collecting the target audio; the collection information includes the identification of the microphone and the collection time of the target audio; based on the collection information and the abnormal scene label, generate an abnormal warning prompt.

[0178] As an optional implementation of the embodiment of the present application, the separation unit 501 is specifically used to convert the target audio from time domain data into frequency domain data, and separate multiple audio data from the target audio according to timbre based on the frequency domain data corresponding to the target audio.

[0179] For the description of the features in the embodiment corresponding to the device for abnormal scene recognition, please refer to the relevant description of the embodiment corresponding to the method for abnormal scene recognition, and will not be repeated here.

[0180] An embodiment of the present application further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any of the above-mentioned abnormal scene recognition method embodiments.

[0181] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. The computer program is configured to execute the steps of any of the above-mentioned abnormal scene recognition method embodiments when running.

[0182] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0183] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the training method of any of the above-mentioned abnormal scene recognition models and the steps of any of the abnormal scene recognition methods.

[0184] An embodiment of the present application also provides another computer program product, including a non-volatile computer-readable storage medium, the non-volatile computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, implementing any of the above-mentioned training methods for abnormal scene recognition models and the steps of any of the abnormal scene recognition methods.

[0185] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0186] The above is a detailed introduction to the training method of the abnormal scene recognition model, the abnormal scene recognition method and the device provided by the present application. This article uses specific examples to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

Claims

1. A method for identifying abnormal scenes, characterized in that: include: When the text corresponding to the target audio collected by the microphone contains the target keyword, multiple audio data are separated from the target audio according to the timbre; Performing text recognition on the multiple audio data to determine target audio data containing the target keyword; Obtaining a target text corresponding to the target audio data; The target text is input into a target abnormal scene recognition model to obtain an abnormal scene label corresponding to the target text; the target abnormal scene recognition model is obtained based on the following training method: Obtain a sample text data set; the sample text data set includes multiple sample texts and a sample abnormal scene label corresponding to each sample text data; Inputting the sample text dataset into a pre-trained language model, and fine-tuning the pre-trained language model based on a dynamic rank and control gate mechanism to obtain a target abnormal scene recognition model; The pre-trained language model includes an embedding layer fine-tuning module, a multi-head attention layer fine-tuning module, and a feedforward network layer fine-tuning module; The pre-trained language model further includes an embedding layer, an embedding layer fine-tuning module, a multi-head attention layer, a multi-head attention layer fine-tuning module, a feedforward network layer, a feedforward network layer fine-tuning module, and a decoding output layer; The embedding layer is used to map the sample text into an initial feature vector; The embedding layer fine-tuning module is used to fine-tune the original weight matrix of the embedding layer based on the dynamic rank and control gate mechanism to obtain a first weight matrix, and to adjust the initial eigenvector according to the first weight matrix to obtain a first eigenvector; The multi-head attention layer is used to perform a linear transformation on the first feature vector to generate a query vector, a key vector, and a value vector; and obtain a second feature vector based on the query vector, the key vector, and the value vector; The multi-head attention layer fine-tuning module is used to fine-tune the original weight matrix in the multi-head attention layer based on the dynamic rank and control gate mechanism to obtain a second weight matrix, and adjust the second eigenvector according to the second weight matrix to obtain a third eigenvector; The feedforward network layer is used to fuse the first eigenvector and the third eigenvector to obtain a fourth eigenvector, and perform a linear transformation on the fourth eigenvector to obtain a fifth eigenvector; The feedforward network layer fine-tuning module is used to fine-tune the original weight matrix corresponding to the feedforward network layer based on the dynamic rank sum control gate mechanism to obtain a third weight matrix, so as to adjust the fifth eigenvector according to the third weight matrix to obtain a target eigenvector; The decoding output layer is used to output a predicted abnormal scene label based on the target feature vector; The pre-trained language model further includes: a loss optimization module for calculating a model loss value between the sample abnormal scene label and the predicted abnormal scene label corresponding to the sample text based on a target loss function calculation formula; and optimizing the pre-trained language model based on the model loss value to generate a target abnormal scene recognition model; the target loss function calculation formula is: Among them, L represents the model loss value, n represents the number of sample texts involved in the calculation, is the sample abnormal scene label corresponding to the sample text i, and x represents the initial feature vector corresponding to the sample text; is the second eigenvector obtained after processing x through the multi-head attention unit; is a feedforward neural network, α is a hyperparameter, is the gradient direction value of the weight reduction.

2. The method according to claim 1, characterized in that Before acquiring the initial audio collected by the target microphone, the method further includes: Get the initial audio collected by the target pickup; Performing text recognition on the initial audio to obtain an initial text corresponding to the initial audio; Determining whether the initial text contains target keywords; If so, the initial audio is used as the target audio.

3. The method according to claim 1, characterized in that The obtaining of the target text corresponding to the target audio data includes: Obtaining a text recognition result corresponding to the target audio data, and determining a position of the target keyword in the target audio data based on the text recognition result; The audio data within a preset time range before and after the position of the target keyword in the target audio data is intercepted to generate a target text corresponding to the target audio data.

4. The method according to claim 1, wherein After inputting the target text into a target abnormal scene recognition model and obtaining an abnormal scene label corresponding to the target text, the method further includes: Acquiring acquisition information when the microphone acquires the target audio; the acquisition information includes an identifier of the microphone and a time when the target audio is acquired; Based on the collected information and the abnormal scene label, an abnormal warning prompt is generated.

5. The method according to claim 1, wherein The step of separating a plurality of audio data from the target audio according to timbre includes: The target audio is converted from time domain data to frequency domain data, and multiple audio data are separated from the target audio according to timbre based on the frequency domain data corresponding to the target audio.

6. The method according to claim 1, characterized in that The multi-head attention layer fine-tuning module is specifically used for: Fine-tune the original weight matrix corresponding to the query vector in the multi-head attention layer based on the dynamic rank sum control gate mechanism, obtain a first sub-weight matrix corresponding to the query vector, and adjust the query vector according to the first sub-weight matrix; Based on the dynamic rank and control gate mechanism, the original weight matrix corresponding to the value vector in the multi-head attention layer is fine-tuned to obtain the second sub-weight matrix corresponding to the value vector, so as to adjust the value vector according to the second sub-weight matrix.

7. The method according to claim 1, characterized in that The embedding layer fine-tuning module, the multi-head attention layer fine-tuning module, and the feedforward network layer fine-tuning module all include: A weight matrix block module is used to perform block processing on the original weight matrix to obtain weight matrix sub-blocks; The dynamic rank adjustment and control gate module is used to calculate the weight increment based on the preset rank value range, the control gate parameter vector and the weight matrix sub-block, and fine-tune the original weight matrix based on the weight increment to obtain the fine-tuned weight matrix.

8. An abnormal scene recognition device, characterized in that: include: a separation unit, configured to separate a plurality of audio data from the target audio collected by the microphone according to timbre when the text corresponding to the target audio contains a target keyword; a determining unit, configured to perform text recognition on the plurality of audio data to determine target audio data containing the target keyword; A text acquisition unit, configured to acquire a target text corresponding to the target audio data; a label acquisition unit, configured to input the target text into a target abnormal scene recognition model to acquire an abnormal scene label corresponding to the target text; The target abnormal scene recognition model is obtained based on the following training method; Obtain a sample text data set; the sample text data set includes multiple sample texts and a sample abnormal scene label corresponding to each sample text data; Inputting the sample text dataset into a pre-trained language model, and fine-tuning the pre-trained language model based on a dynamic rank and control gate mechanism to obtain a target abnormal scene recognition model; The pre-trained language model includes an embedding layer fine-tuning module, a multi-head attention layer fine-tuning module, and a feedforward network layer fine-tuning module; The pre-trained language model further includes an embedding layer, an embedding layer fine-tuning module, a multi-head attention layer, a multi-head attention layer fine-tuning module, a feedforward network layer, a feedforward network layer fine-tuning module, and a decoding output layer; The embedding layer is used to map the sample text into an initial feature vector; The embedding layer fine-tuning module is used to fine-tune the original weight matrix of the embedding layer based on the dynamic rank and control gate mechanism to obtain a first weight matrix, and to adjust the initial eigenvector according to the first weight matrix to obtain a first eigenvector; The multi-head attention layer is used to perform a linear transformation on the first feature vector to generate a query vector, a key vector, and a value vector; and obtain a second feature vector based on the query vector, the key vector, and the value vector; The multi-head attention layer fine-tuning module is used to fine-tune the original weight matrix in the multi-head attention layer based on the dynamic rank and control gate mechanism to obtain a second weight matrix, and adjust the second eigenvector according to the second weight matrix to obtain a third eigenvector; The feedforward network layer is used to fuse the first eigenvector and the third eigenvector to obtain a fourth eigenvector, and perform a linear transformation on the fourth eigenvector to obtain a fifth eigenvector; The feedforward network layer fine-tuning module is used to fine-tune the original weight matrix corresponding to the feedforward network layer based on the dynamic rank sum control gate mechanism to obtain a third weight matrix, so as to adjust the fifth eigenvector according to the third weight matrix to obtain a target eigenvector; The decoding output layer is used to output a predicted abnormal scene label based on the target feature vector; The pre-trained language model further includes: a loss optimization module for calculating a model loss value between the sample abnormal scene label and the predicted abnormal scene label corresponding to the sample text based on a target loss function calculation formula; and optimizing the pre-trained language model based on the model loss value to generate a target abnormal scene recognition model; the target loss function calculation formula is: Among them, L represents the model loss value, n represents the number of sample texts involved in the calculation, is the sample abnormal scene label corresponding to the sample text i, and x represents the initial feature vector corresponding to the sample text; is the second eigenvector obtained after processing x through the multi-head attention unit; is a feedforward neural network, α is a hyperparameter, is the gradient direction value of the weight reduction.

9. An electronic device, characterized in that: include: memory for storing computer programs; A processor, configured to implement the abnormal scene recognition method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the abnormal scene recognition method according to any one of claims 1 to 7 is implemented.

11. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the abnormal scene recognition method according to any one of claims 1 to 7 is implemented.

Images