Abnormal video identification method, device and equipment and computer readable storage medium

Abstract

This application discloses a method, apparatus, device, and computer-readable storage medium for identifying abnormal videos, relating to the field of deep learning. The method includes: acquiring n sample video segments; during the i-th iteration of training, analyzing the sample video segments using the i-th analysis model obtained from the (i-1)-th iteration to obtain the i-th set of anomaly prediction results, where i ≥ 1 and i is an integer; performing balanced distribution processing on the i-th set of anomaly prediction results to obtain the i-th set of pseudo-labels corresponding to the n sample video segments; training the i-th analysis model based on the difference between the i-th set of anomaly prediction results and the i-th set of pseudo-labels to obtain the (i+1)-th analysis model during the i-th iteration of training, until the training of the analysis model meets the training result conditions, thus obtaining the anomaly analysis model. This not only saves the manpower required for manual labeling but also improves the recognition accuracy of the anomaly analysis model.

Description

Technical Field

[0001] This application relates to the field of deep learning, and in particular to a method, apparatus, device, and computer-readable storage medium for identifying abnormal videos. Background Technology

[0002] Automated video anomaly detection methods refer to methods that can detect abnormal events in video data without human supervision, facilitating timely handling of these events.

[0003] In related technologies, a weakly supervised learning method is used to train a normal video data model for identifying abnormal videos by training normal video data as input. When the identified video data does not match the normal video data model, the video data will be marked as an abnormal video.

[0004] However, in the above methods, the normal video data model cannot cover all normal video data, and the method for identifying abnormal videos only trains the detection model on the input normal video data. Therefore, there are problems such as low sample capacity for abnormal video identification and poor accuracy of abnormal video identification. Summary of the Invention

[0005] This application provides a method, apparatus, device, and computer-readable storage medium for identifying abnormal videos, capable of covering various types of video data for model training, thereby improving the accuracy of abnormal video identification. The technical solution is as follows:

[0006] On the one hand, a method for identifying abnormal videos is provided, the method comprising:

[0007] Obtain n sample video segments, wherein the sample video segments are segments composed of video frames used to train the anomaly analysis model, and n≥2 and n is an integer;

[0008] During the i-th iteration training process, the i-th analysis model obtained from the (i-1)-th iteration training is used to analyze the sample video segment to obtain the i-th group of anomaly prediction results, where i ≥ 1 and i is an integer;

[0009] The i-th group of anomaly prediction results are processed to achieve a balanced distribution, resulting in the i-th group of pseudo-labels corresponding to the n sample video segments.

[0010] Based on the difference between the i-th anomaly prediction results and the i-th pseudo-labels, the i-th analysis model is trained to obtain the (i+1)-th analysis model in the i-th iteration training process, until the training of the analysis model meets the training result conditions, thus obtaining the anomaly analysis model, which is used to identify abnormal videos.

[0011] On the other hand, an abnormal video identification device is provided, the device comprising:

[0012] The acquisition module is used to acquire n sample video segments, wherein the sample video segments are segments composed of video frames used to train the anomaly analysis model, and n≥2 and n is an integer;

[0013] The analysis module is used to analyze the sample video segment through the i-th analysis model obtained by the (i-1)-th iteration training during the i-th iteration training process, and obtain the i-th group of anomaly prediction results, where i≥1 and i is an integer;

[0014] The balancing module is used to perform balanced distribution processing on the i-th group of anomaly prediction results to obtain the i-th group of pseudo-labels corresponding to the n sample video segments;

[0015] The training module is used to train the i-th analysis model based on the difference between the i-th group of anomaly prediction results and the i-th group of pseudo-labels, to obtain the (i+1)-th analysis model in the i-th iteration training process, until the training of the analysis model meets the training result conditions, and an anomaly analysis model is obtained. The anomaly analysis model is used to identify abnormal videos.

[0016] On the other hand, a computer device is provided, the computer device including a processor and a memory, the memory storing at least one instruction, at least one program, code set or instruction set, the at least one instruction, the at least one program, the code set or instruction set being loaded and executed by the processor to implement the abnormal video identification method as described in any of the above embodiments of this application.

[0017] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction, at least one program, code set, or instruction set is stored therein, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the abnormal video identification method as described in any of the embodiments of this application above.

[0018] On the other hand, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the abnormal video identification method described in any of the above embodiments.

[0019] The beneficial effects of the technical solutions provided in this application include at least the following:

[0020] By performing balanced distribution processing on the anomaly prediction results obtained from the analysis of sample video segments, pseudo-labels for the sample video segments are obtained, which are then used to train the analysis model. Finally, an anomaly analysis model that can be used to identify abnormal videos is obtained. It can not only perform iterative training on all types of sample video segments, but also improve the recognition accuracy of the anomaly analysis model through iterative training. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a schematic diagram of an implementation environment provided by an exemplary embodiment of this application;

[0023] Figure 2 This is a schematic diagram illustrating the process of identifying abnormal videos provided in an exemplary embodiment of this application;

[0024] Figure 3 This is a flowchart illustrating the identification of abnormal videos provided in an exemplary embodiment of this application;

[0025] Figure 4 This is a flowchart illustrating the process of obtaining a sample video clip, provided in an exemplary embodiment of this application.

[0026] Figure 5 This is a flowchart illustrating the process of identifying abnormal videos provided in another exemplary embodiment of this application;

[0027] Figure 6 This is a flowchart illustrating the identification of abnormal videos provided in another exemplary embodiment of this application;

[0028] Figure 7 This is a schematic diagram of the initial pseudo-tag generation process provided in an exemplary embodiment of this application;

[0029] Figure 8 This is a flowchart of an abnormal video recognition device provided in an exemplary embodiment of this application;

[0030] Figure 9 This is a schematic diagram of an object texture acquisition process provided in another exemplary embodiment of this application;

[0031] Figure 10 This is a structural block diagram of a server provided in an exemplary embodiment of this application. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0033] First, a brief introduction to the terms used in the embodiments of this application will be given.

[0034] K-means clustering algorithm is an iterative clustering analysis algorithm. It randomly selects k objects as initial cluster centers, and assigns each object to the nearest cluster center by calculating the distance between each object and each initial cluster center. Each time an object is assigned, the cluster centers itself based on the existing objects in its own cluster. This process is repeated until a certain condition is met and then terminates.

[0035] Backbone Network: Also known as the backbone network, it is a key component of a computer architecture that contributes to network stability. Its main function is to connect the various components of the network into a cohesive interactive unit, while ensuring that each part maintains its independent operational integrity. In this embodiment, the backbone network is mainly used to extract features from sample video clips and obtain the video features of the sample video clips.

[0036] Isolation Forest: Also known as isolated forest, it is a method mainly used to detect outliers in continuous structured data samples. The specific detection steps are to continuously and randomly cut and separate data points that account for a small proportion of the total sample size and whose feature values ​​differ significantly from those of normal data, and calculate their outlier scores until the target sample is obtained. These outlier data points are generally characterized by sparse distribution and distance from high-density groups.

[0037] Focal Loss (FL): Generally used to solve the problem of imbalanced sample size. Its main calculation method is to ensure sample balance by allocating more weight to examples that are difficult to classify or are prone to misclassification (such as backgrounds or parts of objects with noisy textures) and reducing the weight of simple examples (such as background objects).

[0038] The Long Tail Effect is a statistical term. "Head" and "tail" are two statistical terms. The bulge in the middle of a normal distribution curve is called the "head," and the relatively flattened portions on both sides are called the "tail." Specifically, it refers to the portion of the market with high demand (the "head"), where the number of items is relatively small, while the portion with low demand (the "tail") has a large number of items. Therefore, the large quantity of low-value items in the "tail" forms a long "tail" on the demand curve, hence the "long tail effect." In this application, "head" refers to abnormal video clips, and "tail" refers to normal video clips, because in real life, the probability of normal video clips is much higher than that of abnormal video clips. Since this application is based on an abnormal video identification method, this definition is used.

[0039] Total Direct Effect (TDE): This refers to the phenomenon where, due to sample quantity bias, the actual number of samples required is less than the actual number required. The model trained on these samples will produce a certain bias during the detection process. To eliminate this bias, the difference between the model's output score and the output score of the input sample is used as the output result of the input sample.

[0040] When processing video clips, it is necessary to promptly identify and address any abnormal events or phenomena (such as crowd disturbances, violent crimes, or vehicles driving in the wrong direction). However, non-automated video clip anomaly identification methods typically require significant manpower and resources. Therefore, in related technologies, automated video clip identification technology based on deep neural networks offers higher efficiency and accuracy compared to manual video inspection. This involves pre-labeling video clips for model training to automatically identify them. However, it also has drawbacks: firstly, the training process requires each frame of the video clip to contain a normal or abnormal label. Since the probability of abnormal events is much lower than that of normal events, collecting abnormal video clips with abnormal labels is challenging. Secondly, labeling the video clips for training also requires considerable manpower.

[0041] Therefore, weakly supervised learning methods are widely used in video anomaly detection. This method reduces the cost of manual annotation to some extent. In related technologies, only normal video data needs to be input and trained to form a normal video database. When the video data to be identified does not match the normal video database, it can be marked as abnormal video data. However, this method also has some problems: there are many types of normal video data in real life, and the normal video data trained cannot fully cover all normal video data. Therefore, in the process of anomaly video detection, problems such as the normal video data being identified not matching the trained model and being marked as abnormal video data, poor accuracy of the identification model, and small sample size of the identification model may occur.

[0042] This application provides a method for identifying abnormal videos. In the implementation process, the method analyzes sample video segments through an analysis model to obtain the anomaly prediction results corresponding to the sample video segments. The anomaly prediction results are then processed to achieve a balanced distribution to obtain pseudo-labels corresponding to the sample video segments. Based on the difference between the pseudo-labels and the anomaly prediction results, the analysis model is iteratively trained to finally obtain an anomaly analysis model for identifying abnormal videos. This method not only saves the manpower required for manual labeling of video segments but also improves the recognition accuracy of the anomaly analysis model.

[0043] Figure 1 This is a schematic diagram of an implementation environment provided by an exemplary embodiment of this application, such as... Figure 1 As shown, the implementation environment includes a terminal 110 and a server 120, wherein the terminal 110 and the server 120 are connected through a communication network 130.

[0044] The terminal 110 is equipped with an application that provides video analysis functions. When the user runs the application, the terminal 110 sends the stored video segment data to the server 120. The anomaly analysis model in the server 120 extracts and analyzes the features of the video segments sent by the terminal 110 and sends the obtained anomaly identification results back to the terminal 110.

[0045] The anomaly analysis model in server 120 is obtained by iteratively training analysis model 122 by updating the pseudo-labels of sample video segment 121.

[0046] In some embodiments, terminal 110 sends video segment data to server 120 via communication network 130. Server 120 samples and analyzes the received video segment data and sends the analysis results back to terminal 110.

[0047] The terminal can be a smartphone, tablet, desktop computer, laptop, etc., but is not limited to these. The terminal and the server can be connected directly or indirectly through wired or wireless communication, which is not limited herein.

[0048] It is worth noting that the aforementioned servers can be independent physical servers, server clusters or distributed systems composed of multiple physical servers, or cloud servers that provide basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0049] Cloud technology refers to a hosting technology that unifies hardware, software, and network resources within a wide area network (WAN) or local area network (LAN) to achieve data computation, storage, processing, and sharing. Based on the cloud computing business model, cloud technology encompasses network technology, information technology, integration technology, management platform technology, and application technology. It can form resource pools, providing flexible and convenient on-demand access. Cloud computing technology will become a crucial support. Backend services of technical network systems require substantial computing and storage resources, such as websites, image websites, and many portal websites. With the rapid development and application of the internet industry, every item may have its own identification mark in the future, requiring transmission to backend systems for logical processing. Data at different levels will be processed separately, and various industry data will require robust system support, which can only be achieved through cloud computing.

[0050] In some embodiments, the server described above can also be implemented as a node in a blockchain system. Blockchain is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and cryptographic algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include a blockchain underlying platform, a platform product service layer, and an application service layer.

[0051] In some embodiments, please refer to Figure 2 , Figure 2 This is a schematic diagram illustrating the abnormal video recognition process provided in an exemplary embodiment of this application, such as... Figure 2As shown, in the first iteration of training, a sample video stream is acquired, and n sample video segments 210 are extracted from the sample video stream, where n≥2 and n is an integer. The n sample video segments are input into the scene clusterer 220 and the pseudo-label generator 230, respectively. In the scene clusterer 220, the feature extraction network 221 is used to extract the video features corresponding to the sample video segments 210. Scene clustering analysis 211 is performed on the video features to obtain the scene clustering results corresponding to the video features. In the pseudo-label generator 230, the feature extraction network 221 is also used to extract the video features corresponding to the sample video segments 210. The isolation forest method is used to obtain the outlier score corresponding to the video feature 211. The initial pseudo-label 231 of the sample video segment 210 is obtained based on the outlier score. A sample video segment 210, its corresponding initial pseudo-label 231, and the scene clustering result are input into the model trainer 240.

[0052] In the model trainer 240, the feature extraction network 221 is used to extract features from an input sample video segment 210 to obtain the video features corresponding to the sample video segment 210. Based on the attention mechanism, attention is calculated on the input scene clustering results and video features to obtain the attention vector 241 corresponding to the input sample video segment 210. The analysis model 242 analyzes the attention vector 241 and video features to obtain the anomaly prediction result 243 corresponding to the sample video segment 210. Based on the difference between the anomaly prediction result 243 and the initial pseudo label 231, the analysis model 242 and the feature extraction network 221 are trained and updated to obtain the updated analysis model and feature extraction network. The updated feature extraction network is then input into the scene clusterer 220 to update the scene clustering results.

[0053] The updated analysis model and feature extraction network are input into the model inferr 250. An intermediate sample video segment and the sample video segments before and after it are input into the model inferr 250. The updated feature extraction network extracts features from the input sample video segment to obtain updated video features. The updated analysis model is used to analyze the updated video features to obtain anomaly prediction results for the next iteration of training. The anomaly prediction results are balanced to obtain pseudo-labels 251. The pseudo-labels 251 are input into the model trainer 240. The analysis model 242 is then iteratively trained until it meets the training result conditions, resulting in the anomaly analysis model 260.

[0054] It is worth noting that during the first loop, the sample video clips with initial pseudo-labels input to the model trainer 240 are obtained through an isolation forest. However, starting from the second loop, the pseudo-labels of the input sample video clips are updated by the model inferrer 250. The two sample video clips may be the same sample video clip, or they may be different sample video clips with overlapping samples. After the model trainer 240 has been trained a certain number of times, the scene clusterer 230 will update the scene clustering results again.

[0055] In some embodiments, the anomaly analysis model 260 is a model for identifying abnormal video segments of sample video clips. For example, in crowded public places such as shopping malls and supermarkets, it is necessary to identify abnormal video segments in the videos of these public places. The abnormal situations covered in the abnormal videos include stampedes, theft and robbery, fighting, etc., and the identification results are fed back to the management office of the public place for timely processing. Alternatively, in road traffic places such as roads and pedestrian crossings, it is necessary to identify abnormal video segments in the videos of road traffic places. The abnormal situations covered in the abnormal videos include pedestrians crossing the road, car rear-end collisions, car accidents, etc., and the identification results are fed back to the traffic management center for timely processing.

[0056] In light of the aforementioned implementation environment, the abnormal video identification method provided in this application embodiment will be described. This method can be executed by a terminal or a server, or by both a terminal and a server, and can be applied to situations such as... Figure 1 Taking server 120 as an example, the following explanation will be provided. Figure 3 As shown, the method includes:

[0057] Step 301: Obtain n sample video segments.

[0058] The sample video segments are segments composed of video frames used to train the anomaly analysis model, where n ≥ 2 and n is an integer.

[0059] The process of obtaining n sample video segments includes obtaining a sample video stream, which contains video frames; extracting sample video frames from the sample video stream, which are used to train the analysis model; and obtaining n sample video segments based on the connection relationships between the sample video frames.

[0060] In some embodiments, a sample video stream is first input into the server. Optionally, the sample video stream includes normal video frames and abnormal video frames, or it may only contain normal video frames, or it may only contain abnormal video frames. Next, sample video frames are extracted from the sample video stream. For an illustrative example, please refer to [reference needed]. Figure 4 , Figure 4This is a schematic diagram of a sample video segment extraction process provided in an exemplary embodiment of this application, as shown below. Figure 4 As shown, in the sample video segment acquisition process 400, the server extracts sample video frames 420 from the video stream through a sampling interval 410. The sample video frames 420 are video frames used to train the analysis model. The sampling interval may be one video frame; or it may be multiple video frames; or the sampling of the sample video frames may be continuous. The sampling interval for extracting sample video frames for each video stream may be the same or different. It is worth noting that the sampling length is variable. When the sampling length is 1, the input is only one image; the smaller the sampling interval, the denser the input data, resulting in higher training accuracy, but lower frames per second. When the server trains the analysis model, it needs to ensure that the sampling length and sampling interval of the same sample video segment remain consistent in each iteration of training. The connection relationship between sample video frames 420 is analyzed to determine the sampling length 430 for the sample video stream. Sample video frames 420 are sampled to obtain sample video segments 440. Optionally, the sampling length can be determined in at least one of the following ways:

[0061] 1. Set a fixed number of sample video frames, and determine the sampling length based on the number of sample video frames. That is, each sample video segment contains the same number of sample video frames.

[0062] 2. Set a scene similarity probability threshold, extract the pixels of the sample video frames, analyze the pixels of the sample video frames, calculate the scene similarity probability of the sample video frames, and select the sample video frames whose similarity probability is greater than or equal to the scene similarity threshold as the video frames to be sampled.

[0063] It is worth noting that the above-mentioned method of obtaining sample video clips is only an illustrative example, and the specific method of obtaining sample video clips is not limited in the embodiments of this application.

[0064] Step 302: During the i-th iteration training process, the i-th analysis model obtained from the (i-1)-th iteration training is used to analyze the sample video segments to obtain the i-th group of anomaly prediction results, where i ≥ 1 and i is an integer.

[0065] In some embodiments, during the i-th iteration training process, the i-th video feature of the sample video segment is extracted. The video feature is used to analyze the sample video segment. The i-th video feature is analyzed by the i-th analysis model obtained by the (i-1)-th iteration training to obtain the i-th set of anomaly prediction results.

[0066] Schematic illustration: During the i-th iteration of training, the input set of sample video segments is used. The i-th feature extraction network (e.g., a pillar network) trained through the (i-1)-th iteration is used to extract features from the sample video segments, resulting in the video features corresponding to the sample video segments. Here, a video feature is a variable of length... D Vector features (here for) D (Specific numerical values ​​are not limited), and the specific extraction algorithm is shown in Formula 1:

[0067] Formula 1: ,

[0068] in, For video features, video features It is a length of D Vector features, For sample video clips, It serves as a pillar network.

[0069] Optionally, when the sampling length of the sample video segment is equal to 1, the backbone network is a two-dimensional convolutional neural classification network, such as a Residual Network (ResNet), which extracts features from the images contained in the sample video segment. When the sampling length of the sample video segment is greater than or equal to 2, the backbone network is a three-dimensional convolutional neural classification network, such as Inflated 3D ConvNet, which extracts features from the video actions contained in the sample video segment. The video features obtained by using the feature extraction network to extract features from the sample video segment are the features used to analyze the sample video segment.

[0070] In some embodiments, the video features are analyzed by the i-th analysis model obtained through the (i-1)th iteration training to obtain the anomaly prediction result corresponding to the video features; based on the i-th video feature, the i-th scene clustering result corresponding to the sample video segment is determined, and the scene clustering result is the result obtained by performing cluster analysis on the sample video segment.

[0071] Indicatively, the video features used for cluster training are n sample video segments ( The set of n video features extracted by the i-th feature extraction network obtained through the (i-1)-th iteration is used. The server uses the K-means algorithm to perform cluster analysis on the video features, and defines a cluster feature for each video feature. The output of the K-means algorithm is shown in Formula 2.

[0072] Formula 2:

[0073] in, For clustering feature set, For the first k Clustering features This represents the number of cluster features (or the number of clusters), illustratively speaking, that is, each video feature corresponds to a certain number of clusters. k Clustering features.

[0074] The prototype of each cluster is calculated using the method shown in Formula 3:

[0075] Formula 3:

[0076] in, This serves as a clustering prototype. The proportion of each cluster can be statistically analyzed using this prototype, as shown in Formula 4.

[0077] Formula 4:

[0078] in, For the set of cluster proportions, This represents the percentage of clusters.

[0079] Cluster percentage and cluster feature are scene clustering results obtained by the server through cluster analysis of sample video segments. That is, by performing cluster analysis on the i-th video feature through mean clustering, the i-th cluster feature and the i-th cluster percentage of the sample video segment are obtained; the i-th cluster feature and the i-th cluster percentage are used to determine the i-th scene clustering result corresponding to the sample video segment.

[0080] In some embodiments, the i-th scene clustering result and the i-th video feature are analyzed by the i-th analysis model obtained by training in the (i-1)-th iteration to obtain the i-th group of anomaly prediction results.

[0081] As an example, the server inputs a sample video clip. The sample video segment is obtained by using the i-th feature extraction network trained through the (i-1)-th iteration to extract features from the sample video segment. Corresponding video features Scene clustering results based on attention mechanism of video features and sample video segments (including clustering features) and the proportion of clusters Attention is calculated using the formula shown in Formula 5:

[0082] Formula 5:

[0083] in, For attention vectors, This is a fully connected layer, whose main function is to project a feature vector of length D (i.e., video features) onto a layer of length D. k The feature vectors, here k The value of is not fixed; when the sample data is too large during training, then... k The value is relatively small; when the fitting ability of the sample data is insufficient during training, then... k The value is relatively large. k The main purpose of setting the value of is to balance the fitting ability of the sample data.

[0084] By analyzing video features and attention vectors using the analysis model, the anomaly prediction results corresponding to the sample video segments are obtained. The specific analysis formula is shown in Formula Six:

[0085] Formula Six:

[0086] in, For abnormal prediction results, For analysis model. It is a defined model as The function, input video features and attention vector Output anomaly prediction results ,in, Given a continuous value between [0,1], the anomaly prediction result is... The anomaly probability value represents the sample video segment. The larger the value, the greater the probability that the sample video segment is an abnormal video. That is, the attention model is used to analyze the clustering result of the i-th scene and the i-th video feature based on the attention mechanism to obtain the i-th attention vector. The attention model is the model used to determine the attention vector. The i-th attention vector and the i-th video feature are analyzed by the i-th analysis model obtained by training in the (i-1)-th iteration to obtain the i-th group of abnormal prediction results.

[0087] Step 303: Perform balanced distribution processing on the i-th group of anomaly prediction results to obtain the i-th group of pseudo-labels corresponding to the n sample video segments.

[0088] As an illustration, the server processes one sample video segment at a time. The model inference is performed on the video segments before and after the sample video segment, and the anomaly prediction results of the i-th group are balanced by using all direct effects to obtain the sample video segment. The abnormal score is calculated using the formula shown in Formula 7:

[0089] Formula 7:

[0090] in, The average video feature is the feature of all input sample videos. Optionally, the preceding and following video segments may be one video segment before and after the sample video segment, or several video segments before and after the sample video segment. After obtaining the abnormal score set of n sample video segments, the sample video segments corresponding to the highest 10% scores are selected from the score set as abnormal video segments and labeled with an abnormal label (i.e., 1). The sample video segments corresponding to the lowest 20% scores are selected from the score set as normal video segments and labeled with a normal label (i.e., 0). The set of normal labels and abnormal labels is used as the i-th pseudo-label corresponding to the sample video segment.

[0091] Step 304: Based on the difference between the i-th anomaly prediction results and the i-th pseudo-labels, train the i-th analysis model to obtain the (i+1)-th analysis model in the i-th iteration training process, until the training of the analysis model meets the training result conditions, and obtain the anomaly analysis model.

[0092] Among them, the anomaly analysis model is used to identify abnormal videos.

[0093] In some embodiments, after obtaining the i-th group of anomaly prediction results, the server trains the i-th analysis model using the results calculated by the focus loss function from the i-th group of prediction results and the i-th group of pseudo-labels. The specific calculation results are shown in Formula 8.

[0094] Formula 8:

[0095] in, and These are two hyperparameters that can be customized before training as needed. Their function is to reduce the weight of sample video clips as normal video clips. For the i-th group of anomaly prediction results, Let i be the i-th pseudo-label of the sample video segment.

[0096] The focus loss function can mitigate the impact of the imbalance in the number of normal and abnormal video segments on the training process. Furthermore, the focus loss function is used to update the analysis model and the fully connected layer, resulting in the (i+1)th analysis model in the i-th iteration of training. Specifically, the loss function is used to analyze the i-th group of abnormal prediction results and the i-th group of pseudo-labels to obtain the loss result; this loss result is then used to train the i-th analysis model.

[0097] During iterative training, training terminates when the analysis model meets the training result conditions, and the trained analysis model serves as the anomaly analysis model for identifying abnormal videos. Optionally, the anomaly analysis model can identify abnormal human activities in the sample video clips, or it can identify abnormal scene images in the sample video clips; this is not limited to these specific requirements.

[0098] In summary, the embodiments of this application provide a method for identifying abnormal videos. During implementation, this method analyzes sample video segments using an analysis model to obtain anomaly prediction results corresponding to the sample video segments. The anomaly prediction results are then processed to achieve a balanced distribution, resulting in pseudo-labels for the sample video segments. Based on the difference between the pseudo-labels and the anomaly prediction results, the analysis model is iteratively trained, ultimately yielding an anomaly analysis model for identifying abnormal videos. This method not only saves the manpower required for manual data annotation of video segments but also improves the recognition accuracy of the anomaly analysis model.

[0099] In some embodiments, during the i-th iteration training process, extracting the i-th video feature of the sample video segment further includes: during the i-th iteration training process, extracting the i-th video feature using the i-th feature extraction network obtained from the (i-1)-th iteration training. For illustrative examples, please refer to [reference needed]. Figure 5 , Figure 5 This is a flowchart of an abnormal video recognition method provided in an exemplary embodiment of this application, in which the method is applied to, for example... Figure 1 Taking server 120 as an example, the following explanation will be provided. Figure 5 As shown, the method includes:

[0100] Step 501: Obtain n sample video segments.

[0101] The sample video segments are segments composed of video frames used to train the anomaly analysis model, where n ≥ 2 and n is an integer.

[0102] First, the server receives a sample video stream containing video frames from the terminal, extracts the sample video frames needed to train the analysis model based on the sample video stream, and obtains n sample video segments based on the connection relationships between the sample video frames. Optionally, the server obtains the sample video segments in at least one of the following ways:

[0103] 1. The server selects video frames containing the activities of a specified person as sample video frames, and obtains n sample video segments based on the relationship between the sample video frames containing the activities of the specified person. The analysis model is trained using the analysis results of the activities of the specified person.

[0104] 2. The server selects video frames containing specified scene images as sample video frames, and obtains n sample video segments based on the relationship between sample video frames containing specified scene images. The analysis model is trained using the analysis results of the specified scene images.

[0105] It is worth noting that the above-described method for obtaining sample video clips is merely an illustrative example, and the specific method for obtaining sample video clips is not limited in the embodiments of this application.

[0106] Step 502: During the i-th iteration training process, the i-th video feature is extracted using the i-th feature extraction network obtained from the (i-1)-th iteration training.

[0107] Among them, video features are the features used to analyze sample video clips.

[0108] As an illustration, during the i-th iteration of training, n sample video segments are input, and the i-th feature extraction network obtained from the (i-1)-th iteration is used to extract features from the sample video segments to obtain the video features corresponding to the sample video segments. For the specific algorithm, please refer to Formula 1.

[0109] In some embodiments, during the i-th iteration of training, cluster analysis is performed on the i-th video feature to obtain the i-th scene clustering result. An attention model is then used to analyze the i-th scene clustering result and the i-th video feature based on an attention mechanism to obtain the i-th attention vector. The i-th attention vector and the i-th video feature are then analyzed using the i-th analysis model obtained from the (i-1)-th iteration to obtain the i-th set of anomaly prediction results. The specific steps are the same as in step 302.

[0110] Step 503: The i-th anomaly prediction results are balanced and distributed using the i-th attention vector and the i-th video feature, and the i-th pseudo-labels corresponding to the n sample video segments are updated.

[0111] In some embodiments, the server takes a sample video segment as input for each training iteration. and sample video clips front and back b The model performs inference on a video clip, which is optional. b It could be 1, meaning the sample video clip. One video clip before and one after, or, b It may be greater than 1, that is, the sample video segment Multiple video clips before and after.

[0112] Using feature extraction networks f For all input sample video segments (i.e., sample video segments) and sample video clips front and back b Feature extraction is performed on each video segment to obtain sample video segments. Corresponding video features and the video features corresponding to all sample video segments. The i-th group of anomaly prediction results are balanced using all direct effects to obtain sample video segments. The anomaly score is calculated using Formula 7. After obtaining the set of anomaly scores for n sample video segments, the i-th pseudo-label corresponding to each sample video segment is obtained based on its anomaly score.

[0113] It is worth noting that the i-th pseudo-label corresponding to the sample video segment is a pseudo-label of a portion of the sample video segments out of the n sample video segments, and the pseudo-label updated in each training session is the pseudo-label of a portion of the sample video segments.

[0114] Step 504: Based on the difference between the i-th group of anomaly prediction results and the i-th group of pseudo-labels, update the i-th feature extraction network to obtain the (i+1)-th feature extraction network in the i-th iteration training process.

[0115] In some embodiments, after obtaining the i-th group of anomaly prediction results, the server updates the i-th feature extraction network with the result calculated by the focus loss function based on the i-th group of prediction results and the i-th group of pseudo-labels. The specific calculation result can be found in Formula 8.

[0116] The feature extraction network is updated using the focus loss function to obtain the (i+1)th feature extraction network in the i-th iteration training process.

[0117] In illustrative terms, after one or several iterations of training, the updated feature extraction network will update the scene clustering results of the sample video clips, including updating the clustering features and the proportion of clusters. Alternatively, after iterative training, once the scene clustering results meet certain update conditions, the updated feature extraction network will update the scene clustering results of the sample video clips, including updating the clustering features and the proportion of clusters. No further limitations are specified here.

[0118] In summary, the embodiments of this application provide a method for identifying abnormal videos. During implementation, this method analyzes sample video segments using an analysis model to obtain anomaly prediction results corresponding to the sample video segments. The anomaly prediction results are then processed to achieve a balanced distribution, resulting in pseudo-labels for the sample video segments. Based on the difference between the pseudo-labels and the anomaly prediction results, the analysis model is iteratively trained, ultimately yielding an anomaly analysis model for identifying abnormal videos. This method not only saves the manpower required for manual data annotation of video segments but also improves the recognition accuracy of the anomaly analysis model.

[0119] In some embodiments, the method for calculating outlier scores during iterative training also includes isolation forests. Please refer to [link / reference]. Figure 6 , Figure 6 This is a flowchart illustrating the identification of abnormal videos provided in an exemplary embodiment of this application, in which the method is applied to, for example... Figure 1 Taking server 120 as an example, the following explanation will be provided. Figure 6 As shown, the method includes:

[0120] Step 601: Obtain n sample video segments.

[0121] The sample video segments are segments composed of video frames used to train the anomaly analysis model, where n ≥ 2 and n is an integer.

[0122] The sample video clip is extracted from the sample video stream; a single sample video clip... V It is a three-dimensional structure W × H × T The video clip, in which, W and H It is the resolution of the sample video clip. T The sampling length of a sample video segment is... ,in, The resolution is W × H Sample video frames.

[0123] Step 602: At the beginning of the iteration loop, the sample video segments are analyzed by the isolated forest to obtain the outlier results of the sample video segments.

[0124] At the beginning of the iteration loop, the server inputs a set of sample video clips. A feature set is obtained by extracting video features from sample video segments using a feature extraction network. .

[0125] Using an isolated forest approach, we analyze n video feature sets, and for each video feature... Each is given its corresponding outlier score. Therefore, the set of outlier scores corresponding to the video feature set can be obtained. .

[0126] Step 603: Based on the outlier results, obtain the initial pseudo-labels for the sample video segments.

[0127] In this application, the definition is... Let j be the set of pseudo-labels, where j is the number of times the pseudo-labels have been updated. For sample video clips The tags are illustrative, representing sample video clips taken during the server's training of the analytical model. The intermediate frames are used as sample video segments Tags, or sample video clips. The first or last frame as a sample video segment The tag, A value of 1 represents a sample video segment. This is an abnormal video clip. A value of 0 represents a sample video segment. This is a normal video clip. n For the number of tags, For initial pseudo-tags, These are the pseudo-tags after being updated j times.

[0128] The process of generating initial pseudo-tags can be found in [reference]. Figure 7 , Figure 7 This is a schematic diagram of the initial pseudo-tag generation process provided in an exemplary embodiment of this application, such as... Figure 7 As shown, a feature extraction network is used to extract features from n sample video segments 710. An isolation forest is used to perform outlier analysis on the video feature set 720, assigning a corresponding outlier score to each video feature. After obtaining the outlier score set 730 corresponding to the video feature set, the sample video segments corresponding to the highest 10% scores are selected as anomalous video segments and labeled with an anomalous label (i.e., 1). The sample video segments corresponding to the lowest 20% scores are selected as normal video segments and labeled with a normal label (i.e., 0). The set of normal labels and anomalous labels is used as the initial pseudo-labels for the sample video segments. 740.

[0129] Step 604: Analyze the sample video segments using the initial analysis model at the beginning of the iterative loop to obtain the first set of anomaly prediction results.

[0130] In some embodiments, the first analysis model is defined as F The first analysis model was used to analyze the sample video clips.V Analysis yielded the first set of anomaly prediction results, which is a continuous value between [0,1]. ,Right now Anomaly prediction results This represents the anomaly probability value of the sample video segment, that is, the probability that the sample video segment is an anomalous video segment. The larger the value, the greater the probability that the sample video segment is an abnormal video segment.

[0131] Step 605: Based on the difference between the first set of anomaly prediction results and the initial pseudo-labels, train the initial analysis model to obtain the first analysis model in the first iteration training process.

[0132] In some embodiments, the first analysis model is obtained by calculating the focus loss function using the first set of anomaly prediction results and the initial pseudo-labels, and then using the calculation results to train the initial analysis model.

[0133] In summary, the embodiments of this application provide a method for identifying abnormal videos. During implementation, this method analyzes sample video segments using an analysis model to obtain anomaly prediction results corresponding to the sample video segments. The anomaly prediction results are then processed to achieve a balanced distribution, resulting in pseudo-labels for the sample video segments. Based on the difference between the pseudo-labels and the anomaly prediction results, the analysis model is iteratively trained, ultimately yielding an anomaly analysis model for identifying abnormal videos. This method not only saves the manpower required for manual data annotation of video segments but also improves the recognition accuracy of the anomaly analysis model.

[0134] This application's embodiments are based on the prior knowledge that "outliers are more likely to be anomalous data." It assumes that outlier data is more likely to be identified as anomalous data compared to normal data; therefore, scenes or character activities that occur less frequently can be defined as outliers. Furthermore, this application's embodiments also propose methods to address the noise and imbalance between normal and anomalous data ratios that arise when outlier data is treated as anomalous data.

[0135] The basic idea of ​​this application embodiment is to use an isolation forest to analyze sample video segments, obtain outlier results of the sample video segments, label the parts with the highest degree of outlier in the sample video segments as outliers, and label the parts with the lowest degree of outliers as normal. The set of these labels is called the initial pseudo-label. The sample video segments with the initial pseudo-labels are input into the network for training. After training, the model re-labels the sample video segments, and the data with pseudo-labels is input into the network for training again. This process is repeated, training the network from coarse to fine, until a certain number of training iterations are reached or the network converges.

[0136] Furthermore, this application embodiment discovers through causal inference theory that initial pseudo-labels often contain noise (i.e., errors exist in the label data), and the degree of noise typically affects the entire training process. Therefore, this application embodiment proposes a method to mitigate noise based on an attention mechanism. In most existing datasets and real-world situations, due to the often imbalanced labels—that is, abnormal video clips are usually far fewer than normal video clips—this application embodiment uses a focus loss function during training and all direct effects during inference to mitigate this long-tail effect, thereby improving the sample balance of the analysis model.

[0137] This application's embodiments introduce an isolated forest and a coarse-to-fine training process to address the issue of unlabeled training. It also proposes variable sampling schemes to meet different needs. When real-time performance requirements are not high but accuracy is less critical, the sampling interval can be reduced or the sampling length increased to meet the requirements. Conversely, when real-time performance is somewhat critical, the sampling interval can be increased or the sampling length decreased to meet the requirements.

[0138] During iterative training, an attention mechanism was used to mitigate the noise associated with obtaining initial pseudo-labels through isolated forests, thus improving the training process and increasing the accuracy of the analysis model. Furthermore, in updating pseudo-labels, this embodiment used all direct effects to balance the distribution of anomaly prediction results, further improving the model's accuracy by addressing the sample imbalance problem between abnormal and normal video segments.

[0139] Figure 8 This is a structural diagram of an abnormal video recognition device provided in an exemplary embodiment of this application, as shown below. Figure 8 As shown, the device includes:

[0140] The acquisition module 810 is used to acquire n sample video segments, wherein the sample video segments are segments composed of video frames used to train the anomaly analysis model, and n≥2 and n is an integer;

[0141] The analysis module 820 is used to analyze the sample video segment through the analysis model obtained by the (i-1)th iteration training during the i-th iteration training process, and obtain the i-th group of anomaly prediction results, where i≥1 and i is an integer;

[0142] The balancing module 830 is used to perform balanced distribution processing on the i-th group of anomaly prediction results to obtain the i-th group of pseudo-labels corresponding to the n sample video segments;

[0143] The training module 840 is used to train the i-th analysis model based on the difference between the i-th group of anomaly prediction results and the i-th group of pseudo-labels, to obtain the (i+1)-th analysis model in the i-th iteration training process, until the training of the analysis model meets the training result conditions, and an anomaly analysis model is obtained. The anomaly analysis model is used to identify abnormal videos.

[0144] In an optional embodiment, such as Figure 9 As shown, the analysis module 820 also includes:

[0145] Extraction unit 821 is used to extract the i-th video feature of the sample video segment during the i-th iteration training process, wherein the video feature is a feature used to analyze the sample video segment;

[0146] Analysis unit 822 is used to analyze the i-th video feature through the i-th analysis model obtained by training in the (i-1)th iteration, and obtain the i-th set of anomaly prediction results;

[0147] The analysis module 820 is further configured to determine the i-th scene clustering result corresponding to the sample video segment based on the i-th video feature, wherein the scene clustering result is the result obtained by performing clustering analysis on the sample video segment; and to analyze the i-th scene clustering result and the i-th video feature through the i-th analysis model obtained by training in the (i-1)-th iteration to obtain the i-th group of anomaly prediction results.

[0148] The analysis module 820 is further configured to perform cluster analysis on the i-th video feature using mean clustering to obtain the i-th cluster feature and the i-th cluster percentage of the sample video segment; the i-th cluster feature and the i-th cluster percentage are used to determine the i-th scene clustering result corresponding to the sample video segment;

[0149] The analysis module 820 is further configured to analyze the i-th scene clustering result and the i-th video feature based on the attention mechanism using an attention model to obtain the i-th attention vector, wherein the attention model is a model used to determine the attention vector; and to analyze the i-th attention vector and the i-th video feature using the i-th analysis model obtained through the (i-1)-th iteration training to obtain the i-th group of anomaly prediction results.

[0150] The balancing module 830 is further configured to perform balanced distribution processing on the i-th group of anomaly prediction results using the i-th attention vector and the i-th video features, and update the i-th group of pseudo-labels corresponding to the n sample video segments.

[0151] In an optional embodiment, the extraction unit 821 is further configured to update the i-th feature extraction network based on the difference between the i-th group of anomaly prediction results and the i-th group of pseudo-labels, to obtain the (i+1)-th feature extraction network in the i-th iteration training process.

[0152] The analysis module 820 is further configured to, at the beginning of the iteration loop, analyze the sample video segments using an isolated forest to obtain outlier results for the sample video segments; based on the outlier results, obtain initial pseudo-labels for the sample video segments; analyze the sample video segments using the initial analysis model at the beginning of the iteration loop to obtain a first set of anomaly prediction results; and train the initial analysis model based on the difference between the first set of anomaly prediction results and the initial pseudo-labels to obtain the first analysis model in the first iteration loop training process.

[0153] The acquisition module 810 is further configured to acquire a sample video stream, the sample video stream containing video frames; extract sample video frames from the sample video stream, the sample video frames being video frames used to train the analysis model; and acquire n sample video segments based on the connection relationships between the sample video frames.

[0154] The training module 840 is further configured to analyze the i-th group of anomaly prediction results and the i-th group of pseudo-labels using a loss function to obtain a loss result; and to train the i-th analysis model using the loss result.

[0155] In summary, the embodiments of this application provide an abnormal video identification device. During implementation, the device analyzes sample video segments using an analysis model to obtain anomaly prediction results corresponding to the sample video segments. The anomaly prediction results are then processed to achieve a balanced distribution, resulting in pseudo-labels for the sample video segments. Based on the difference between the pseudo-labels and the anomaly prediction results, the analysis model is iteratively trained to ultimately obtain an anomaly analysis model for identifying abnormal videos. This not only saves the manpower required for manual data annotation of video segments but also improves the identification accuracy of the anomaly analysis model.

[0156] It should be noted that the abnormal video recognition device provided in the above embodiments is only an example of the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the abnormal video recognition device provided in the above embodiments belongs to the same concept as the abnormal video recognition method embodiments, and its specific implementation process can be found in the method embodiments, which will not be repeated here.

[0157] Figure 10 This illustration shows a structural block diagram of a computer device 1000 provided in an exemplary embodiment of this application. The computer device 1000 may be a portable mobile terminal, such as a smartphone, tablet computer, MP3 player (Moving Picture Experts Group Audio Layer III), MP4 player (Moving Picture Experts Group Audio Layer IV), laptop computer, or desktop computer. The computer device 1000 may also be referred to as a user device, portable terminal, laptop terminal, desktop terminal, or other names.

[0158] Typically, computer device 1000 includes a processor 1001 and a memory 1002.

[0159] Processor 1001 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 1001 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Processor 1001 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 1001 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content required to be displayed on the screen. In some embodiments, processor 1001 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0160] The memory 1002 may include one or more computer-readable storage media, which may be non-transitory. The memory 1002 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 1002 is used to store at least one instruction, which is executed by the processor 1001 to implement the video material image selection method provided in the method embodiments of this application.

[0161] In some embodiments, the computer device 1000 may optionally include a peripheral device interface 1003 and at least one peripheral device. The processor 1001, memory 1002, and peripheral device interface 1003 can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface 1003 via a bus, signal line, or circuit board. Specifically, the peripheral device includes at least one of the following: a radio frequency circuit 1004, a display screen 1005, a camera assembly 1006, an audio circuit 1007, a positioning assembly 1008, and a power supply 1009.

[0162] Peripheral device interface 1003 can be used to connect at least one I / O (Input / Output) related peripheral device to processor 1001 and memory 1002. In some embodiments, processor 1001, memory 1002 and peripheral device interface 1003 are integrated on the same chip or circuit board; in some other embodiments, any one or two of processor 1001, memory 1002 and peripheral device interface 1003 can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0163] The radio frequency (RF) circuit 1004 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The RF circuit 1004 communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 1004 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals back into electrical signals. Optionally, the RF circuit 1004 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, etc. The RF circuit 1004 can communicate with other terminals via at least one wireless communication protocol. This wireless communication protocol includes, but is not limited to: the World Wide Web, metropolitan area networks, intranets, various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and / or WiFi (Wireless Fidelity) networks. In some embodiments, the RF circuit 1004 may also include circuitry related to NFC (Near Field Communication), which is not limited in this application.

[0164] Display screen 1005 is used to display a UI (User Interface). This UI may include graphics, text, icons, videos, and any combination thereof. When display screen 1005 is a touch display screen, it also has the ability to collect touch signals on or above its surface. These touch signals can be input as control signals to processor 1001 for processing. In this case, display screen 1005 can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments, display screen 1005 may be a single screen, disposed on the front panel of computer device 1000; in other embodiments, display screen 1005 may be at least two screens, disposed on different surfaces of computer device 1000 or in a folded design; in still other embodiments, display screen 1005 may be a flexible display screen, disposed on a curved or folded surface of computer device 1000. Furthermore, display screen 1005 may also be configured as a non-rectangular, irregular shape, i.e., a non-rectangular screen. The display screen 1005 can be made of materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0165] The camera assembly 1006 is used to acquire images or videos. Optionally, the camera assembly 1006 includes a front-facing camera and a rear-facing camera. Typically, the front-facing camera is located on the front panel of the terminal, and the rear-facing camera is located on the back of the terminal. In some embodiments, there are at least two rear-facing cameras, which are any one of a main camera, a depth-sensing camera, a wide-angle camera, and a telephoto camera, to achieve background blurring by fusion of the main camera and the depth-sensing camera, panoramic shooting by fusion of the main camera and the wide-angle camera, VR (Virtual Reality) shooting, or other fusion shooting functions. In some embodiments, the camera assembly 1006 may also include a flash. The flash can be a single-color temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm-light flash and a cool-light flash, which can be used for light compensation at different color temperatures.

[0166] The audio circuit 1007 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, converting the sound waves into electrical signals that are input to the processor 1001 for processing, or input to the radio frequency circuit 1004 for voice communication. For stereo sound acquisition or noise reduction purposes, multiple microphones may be used, each located in a different part of the computer device 1000. The microphone may also be an array microphone or an omnidirectional microphone. The speaker is used to convert electrical signals from the processor 1001 or the radio frequency circuit 1004 into sound waves. The speaker may be a conventional diaphragm speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can convert electrical signals not only into audible sound waves but also into inaudible sound waves for purposes such as distance measurement. In some embodiments, the audio circuit 1007 may also include a headphone jack.

[0167] The positioning component 1008 is used to locate the current geographical location of the computer device 1000 in order to enable navigation or LBS (Location Based Service). The positioning component 1008 can be a positioning component based on GPS (Global Positioning System), BeiDou system, or Galileo system.

[0168] Power supply 1009 is used to supply power to the various components in computer device 1000. Power supply 1009 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery. When power supply 1009 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged via a wired line, and a wireless rechargeable battery is a battery that is charged via a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0169] In some embodiments, the computer device 1000 further includes one or more sensors 1010. The one or more sensors 1010 include, but are not limited to: an accelerometer 1011, a gyroscope 1012, a pressure sensor 1013, a fingerprint sensor 1014, an optical sensor 1015, and a proximity sensor 1016.

[0170] Accelerometer 1011 can detect the magnitude of acceleration along the three coordinate axes of a coordinate system established by computer device 1000. For example, accelerometer 1011 can be used to detect the components of gravitational acceleration along the three coordinate axes. Processor 1001 can control display screen 1005 to display the user interface in either a landscape or portrait view based on the gravitational acceleration signal acquired by accelerometer 1011. Accelerometer 1011 can also be used for games or for acquiring user motion data.

[0171] The gyroscope sensor 1012 can detect the orientation and rotation angle of the computer device 1000. The gyroscope sensor 1012 can work in conjunction with the accelerometer sensor 1011 to acquire 3D motion data from the user on the computer device 1000. Based on the data acquired by the gyroscope sensor 1012, the processor 1001 can perform the following functions: motion sensing (e.g., changing the UI based on the user's tilt), image stabilization during shooting, game control, and inertial navigation.

[0172] The pressure sensor 1013 can be disposed on the side bezel of the computer device 1000 and / or on the lower layer of the display screen 1005. When the pressure sensor 1013 is disposed on the side bezel of the computer device 1000, it can detect the user's grip signal on the computer device 1000, and the processor 1001 can perform left / right hand recognition or quick operation based on the grip signal collected by the pressure sensor 1013. When the pressure sensor 1013 is disposed on the lower layer of the display screen 1005, the processor 1001 can control the operable controls on the UI interface based on the user's pressure operation on the display screen 1005. The operable controls include at least one of button controls, scroll bar controls, icon controls, and menu controls.

[0173] The fingerprint sensor 1014 is used to collect a user's fingerprint. The processor 1001 identifies the user based on the fingerprint collected by the fingerprint sensor 1014, or vice versa. When the user's identity is identified as trusted, the processor 1001 authorizes the user to perform relevant sensitive operations, including unlocking the screen, viewing encrypted information, downloading software, making payments, and changing settings. The fingerprint sensor 1014 can be located on the front, back, or side of the computer device 1000. When the computer device 1000 has physical buttons or a manufacturer's logo, the fingerprint sensor 1014 can be integrated with the physical buttons or the manufacturer's logo.

[0174] An optical sensor 1015 is used to collect ambient light intensity. In one embodiment, the processor 1001 can control the display brightness of the display screen 1005 based on the ambient light intensity collected by the optical sensor 1015. Specifically, when the ambient light intensity is high, the display brightness of the display screen 1005 is increased; when the ambient light intensity is low, the display brightness of the display screen 1005 is decreased. In another embodiment, the processor 1001 can also dynamically adjust the shooting parameters of the camera assembly 1006 based on the ambient light intensity collected by the optical sensor 1015.

[0175] The proximity sensor 1016, also known as a distance sensor, is typically installed on the front panel of the computer device 1000. The proximity sensor 1016 is used to detect the distance between the user and the front of the computer device 1000. In one embodiment, when the proximity sensor 1016 detects that the distance between the user and the front of the computer device 1000 is gradually decreasing, the processor 1001 controls the display screen 1005 to switch from a screen-on state to a screen-off state; when the proximity sensor 1016 detects that the distance between the user and the front of the computer device 1000 is gradually increasing, the processor 1001 controls the display screen 1005 to switch from a screen-off state to a screen-on state.

[0176] Those skilled in the art will understand that Figure 10 The structure shown does not constitute a limitation on the computer device 1000, and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0177] It should be noted that the abnormal video recognition device provided in the above embodiments is only an example of the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the virtual scene object drawing device and the virtual scene object drawing method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0178] Embodiments of this application also provide a computer device, which includes a processor and a memory. The memory stores at least one instruction, at least one program, code set, or instruction set. The processor loads and executes the at least one instruction, at least one program, code set, or instruction set to implement the abnormal video identification method provided in the above-described method embodiments.

[0179] Embodiments of this application also provide a computer-readable storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, at least one program, code set, or instruction set is loaded and executed by a processor to implement the abnormal video identification method provided in the above-described method embodiments.

[0180] Embodiments of this application also provide a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform any of the abnormal video identification methods described in the above embodiments.

[0181] Optionally, the computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), solid-state drives (SSDs), or optical discs, etc. The random access memory may include resistive random access memory (ReRAM) and dynamic random access memory (DRAM). The sequence numbers of the embodiments in this application are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0182] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0183] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for identifying abnormal videos, characterized in that, The method includes: Obtain n sample video segments, wherein the sample video segments are segments composed of video frames used to train the anomaly analysis model, and n≥2 and n is an integer; During the i-th iteration of training, the i-th video feature of the sample video segment is extracted. The video feature is used to analyze the sample video segment, where i ≥ 1 and i is an integer. Based on the k clustering features corresponding to the i-th video feature, the i-th scene clustering result corresponding to the sample video segment is determined. The scene clustering result is the result obtained by performing cluster analysis on the sample video segment, where k is the number of clusters. The i-th analysis model, obtained through the (i-1)th iteration training, is used to analyze the i-th scene clustering result and the i-th video feature to obtain the i-th group of anomaly prediction results; The i-th group of anomaly prediction results are processed to achieve a balanced distribution, resulting in the i-th group of pseudo-labels corresponding to the n sample video segments. Based on the difference between the i-th anomaly prediction results and the i-th pseudo-labels, the i-th analysis model is trained to obtain the (i+1)-th analysis model in the i-th iteration training process, until the training of the analysis model meets the training result conditions, thus obtaining the anomaly analysis model, which is used to identify abnormal videos.

2. The method according to claim 1, characterized in that, The step of determining the i-th scene clustering result corresponding to the sample video segment based on the i-th video feature includes: Cluster analysis is performed on the i-th video feature using mean clustering to obtain the i-th cluster feature and the proportion of the i-th cluster in the sample video segment; The i-th clustering feature and the i-th cluster percentage are used to determine the i-th scene clustering result corresponding to the sample video segment.

3. The method according to claim 1, characterized in that, The analysis model, trained through the (i-1)th iteration, analyzes the i-th scene clustering result and the i-th video feature to obtain the i-th set of anomaly prediction results, including: The attention model is used to analyze the clustering result of the i-th scene and the i-th video feature based on the attention mechanism to obtain the i-th attention vector. The attention model is a model used to determine the attention vector. The i-th attention vector and the i-th video feature are analyzed by the i-th analysis model obtained through the (i-1)-th iteration training to obtain the i-th set of anomaly prediction results.

4. The method according to claim 3, characterized in that, The step of performing balanced distribution processing on the i-th group of anomaly prediction results to obtain the i-th group of pseudo-labels corresponding to the n sample video segments includes: The i-th anomaly prediction results are balanced and distributed using the i-th attention vector and the i-th video features, and the i-th pseudo-labels corresponding to the n sample video segments are updated.

5. The method according to any one of claims 1 to 4, characterized in that, During the i-th iteration of training, extracting the i-th video feature of the sample video segment includes: During the i-th iteration training process, the i-th video feature is extracted by the i-th feature extraction network obtained from the (i-1)-th iteration training; The method further includes: Based on the difference between the i-th group of anomaly prediction results and the i-th group of pseudo-labels, the i-th feature extraction network is updated to obtain the (i+1)-th feature extraction network in the i-th iteration training process.

6. The method according to any one of claims 1 to 4, characterized in that, The step of analyzing the sample video segment using the analysis model obtained from the (i-1)th iteration training process to obtain the i-th set of anomaly prediction results further includes: At the beginning of the iterative loop, the sample video segments are analyzed using an isolated forest to obtain the outlier results of the sample video segments; Based on the outlier results, the initial pseudo-labels of the sample video segments are obtained; The sample video segments are analyzed using the initial analysis model at the beginning of the iterative loop to obtain the first set of anomaly prediction results; Based on the difference between the first set of anomaly prediction results and the initial pseudo-label, the initial analysis model is trained to obtain the first analysis model in the first iteration training process.

7. The method according to any one of claims 1 to 4, characterized in that, The acquisition of n sample video segments includes: Acquire a sample video stream, the sample video stream containing video frames; Sample video frames are extracted from the sample video stream, and the sample video frames are used to train the analysis model. Based on the connection relationship between the sample video frames, n sample video segments are obtained.

8. The method according to any one of claims 1 to 4, characterized in that, The step of training the i-th analysis model based on the difference between the i-th group of anomaly prediction results and the i-th group of pseudo-labels includes: The loss result is obtained by analyzing the i-th group of anomaly prediction results and the i-th group of pseudo-labels using a loss function; The i-th analysis model is trained using the loss result.

9. An abnormal video identification device, characterized in that, The device includes: The acquisition module is used to acquire n sample video segments, wherein the sample video segments are segments composed of video frames used to train the anomaly analysis model, and n≥2 and n is an integer; An extraction unit is used to extract the i-th video feature of the sample video segment during the i-th iteration of training. The video feature is a feature used to analyze the sample video segment, where i ≥ 1 and i is an integer. The analysis module is used to determine the i-th scene clustering result corresponding to the sample video segment based on the k clustering features corresponding to the i-th video feature. The scene clustering result is the result obtained by performing cluster analysis on the sample video segment, where k is the number of clusters. The analysis module is also used to analyze the i-th scene clustering result and the i-th video feature through the i-th analysis model obtained by the (i-1)th iteration training, and obtain the i-th group of anomaly prediction results; The balancing module is used to perform balanced distribution processing on the i-th group of anomaly prediction results to obtain the i-th group of pseudo-labels corresponding to the n sample video segments; The training module is used to train the i-th analysis model based on the difference between the i-th group of anomaly prediction results and the i-th group of pseudo-labels, to obtain the (i+1)-th analysis model in the i-th iteration training process, until the training of the analysis model meets the training result conditions, and an anomaly analysis model is obtained. The anomaly analysis model is used to identify abnormal videos.

10. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing at least one instruction, at least one program, code set, or instruction set, the at least one instruction, the at least one program, the code set, or the instruction set being loaded and executed by the processor to implement the abnormal video identification method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, The storage medium stores at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, the at least one program, the code set, or instruction set is loaded and executed by a processor to implement the abnormal video identification method as described in any one of claims 1 to 8.

Images