Training data acquisition method and abnormal behavior recognition network training method

Abstract

This specification provides a training data acquisition method and an abnormal behavior recognition network training method. For specific abnormal behaviors, the collected data containing the specific abnormal behaviors are acquired, and the network data is acquired, and the action features of the collected data and the network data are acquired. According to the similarity between the action features of each network data and the action features of the collected data, several similar network data of the collected data are determined, and the similar network data and the collected data are used as positive sample training data of the specific abnormal behavior. Through the similarity of the action features, the training data that can be used as positive samples are determined from several cheap network data, which improves the efficiency of training data acquisition and further speeds up the acquisition of the abnormal behavior recognition network.

Description

Technical Field

[0001] The present invention relates to the field of computer application technology, specifically to the field of machine learning technology, and in particular to a method for acquiring training data and a method for training an abnormal behavior recognition network. Background Art

[0002] In the scenario of smart city management, it is often necessary to automatically identify abnormal behaviors (such as fighting, climbing, etc.) through video data to prevent abnormal behaviors from endangering the safety and harmony of the city.

[0003] Automatically identifying abnormal behaviors often requires training the abnormal behavior recognition network with labeled abnormal behavior data. However, data collection and labeling often take a long time, which makes it impossible to quickly train the abnormal behavior recognition network in a short period of time. Summary of the invention

[0004] This specification provides a training data acquisition method and an abnormal behavior recognition network training method.

[0005] According to a first aspect of an embodiment of this specification, a training data acquisition method is provided, including:

[0006] Obtain network data and collected data containing specific abnormal behaviors;

[0007] Acquire action features of each of the network data and each of the collected data;

[0008] According to the similarity between the action features of each of the network data and the action features of the collected data, similar network data of the collected data is selected from the network data, and the collected data and the similar network data are used as positive sample training data for specific abnormal behavior.

[0009] In some embodiments, the network data includes: data sets publicly available on the Internet, and / or web crawler data, and / or data generated based on a virtual game engine.

[0010] Such network data is more comprehensive, making the training data more diverse.

[0011] In some implementations, the obtaining of the action characteristics of each of the network data and each of the collected data includes:

[0012] Get the backbone network;

[0013] Extracting action features of each of the collected data through the backbone network;

[0014] Obtain pre-stored action features of each of the network data extracted through the backbone network.

[0015] Through the backbone network, the action features of the collected data can be obtained, and the action features of the network data can be extracted in advance. When faced with the requirements of multiple abnormal behavior training, there is no need to obtain the action features of the network data multiple times, which improves the training efficiency.

[0016] In some implementations, selecting similar network data to the collected data according to the similarity between the action feature of each of the network data and the action feature of the collected data includes:

[0017] The action features of all acquired data are synthesized into the data collection center features;

[0018] According to the similarity between the action feature of each of the network data and the feature of the collection data center, similar network data of the collection data is selected.

[0019] In this way, the action features of all collected data are synthesized into central features, which can better reflect the characteristics of the abnormal behavior, and the selected similar network data is more accurate.

[0020] In some implementations, selecting similar network data to the collected data according to the similarity between the action feature of each of the network data and the action feature of the collected data includes:

[0021] Determine the number N of similar network data that needs to be collected according to the preset number ratio and the number of collected data;

[0022] N similar network data are selected from the network data; wherein the similarity between the action characteristics of any of the similar network data and the action characteristics of the collected data is not less than the similarity between the action characteristics of any of the unselected network data and the action characteristics of the collected data.

[0023] In this way, the ratio of the selected similar network data and the collected data meets certain conditions, so that the abnormal behavior recognition network will not be biased by the similar network data, and has a better effect on the collected data.

[0024] According to a second aspect of an embodiment of this specification, a method for training an abnormal behavior recognition network is provided, the method comprising:

[0025] Acquire training data, wherein the training data includes positive sample training data and negative sample training data, and the positive sample training data is acquired based on the above-mentioned training data acquisition method;

[0026] The abnormal behavior recognition network is iteratively trained using the positive sample training data and the negative sample training data until the loss of the network output is less than a preset first loss threshold, or the number of iterations is greater than a preset number threshold.

[0027] In some embodiments, the positive sample training data includes a first label, and the negative sample training data includes a second label;

[0028] Each iteration of training the abnormal behavior recognition network includes:

[0029] Obtaining a determination result for each of the training data; the determination result is used to indicate whether the training data includes the specific abnormal behavior;

[0030] For each similar network data, if the determination result of the similar network data does not match the label, the similar network data is discarded;

[0031] Obtaining the loss of the network output according to the determination results and label values ​​of the training data that has not been discarded;

[0032] The weight of the abnormal behavior recognition network is updated according to the loss output by the network.

[0033] In this way, similar network data that are not similar to the collected data are discarded, preventing the abnormal behavior recognition network from being biased by these data.

[0034] In some implementations, in each iteration, before obtaining the determination result of each training data, the method further includes:

[0035] According to the backbone network, obtaining action features of each of the training data; the action features are used to determine whether the training data includes the specific abnormal behavior;

[0036] In each iteration, after obtaining the loss of the network output according to the determination result and label of the training data that has not been discarded, the method further includes:

[0037] According to the loss, the weight of the backbone network is updated.

[0038] In this way, by updating the backbone network, the backbone network can better extract the characteristics of abnormal behavior, thereby improving the accuracy of the abnormal behavior recognition network.

[0039] In some implementations, in each iteration, before discarding each similar network data if the determination result of the similar network data does not match the label, the method further includes:

[0040] Input each of the training data into a discriminator; the discriminator is used to determine whether the training data is collected data;

[0041] For each similar network data, if the determination result of the network data does not match the label, the network data is discarded, including:

[0042] For each training data, in response to the discriminator outputting that the training data is not collected data, and the determination result of the training data does not match the label of the training data, the training data is discarded.

[0043] In this way, the input training data does not need to contain features that identify whether it is collected data, reducing the complexity of the model.

[0044] According to a third aspect of an embodiment of this specification, there is provided a training data acquisition device, including:

[0045] A data acquisition module is used to acquire network data and collected data including specific abnormal behaviors;

[0046] An action feature acquisition module, used to acquire the action features of each of the network data and each of the collected data;

[0047] The training data selection module is used to select similar network data of the collected data from the network data according to the similarity between the action features of each network data and the action features of the collected data, and use the collected data and the similar network data as positive sample training data for specific abnormal behavior.

[0048] In some embodiments, the network data includes: data sets publicly available on the Internet, and / or web crawler data, and / or data generated based on a virtual game engine.

[0049] Such network data is more comprehensive, making the training data more diverse.

[0050] In some embodiments, the data acquisition module is specifically used to: acquire a backbone network; extract action features of each of the collected data through the backbone network; and acquire pre-stored action features of each of the network data extracted through the backbone network.

[0051] Through the backbone network, the action features of the collected data can be obtained, and the action features of the network data can be extracted in advance. When faced with the requirements of multiple abnormal behavior training, there is no need to obtain the action features of the network data multiple times, which improves the training efficiency.

[0052] In some embodiments, in the training data selection module, similar network data of the collected data is selected based on the similarity between the action features of each of the network data and the action features of the collected data, including: synthesizing the acquired action features of all the collected data into a collection data center feature; selecting similar network data of the collected data based on the similarity between the action features of each of the network data and the collection data center feature.

[0053] In this way, the action features of all collected data are synthesized into central features, which can better reflect the characteristics of the abnormal behavior, and the selected similar network data is more accurate.

[0054] In some embodiments, in the training data selection module, similar network data of the collected data is selected according to the similarity between the action features of each of the network data and the action features of the collected data, including: determining the number N of similar network data that need to be collected according to a preset quantity ratio and the number of the collected data; selecting N similar network data from the network data; wherein the similarity between the action features of any of the similar network data and the action features of the collected data is not less than the similarity between the action features of any of the network data that have not been selected and the action features of the collected data.

[0055] In this way, the ratio of the selected similar network data and the collected data meets certain conditions, so that the abnormal behavior recognition network will not be biased by the similar network data, and has a better effect on the collected data.

[0056] According to a fourth aspect of the embodiments of this specification, there is provided an abnormal behavior recognition network training device, the device comprising:

[0057] A training data acquisition module, used to acquire training data, wherein the training data includes positive sample training data and negative sample training data, and the positive sample training data is acquired based on the above-mentioned training sample acquisition method;

[0058] The network training module is used to iteratively train the abnormal behavior recognition network through the positive sample training data and the negative sample training data until the loss of the network output is less than a preset first loss threshold, or the number of iterations is greater than a preset number threshold.

[0059] In some implementations, the positive sample training data includes a first label, and the negative sample training data includes a second label.

[0060] In the network training module, each iteration of training the abnormal behavior recognition network includes: obtaining the judgment result of each of the training data; the judgment result is used to characterize whether the training data includes the specific abnormal behavior; for each similar network data, if the judgment result of the similar network data does not match the label, the similar network data is discarded; according to the judgment result and label value of the training data that is not discarded, the loss of the network output is obtained; according to the loss of the network output, the weight of the abnormal behavior recognition network is updated.

[0061] In this way, similar network data that are not similar to the collected data are discarded, preventing the abnormal behavior recognition network from being biased by these data.

[0062] In some embodiments, in each iteration of the network training module, before obtaining the judgment result of each of the training data, it also includes: obtaining the action feature of each of the training data according to the backbone network; the action feature is used to determine whether the training data includes the specific abnormal behavior; in each iteration, after obtaining the loss of the network output according to the judgment result and label of the training data that has not been discarded, the method also includes: updating the weight of the backbone network according to the loss.

[0063] In this way, by updating the backbone network, the backbone network can better extract the characteristics of abnormal behavior, thereby improving the accuracy of the abnormal behavior recognition network.

[0064] In some embodiments, in each iteration of the network training module, before discarding each similar network data if the determination result of the similar network data does not match the label, the network training module further includes: inputting each training data into a discriminator; the discriminator is used to determine whether the training data is collected data.

[0065] For each similar network data, if the determination result of the network data does not match the label, the network data is discarded, including: for each training data, in response to the discriminator outputting that the training data is not collected data, and the determination result of the training data does not match the label of the training data, the training data is discarded.

[0066] In this way, the input training data does not need to contain features that identify whether it is collected data, reducing the complexity of the model.

[0067] According to a fifth aspect of the embodiments of this specification, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the above-mentioned training data acquisition method or abnormal behavior recognition network training method is implemented.

[0068] According to a sixth aspect of an embodiment of this specification, a computer device is provided, the computer device comprising:

[0069] one or more processors;

[0070] A memory for storing one or more programs;

[0071] When the one or more programs are executed by the one or more processors, the one or more processors implement the above-mentioned training data acquisition method or abnormal behavior recognition network training method.

[0072] This specification provides a training data acquisition method, an abnormal behavior recognition network training method, an apparatus, a device and a storage medium. For a specific abnormal behavior, the collected data containing the specific abnormal behavior is obtained, and the network data is obtained, and the action features of the collected data and the network data are obtained. According to the similarity between the action features of each network data and the action features of the collected data, several similar network data of the collected data are determined, and the similar network data and the collected data are used as positive sample training data of the specific abnormal behavior. Through the similarity of the action features, the training data that can be used as positive samples is determined from several cheap network data, which improves the efficiency of training data acquisition and further speeds up the acquisition of the abnormal behavior recognition network.

[0073] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present specification. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the specification and, together with the description, serve to explain the principles of the specification.

[0075] Figure 1 It is a flowchart of a method for obtaining training data according to an exemplary embodiment of the present specification.

[0076] Figure 2 It is a flowchart of a method for obtaining similar network data according to an exemplary embodiment of this specification.

[0077] Figure 3 is a schematic diagram of similar network data shown in this specification.

[0078] Figure 4 This is a flowchart of an abnormal behavior recognition network training method shown in this specification according to an exemplary embodiment.

[0079] Figure 5 The present specification is a flow chart showing an iterative method according to an exemplary embodiment.

[0080] Figure 6 It is a block diagram of a training data acquisition device shown in this specification according to an exemplary embodiment.

[0081] Figure 7 It is a block diagram of an abnormal behavior recognition network training device shown in this specification according to an exemplary embodiment.

[0082] Figure 8 It is a hardware structure diagram of a computer device where a training data acquisition device or an abnormal behavior recognition network training device is located according to an exemplary embodiment of this specification. DETAILED DESCRIPTION

[0083] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with this specification. Instead, they are merely examples of devices and methods consistent with some aspects of this specification as detailed in the appended claims.

[0084] The terms used in this specification are for the purpose of describing specific embodiments only and are not intended to limit this specification. The singular forms "a", "the" and "the" used in this specification and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0085] It should be understood that although the terms first, second, third, etc. may be used in this specification to describe various information, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this specification, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0086] In the management of smart cities, in order to maintain the safety and harmony of the city, it is necessary to identify abnormal behaviors in the city (such as fighting, climbing, etc.) through video data, etc. In order to identify abnormal behaviors, usually for each abnormal behavior, a large amount of labeled data is used to train an abnormal behavior recognition network, so that the trained abnormal behavior recognition network can identify the abnormal behavior.

[0087] However, there are some problems in this process. Training an abnormal behavior recognition network requires a large amount of labeled data, which usually takes a long time to obtain and requires a lot of manpower to label. As a result, when receiving a training task for an abnormal behavior recognition network, it is impossible to quickly obtain the abnormal behavior recognition network due to only a small amount of labeled data. If training is only based on a small amount of data, the accuracy of the abnormal behavior recognition network will be difficult to meet the accuracy requirements for use.

[0088] In order to solve the above problems, this specification provides a training data acquisition method, for specific abnormal behavior, to obtain the collected data including the specific abnormal behavior, and to obtain network data, and to obtain the action features of the collected data and the network data, and according to the similarity between the action features of each network data and the action features of the collected data, to determine a number of similar network data of the collected data, and to use the similar network data and the collected data as positive sample training data of the specific abnormal behavior. Through the similarity of the action features, the training data that can be used as positive samples is determined from a number of cheap network data, which improves the efficiency of training data acquisition and further speeds up the acquisition of the abnormal behavior recognition network.

[0089] Next, the embodiments of this specification are described in detail.

[0090] In a first aspect, the embodiments of this specification provide a method for obtaining training data, which is used to obtain positive sample training data of a specific abnormal behavior. The method can be applied to any system device suitable for implementation, such as a server, a smart city management system, or other processing device. Figure 1 As shown, Figure 1 This is a flowchart of a method for obtaining training data according to an exemplary embodiment of the present specification. Figure 1 The method of obtaining training data is explained in detail.

[0091] The training data acquisition method provided in this specification may include:

[0092] Step 101, obtaining network data and collected data including specific abnormal behaviors.

[0093] First of all, it should be noted that the training data acquisition method in this specification is for a specific abnormal behavior. If you want to obtain training data for multiple specific abnormal behaviors, you can execute this method multiple times. Unless otherwise specified below, the collected data refers to the collected data for a specific abnormal behavior.

[0094] The collected data may be video or dynamic image data captured by the smart city management system, and the collected data may be data containing specific abnormal behaviors. In addition, the collected data may be labeled data, including a first label for the specific abnormal behavior.

[0095] Network data includes: data sets disclosed on the Internet, and / or web crawler data, and / or data generated based on a virtual game engine. Data sets disclosed on the Internet refer to training data sets disclosed on the Internet. In other words, data sets for certain tasks that have been sorted out by other Internet users, such as K400 data sets, etc. Web crawler data refers to data sets searched and sorted out from the Internet. Compared with Internet public data sets, Internet public data sets refer to data sets for training that have been sorted out by other Internet users, while web crawler data is a data set obtained by crawling and sorting out multiple crawled video data. Data generated based on a virtual game engine refers to data generated by a virtual game engine, which can generate data that does not contain scenes. The generated data is more flexible, and more cheap data can be generated through the virtual game engine. In addition, the network data obtained in step 101 are all unlabeled data and cannot be used directly.

[0096] Since specific abnormal behaviors are generally performed by humans and require the participation of at least one person, each collected data or network data contains data of at least one person. Data in the collected data and network data that are not related to specific abnormal behaviors, such as large areas of background, can be cut out through portrait recognition to reduce the noise input into the abnormal behavior recognition network.

[0097] In addition, considering that the training of the abnormal behavior recognition network cannot only have positive samples (data containing specific abnormal behaviors), but also requires negative samples (data that do not contain specific abnormal behaviors), the method provided in this specification is only for the acquisition of positive sample training data. Negative sample training data also consists of two parts, one part is the data that does not contain specific abnormal behaviors and includes at least one person in the city, and the other part is the data obtained from the Internet that does not contain specific abnormal behaviors. The specific method for obtaining negative sample training data is explained below and will not be repeated here.

[0098] Since the method provided in this specification is used to obtain positive sample data, the collected data is positive sample data containing a label, then the first label represents that the data carrying the label is data containing specific abnormal behavior. Correspondingly, the negative sample has a second label.

[0099] Step 103, obtaining action features of each of the network data and each of the collected data.

[0100] Step 105 : selecting similar network data to the collected data according to the similarity between the action feature of each network data and the action feature of the collected data.

[0101] Next, step 103 and step 105 will be described uniformly. Specifically, the actions of different abnormal behaviors are different, and for the same abnormal behavior, the actions of different data are similar. Therefore, in order to obtain similar network data of the collected data, it is necessary to find similar network data based on action features, so firstly, it is necessary to obtain the action features of the collected data and the network data.

[0102] The network data may be collected in advance, and the corresponding action features may be extracted after the collection. The action features of the network data may be data that has been acquired and stored before the training task of training the abnormal behavior recognition network is acquired. The action features of the network data may be acquired by acquiring the action features of the pre-stored network data.

[0103] For the collected data, its motion features need to be extracted through the backbone network. The backbone network is a network trained according to a specific data set and used to extract motion features. The specific data set refers to the data set required for training the network to extract motion features. This data set generally uses the data set publicly available on the Internet, such as the K400 data set.

[0104] A collection data or a network data generally corresponds to an action feature, and the action feature is generally in the form of a vector. In other words, the input of the backbone network is a video data or a dynamic image data, and the output is a feature vector.

[0105] In the case where the action features of the collected data are extracted through the backbone network, the step of obtaining the action features of each of the network data and each of the collected data includes: obtaining the backbone network; extracting the action features of each of the collected data through the backbone network; and obtaining the pre-stored action features of each of the network data extracted through the backbone network.

[0106] Among them, the action features of the collected data for similarity calculation can be selected from any action features of the collected data, or the action features of several collected data can be synthesized into one action feature, and the synthesized action feature can be used as the action feature for similarity calculation. Of course, the user can also specify the most representative action feature of the collected data as the action feature for similarity calculation, and this specification does not limit it here. The specific method for obtaining similar network data will be described in detail below, and will not be repeated here.

[0107] Step 107: adding a first label to the similar network data, and using the collected data and the selected similar network data as positive sample training data for specific abnormal behavior.

[0108] Specifically, in order to enable similar network data to be used for training the abnormal behavior recognition network, it is necessary to add a first label to the similar network data so that supervised learning can be performed using the similar network data. Through the above method, positive sample training data for training the abnormal behavior recognition network for specific abnormal behaviors is obtained.

[0109] Next, we will combine Figure 2 , the method of obtaining similar network data is described in detail, such as Figure 2 As shown, Figure 2 This is a flowchart of a method for obtaining similar network data according to an embodiment of the present specification, comprising the following steps:

[0110] Step 201, obtaining a backbone network.

[0111] As mentioned above, the backbone network is trained by a public network data set, and the backbone network may be pre-trained or trained during the execution of step 201. The backbone network is used to extract motion features of video or dynamic image data.

[0112] Step 203: input each collected data into the backbone network to obtain the action features of each collected data.

[0113] The input of the backbone network is video or dynamic image data, and the output is a feature vector.

[0114] Step 205: Acquire the characteristics of pre-stored network data.

[0115] The features of network data are extracted through the pre-trained backbone network. Network data can be used to train abnormal behavior recognition networks for a variety of different abnormal behaviors. Therefore, when there is a need to train abnormal behavior recognition networks for a variety of different abnormal behaviors, pre-extracting the action features of network data eliminates the need to repeatedly extract the same action features of network data through the backbone network every time a training sample of a certain abnormal behavior is obtained, which can reduce the amount of calculation.

[0116] Step 207: synthesize the action features of all acquired data into a collection data center feature.

[0117] Specifically, the action features of all collected data are normalized and averaged to form a collection data center feature. The collection data center feature can integrate the action features of each collected data and accurately reflect the characteristics of specific abnormal behaviors.

[0118] Step 209 : selecting similar network data to the collected data according to the similarity between the action feature of each of the network data and the feature of the collected data center.

[0119] Specifically, for each network data, the feature similarity between its action feature and the feature of the data center is calculated. The feature similarity can be cosine similarity or Euclidean distance, or other values ​​used to represent the similarity between two vectors. This manual does not limit this.

[0120] The selected similar network data may be network data whose similarity to the collected data is greater than a preset threshold, or the network data may be sorted according to the similarity and the top N network data may be selected, where N may be a value preset by the user.

[0121] In addition, the number N of similar network data selected can also be obtained according to other methods, such as being determined according to the ratio of the number of preset similar network data and the number of collected data; in other words, the similar network data of the collected data is selected according to the similarity between the action characteristics of each of the network data and the action characteristics of the collected data, including: determining the number N of similar network data to be collected according to the preset ratio and the number of collected data; selecting N similar network data from the network data; wherein the similarity between the action characteristics of any similar network data and the action characteristics of the collected data is not less than the similarity between the action characteristics of any network data not selected and the action characteristics of the collected data. The preset ratio is the ratio of the number of similar network data to be collected and the collected data.

[0122] Furthermore, when there are multiple categories of network data (for example, different categories may correspond to data crawled from different video websites, and different categories may also correspond to different network data sets), the selected similar network data should be selected from various categories as much as possible. Specifically, after determining the number N of similar network data to be selected, you can first filter out network data with a similarity threshold greater than a preset similarity threshold based on a preset similarity threshold, and then uniformly sample from the filtered network data of each category to select similar network data.

[0123] Furthermore, considering that the trained abnormal behavior recognition network needs to have a higher recognition ability for the collected data, and does not care whether the abnormal behavior recognition network has a good recognition ability for similar network data, in order to enable the abnormal behavior recognition network to have a better recognition ability for the collected data, the number of similar network data in the training data needs to be no more than the number of collected data. If similar network data is selected according to a certain ratio, sampling can be performed at a ratio of 1:2 for similar network data: collected data to obtain positive sample training data.

[0124] The schematic diagram of the selected similar network data is as follows: Figure 3As shown. When there are multiple categories of network data, when retrieving similar network data, if each collected data is searched separately, then for each collected data, in each category of network data, the collected data that is most similar to the collected data can be retrieved. In addition, some actions may carry special scenes, such as climbing, which requires interaction between people and climbing objects. Since the extracted features are action features, network data that does not contain special scenes will generally be retrieved (for example Figure 3 Although these network data do not contain specific abnormal behaviors, for the abnormal behavior recognition network, this type of network data does not contain scenes, but can more clearly represent the action and reduce noise. In other words, although this type of network data does not contain specific abnormal behaviors, it can make it easier for the abnormal behavior recognition network to learn the characteristics of action features, which is beneficial for training. This type of network data should be retained.

[0125] After obtaining the positive sample training data, it is also necessary to obtain the negative sample training data. The specific method of obtaining the negative sample training data can be to obtain data that does not contain specific abnormal behaviors and is actually captured by the smart city management system and contains the second label. The data is obtained in the same way as the collected data. In addition, this data often contains human actions, so that the action features can be extracted. The second label is a label added to the data that does not contain specific abnormal behaviors. It is also possible to obtain several network data with the lowest similarity to the action features of the collected data from the network data as negative samples.

[0126] Secondly, after obtaining the positive sample training data and the negative sample training data, it is necessary to train the abnormal behavior recognition network through the positive sample training data and the negative sample training data. Therefore, this specification also provides an abnormal behavior recognition network training method. Figure 4 The following is a detailed description of an abnormal behavior recognition network training method provided in this specification.

[0127] Figure 4 This specification is an abnormal behavior recognition network training method according to an exemplary embodiment, the method is used to train an abnormal behavior recognition network for the specific abnormal behavior based on the positive sample training data acquired by the training data acquisition method of the first aspect of this specification; the method includes:

[0128] Step 401: Acquire training data, where the training data includes positive sample training data and negative sample training data.

[0129] The positive sample training data is obtained based on the above training data acquisition method, and the method for obtaining the negative sample data is as described above and will not be repeated here. In addition, in order to distinguish between the positive sample training data and the negative sample training data, and to perform supervised learning, the positive sample training data includes a first label, and the negative sample training data includes a second label.

[0130] Step 403, iteratively training the abnormal behavior recognition network using the positive sample training data and the negative sample training data until the loss output by the network is less than a preset first loss threshold, or the number of iterations is greater than a preset number threshold.

[0131] The first loss threshold and the number threshold can be selected by the user according to actual conditions. The smaller the first loss threshold and the larger the number threshold, the better the network training effect. However, at the same time, the amount of calculation will increase. Therefore, it is necessary to consider both the accuracy requirements and the amount of calculation requirements of the network and select the appropriate first loss threshold and number threshold.

[0132] Next, step 403 will be described in detail. Figure 5 As shown, Figure 5 is a flowchart showing an iterative method according to an exemplary embodiment of the present specification, comprising:

[0133] Step 511, obtaining a determination result of each of the training data; the determination result is used to indicate whether the training data includes the specific abnormal behavior.

[0134] Step 513, for each similar network data, if the determination result of the similar network data does not match the label, the similar network data is discarded;

[0135] Step 515, obtaining the loss of the network output according to the determination result and label of the training data that is not discarded.

[0136] Step 517: Update the weight of the abnormal behavior recognition network according to the loss output by the network.

[0137] Next, steps 511 to 517 will be described in a unified manner.

[0138] In step 511, the determination result of each training data is obtained by inputting the action features of each training data. The input action features may be the action features used when selecting the training data, or may be obtained again according to the updated backbone network in each iteration. The backbone network may be updated according to the output loss, so that the vector output by the backbone network can better characterize the specific abnormal behavior, so that the fully connected layer can better distinguish the training data containing the specific abnormal behavior from the training data not containing the specific abnormal behavior, thereby making the training effect better.

[0139] In other words, in each iteration, before inputting the features of each training data into the fully connected layer network, the method further includes: obtaining the action features of each training data according to the backbone network; the action features are used to determine whether the training data includes the specific abnormal behavior; in each iteration, after obtaining the loss of the network output based on the determination results and label values ​​of the training data that have not been discarded, the method further includes: updating the weights of the backbone network according to the loss.

[0140] Among them, in step 511, the judgment result can be obtained through the fully connected layer network. The classifier formed by the fully connected layer network classifies the input action feature vector to obtain the output judgment result. What needs to be trained in this specification is a binary classifier for obtaining the judgment result. Through cheap data, a binary classifier with sufficient accuracy can be obtained. In this way, the binary classifier, that is, the abnormal behavior recognition network, is quickly obtained.

[0141] In step 513, similar network data whose judgment results do not match the labels are discarded to prevent the final trained binary classifier from being inaccurate due to these similar network data during the subsequent training. When it is necessary to update the weights of the backbone network, discarding these similar network data can also prevent these similar network data from biasing the representation space of the backbone network, further preventing the final binary classifier from being affected.

[0142] Among them, since the abnormal behavior recognition network is a binary classifier, the corresponding labels are only the first label and the second label. Any two numbers can be used to represent the judgment result, and any two numbers can be used to represent the label value. When the obtained judgment result and the corresponding label both indicate that a specific abnormal behavior is included, or both indicate that a specific abnormal behavior is not included, the two are consistent. In other cases, the judgment result and the label are inconsistent. For example, for the judgment result, 0 indicates that the training data is training data that includes a specific abnormal behavior, 1 indicates that the training data is training data that does not include a specific abnormal behavior, 0 indicates the first label, and 1 indicates the second label. If the judgment result is 0 and the label is 0, according to the judgment result and the meaning of the first label 0, it can be seen that the two are consistent. If the judgment result is 1 and the label is 0, it means that the two are inconsistent.

[0143] In addition, considering that the data to be discarded are similar network data and the collected data cannot be discarded, before discarding the training data, it is also necessary to determine whether the data is similar network data. The method for determining whether it is similar network data can be to judge whether each pre-stored training data is similar network data, or to train a discriminator based on generative adversarial, and use the discriminator to judge whether the training data is network data. In other words, in each iteration, before discarding the similar network data if the judgment result of the similar network data does not match the label for each similar network data, the method further includes: inputting each training data into the discriminator; the discriminator is used to judge whether the training data is collected data; for each similar network data, if the judgment result of the similar network data does not match the label, the similar network data is discarded, including: for each training data, in response to the discriminator outputting that the training data is not collected data, and the judgment result of the training data does not match the label of the training data, the training data is discarded.

[0144] In step 515, the loss of the abnormal behavior recognition network can be obtained according to the loss function, which is a function preset by the user, with the input being the judgment result and label of each training data, and the output being the loss of the network, which is used to evaluate the quality of the abnormal behavior recognition network. The loss function can select the loss function of the binary classifier.

[0145] Since the abnormal behavior recognition network in this specification pays more attention to the recognition results of the collected data, when calculating the loss, a weighted calculation can be performed to increase the weight of the collected data and reduce the weight of the similar network data. If the number of similar network data is less than the collected data when obtaining training data, since the obtained training data has been obtained according to a certain ratio, the contribution of the collected data to the loss is greater than the loss of the similar network data, and the weighted calculation method can be used to calculate the loss.

[0146] The weights updated in step 517 are the weights of the abnormal behavior recognition network. In other words, what is updated are the weights of the network (which can be a fully connected layer) used in step 511. The weight update can be based on the gradient descent method or other weight update methods, and this manual does not limit this.

[0147] In addition, in the above embodiment, only an abnormal behavior recognition network for a specific abnormal behavior is described. When multiple abnormal behavior recognition networks for different abnormal behaviors need to be trained at the same time, multiple networks can be trained at the same time, sharing the same backbone network. The weights of each abnormal behavior recognition network are updated according to the loss of each network, and the weights of the backbone network are updated according to the loss of all abnormal behavior recognition networks. In this way, the training efficiency is improved.

[0148] The above description of the various embodiments tends to emphasize the differences between the various embodiments. The same or similar points can be referenced to each other, and for the sake of brevity, this specification will not go into details.

[0149] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

[0150] Corresponding to the embodiments of the aforementioned training data acquisition method and abnormal behavior recognition network training method, this specification also provides embodiments of a training data acquisition device, an abnormal behavior recognition network training device, and terminals used therein.

[0151] like Figure 6 As shown, Figure 6 is a block diagram of a training data acquisition device according to an exemplary embodiment of the present specification, the device comprising:

[0152] The data acquisition module 610 is used to acquire network data and collected data including specific abnormal behaviors.

[0153] The action feature acquisition module 620 is used to acquire the action features of each of the network data and each of the collected data.

[0154] The training data selection module 630 is used to select similar network data of the collected data from the network data according to the similarity between the action features of each network data and the action features of the collected data, and use the collected data and the similar network data as positive sample training data for specific abnormal behavior.

[0155] In some embodiments, the network data includes: data sets publicly available on the Internet, and / or web crawler data, and / or data generated based on a virtual game engine. Such network data is more comprehensive, making the training data more diverse.

[0156] In some embodiments, the data acquisition module 610 is specifically used to: acquire the backbone network; extract the action features of each of the collected data through the backbone network; and acquire the pre-stored action features of each of the network data extracted through the backbone network. The action features of the collected data can be acquired through the backbone network, and the action features of the network data are pre-extracted. When faced with the requirements of multiple abnormal behavior training, there is no need to acquire the action features of the network data multiple times, thereby improving the training efficiency.

[0157] In some embodiments, in the training data selection module 630, based on the similarity between the action feature of each of the network data and the action feature of the collected data, similar network data of the collected data are selected, including: synthesizing the action features of all the collected data into a collection data center feature; based on the similarity between the action feature of each of the network data and the collection data center feature, similar network data of the collected data are selected. In this way, the action features of all the collected data are synthesized into a central feature, the central feature can better reflect the characteristics of the abnormal behavior, and the selected similar network data is more accurate.

[0158] In some embodiments, in the training data selection module 630, similar network data of the collected data is selected according to the similarity between the action features of each of the network data and the action features of the collected data, including: determining the number N of similar network data to be collected according to a preset number ratio and the number of the collected data; selecting N similar network data from the network data; wherein the similarity between the action features of any of the similar network data and the action features of the collected data is not less than the similarity between the action features of any of the network data that are not selected and the action features of the collected data. In this way, the ratio of the selected similar network data to the collected data meets certain conditions, so that the abnormal behavior recognition network will not be biased by the similar network data, and has a better effect on the collected data.

[0159] like Figure 7 As shown, Figure 7 is a block diagram of an abnormal behavior recognition network training device according to an exemplary embodiment of the present specification, the device comprising:

[0160] The training data acquisition module 710 is used to acquire training data, where the training data includes positive sample training data and negative sample training data. The positive sample training data is acquired based on the above-mentioned training sample acquisition method.

[0161] The network training module 720 is used to iteratively train the abnormal behavior recognition network through the positive sample training data and the negative sample training data until the loss output by the network is less than a preset first loss threshold, or the number of iterations is greater than a preset number threshold.

[0162] In some embodiments, the positive sample training data includes a first label, and the negative sample training data includes a second label. In the network training module 720, each iteration of training the abnormal behavior recognition network includes: obtaining the judgment result of each of the training data; the judgment result is used to characterize whether the training data includes the specific abnormal behavior; for each similar network data, if the judgment result of the similar network data does not match the label, the similar network data is discarded; according to the judgment result and label value of the training data that has not been discarded, the loss of the network output is obtained; according to the loss of the network output, the weight of the abnormal behavior recognition network is updated. In this way, similar network data that is not similar to the collected data is discarded, avoiding the abnormal behavior recognition network from being biased by these data.

[0163] In some embodiments, in each iteration of the network training module 720, before obtaining the judgment result of each training data, it also includes: obtaining the action feature of each training data according to the backbone network; the action feature is used to determine whether the training data includes the specific abnormal behavior; in each iteration, after obtaining the loss of the network output according to the judgment result and label of the training data that has not been discarded, the method also includes: updating the weight of the backbone network according to the loss. In this way, by updating the backbone network, the backbone network can better extract the characteristics of abnormal behavior, thereby improving the accuracy of the abnormal behavior recognition network.

[0164] In some embodiments, in each iteration of the network training module 720, before discarding the similar network data if the determination result of the similar network data does not match the label for each similar network data, the network training module 720 further includes: inputting each of the training data into a discriminator; the discriminator is used to determine whether the training data is collected data. The method of discarding the network data if the determination result of the network data does not match the label for each similar network data includes: for each training data, in response to the discriminator outputting that the training data is not collected data, and the determination result of the training data does not match the label of the training data, the training data is discarded. In this way, the input training data does not need to contain features that identify whether it is collected data, thereby reducing the complexity of the model.

[0165] The implementation process of the functions and effects of each module in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, which will not be repeated here.

[0166] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this specification. A person of ordinary skill in the art can understand and implement it without paying creative labor.

[0167] like Figure 8 As shown, Figure 8 A hardware structure diagram of a computer device in which the above device is located is shown. The computer device may include only a training data acquisition device, or only an abnormal behavior recognition network training device, or may include a training data acquisition device and an abnormal behavior recognition network training device. The device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other in communication within the device through the bus 1050.

[0168] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0169] The memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 may store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 1020 and are called and executed by the processor 1010.

[0170] The input / output interface 1030 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0171] The communication interface 1040 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0172] The bus 1050 includes a path that transmits information between the various components of the device (eg, the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040).

[0173] It should be noted that, although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.

[0174] The embodiments of this specification also provide a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the above-mentioned training data acquisition method or abnormal behavior recognition network training method is implemented. The computer-readable storage medium may only store the computer program corresponding to the training data set acquisition method, or may only store the computer program corresponding to the abnormal behavior recognition network training method.

[0175] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0176] Those skilled in the art will readily appreciate other embodiments of the specification after considering the specification and practicing the invention claimed herein. The specification is intended to cover any variations, uses or adaptations of the specification that follow the general principles of the specification and include common knowledge or customary techniques in the art that are not claimed in the specification. The specification and examples are to be considered exemplary only, and the true scope and spirit of the specification are indicated by the following claims.

[0177] It should be understood that the present description is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present description is limited only by the appended claims.

[0178] The above description is only a preferred embodiment of this specification and is not intended to limit this specification. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of this specification should be included in the scope of protection of this specification.

Claims

1. A method for abnormal behavior recognition network training, characterized in that: The method comprises: Acquire training data, where the training data includes positive sample training data and negative sample training data; Iteratively training the abnormal behavior recognition network through the positive sample training data and the negative sample training data until the loss of the network output is less than a preset first loss threshold, or the number of iterations is greater than a preset number threshold; Wherein, the positive sample training data includes a first label, and the negative sample training data includes a second label; Each iteration of training the abnormal behavior recognition network includes: obtaining the judgment result of each training data; the judgment result is used to characterize whether the training data includes a specific abnormal behavior; for each similar network data, if the judgment result of the similar network data does not match the label, the similar network data is discarded; according to the judgment result and label value of the training data that is not discarded, the loss of the network output is obtained; according to the loss of the network output, the weight of the abnormal behavior recognition network is updated; The positive sample training data is obtained by at least the following method: Obtain network data and collected data containing specific abnormal behaviors; Based on the backbone network, obtaining action features of each of the network data and each of the collected data; According to the similarity between the action features of each of the network data and the action features of the collected data, similar network data of the collected data is selected from the network data, and the collected data and the similar network data are used as positive sample training data for the specific abnormal behavior.

2. The method according to claim 1, characterized in that: The network data includes: Data sets publicly available on the Internet, and / or web crawler data, and / or data generated based on a virtual game engine.

3. The method according to claim 1 or 2, characterized in that: The step of obtaining the action characteristics of each of the network data and each of the collected data includes: Get the backbone network; Extracting action features of each of the collected data through the backbone network; Obtain pre-stored action features of each of the network data extracted through the backbone network.

4. The method according to claim 1 or 2, characterized in that: The selecting similar network data to the collected data according to the similarity between the action feature of each of the network data and the action feature of the collected data comprises: The action features of all acquired data are synthesized into the data collection center features; According to the similarity between the action feature of each of the network data and the feature of the collection data center, similar network data of the collection data is selected.

5. The method according to claim 1 or 2, characterized in that: The selecting similar network data to the collected data according to the similarity between the action feature of each of the network data and the action feature of the collected data comprises: Determine the number N of similar network data that needs to be collected according to the preset number ratio and the number of collected data; N similar network data are selected from the network data; wherein the similarity between the action characteristics of any of the similar network data and the action characteristics of the collected data is not less than the similarity between the action characteristics of any of the unselected network data and the action characteristics of the collected data.

6. The method according to claim 1 or 2, characterized in that: In each iteration, before obtaining the determination result of each training data, the method further includes: According to the backbone network, obtaining action features of each of the training data; the action features are used to determine whether the training data includes the specific abnormal behavior; In each iteration, after obtaining the loss of the network output according to the determination result and label of the training data that has not been discarded, the method further includes: According to the loss, the weight of the backbone network is updated.

7. The method according to claim 1 or 2, characterized in that: In each iteration, before discarding each similar network data if the determination result of the similar network data does not match the label, the method further includes: Input each of the training data into a discriminator; the discriminator is used to determine whether the training data is collected data; For each similar network data, if the determination result of the network data does not match the label, the network data is discarded, including: For each training data, in response to the discriminator outputting that the training data is not collected data, and the determination result of the training data does not match the label of the training data, the training data is discarded.

8. An abnormal behavior recognition network training device, characterized in that: The device comprises: A training data acquisition module, used to acquire training data, wherein the training data includes positive sample training data and negative sample training data; A network training module, used for iteratively training the abnormal behavior recognition network through the positive sample training data and the negative sample training data, until the loss of the network output is less than a preset first loss threshold, or the number of iterations is greater than a preset number threshold; Wherein, the positive sample training data includes a first label, and the negative sample training data includes a second label; In the network training module, each iteration of training the abnormal behavior recognition network includes: obtaining the judgment result of each training data; the judgment result is used to characterize whether the training data includes a specific abnormal behavior; for each similar network data, if the judgment result of the similar network data does not match the label, the similar network data is discarded; according to the judgment result and label value of the training data that is not discarded, the loss of the network output is obtained; according to the loss of the network output, the weight of the abnormal behavior recognition network is updated; The positive sample training data is obtained by at least the following method: Obtain network data and collected data containing specific abnormal behaviors; Based on the backbone network, obtaining action features of each of the network data and each of the collected data; According to the similarity between the action features of each of the network data and the action features of the collected data, similar network data of the collected data is selected from the network data, and the collected data and the similar network data are used as positive sample training data for the specific abnormal behavior.

9. A computer-readable storage medium, characterized in that: A computer program is stored, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer device, characterized in that: The computer device comprises: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

Images