Anomaly detection model training method and device

Abstract

Embodiments of the present description provide an anomaly detection model training method and device. The anomaly detection model training method comprises: after obtaining first institution description data for model training under a preset institution category, on the basis of the preset institution category and at least one of the data type and description type of the first institution description data, selecting an anomaly detection unit in an anomaly detection unit set, and on the basis of the anomaly detection unit, constructing a model to be trained; and on the basis of the first institution description data, performing model training on said model to obtain an anomaly detection model, so as to perform anomaly detection processing on second institution description data under the preset institution category by means of the anomaly detection model.

Description

Anomaly detection model training method and device Technical Field

[0001] This article relates to the field of data processing technology, and in particular to a method and device for training an anomaly detection model. Background Art

[0002] With the continuous development of Internet technology, more and more institutions have the need to disclose institutional label data to users. For example, when an institution has the need to recruit employees, it discloses institutional label data to users so that users can understand the institution from all aspects, making it easier for users to choose the intended institution for further in-depth understanding.

[0003] Institutional label data represents the image of an institution and uses short label data to describe the attribute characteristics of an institution. Therefore, the importance of institutional label data is self-evident. In this process, the management and improvement of institutional label data puts higher requirements on the managers of institutional label data.

[0004] Summary of the Invention

[0005] One or more embodiments of this specification provide an anomaly detection model training method, comprising: obtaining first mechanism description data for model training under a preset mechanism classification. Based on the preset mechanism classification and at least one of the data type and description type of the first mechanism description data, selecting an anomaly detection unit from an anomaly detection unit set, and constructing a model to be trained based on the anomaly detection unit. Model training is performed on the model to be trained based on the first mechanism description data to obtain an anomaly detection model, so as to perform anomaly detection processing on the second mechanism description data under the preset mechanism classification using the anomaly detection model. Each anomaly detection unit in the anomaly detection unit set is extracted from a pre-trained detection model.

[0006] One or more embodiments of this specification provide an anomaly detection processing method, comprising: obtaining organization description data to be detected. Inputting the organization description data into an anomaly detection model that matches the organization classification of the organization description data and at least one of the data type and description type to perform anomaly detection processing, thereby obtaining an anomaly detection result for the organization description data. The anomaly detection model is obtained by selecting an anomaly detection unit from an anomaly detection unit set based on the organization classification and at least one of the data type and description type, constructing a model to be trained based on the anomaly detection unit, and training the model to be trained based on a sample of the organization description data.

[0007] One or more embodiments of the present specification provide an anomaly detection model training device, comprising: a data acquisition module, configured to acquire first mechanism description data for model training under a preset mechanism classification. A model construction module, configured to select an anomaly detection unit from an anomaly detection unit set based on the preset mechanism classification and at least one of the data type and description type of the first mechanism description data, and to construct a model to be trained based on the anomaly detection unit. A model training module, configured to perform model training on the model to be trained based on the first mechanism description data to obtain an anomaly detection model, so as to perform anomaly detection processing on the second mechanism description data under the preset mechanism classification through the anomaly detection model. Each anomaly detection unit in the anomaly detection unit set is extracted and obtained from a pre-trained detection model.

[0008] One or more embodiments of this specification provide an anomaly detection and processing device, comprising: a description data acquisition module configured to acquire organization description data to be detected. An anomaly processing module configured to input the organization description data into an anomaly detection model that matches the organization classification of the organization description data and at least one of the data type and description type, to perform anomaly detection processing and obtain an anomaly detection result for the organization description data. The anomaly detection model is obtained by selecting an anomaly detection unit from an anomaly detection unit set based on the organization classification and at least one of the data type and description type, constructing a model to be trained based on the anomaly detection unit, and training the model to be trained based on a sample of the organization description data.

[0009] One or more embodiments of the present specification provide an anomaly detection model training device, comprising: a processor; and a memory configured to store computer-executable instructions, wherein the computer-executable instructions, when executed, cause the processor to: obtain first mechanism description data for model training under a preset mechanism classification. According to the preset mechanism classification and at least one of the data type and description type of the first mechanism description data, an anomaly detection unit is selected from an anomaly detection unit set, and a model to be trained is constructed based on the anomaly detection unit. The model to be trained is trained based on the first mechanism description data to obtain an anomaly detection model, so as to perform anomaly detection processing on the second mechanism description data under the preset mechanism classification through the anomaly detection model. Each anomaly detection unit in the anomaly detection unit set is extracted and obtained from a pre-trained detection model.

[0010] One or more embodiments of the present specification provide an anomaly detection processing device, comprising: a processor; and a memory configured to store computer-executable instructions, wherein the computer-executable instructions, when executed, cause the processor to: obtain mechanism description data to be detected. Input the mechanism description data into an anomaly detection model that matches the mechanism classification of the mechanism description data and at least one of the data type and the description type, and perform anomaly detection processing to obtain an anomaly detection result for the mechanism description data. The anomaly detection model is obtained by selecting an anomaly detection unit from an anomaly detection unit set based on the mechanism classification and at least one of the data type and the description type, and constructing a model to be trained based on the anomaly detection unit, and performing model training on the model to be trained based on a sample of the mechanism description data.

[0011] One or more embodiments of the present specification provide a storage medium for storing computer-executable instructions, which implement the following process when executed by a processor: obtaining first mechanism description data for model training under a preset mechanism classification. According to the preset mechanism classification and at least one of the data type and description type of the first mechanism description data, select an anomaly detection unit from an anomaly detection unit set, and construct a model to be trained based on the anomaly detection unit. Perform model training on the model to be trained based on the first mechanism description data to obtain an anomaly detection model, so as to perform anomaly detection processing on the second mechanism description data under the preset mechanism classification through the anomaly detection model. Wherein, each anomaly detection unit in the anomaly detection unit set is extracted and obtained in a pre-trained detection model.

[0012] One or more embodiments of the present specification provide another storage medium for storing computer-executable instructions, which, when executed by a processor, implement the following process: obtaining the mechanism description data to be detected. Inputting the mechanism description data into the mechanism classification of the mechanism description data and an anomaly detection model that matches at least one of the data type and the description type to perform anomaly detection processing, thereby obtaining an anomaly detection result of the mechanism description data. The anomaly detection model is obtained by selecting an anomaly detection unit from an anomaly detection unit set based on the mechanism classification and at least one of the data type and the description type, and constructing a model to be trained based on the anomaly detection unit, and performing model training on the model to be trained based on a sample of the mechanism description data. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the technical solutions in one or more embodiments of this specification, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0014] FIG1 is a schematic diagram of an implementation environment of an anomaly detection model training method provided by one or more embodiments of this specification.

[0015] FIG2 is a processing flow chart of an anomaly detection model training method provided by one or more embodiments of this specification.

[0016] FIG3 is a processing flow chart of an anomaly detection model training method applied to a model training scenario provided by one or more embodiments of this specification.

[0017] FIG4 is a flowchart of an anomaly detection processing method provided by one or more embodiments of this specification.

[0018] FIG5 is a schematic diagram of an embodiment of an anomaly detection model training device provided by one or more embodiments of this specification.

[0019] FIG6 is a schematic diagram of an embodiment of an abnormality detection and processing device provided by one or more embodiments of this specification.

[0020] FIG7 is a schematic diagram of the structure of an anomaly detection model training device provided by one or more embodiments of this specification.

[0021] FIG8 is a schematic diagram of the structure of an anomaly detection and processing device provided by one or more embodiments of this specification. DETAILED DESCRIPTION

[0022] In order to enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the technical solutions in one or more embodiments of this specification will be clearly and completely described below in conjunction with the drawings in one or more embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this document.

[0023] The anomaly detection model training method provided in one or more embodiments of this specification can be applied to the implementation environment of model training of anomaly detection models. Referring to Figure 1, the implementation environment of this method at least includes: a training system 101 for training a model to be trained, and a pre-trained detection model 102 for performing anomaly detection processing; in addition, the implementation environment may also include an anomaly detection unit set 103 and a model to be trained 104.

[0024] In this implementation environment, during the process of training the anomaly detection model, first organization description data for model training under a preset organization classification is obtained, an anomaly detection unit is selected from the anomaly detection unit set 103 according to the preset organization classification and at least one of the data type and description type of the first organization description data, and a to-be-trained model 104 is constructed based on the selected anomaly detection unit. Then, the training system 101 uses the first organization description data as a training sample to perform model training on the to-be-trained model 104 to obtain an anomaly detection model, so as to perform anomaly detection processing on the second organization description data under the preset organization classification through the anomaly detection model;

[0025] Among them, the training system 101 can obtain the first mechanism description data for model training, and the training system 101 can also select an anomaly detection unit in the anomaly detection unit set 103 according to the preset mechanism classification and at least one of the data type and description type of the first mechanism description data, and construct the model to be trained 104 based on the anomaly detection unit; the training system 101 can also extract each anomaly detection unit in the anomaly detection unit set from the pre-trained detection model 102.

[0026] One or more embodiments of an anomaly detection model training method provided in this specification are as follows.

[0027] The anomaly detection model training method provided in this embodiment, after obtaining the first organization description data for model training under the preset organization classification, selects an anomaly detection unit from the anomaly detection unit set according to the preset organization classification and at least one of the data type and description type of the first organization description data, and constructs a model to be trained based on the selected anomaly detection unit, thereby selecting an anomaly detection unit adapted to the first organization description data from multiple dimensions of the preset organization classification, data type and description type, and constructing the model to be trained, thereby improving the comprehensiveness and flexibility of constructing the model to be trained, further, the model to be trained is trained on the model to be trained according to the first organization description data to obtain an anomaly detection model, so as to perform anomaly detection processing on the second organization description data under the preset organization classification through the anomaly detection model, thereby enabling the trained anomaly detection model to perform anomaly detection on the adapted second organization description data, thereby improving the pertinence and flexibility of anomaly detection, and improving the accuracy and effectiveness of anomaly detection through a more adapted anomaly detection model.

[0028] 2 , the anomaly detection model training method provided in this embodiment specifically includes steps S202 to S206 .

[0029] Step S202: obtaining first mechanism description data for model training under a preset mechanism classification.

[0030] The institutions in this embodiment include various forms of organizations such as enterprises, institutions, and social groups. In addition, the institutions include not only organizations at the same level, but also internal organizations of the institutions, such as enterprise departments or branches.

[0031] The preset institutional classification refers to a pre-set institutional classification; the preset institutional classification includes the institutional classification obtained by dividing according to the field in which the institution is engaged, for example, the preset institutional classification can be one of medical institutions, financial institutions, catering institutions, etc.; in addition, the preset institutional classification can also include the institutional classification obtained by dividing according to the size of the institution or the institutional classification obtained by dividing according to the geographical area to which the institution belongs. The preset institutional classification can also include the institutional classification obtained by dividing according to the field in which the institution is engaged, the size of the institution and / or the geographical area to which the institution belongs; wherein the size of the institution can be the size of the institution's personnel, the size of the institution's assets and / or the size of the institution's office space.

[0032] The first institution description data refers to description data describing an institution in one or more description types, such as the credit rating and credit score of an institution described in the credit description type; revenue description data of an institution described in the revenue description type; disciplinary description data of an institution described in the disciplinary description type, etc. The first institution description data may be a first institution label. The first institution description data may be the description data of a single institution or the description data of multiple institutions.

[0033] The description type of the first institution description data includes description dimensions for describing the institution; for example, the description type of the first institution description data includes a credit description type, a revenue description type, a disciplinary description type and / or a scale description type. The examples here are merely illustrative, and the description type of the first institution description data may be one or more; in addition, the description type may also include a statistical type and / or a predictive type. The statistical type refers to a statistical type that performs statistics on data that has currently occurred in the institution. For example, if the first institution description data is the size of personnel, then the description type of the first institution description data is a statistical type. The predictive type refers to a predictive type that predicts data that has not occurred in the institution. For example, if the first institution description data is whether the company is suitable for personnel development, then the description type of the first institution description data is a predictive type.

[0034] The data type of the first mechanism description data includes an image type, a text type, a voice type and / or a video type.

[0035] During specific implementation, in order to improve the flexibility of model training, the first institution description data for model training under the preset institution classification can be obtained; for example, revenue description data and punishment description data for model training under financial institutions and / or revenue description data and credit rating for model training under catering institutions can be obtained.

[0036] It should be noted that the above-mentioned step S202 can be replaced by obtaining the first mechanism description data for model training, and form a new implementation method together with other processing steps provided in this embodiment.

[0037] Step S204: selecting an anomaly detection unit from an anomaly detection unit set according to the preset mechanism classification and at least one of the data type and description type of the first mechanism description data, and constructing a model to be trained based on the anomaly detection unit.

[0038] After obtaining the first mechanism description data for model training under the preset mechanism classification, in this step, an anomaly detection unit is selected from the anomaly detection unit set based on the preset mechanism classification and at least one of the data type and description type of the first mechanism description data, and the model to be trained is constructed based on the anomaly detection unit.

[0039] The anomaly detection unit set described in this embodiment refers to a set consisting of one or more anomaly detection units; each anomaly detection unit in the anomaly detection unit set can be an anomaly detection unit obtained through preliminary training, that is, each anomaly detection unit can also perform anomaly detection processing on the mechanism description data.

[0040] In order to improve the accuracy of the anomaly detection model obtained by training and reduce the difficulty of training the anomaly detection model, the anomaly detection units that construct the model to be trained can be trained in advance; optionally, each anomaly detection unit in the anomaly detection unit set is extracted from a pre-trained detection model. The pre-trained detection model includes a pre-trained anomaly detection model, that is, an anomaly detection model obtained by pre-training; the pre-trained detection model can be a large model LLM (Large Language Model). The anomaly detection units in the anomaly detection unit set include decision trees, random forests and / or neural networks. In addition, the anomaly detection units in the anomaly detection unit set can also include other types of anomaly detection units; the model structure of the pre-trained detection model can be composed of decision trees, random forests and / or neural networks.

[0041] In addition, each of the anomaly detection units in the anomaly detection unit set can also be obtained by pre-training separately; that is, the first unit to be trained can be pre-trained to obtain the first anomaly detection unit, the second unit to be detected can be pre-trained to obtain the second anomaly detection unit, and / or the third unit to be trained can be pre-trained to obtain the third anomaly detection unit. The structure of the first unit to be trained can be a decision tree, the structure of the second unit to be trained can be a random forest, and the structure of the third unit to be trained can be a neural network.

[0042] In actual applications, since the data types of organization description data are diverse, the description types of organization description data are also complex and changeable, and there are many organization classifications of the organization to which the organization description data belongs, if the same anomaly detection model is used to perform anomaly detection on the organization description data, the detection accuracy of the anomaly detection model may not be high, and the training requirements for the anomaly detection model are also high. In view of this, in order to reduce the training difficulty of the anomaly detection model and at the same time improve the detection accuracy and detection flexibility of the anomaly detection model, at least one anomaly detection unit can be selected from the anomaly detection unit set according to the preset organization classification and at least one of the data type and description type of the first organization description data, and a model to be trained is constructed based on at least one anomaly detection unit.

[0043] In an optional implementation manner provided by this embodiment, in the process of selecting an abnormality detection unit from the abnormality detection unit set according to the preset mechanism classification and at least one of the data type and description type of the first mechanism description data, and constructing a model to be trained based on the abnormality detection unit, the following operations are performed: if the preset mechanism classification is a specific mechanism classification, the first abnormality detection unit is selected from the abnormality detection unit set; when the description type is a prediction type, the second abnormality detection unit is selected from the abnormality detection unit set, and the first abnormality detection unit and the second abnormality detection unit are used as the abnormality detection units; when the description type is not a prediction type, the first abnormality detection unit is used as the abnormality detection unit.

[0044] Among them, the specific institution classification refers to a designated institution classification, which may include medical institutions or financial institutions. In addition, the specific institution classification may also be determined according to the actual application scenario, which is not specifically limited in this embodiment. The first anomaly detection unit may be a decision tree, and the second anomaly detection unit may be a random forest. The prediction type refers to the description type of the first institution description data being a prediction type for predicting something that has not happened. For example, the first institution description data is that xx company will be suitable for personnel development in the future or that xx company will not be suitable for investment in the future. These description types of institution description data that predict future or non-event things all belong to the prediction type.

[0045] In the above-mentioned case where the description type is a prediction type, the second abnormality detection unit is selected from the abnormality detection unit set, and the first abnormality detection unit and the second abnormality detection unit are used as abnormality detection units. In an optional implementation provided by this embodiment, in the process of selecting an abnormality detection unit from the abnormality detection unit set according to the preset organization classification and at least one of the data type and description type of the first organization description data, and constructing a model to be trained based on the abnormality detection unit, if the preset organization classification is not the specific organization classification and the data type of the first organization description data is a text type, then the second abnormality detection unit is selected from the abnormality detection unit set as the abnormality detection unit; if the preset organization classification is not the specific organization classification and the data type of the first organization description data is not a text type, then the third abnormality detection unit is selected from the abnormality detection unit set as the abnormality detection unit. Specifically, the following operation can be performed: if the preset organization classification is not the specific organization classification, determine whether the data type of the first organization description data is a text type; if so, select the second abnormality detection unit from the abnormality detection unit set as the abnormality detection unit; if not, select the third abnormality detection unit from the abnormality detection unit set as the abnormality detection unit.

[0046] Among them, the third abnormality detection unit can be a neural network.

[0047] In addition, instead of selecting the second abnormality detection unit from the abnormality detection unit set when the description type is a prediction type, and using the first abnormality detection unit and the second abnormality detection unit as the abnormality detection units, the above operation can be performed directly when the preset mechanism classification is not a specific mechanism classification.

[0048] In addition, in the process of selecting an abnormality detection unit in the abnormality detection unit set based on the preset organization classification and at least one of the data type and description type of the first organization description data, the following operations can also be performed: if the preset organization classification is a specific organization classification, the first abnormality detection unit is selected as the abnormality detection unit in the abnormality detection unit set; if the preset organization classification is not a specific organization classification, and the data type of the first organization description data is a text type, the second abnormality detection unit is selected as the abnormality detection unit in the abnormality detection unit set; if the preset organization classification is not a specific organization classification, and the data type of the first organization description data is an image type, a voice type and / or a video type, the third abnormality detection unit is selected as the abnormality detection unit in the abnormality detection unit set.

[0049] It should be noted that if the preset organization classification is a specific organization classification or if the preset organization classification is not a specific organization classification, they belong to different execution processes, so any one or more of the three execution processes can be selected to select the abnormality detection unit.

[0050] For example, a specific institution is classified as a financial institution or a medical institution. If the preset institution is classified as a financial institution or a medical institution, a decision tree is selected in the anomaly detection unit set, and when the description type of the first institution description data is a prediction type, a random forest is selected in the anomaly detection unit set, and the decision tree and the random forest are used as anomaly detection units; if the preset institution type is not a financial institution or a medical institution, when the data type of the first institution description data is a text type, a random forest is selected as the anomaly detection unit in the anomaly detection unit set; if the preset institution type is not a financial institution or a medical institution, when the data type is an image type, a voice type and / or a video type, a neural network is selected as the anomaly detection unit in the anomaly detection unit set.

[0051] After selecting an anomaly detection unit from the anomaly detection unit set, a model to be trained can be constructed based on the anomaly detection unit. In the process of constructing the model to be trained based on the anomaly detection unit, in order to improve the flexibility of constructing the model to be trained, if only one anomaly detection unit is selected, the anomaly detection unit is used as the model to be trained; if multiple anomaly detection units are selected, the multiple selected anomaly detection units are assembled to obtain the model to be trained.

[0052] In addition, in order to improve the comprehensiveness of the model to be trained and improve the performance of the model to be trained, in the process of constructing the model to be trained based on the anomaly detection unit, an input unit and / or an output unit can be added, and the model to be trained can be constructed based on the input unit, the anomaly detection unit and the output unit, or the model to be trained can be constructed based on the input unit and the anomaly detection unit, or the model to be trained can be constructed based on the anomaly detection unit and the output unit; in this way, by adding the input unit and / or the output unit, the execution operations of the anomaly detection unit are reduced, and simple execution operations are handed over to the input unit and / or the output unit, which helps to improve the anomaly detection unit's focus on anomaly detection and improve the detection accuracy of anomaly detection of the model to be trained obtained through training; wherein, the input unit and the output unit can be deployed based on the actual needs of constructing the model to be trained, for example, the input unit can include a feature extraction unit and / or a feature fusion unit, etc., and the output unit can include a data output unit, etc.

[0053] It should be noted that the above step S204 can be replaced by selecting at least one anomaly detection unit from the anomaly detection unit set according to the preset mechanism classification, at least one of the data type and description type of the first mechanism description data (at least one of the preset mechanism classification, data type and description type), and constructing a to-be-trained model based on the at least one anomaly detection unit;

[0054] Alternatively, it can also be replaced by selecting an abnormality detection unit from the abnormality detection unit set based on at least one of the preset mechanism classification and the data type of the first mechanism description data (at least one of the two), and constructing a model to be trained based on the abnormality detection unit, and forming a new implementation method with other processing steps provided in this embodiment; the abnormality detection unit here can also be at least one.

[0055] Step S206: training the model to be trained according to the first organization description data to obtain an anomaly detection model, so as to perform anomaly detection processing on the second organization description data under the preset organization classification through the anomaly detection model.

[0056] In the above-mentioned step, with the help of the preset organization classification and at least one of the data type and description type of the first organization description data, an anomaly detection unit is selected from the anomaly detection unit set, and a model to be trained is constructed based on the anomaly detection unit. In this step, the model to be trained is trained according to the first organization description data to obtain an anomaly detection model, so that the second organization description data under the preset organization classification is subjected to anomaly detection processing through the anomaly detection model, specifically, the second organization description data under the preset organization classification and at least one of the data type and description type is subjected to anomaly detection processing through the anomaly detection model.

[0057] In this embodiment, the first organization description data refers to the organization description data used as training samples for training the to-be-trained model; the second organization description data refers to the organization description data to be tested after the anomaly detection model is trained. The preset organization classification of the first organization description data is the same as that of the second organization description data, and at least one of the data type and description type of the first organization description data is the same as that of the second organization description data.

[0058] During specific implementation, during each round of model training, the first organization description data can be subjected to anomaly detection processing by the model to be trained to obtain anomaly detection results of the first organization description data, and then the training loss is calculated based on the anomaly detection results and the label data, and the parameters of the model to be trained are adjusted based on the training loss; in an optional implementation manner provided by this embodiment, during model training of the model to be trained based on the first organization description data, the following operations are performed: the first organization description data is input into the model to be trained to perform anomaly detection processing to obtain anomaly detection results of the first organization description data; the training loss is calculated based on the anomaly detection results and the label data corresponding to the first organization description data, and the parameters of the model to be trained are adjusted based on the training loss.

[0059] Among them, the anomaly detection result may include the anomaly category of the first institution's description data, null value indicators, quality anomaly indicators, invalid marking results and / or the association results of asset transaction open information. In addition, the anomaly detection result may also include other types of detection results. The anomaly category here refers to the category in which the first institution's description data has an anomaly, such as the anomaly category including forecast direction error (revenue growth forecast is revenue decline), numerical error, etc. The null value indicator includes the null value rate, that is, the null value ratio, which specifically represents the ratio of null values ​​in the first institution's description data, that is, the ratio of null values ​​in the first institution's description data; the invalid marking result includes the marking result that the first institution's description data belongs to invalid data; the asset transaction open information includes listing information.

[0060] The label data corresponding to the first organization description data refers to the label data corresponding to the anomaly detection result. For example, if the anomaly detection result is an anomaly category, the label data corresponding to the first organization description data is the anomaly category label corresponding to the first organization description data.

[0061] It should be noted that the above model training process can represent the process of each round of model training for the training model. In order to improve the model accuracy and model performance of the anomaly detection model obtained by training, the above model training process for the training model can be referred to, and the training model can be iteratively trained until the training loss converges to obtain the anomaly detection model.

[0062] The above-mentioned anomaly detection results of the first institution description data may include a variety of different anomaly detection results; in actual applications, the first institution description data may be one or more pieces, and the first institution description data can be detected for anomaly from a local perspective, such as detecting the anomaly category of each institution description data in the first institution description data, and performing anomaly detection on the first institution description data from a fine-grained level; in the first optional implementation provided by this embodiment, the model to be trained can determine the anomaly category of the first institution description data as the anomaly detection result based on the description type of the first institution description data, and specifically, the anomaly detection processing can be performed in the following manner: according to the description type of the first institution description data, key data is extracted from the initial data of the institution to which the first institution description data belongs; based on the key data, the anomaly category of the first institution description data is determined as the anomaly detection result.

[0063] The initial data may be the current organization data of the organization to which the first organization description data belongs, that is, the organization data currently related to the organization.

[0064] Specifically, according to the description type of the first organization description data, key data matching the description type can be extracted from the initial data of the organization to which the first organization description data belongs, and the abnormal category of the first organization description data can be determined based on the key data as the abnormality detection result.

[0065] For example, the description type of the first institution description data is the prediction type, and the first institution description data is "the punishment data decreases year by year." The key data "5 punishment data, 8 punishment data, 10 punishment data" are extracted from the initial data of the institution to which the first institution description data belongs, "5 punishment data in the first year, 8 punishment data in the second year, and 10 punishment data in the third year." Then, the punishment data of the institution increases year by year, and the abnormal category of the first institution description data is the prediction direction error.

[0066] In addition to the above-mentioned method of detecting the abnormal category of the first institution description data, the first institution description data can also be detected for abnormalities from a holistic perspective, that is, the first institution description data can be detected for abnormalities from a coarse-grained level, such as calculating the abnormality index of the entire first institution description data; specifically, the null value index of the first institution description data can be calculated, and the intermediate institution description data without null values ​​can be filtered out from the first institution description data, and the quality abnormality index of the intermediate institution description data can be calculated, and the null value index and the quality abnormality index can be used as the abnormality detection results; specifically, in another optional implementation provided by this embodiment, the model to be trained adopts the following method to perform abnormality detection processing: determine the intermediate institution description data without null values ​​in the first institution description data; based on the first institution description data and the intermediate institution description data, calculate the null value index of the first institution description data; calculate the quality abnormality index of the intermediate institution description data, and use the null value index and the quality abnormality index as the abnormality detection result.

[0067] The null value refers to a blank value in the first organization description data. For example, the specific value of the staff size in the first organization description data is blank, that is, the organization description data with a null value is the staff size. The quality abnormality indicator includes an error rate, that is, an error ratio.

[0068] Furthermore, in an optional implementation provided by this embodiment, after calculating the quality anomaly index of the intermediate institution description data and executing the null value index and the quality anomaly index as the anomaly detection result, the following operation is also performed: if the null value index is greater than the null value index threshold, the null value institution description data other than the intermediate institution description data in the first institution description data is determined, and the null value institution description data is filled with null values ​​to obtain the target institution description data, and the target institution description data is stored in the training sample set; if the quality anomaly index is greater than the anomaly index threshold, the data generation model that generates the first institution description data is trained to obtain a trained data generation model.

[0069] The empty value organization description data includes organization description data having an empty value in the first organization description data.

[0070] Specifically, when the null value index is greater than the null value index threshold, after filling the null value organization description data with null values ​​to obtain the target organization description data and storing the target organization description data in the training sample set, the organization description data can be read from the training sample set to continue training the model to be trained or the anomaly detection model obtained through training can be fine-tuned; when the quality anomaly index is greater than the anomaly index threshold, the accuracy of the data generation model representing the generation of the first organization description data is low, so the data generation model generating the first organization description data can be trained to obtain the trained data generation model, and the organization description data can be generated through the trained data generation model to achieve a virtuous circle, so that the organization description data becomes more and more accurate, and the anomaly detection results of the organization description data also become more and more accurate.

[0071] In addition, the model to be trained may also be used for anomaly detection processing in the following manner: according to the description type of the first institution description data, key data is extracted from the initial data of the institution to which the first institution description data belongs, and the detection category of the first institution description data is determined based on the key data as the anomaly detection result; wherein, the detection category includes whether the first institution description data is correct or incorrect, and the detection category includes a data anomaly category and a data normal category, the data anomaly category represents that the first institution description data is incorrect, and the data normal category represents that the first institution description data is correct; or, the model to be trained may also be used for anomaly detection processing in the following manner: calculate the null value index of the first institution description data as the anomaly detection result or calculate the quality anomaly index of the first institution description data as the anomaly detection result; or, the model to be trained may also be used for anomaly detection processing in the following manner Processing: When the cancellation information of the institution to which the first institution description data belongs is detected, the first institution description data is marked as invalid, and the invalid marking result is used as the anomaly detection result; or, when the asset transaction opening information of the institution to which the first institution description data belongs is detected, the first institution description data and the asset transaction opening information are associated with each other, and the association result is used as the anomaly detection result; or, the model to be trained can also perform anomaly detection processing in the following manner: according to the remaining institution description data in the first institution description data except the current institution description data, the quality anomaly index of the current institution description data is calculated, and according to the correlation between the current institution description data and the remaining institution description data, the quality anomaly index is weightedly calculated to obtain the target quality anomaly index of the current institution description data, and the target quality anomaly index of the first institution description data is used as the anomaly detection result.

[0072] It should be noted that the above-mentioned several implementation methods for anomaly detection processing of the model to be trained can be combined and referenced with each other in the process of anomaly detection processing of the model to be trained, that is, in the process of anomaly detection processing of the model to be trained, any one or more of the above-mentioned implementation methods can be adopted.

[0073] After model training is performed to obtain the anomaly detection model, the anomaly detection model can be used to perform anomaly detection processing on the second organization description data under the preset organization classification. The process of the anomaly detection model performing anomaly detection processing on the second organization description data under the preset organization classification and the process of the above-mentioned model to be trained performing anomaly detection processing on the first organization description data belong to the same technical concept, so the process of the anomaly detection model performing anomaly detection processing on the second organization description data here can refer to the process of the above-mentioned model to be trained performing anomaly detection processing on the first organization description data.

[0074] In an optional implementation manner provided by this embodiment, the anomaly detection model performs anomaly detection processing in the following manner: when the cancellation information of the institution to which the second institution description data belongs is detected, the second institution description data is marked invalid, and the invalid marking result is used as the anomaly detection result; and / or, when the asset transaction opening information of the institution to which the second institution description data belongs is detected, the second institution description data is associated with the asset transaction opening information, and the association result is used as the anomaly detection result.

[0075] The cancellation information of the institution in this embodiment includes relevant information on the cancellation of the institution; and the asset transaction opening message includes the listing message of the institution.

[0076] In addition, in order to improve the detection diversity and comprehensiveness of anomaly detection performed by the anomaly detection model; in another optional implementation provided by this embodiment, the anomaly detection model adopts the following method to perform anomaly detection processing: based on the remaining organization description data other than the current organization description data in the second organization description data, calculate the quality anomaly index of the current organization description data; based on the correlation between the current organization description data and the remaining organization description data, perform weighted calculation on the quality anomaly index to obtain the target quality anomaly index of the current organization description data; thereafter, the target quality anomaly index of the second organization description data can be used as the anomaly detection result of the second organization description data.

[0077] Furthermore, in order to improve the convenience of model training of the anomaly detection model; in an optional implementation provided by this embodiment, after performing a weighted calculation on the quality anomaly index based on the correlation between the current organization description data and the remaining organization description data to obtain the target quality anomaly index of the current organization description data, the following operation is also performed: if the target quality anomaly index of the second organization description data is greater than the preset anomaly index threshold, calculate the difference between the target quality anomaly index of the second organization description data and the preset anomaly index threshold; determine whether the difference is less than the difference threshold, and if so, store the second organization description data in the training sample set to perform model training on the anomaly detection model.

[0078] Specifically, when the target quality anomaly index of the second institution description data is greater than the preset anomaly index threshold, the difference between the target quality anomaly index and the preset anomaly index threshold can be calculated. If the difference is less than the difference threshold, it means that the target quality anomaly index of the second institution description data meets the requirements, and the second institution description data can be stored in the training sample set to train the anomaly detection model; if the difference is greater than or equal to the difference threshold, it means that there may be more quality anomaly problems in the second institution description data. In fact, model training can be carried out based on the institution description data with moderate target quality anomaly index, which helps to improve the efficiency of model training, so no processing is required.

[0079] As described above, the process of the anomaly detection model performing anomaly detection processing on the second organization description data under the preset organization classification and the process of the above-mentioned model to be trained performing anomaly detection processing on the first organization description data belong to the same technical concept, so in the process of the anomaly detection processing on the second organization description data, the anomaly detection model can also determine the anomaly category of the second organization description data, determine the detection category of the second organization description data, calculate the null value index and quality anomaly index of the second organization description data, calculate the null value index or quality anomaly index of the second organization description data, or any one or more of these.

[0080] During the specific execution process, after the model to be trained is trained based on the first organization description data to obtain an anomaly detection model, the organization description data to be detected can also be obtained, and the organization description data is input into an anomaly detection model that matches at least one of the data type, organization classification and description type of the organization description data for anomaly detection processing to obtain anomaly detection results of the organization description data.

[0081] To summarize, the present embodiment provides one or more anomaly detection model training methods. First, first, first organization description data for model training under a preset organization classification is obtained. If the preset organization classification is a specific organization classification, the first anomaly detection unit is selected from the anomaly detection unit set. When the description type is a prediction type, the second anomaly detection unit is selected from the anomaly detection unit set, and the first anomaly detection unit and the second anomaly detection unit are used as anomaly detection units. When the description type is not a prediction type, the first anomaly detection unit is used as an anomaly detection unit. Secondly, if the preset organization classification is not a specific organization classification and the data type of the first organization description data is a text type, the second anomaly detection unit is selected from the anomaly detection unit set as the anomaly detection unit. If the preset organization classification is not a specific organization classification and the data type of the first organization description data is not a text type, the third anomaly detection unit is selected from the anomaly detection unit set as the anomaly detection unit. Finally, a model to be trained is constructed based on the anomaly detection unit, and the model to be trained is trained according to the first organization description data to obtain an anomaly detection model, so as to perform anomaly detection processing on the second organization description data under the preset organization classification through the anomaly detection model.

[0082] The following further illustrates the anomaly detection model training method provided in this embodiment by taking the application of an anomaly detection model training method provided in this embodiment in a model training scenario as an example. Referring to Figure 3, the anomaly detection model training method applied to the model training scenario specifically includes steps S302 to S316.

[0083] Step S302: Acquire first mechanism description data for model training under a preset mechanism classification.

[0084] In step S304, if the preset organization classification is a specific organization classification, a first abnormality detection unit is selected from the abnormality detection unit set. If the description type is a prediction type, a second abnormality detection unit is selected from the abnormality detection unit set, and the first abnormality detection unit and the second abnormality detection unit are used as abnormality detection units.

[0085] Step S306: When the description type is not a prediction type, the first anomaly detection unit is used as an anomaly detection unit.

[0086] Step S308: If the preset organization classification is not a specific organization classification and the data type of the first organization description data is a text type, a second anomaly detection unit is selected as the anomaly detection unit in the anomaly detection unit set.

[0087] Step S310: If the preset organization classification is not a specific organization classification and the data type of the first organization description data is not a text type, a third anomaly detection unit is selected from the anomaly detection unit set as the anomaly detection unit.

[0088] Step S312: construct a model to be trained based on the anomaly detection unit.

[0089] Step S314: input the first organization description data into the to-be-trained model for anomaly detection processing to obtain an anomaly detection result of the first organization description data.

[0090] In step S316, a training loss is calculated based on the anomaly detection result and the label data corresponding to the first organization description data, and parameters of the to-be-trained model are adjusted based on the training loss, so as to perform anomaly detection processing on the second organization description data under the preset organization classification through the obtained anomaly detection model.

[0091] One or more embodiments of an anomaly detection and processing method provided in this specification are as follows.

[0092] 4 , the anomaly detection and processing method provided in this embodiment specifically includes steps S402 to S404 .

[0093] Step S402: Obtain description data of the organization to be detected.

[0094] The institutions in this embodiment include various forms of organizations such as enterprises, institutions, and social groups. In addition, the institutions include not only organizations at the same level, but also internal organizations of the institutions, such as enterprise departments or branches.

[0095] The organization description data refers to descriptive data describing an organization in one or more description types, such as the organization's credit rating and credit score described in the credit description type; the organization's revenue description data described in the revenue description type; the organization's disciplinary description data described in the disciplinary description type, etc. The organization description data may be an organization label. The organization description data may be the organization description data of a single organization or the organization description data of multiple organizations.

[0096] Step S404: input the organization description data into an anomaly detection model whose organization classification and data type match at least one of the description type and the organization description data to perform anomaly detection processing to obtain an anomaly detection result of the organization description data.

[0097] The description type of the organization description data described in this embodiment includes description dimensions for describing the organization; for example, the description type of the organization description data includes a credit description type, a revenue description type, a disciplinary description type and / or a scale description type. The examples here are merely illustrative, and the description type of the organization description data can be one or more; in addition, the description type can also include a statistical type and / or a predictive type. The statistical type refers to a statistical type that performs statistics on data that has currently occurred in the organization. For example, if the organization description data is the size of personnel, then the description type of the organization description data is a statistical type. The predictive type refers to a predictive type that predicts data that has not occurred in the organization. For example, if the organization description data is the company's suitability for personnel development, then the description type of the organization description data is a predictive type.

[0098] The data type of the organization description data includes image type, text type, voice type and / or video type. The organization classification refers to a pre-set organization classification; the organization classification includes the organization classification obtained by dividing according to the field in which the organization is engaged, for example, the organization classification can be one of medical institutions, financial institutions, catering institutions, etc.; in addition, the organization classification can also include the organization classification obtained by dividing according to the scale of the organization or the organization classification obtained by dividing according to the geographical area to which the organization belongs, and the organization classification can also include the organization classification obtained by dividing according to the field in which the organization is engaged, the scale of the organization and / or the geographical area to which the organization belongs; wherein the scale of the organization can be the scale of the organization's personnel, the scale of the organization's assets and / or the scale of the organization's office space.

[0099] Optionally, the anomaly detection model is obtained by selecting an anomaly detection unit from an anomaly detection unit set based on the organization classification and at least one of the data type and the description type, and constructing a model to be trained based on the anomaly detection unit, and training the model to be trained based on the organization description data sample.

[0100] In actual applications, since the data types of the organization description data are diverse, the description types of the organization description data are also complex and changeable, and the organization classifications of the organization to which the organization description data belongs are also numerous, if the same anomaly detection model is used to perform anomaly detection on the organization description data, the detection accuracy of the anomaly detection model may not be high, and the training requirements for the anomaly detection model are also high. In view of this, in order to reduce the detection difficulty of the anomaly detection model and at the same time improve the detection accuracy and detection flexibility of the anomaly detection model, the organization description data can be input into the anomaly detection model whose organization classification of the organization description data and data type and description type at least one match to perform anomaly detection processing to obtain the anomaly detection result of the organization description data; specifically, the anomaly detection model whose organization classification of the organization description data and data type and description type at least one match (at least one of the organization classification of the organization description data and data type and description type) can be determined, and then the organization description data can be input into the anomaly detection model for anomaly detection processing to obtain the anomaly detection result of the organization description data.

[0101] In the process of determining an anomaly detection model that matches at least one of the organization classification and the data type and the description type of the organization description data, the following operations may be performed: if the organization classification is a specific organization classification and the description type is a prediction type, determining that the matching anomaly detection model is a first anomaly detection model; if the organization classification is a specific organization classification and the description type is not a prediction type, determining that the matching anomaly detection model is a second anomaly detection model; optionally, the first anomaly detection model is constructed based on the first anomaly detection unit and the second anomaly detection unit, and the second anomaly detection model is constructed based on the first anomaly detection unit. The first anomaly detection unit may be a decision tree, and the second anomaly detection unit may be a random forest.

[0102] The specific institution classification refers to a designated institution classification, which may include medical institutions or financial institutions. In addition, the specific institution classification may also be determined based on the actual application scenario, which is not specifically limited in this embodiment. The prediction type refers to a prediction type in which the description type of the institution description data is a prediction type for something that has not yet occurred. For example, the institution description data is that xx company will be suitable for personnel development in the future or xx company will not be suitable for investment in the future. These description types of institution description data that predict future or future events are all prediction types.

[0103] In the process of determining an anomaly detection model that matches at least one of the organization classification and the data type of the organization description data and the description type, the following operation may also be performed: if the organization classification is not the specific organization classification and the data type of the organization description data is a text type, determining that the matching anomaly detection model is a third anomaly detection model; if the organization classification is not the specific organization classification and the data type of the organization description data is not a text type, determining that the matching anomaly detection model is a fourth anomaly detection model. Optionally, the third anomaly detection model is constructed based on the second anomaly detection unit; the fourth anomaly detection model is constructed based on the third anomaly detection unit, and the third anomaly detection unit may be a neural network.

[0104] In addition, in the process of determining the organization classification of the organization description data and the anomaly detection model that matches at least one of the data type and the description type, the following operations can also be performed: if the organization classification is a specific organization classification, the matching anomaly detection model is determined to be the second anomaly detection model; if the organization classification is not a specific organization classification, when the data type of the organization description data is a text type, the matching anomaly detection model is determined to be the third anomaly detection model; if the organization classification is not a specific organization classification, when the data type of the organization description data is an image type, a voice type and / or a video type, the matching anomaly detection model is determined to be the fourth anomaly detection model.

[0105] It should be noted that the above-mentioned if the institution classification is a specific institution classification and if the institution classification is not a specific institution classification belong to different execution processes, so any one or more of the three execution processes can be selected to determine the anomaly detection model.

[0106] In specific implementation, in an optional implementation provided by this embodiment, the anomaly detection processing includes: extracting key data from the initial data of the organization to which the organization description data belongs according to the description type of the organization description data; and determining the anomaly category of the organization description data based on the key data as the anomaly detection result.

[0107] On this basis, in an optional implementation manner provided by this embodiment, the organization description data is input into an anomaly detection model whose organization classification of the organization description data and data type matches at least one of the description types for anomaly detection processing, and after the anomaly detection result of the organization description data is obtained, it also includes: determining the correction content of the organization description data according to the anomaly category of the organization description data; and performing correction processing on the organization description data according to the correction content to obtain the target organization description data.

[0108] In another optional implementation manner provided by this embodiment, the anomaly detection processing includes: determining that there is no intermediate organization description data with null values ​​in the organization description data; calculating the null value index of the organization description data based on the organization description data and the intermediate organization description data; calculating the quality anomaly index of the intermediate organization description data, and using the null value index and the quality anomaly index as the anomaly detection result.

[0109] On this basis, in an optional implementation manner provided by this embodiment, the organization description data is input into the organization classification of the organization description data and the anomaly detection model that matches at least one of the data type and the description type for anomaly detection processing, and after the anomaly detection result of the organization description data is obtained, it also includes: if the null value index is greater than the null value index threshold, determining the null value organization description data in the organization description data other than the intermediate organization description data, and filling the null value organization description data with null values ​​to obtain the target organization description data, and storing the target organization description data in the training sample set; if the quality anomaly index is greater than the anomaly index threshold, performing model training on the data generation model that generates the organization description data to obtain a trained data generation model.

[0110] In addition, during the anomaly detection processing, the anomaly detection model can also determine the anomaly category of the institution description data, determine the detection category of the institution description data, calculate the null value index and quality anomaly index of the institution description data, calculate the null value index or quality anomaly index of the institution description data, determine the invalid marking result, and determine the association result of the asset transaction open information.

[0111] The process of anomaly detection processing performed by the anomaly detection model here is similar to the process of anomaly detection processing performed by the above-mentioned trained model. Please refer to it for reading, and this embodiment will not be repeated here.

[0112] It should be noted that, in this embodiment of the present application, the organization description data is input into the organization classification of the organization description data, and the data type is matched with at least one of the description types to perform anomaly detection processing, and the anomaly detection result of the organization description data is obtained. The anomaly detection model is obtained by training the model to be trained according to the first organization description data in the previous embodiment of the present application. It is based on the same inventive concept, so this embodiment can refer to the implementation of the aforementioned anomaly detection model training method, and the repeated parts will not be repeated.

[0113] An embodiment of an anomaly detection model training device provided in this specification is as follows: In the above embodiment, an anomaly detection model training method is provided, and correspondingly, an anomaly detection model training device is also provided, which is described below in conjunction with the accompanying drawings.

[0114] 5 , which shows a schematic diagram of an embodiment of an anomaly detection model training device provided by this embodiment.

[0115] Since the device embodiment corresponds to the method embodiment, the description is relatively simple. For the relevant parts, please refer to the corresponding description of the method embodiment provided above. The device embodiment described below is only illustrative.

[0116] This embodiment provides an anomaly detection model training device, including: a data acquisition module 502, configured to acquire first mechanism description data for model training under a preset mechanism classification; a model construction module 504, configured to select an anomaly detection unit from an anomaly detection unit set according to the preset mechanism classification and at least one of the data type and description type of the first mechanism description data, and to construct a model to be trained based on the anomaly detection unit; a model training module 506, configured to perform model training on the model to be trained according to the first mechanism description data to obtain an anomaly detection model, so as to perform anomaly detection processing on the second mechanism description data under the preset mechanism classification through the anomaly detection model; wherein, each anomaly detection unit in the anomaly detection unit set is extracted and obtained from a pre-trained detection model.

[0117] An embodiment of an abnormality detection and processing device provided in this specification is as follows: In the above embodiment, an abnormality detection and processing method is provided, and correspondingly, an abnormality detection and processing device is also provided, which is described below with reference to the accompanying drawings.

[0118] 6 , which shows a schematic diagram of an abnormality detection and processing device embodiment provided by this embodiment.

[0119] Since the device embodiment corresponds to the method embodiment, the description is relatively simple. For the relevant parts, please refer to the corresponding description of the method embodiment provided above. The device embodiment described below is only illustrative.

[0120] This embodiment provides an anomaly detection and processing device, comprising: a description data acquisition module 602, configured to acquire organization description data to be detected; an anomaly processing module 604, configured to input the organization description data into an anomaly detection model whose organization classification and data type match at least one of the description type and the organization description data, to perform an anomaly detection process, thereby obtaining an anomaly detection result for the organization description data;

[0121] Among them, the anomaly detection model is obtained by selecting an anomaly detection unit from the anomaly detection unit set according to the organization classification and at least one of the data type and the description type, and constructing a model to be trained based on the anomaly detection unit, and training the model to be trained based on the organization description data sample.

[0122] An embodiment of an anomaly detection model training device provided in this specification is as follows: Corresponding to the anomaly detection model training method described above, based on the same technical concept, one or more embodiments of this specification also provide an anomaly detection model training device, which is used to execute the anomaly detection model training method provided above. Figure 7 is a structural schematic diagram of an anomaly detection model training device provided by one or more embodiments of this specification.

[0123] This embodiment provides an anomaly detection model training device, as shown in FIG7 . The anomaly detection model training device may vary significantly due to different configurations or performance. It may include one or more processors 701 and memory 702. The memory 702 may store one or more applications or data. The memory 702 may be either ephemeral or persistent. The application stored in the memory 702 may include one or more modules (not shown), each of which may include a series of computer-executable instructions in the anomaly detection model training device. Furthermore, the processor 701 may be configured to communicate with the memory 702 to execute the series of computer-executable instructions in the memory 702 on the anomaly detection model training device. The anomaly detection model training device may also include one or more power supplies 703, one or more wired or wireless network interfaces 704, one or more input / output interfaces 705, one or more keyboards 706, and the like.

[0124] In a specific embodiment, an anomaly detection model training device includes a memory and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs may include one or more modules, and each module may include a series of computer-executable instructions for the anomaly detection model training device, and is configured to be executed by one or more processors to include the following computer-executable instructions: obtaining first mechanism description data for model training under a preset mechanism classification; selecting an anomaly detection unit from an anomaly detection unit set according to the preset mechanism classification and at least one of the data type and description type of the first mechanism description data, and constructing a model to be trained based on the anomaly detection unit; performing model training on the model to be trained according to the first mechanism description data to obtain an anomaly detection model, so as to perform anomaly detection processing on the second mechanism description data under the preset mechanism classification through the anomaly detection model; wherein each anomaly detection unit in the anomaly detection unit set is extracted from the pre-trained detection model.

[0125] An embodiment of an anomaly detection and processing device provided in this specification is as follows: Corresponding to the anomaly detection and processing method described above, based on the same technical concept, one or more embodiments of this specification also provide an anomaly detection and processing device, which is used to execute the anomaly detection and processing method provided above. Figure 8 is a structural schematic diagram of an anomaly detection and processing device provided in one or more embodiments of this specification.

[0126] This embodiment provides an anomaly detection and processing device, as shown in FIG8 . The anomaly detection and processing device may vary significantly depending on configuration or performance. It may include one or more processors 801 and memory 802. Memory 802 may store one or more applications or data. Memory 802 may be either ephemeral or persistent. The applications stored in memory 802 may include one or more modules (not shown), each of which may include a series of computer-executable instructions for the anomaly detection and processing device. Furthermore, processor 801 may be configured to communicate with memory 802, allowing the anomaly detection and processing device to execute the series of computer-executable instructions in memory 802. The anomaly detection and processing device may also include one or more power supplies 803, one or more wired or wireless network interfaces 804, one or more input / output interfaces 805, one or more keyboards 806, and the like.

[0127] In a specific embodiment, an anomaly detection processing device includes a memory and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs may include one or more modules, and each module may include a series of computer-executable instructions for the anomaly detection processing device, and is configured to be executed by one or more processors, and the one or more programs include the following computer-executable instructions: obtaining the mechanism description data to be detected; inputting the mechanism description data into the mechanism classification of the mechanism description data and an anomaly detection model that matches at least one of the data type and the description type to perform anomaly detection processing, and obtaining an anomaly detection result of the mechanism description data; wherein the anomaly detection model is obtained after selecting an anomaly detection unit from an anomaly detection unit set according to the mechanism classification and at least one of the data type and the description type, and constructing a model to be trained based on the anomaly detection unit, and the model to be trained is trained based on the mechanism description data sample.

[0128] An embodiment of a storage medium provided in this specification is as follows: Corresponding to the above-described anomaly detection model training method, based on the same technical concept, one or more embodiments of this specification also provide a storage medium.

[0129] The storage medium provided in this embodiment is used to store computer-executable instructions, which implement the following process when executed by a processor: obtaining first mechanism description data for model training under a preset mechanism classification; selecting an abnormality detection unit from an abnormality detection unit set based on the preset mechanism classification and at least one of the data type and description type of the first mechanism description data, and constructing a model to be trained based on the abnormality detection unit; performing model training on the model to be trained based on the first mechanism description data to obtain an abnormality detection model, so as to perform abnormality detection processing on the second mechanism description data under the preset mechanism classification through the abnormality detection model; wherein, each abnormality detection unit in the abnormality detection unit set is extracted and obtained from the pre-trained detection model.

[0130] It should be noted that the embodiment of a storage medium in this specification and the embodiment of an anomaly detection model training method in this specification are based on the same inventive concept. Therefore, the specific implementation of this embodiment can refer to the implementation of the aforementioned corresponding method, and the repeated parts will not be repeated.

[0131] Another storage medium embodiment provided in this specification is as follows: Corresponding to the above-described anomaly detection and processing method, based on the same technical concept, one or more embodiments of this specification also provide another storage medium.

[0132] The storage medium provided in this embodiment is used to store computer-executable instructions, which implement the following process when executed by a processor: obtaining the organization description data to be detected; inputting the organization description data into the organization classification of the organization description data and an anomaly detection model that matches at least one of the data type and the description type to perform anomaly detection processing, and obtaining an anomaly detection result of the organization description data; wherein, the anomaly detection model selects an anomaly detection unit from an anomaly detection unit set according to the organization classification and at least one of the data type and the description type, and constructs a model to be trained based on the anomaly detection unit, and is obtained after model training of the model to be trained based on the organization description data sample.

[0133] It should be noted that the embodiment of another storage medium in this specification and the embodiment of an abnormality detection and processing method in this specification are based on the same inventive concept. Therefore, the specific implementation of this embodiment can refer to the implementation of the aforementioned corresponding method, and the repeated parts will not be repeated.

[0134] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments. For example, the device embodiment, equipment embodiment and storage medium embodiment are similar to the method embodiment, so the description is relatively simple. To read the relevant content in the device embodiment, equipment embodiment and storage medium embodiment, please refer to the partial description of the method embodiment.

[0135] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0136] In the 1930s, technological improvements could be clearly distinguished as either hardware improvements (for example, improvements to circuit structures like diodes, transistors, and switches) or software improvements (improvements to process flows). However, with the advancement of technology, many process flow improvements today can now be considered direct improvements to hardware circuit structures. Designers almost always create the corresponding hardware circuit structure by programming the improved process flow into the hardware circuit. Therefore, it cannot be said that a process flow improvement cannot be implemented using hardware modules. For example, a programmable logic device (PLD), such as a field programmable gate array (FPGA), is an integrated circuit whose logical function is determined by user programming. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to hire a chip manufacturer to design and manufacture a dedicated integrated circuit chip. Moreover, nowadays, instead of manually fabricating integrated circuit chips, this programming is mostly done using "logic compiler" software. This is similar to the software compiler used when developing programs. Before compilation, the original code must also be written in a specific programming language, called a hardware description language (HDL). There is not just one HDL, but many, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art will also understand that by simply programming the method flow in one of these hardware description languages ​​and then programming it into an integrated circuit, a hardware circuit that implements the logic method flow can be easily obtained.

[0137] The controller can be implemented in any suitable manner. For example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also know that in addition to implementing the controller in a purely computer-readable program code format, the controller can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, such a controller can be considered a hardware component, and the devices included therein for implementing various functions can also be considered as structures within the hardware component. Or even, the devices for implementing various functions can be considered as both software modules that implement the method and structures within the hardware component.

[0138] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0139] For the convenience of description, the above devices are described as being divided into various units according to their functions. Of course, when implementing the embodiments of this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0140] Those skilled in the art will appreciate that one or more embodiments of this specification may be provided as a method, system, or computer program product. Thus, one or more embodiments of this specification may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0141] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable anomaly detection model training device to produce a machine, so that the instructions executed by the processor of the computer or other programmable anomaly detection model training device generate a device for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0142] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable anomaly detection model training device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0143] These computer program instructions may also be loaded onto a computer or other programmable anomaly detection model training device so that a series of operational steps are executed on the computer or other programmable device to produce computer-implemented processing, whereby the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0144] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0145] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0146] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory media such as modulated data signals and carrier waves.

[0147] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0148] One or more embodiments of this specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. One or more embodiments of this specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communications network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0149] The various embodiments in this specification are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the system embodiments are generally similar to the method embodiments, so the description is relatively simple. For relevant parts, refer to the description of the method embodiments.

[0150] The foregoing is a list of some embodiments of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should be included within the scope of the claims of the present application.

Claims

1. A method for training an anomaly detection model, comprising: Obtaining first institution description data for model training under a preset institution classification; According to the preset mechanism classification and at least one of the data type and the description type of the first mechanism description data, select an anomaly detection unit from the anomaly detection unit set, and construct a model to be trained based on the anomaly detection unit; Performing model training on the model to be trained according to the first organization description data to obtain an anomaly detection model, so as to perform an anomaly detection process on the second organization description data under the preset organization classification through the anomaly detection model; Wherein, each anomaly detection unit in the anomaly detection unit set is extracted from a pre-trained detection model.

2. The anomaly detection model training method according to claim 1, wherein: The performing model training on the to-be-trained model according to the first mechanism description data includes: Inputting the first organization description data into the to-be-trained model for anomaly detection processing to obtain an anomaly detection result of the first organization description data; A training loss is calculated based on the anomaly detection result and the label data corresponding to the first mechanism description data, and parameters of the model to be trained are adjusted based on the training loss.

3. The anomaly detection model training method according to claim 2, wherein: The model to be trained uses the following method to perform anomaly detection processing: extracting key data from initial data of an organization to which the first organization description data belongs according to a description type of the first organization description data; Based on the key data, an abnormal category of the first organization description data is determined as the abnormality detection result.

4. The anomaly detection model training method according to claim 2, wherein: The model to be trained uses the following method to perform anomaly detection processing: Determining that there is no intermediate organization description data with a null value in the first organization description data; Calculating a null value index of the first organization description data based on the first organization description data and the intermediate organization description data; The quality anomaly index of the intermediate agency description data is calculated, and the null value index and the quality anomaly index are used as the anomaly detection result.

5. The anomaly detection model training method according to claim 4, wherein: After calculating the quality anomaly index of the intermediate mechanism description data and taking the null value index and the quality anomaly index as the anomaly detection result operation, the method further includes: If the null value index is greater than the null value index threshold, determine the null value institution description data in the first institution description data except the intermediate institution description data, and perform null value filling on the null value institution description data to obtain the target institution description data, and store the target institution description data into the training sample set; If the quality abnormality index is greater than the abnormality index threshold, model training is performed on the data generation model for generating the first mechanism description data to obtain a trained data generation model.

6. The anomaly detection model training method according to claim 1, wherein: The selecting an abnormality detection unit from the abnormality detection unit set according to the preset mechanism classification and at least one of the data type and the description type of the first mechanism description data comprises: If the preset mechanism classification is a specific mechanism classification, selecting a first abnormality detection unit in the abnormality detection unit set; In the case where the description type is a prediction type, a second abnormality detection unit is selected from the abnormality detection unit set, and the first abnormality detection unit and the second abnormality detection unit are used as the abnormality detection units.

7. The anomaly detection model training method according to claim 6, wherein: The selecting an abnormality detection unit from the abnormality detection unit set according to the preset mechanism classification and at least one of the data type and the description type of the first mechanism description data further includes: If the preset organization classification is not the specific organization classification, determine the data type of the first organization description data Whether the type is a text type; If so, selecting the second anomaly detection unit from the anomaly detection unit set as the anomaly detection unit; If not, a third anomaly detection unit is selected from the anomaly detection unit set as the anomaly detection unit.

8. The anomaly detection model training method according to claim 1, wherein: The abnormality detection process includes: When the deregistration information of the organization to which the second organization description data belongs is detected, the second organization description data is marked invalid, and the invalid marking result is used as the abnormality detection result; When the asset transaction opening information of the institution to which the second institution description data belongs is detected, the second institution description data is associated with the asset transaction opening information, and the association result is used as the abnormality detection result.

9. The anomaly detection model training method according to claim 1, wherein: The abnormality detection process includes: Calculating a quality anomaly index of the current organization description data according to the remaining organization description data except the current organization description data in the second organization description data; According to the correlation between the current organization description data and the remaining organization description data, the quality abnormality index is weightedly calculated to obtain a target quality abnormality index of the current organization description data.

10. The anomaly detection model training method according to claim 9, wherein: After the operation of performing weighted calculation on the quality abnormality index according to the correlation between the current mechanism description data and the remaining mechanism description data to obtain the target quality abnormality index of the current mechanism description data is performed, the method further includes: If the target quality abnormality index of the data described by the second mechanism is greater than a preset abnormality index threshold, calculating the difference between the target quality abnormality index of the data described by the second mechanism and the preset abnormality index threshold; Determine whether the difference is less than a difference threshold; if so, store the second mechanism description data in a training sample set to perform model training on the anomaly detection model.

11. An anomaly detection processing method, comprising: Obtain description data of the organization to be tested; Inputting the organization description data into an anomaly detection model whose organization classification and data type match at least one of the description type of the organization description data to perform anomaly detection processing, and obtaining an anomaly detection result of the organization description data; Among them, the anomaly detection model selects an anomaly detection unit from an anomaly detection unit set according to the organization classification and at least one of the data type and the description type, and constructs a model to be trained based on the anomaly detection unit, and is obtained after model training is performed on the model to be trained based on the organization description data sample.

12. The abnormality detection processing method according to claim 11, wherein: The abnormality detection process includes: Extracting key data from initial data of an institution to which the institution description data belongs according to a description type of the institution description data; Based on the key data, an abnormal category of the organization description data is determined as the abnormality detection result.

13. The abnormality detection processing method according to claim 12, wherein: After the step of inputting the organization description data into an anomaly detection model that matches at least one of the organization classification and data type of the organization description data with the description type to perform anomaly detection processing and obtaining an anomaly detection result of the organization description data is performed, the method further includes: Determining, according to the abnormality category of the organization description data, a correction content to be made to the organization description data; The organization description data is modified according to the modification content to obtain target organization description data.

14. The abnormality detection processing method according to claim 11, wherein: The abnormality detection process includes: Determining that there is no intermediate organization description data with a null value in the organization description data; Based on the organization description data and the intermediate organization description data, a null value index of the organization description data is calculated. mark; The quality anomaly index of the intermediate agency description data is calculated, and the null value index and the quality anomaly index are used as the anomaly detection result.

15. The abnormality detection processing method according to claim 14, wherein: After the step of inputting the organization description data into an anomaly detection model that matches at least one of the organization classification and data type of the organization description data with the description type to perform anomaly detection processing and obtaining an anomaly detection result of the organization description data is performed, the method further includes: If the null value index is greater than the null value index threshold, determine the null value institution description data in the institution description data except the intermediate institution description data, and perform null value filling on the null value institution description data to obtain target institution description data, and store the target institution description data in the training sample set; If the quality abnormality index is greater than the abnormality index threshold, model training is performed on the data generation model for generating the organization description data to obtain a trained data generation model.

16. An anomaly detection model training device, comprising: A data acquisition module is configured to acquire first organization description data for model training under a preset organization classification; a model building module, configured to select an anomaly detection unit from an anomaly detection unit set according to the preset mechanism classification and at least one of a data type and a description type of the first mechanism description data, and to build a model to be trained based on the anomaly detection unit; A model training module is configured to perform model training on the to-be-trained model according to the first organization description data to obtain an anomaly detection model, so as to perform an anomaly detection process on the second organization description data under the preset organization classification through the anomaly detection model; Wherein, each anomaly detection unit in the anomaly detection unit set is extracted from a pre-trained detection model.

17. An abnormality detection processing device, comprising: A description data acquisition module is configured to acquire description data of the organization to be detected; An exception processing module is configured to input the organization description data into an anomaly detection model whose organization classification and data type match at least one of the description type of the organization description data to perform anomaly detection processing, and obtain an anomaly detection result of the organization description data; Among them, the anomaly detection model selects an anomaly detection unit from an anomaly detection unit set according to the organization classification and at least one of the data type and the description type, and constructs a model to be trained based on the anomaly detection unit, and is obtained after model training is performed on the model to be trained based on the organization description data sample.

18. An anomaly detection model training device, comprising: processor; and a memory configured to store computer executable instructions that, when executed, cause the processor to: Obtaining first institution description data for model training under a preset institution classification; According to the preset mechanism classification and at least one of the data type and the description type of the first mechanism description data, select an anomaly detection unit from the anomaly detection unit set, and construct a model to be trained based on the anomaly detection unit; Performing model training on the model to be trained according to the first organization description data to obtain an anomaly detection model, so as to perform an anomaly detection process on the second organization description data under the preset organization classification through the anomaly detection model; Wherein, each anomaly detection unit in the anomaly detection unit set is extracted from a pre-trained detection model.

19. An abnormality detection processing device, comprising: processor; and a memory configured to store computer executable instructions that, when executed, cause the processor to: Obtain description data of the organization to be tested; Inputting the organization description data into an anomaly detection model whose organization classification and data type match at least one of the description type of the organization description data to perform anomaly detection processing, and obtaining an anomaly detection result of the organization description data; The anomaly detection model is based on the organization classification and the data type and description type. At least one of them is to select an anomaly detection unit from an anomaly detection unit set, and to construct a model to be trained based on the anomaly detection unit, and to obtain the model after model training is performed on the model to be trained based on the organization description data sample.

20. A storage medium for storing computer executable instructions, wherein the computer executable instructions, when executed by a processor, implement the following process: Obtaining first institution description data for model training under a preset institution classification; According to the preset mechanism classification and at least one of the data type and the description type of the first mechanism description data, select an anomaly detection unit from the anomaly detection unit set, and construct a model to be trained based on the anomaly detection unit; Performing model training on the model to be trained according to the first organization description data to obtain an anomaly detection model, so as to perform an anomaly detection process on the second organization description data under the preset organization classification through the anomaly detection model; in, Each anomaly detection unit in the anomaly detection unit set is extracted from a pre-trained detection model.

21. A storage medium for storing computer executable instructions, wherein the computer executable instructions, when executed by a processor, implement the following process: Obtain description data of the organization to be tested; Inputting the organization description data into an anomaly detection model whose organization classification and data type match at least one of the description type of the organization description data to perform anomaly detection processing, and obtaining an anomaly detection result of the organization description data; in, The anomaly detection model is obtained by selecting an anomaly detection unit from an anomaly detection unit set according to the organization classification and at least one of the data type and the description type, and constructing a model to be trained based on the anomaly detection unit, and training the model to be trained based on the organization description data sample.

Images