Model training method and device, abnormal information processing method and device, equipment and medium
By oversampling and undersampling samples in the abnormality detection field, a suitable training data set is generated, which solves the problem of positive and negative samples imbalance, improves the training accuracy of the abnormality detection model, and realizes model adaptability update of the slave device terminal.
Patent Information
- Application Number
- CN202411991881.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-31
- Publication Date
- 2025-04-29
AI Technical Summary
In the field of abnormal detection, the amount of data between positive training data and negative training data is large, resulting in poor model training effect of the initial abnormal detection model, and it is impossible to select an adapted training data set for the slave terminal for model training.
By obtaining the anomaly detection sample set related to the target value field from the target sample database, sample preprocessing is performed, including oversampled positive sample sets and undersampled negative sample sets, the combined sample sets are generated and stored in the target sample database. In response to the model update request from the device terminal, the corresponding combined sample set is obtained for model update.
By oversampling the positive sample set and undersampling the negative sample set, the sample imbalance problem is avoided, the accuracy of model training is improved, and the slave device terminal can receive the adaptive training data set for model updates.
Smart Images

Figure CN119903345A_ABST
Abstract
Description
Technical Field
[0001] Embodiments of the present disclosure relate to the field of computer technology, and in particular to a model training method, an abnormal information processing method, an apparatus, a device, and a medium. Background Art
[0002] At present, in the field of target anomaly detection, the processing related to anomaly detection based on deep learning networks has become more and more mature. The processing of samples in the training process of deep learning networks is crucial. For the generation of training sample sets and the update of corresponding anomaly detection models, the usual method is: first, obtain anomaly detection related data sets. Then, perform data preprocessing on the anomaly detection related data sets to generate a preprocessed data set. Finally, use the preprocessed data set as the training data set to train the initial anomaly detection model to generate an anomaly detection model.
[0003] However, when the above method is adopted, the following technical problems often occur:
[0004] In the field of anomaly detection, the data volume difference between positive training data and negative training data is large, resulting in poor model training effect of the initial anomaly detection model. It is not possible to select an adapted training data set for at least one slave device terminal to perform model training on the initial anomaly detection model of the slave device terminal.
[0005] The above information disclosed in this Background section is only for enhancement of understanding of the background of the inventive concept and therefore it may contain information that does not form the prior art that is already known in this country to a person of ordinary skill in the art. Summary of the invention
[0006] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.
[0007] Some embodiments of the present disclosure propose a model training method, an abnormal information processing method, an apparatus, an electronic device, and a computer-readable medium to solve one or more of the technical problems mentioned in the above background technology section.
[0008] In a first aspect, some embodiments of the present disclosure provide a model training method applied to a master device terminal, comprising: obtaining an anomaly detection sample set related to a target value field within a target historical time period from a target sample database, wherein the target database stores anomaly detection sample sets sent by each slave device terminal in a slave device terminal set; performing sample preprocessing on each anomaly detection sample in the anomaly detection sample set to generate a first preprocessed sample set, wherein the first preprocessed sample set includes: a positive sample set representing the presence of anomalies and a negative sample set representing the absence of anomalies; performing oversampling processing on the positive sample set to generate a sampled positive sample set, And under-sampling the above-mentioned negative sample set to generate a sampled negative sample set; combining the above-mentioned sampled positive sample set and the above-mentioned sampled negative sample set to generate a combined sample set; storing the above-mentioned combined sample set and the above-mentioned target historical time period in the above-mentioned target sample database in a target storage manner; in response to receiving a model update request for at least one slave device terminal at a target time, obtaining at least one combined sample set corresponding to the above-mentioned target time from the above-mentioned target sample database as at least one training sample set; and updating the model of the anomaly detection model corresponding to the above-mentioned at least one slave device terminal according to the above-mentioned at least one training sample set.
[0009] In a second aspect, some embodiments of the present disclosure provide a model training device applied to a master device terminal, comprising: a first acquisition unit, configured to acquire an anomaly detection sample set related to a target value field within a target historical time period from a target sample database, wherein the target database stores anomaly detection sample sets sent by each slave device terminal in a slave terminal set; a processing unit, configured to perform sample preprocessing on each anomaly detection sample in the anomaly detection sample set to generate a first preprocessed sample set, wherein the first preprocessed sample set includes: a positive sample set representing the presence of anomalies and a negative sample set representing the absence of anomalies; a sampling unit, configured to perform oversampling processing on the positive sample set to generate a sampled positive sample set, and The negative sample set is under-sampled to generate a sampled negative sample set; the combining unit is configured to perform sample set combination on the above-mentioned sampled positive sample set and the above-mentioned sampled negative sample set to generate a combined sample set; the storage unit is configured to store the above-mentioned combined sample set and the above-mentioned target historical time period in the above-mentioned target sample database in a target storage manner; the second acquisition unit is configured to respond to receiving a model update request for at least one slave device terminal at a target time, and obtain at least one combined sample set corresponding to the above-mentioned target time from the above-mentioned target sample database as at least one training sample set; the first updating unit is configured to perform model update on the anomaly detection model corresponding to the above-mentioned at least one slave device terminal according to the above-mentioned at least one training sample set.
[0010] In a third aspect, some embodiments of the present disclosure provide an abnormality information processing method applied to a slave device terminal, comprising: in response to determining that the abnormality detection model usage information corresponding to the target slave device terminal is different abnormality detection model usage information and the corresponding abnormality detection model update method is the second update method, obtaining a training sample subset sent by the master device terminal corresponding to the target area as a third training sample subset; based on the above-mentioned third training sample subset, updating the model of the abnormality detection model corresponding to the above-mentioned target slave device terminal to generate a trained abnormality detection model; in response to receiving an abnormality information generation request for a target user, inputting the abnormality detection data corresponding to the above-mentioned target user into the above-mentioned trained abnormality detection model to generate abnormality information; generating multimodal abnormality information corresponding to the above-mentioned abnormality information; in response to determining that the above-mentioned target user outputs the abnormality information in the form of voice playback, performing language broadcast according to the voice abnormality information in the above-mentioned multimodal abnormality information, and performing text display of the abnormality information according to the text abnormality information in the above-mentioned multimodal abnormality information.
[0011] In a fourth aspect, some embodiments of the present disclosure provide an abnormal information processing device applied to a slave device terminal, including: a third acquisition unit, configured to, in response to determining that the abnormal detection model usage information corresponding to the target slave device terminal is different abnormal detection model usage information and the corresponding abnormal detection model update method is the second update method, obtain a training sample subset sent by the master device terminal corresponding to the target area as a third training sample subset; a second update unit, configured to update the abnormal detection model corresponding to the target slave device terminal according to the third training sample subset to generate a trained abnormal detection model; a first generation unit, configured to, in response to receiving an abnormal information generation request for a target user, input the abnormal detection data corresponding to the target user into the trained abnormal detection model to generate abnormal information; a second generation unit, configured to generate multimodal abnormal information corresponding to the abnormal information; an execution unit, configured to, in response to determining that the target user outputs the abnormal information in the form of voice playback, perform language broadcast according to the voice abnormal information in the multimodal abnormal information, and perform text display of the abnormal information according to the text abnormal information in the multimodal abnormal information.
[0012] In a fifth aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any one of the implementation methods in the first and third aspects.
[0013] In a sixth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any one of the implementation modes of the first aspect and the third aspect is implemented.
[0014] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through the model training method of some embodiments of the present disclosure, on the basis of avoiding sample imbalance, the model of the anomaly detection model corresponding to at least one slave device terminal can be efficiently and accurately updated. Specifically, the reason for the inaccuracy of the model update is that in the field of anomaly detection, the data volume difference between positive training data and negative training data is large, resulting in poor model training effect of the initial anomaly detection model. It is not possible to select an adapted training data set for at least one slave device terminal to perform model training on the initial anomaly detection model of the slave device terminal. Based on this, the model training method of some embodiments of the present disclosure, first, obtains an anomaly detection sample set related to the target value field within the target historical time period from the target sample database, wherein the above-mentioned target database stores the anomaly detection sample set sent by each slave device terminal in the slave device terminal set, so as to generate a subsequent training sample set. Then, each anomaly detection sample in the above-mentioned anomaly detection sample set is subjected to sample preprocessing to generate a first preprocessed sample set, wherein the above-mentioned first preprocessed sample set includes: a positive sample set representing the presence of anomalies and a negative sample set representing the absence of anomalies, so as to ensure the high quality of the samples. Next, the positive sample set is oversampled to generate a sampled positive sample set, and the negative sample set is undersampled to generate a sampled negative sample set. Here, by oversampling the positive sample set and undersampling the negative sample set, the imbalance of positive and negative samples can be avoided, and the accuracy of subsequent model training can be improved. Then, the sampled positive sample set and the sampled negative sample set are combined to generate a combined sample set, so that the subsequent model can obtain a training sample set for training. Next, the combined sample set and the target historical time period are stored in the target sample database in a target storage manner. Furthermore, in response to receiving a model update request for at least one slave device terminal at the target time, at least one combined sample set corresponding to the target time is obtained from the target sample database as at least one training sample set. Here, after receiving a sample acquisition request from at least one slave device terminal, the corresponding training sample set is adaptively allocated to each slave device terminal to ensure the adaptive training of the model of the corresponding slave device terminal. Finally, based on the at least one training sample set, the anomaly detection model corresponding to the at least one slave device terminal can be accurately updated. In summary, by oversampling the positive sample set and undersampling the negative sample set, the problem of sample imbalance can be avoided. In addition, by matching the master device terminal and at least one slave device terminal, the corresponding training sample set can be accurately adapted for the slave device terminal to ensure accurate training of the anomaly detection model corresponding to the slave device terminal. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.
[0016] Figure 1 is a flowchart of some embodiments of a model training method applied to a master device terminal according to the present disclosure;
[0017] Figure 2 is a flowchart of some embodiments of the abnormal information processing method applied to a slave device terminal according to the present disclosure;
[0018] Figure 3 is a structural schematic diagram of some embodiments of a model training device applied to a master device terminal according to the present disclosure;
[0019] Figure 4 It is a structural schematic diagram of some embodiments of an abnormal information processing device applied to a slave device terminal according to the present disclosure;
[0020] Figure 5 It is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION
[0021] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.
[0022] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.
[0023] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.
[0024] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".
[0025] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.
[0026] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.
[0027] refer to Figure 1 , shows a process 100 of some embodiments of a model training method applied to a master device terminal according to the present disclosure. The model training method applied to the master device terminal includes the following steps:
[0028] Step 101: Acquire anomaly detection sample sets related to a target value domain within a target historical time period from a target sample database.
[0029] In some embodiments, the execution subject (i.e., the master device terminal) of the above-mentioned model training method applied to the master device terminal can obtain an anomaly detection sample set related to the target value field within the target historical time period from the target sample database through a wired connection method or a wireless connection method. Among them, the above-mentioned target database stores the anomaly detection sample set sent by each slave device terminal in the slave device terminal set. Among them, the target sample database can be a database of anomaly detection sample sets for model training of each anomaly detection model in the target area. Anomaly detection samples can be sample data related to anomaly detection. Each anomaly detection sample can be an anomaly detection data set corresponding to each anomaly detection object. The anomaly detection data set can be a value flow data set corresponding to the anomaly detection object in the historical time period. The target value field can be a predetermined value field. In practice, the target value field can be an anomaly detection field. The target historical time period can be a predetermined historical time period of a predetermined length before the current time. The anomaly detection sample set can be each anomaly detection sample received within the target historical time period. The master device terminal can be a device terminal deployed in the target area for coordinating with each slave device terminal to complete various processing in the target area. That is, the target area is monitored and processed by a master device terminal and multiple slave device terminals. For example, the target value area can be an order processing scenario or a credit risk control inspection scenario.
[0030] Step 102: perform sample preprocessing on each anomaly detection sample in the anomaly detection sample set to generate a first preprocessed sample set.
[0031] In some embodiments, the execution subject may perform sample preprocessing on each abnormal detection sample in the abnormal detection sample set to generate a first preprocessed sample set. The sample preprocessing may include but is not limited to at least one of the following: supplementation of missing sample content, deletion of redundant sample content, standardization of sample content, and screening based on age, time period, and institution. The first preprocessed sample set includes: a positive sample set representing the presence of abnormalities and a negative sample set representing the absence of abnormalities.
[0032] Step 103: oversampling the positive sample set to generate a sampled positive sample set, and undersampling the negative sample set to generate a sampled negative sample set.
[0033] In some embodiments, the execution entity may perform oversampling processing on the positive sample set to generate a sampled positive sample set, and may perform undersampling processing on the negative sample set to generate a sampled negative sample set.
[0034] As an example, the execution subject may oversample the positive sample set using the synthetic minority over-sampling technique (SMOTE) to generate a sampled positive sample set. The execution subject may undersample the negative sample set using the SMOTE method to generate a sampled negative sample set.
[0035] In some optional implementations of some embodiments, the execution subject may perform oversampling processing on the positive sample set to generate a sampled positive sample set, including the following steps:
[0036] The first step is to determine the sample attribute set and the corresponding positive sample attribute range set corresponding to the positive sample. Among them, the sample attribute set can represent the attribute set in the sample content corresponding to the positive sample. For example, the sample attribute set can be various attributes related to anomaly detection. For example, for order anomaly detection, the corresponding sample attribute set can include: order duration, order address. That is, for different corresponding anomaly detection scenarios, the various sample attributes in the corresponding sample attribute set are also different. There is a one-to-one correspondence between the positive sample attribute range in the positive sample attribute range set and the sample attribute in the sample attribute set. The positive sample attribute range can be the attribute value range corresponding to the sample attribute in the positive sample. For example, for the sample attribute is order duration, the corresponding attribute value range can be [0 minutes, 120 minutes].
[0037] The second step is to obtain at least one abnormal rule requirement information corresponding to the abnormal detection sample. Among them, the at least one abnormal rule requirement information can be at least one rule requirement information related to abnormal detection. In practice, each abnormal rule requirement information in the at least one abnormal rule requirement information corresponds to a different abnormal detection standard. At least one abnormal rule requirement information can be pre-set by relevant technical personnel for different detection standards. For example, in the field of order anomaly detection, the corresponding abnormal rule requirement information can be that the delivery time must not exceed the specified time and the distance between the delivery object and the delivery object is less than the target distance.
[0038] In the third step, based on the sample attribute set, the positive sample attribute range set and the at least one exception rule requirement information, an initial positive sample set is generated using a pre-generated sample generation script. The sample generation script may be a script developed for generating various positive samples based on the sample attribute set, the positive sample attribute range set and the at least one exception rule requirement information. For example, the sample generation script may be a script developed based on Python or JAVA.
[0039] As an example, the execution entity may substitute the sample attribute set, the positive sample attribute range set and the at least one abnormal rule requirement information into a pre-generated sample generation script to generate an initial positive sample set.
[0040] The fourth step is to perform sample preprocessing on each initial positive sample in the initial positive sample set to generate a second preprocessed sample set. The sample preprocessing will not be described in detail.
[0041] Step 5: For each second preprocessed sample in the second preprocessed sample set, perform the following first generation step:
[0042] Sub-step 1: Determine whether there is a positive sample in the positive sample set that is the same as the second pre-processed sample.
[0043] Sub-step 2, in response to determining that it does not exist, a group of positive samples whose sample similarity with the second pre-processed sample is higher than a target value is selected from the positive sample set. The sample similarity may be the similarity between sample contents. The target value may be a predetermined value. Both the sample similarity and the target value are values between 0 and 1. The higher the sample similarity value, the more similar the sample content between the corresponding positive sample and the second pre-processed sample is.
[0044] Sub-step 3: adding the second preprocessed sample to the positive sample group to obtain an added positive sample group.
[0045] Sub-step 4, based on the added positive sample group, generate a sample probability that the second pre-processed sample is a real sample. The sample probability can represent the probability that the generated second pre-processed sample is a real sample. The larger the corresponding value of the sample probability, the more likely the second pre-processed sample is a real sample, rather than a sample generated by a related script.
[0046] Sub-step 5: in response to determining that the sample probability is higher than the target probability value, determining the second pre-processed sample as a candidate positive sample.
[0047] In the sixth step, a sampled positive sample set is generated based on the obtained candidate positive sample subset and the above positive sample set.
[0048] As an example, the execution entity may combine the candidate positive sample subset with the positive sample set to generate a sampled positive sample set.
[0049] In some optional implementations of some embodiments, the execution subject may generate a sample probability characterizing that the second preprocessed sample is a true sample according to the added positive sample group, including the following steps:
[0050] In the first step, each sample in the added positive sample group is input as a whole into a pre-trained real sample information generation model to generate each real sample probability. The real sample information generation model can be a neural network model for generating real sample information. In practice, the real sample information includes the real sample probability of each sample.
[0051] In the second step, the real sample probability corresponding to the second preprocessed sample is selected from the real sample probabilities as the target real sample probability.
[0052] In the third step, in response to determining that the probability of the target real sample is higher than the target probability value, at least one abnormal task information generation model corresponding to at least one abnormal task information is obtained. Among them, the abnormal task information may be an abnormal identifier corresponding to the abnormal task. The abnormal task may be a task suitable for abnormal judgment corresponding to the second preprocessed sample. The abnormal task is an abnormal judgment task. Specifically, for the sample, different fields can be applied to detect the corresponding abnormal task. For example, for the order sample, the corresponding at least one abnormal task may include: order delivery abnormal detection task, order payment abnormal detection task, and order item sales abnormal detection task. There is a one-to-one correspondence between the abnormal task information generation model in the at least one abnormal task information generation model and the abnormal task information in the at least one abnormal task information. The abnormal task information generation model may be a model for generating abnormal task information. In practice, for different abnormal tasks, the abnormal task information generation model can be used to perform abnormal detection. For different tasks, the model structure of the corresponding abnormal task information generation model may be the same or different. The execution setting is specifically performed according to the scenario and relevant personnel.
[0053] Step 4: for each abnormal task information in the at least one abnormal task information, execute the second generating step:
[0054] Sub-step 1: input the second preprocessed sample into the abnormal task information generation model corresponding to the abnormal task information to generate a task probability. The abnormal task information output by the abnormal task information generation model includes a task probability. The task probability can represent the possibility of the presence of an abnormality. The task probability is a value between 0 and 1. The higher the value, the higher the probability of the abnormality.
[0055] Sub-step 2, determine whether the task probability corresponding to the above abnormal task information is in the probability interval corresponding to the positive sample. Among them, the positive sample under the abnormal task has a corresponding probability interval. That is, within the probability interval, the input sample is characterized as a positive sample. If it is not within the probability interval, the input sample is characterized as a negative sample.
[0056] Sub-step 2: in response to determining that , the above task probability is determined as a candidate abnormal task probability.
[0057] Sub-step 3: in response to determining that it is not, setting the probability value 0 as the sample probability characterizing that the second preprocessed sample is a true sample.
[0058] The fifth step is to obtain at least one abnormal task weight corresponding to the at least one abnormal task information and at least one model accuracy corresponding to the at least one abnormal task information generation model. Among them, each abnormal task information has a corresponding abnormal task weight. The abnormal task weight can characterize the importance of the abnormal task in the process of judging whether the sample is a real sample. Similarly, the abnormal task weight can characterize the degree of match between the abnormal task and the sample. At least one abnormal task weight can be pre-set. The model accuracy can be the accuracy of the model output. The model accuracy can be obtained by comprehensively judging the abnormal task information generation model based on the historical data set.
[0059] In a sixth step, based on the at least one abnormal task weight and the at least one model accuracy, weighted summation is performed on each task probability in the at least one task probability to generate a first weighted probability.
[0060] As an example, first, for each task probability in at least one task probability, the task probability, the corresponding abnormal task weight, and the model accuracy are multiplied to obtain multiplication information. Then, the obtained at least one multiplication information is added to obtain a first weighted probability.
[0061] In the seventh step, based on the model accuracy corresponding to the model generated by the real sample information, the first weighted probability and the real sample probability are weighted and summed to generate a second weighted probability as the sample probability.
[0062] As an example, first, the execution subject may multiply the model accuracy corresponding to the real sample information generation model by the real sample probability to generate a multiplied probability, and then add the multiplied probability to the first weighted probability to generate a second weighted probability as the sample probability.
[0063] In some optional implementations of some embodiments, the execution subject may input all samples in the added positive sample group into a pre-trained real sample information generation model to generate each real sample probability, including the following steps:
[0064] In the first step, each of the above samples is input into the sample feature extraction layer included in the above real sample information generation model to generate sample feature information and obtain each sample feature information. Among them, the sample feature extraction layer can be a network layer for extracting sample feature information. The sample feature information can be information in vector form that characterizes the semantic content of sample features. In practice, the sample feature extraction layer can be selected according to the different abnormality detection fields corresponding to the samples. For example, the sample feature extraction layer can be 11 layers of convolutional layers connected in series. It can also be a network layer that extracts features by fusing the attention layer and the convolutional layer.
[0065] In the second step, each of the above-mentioned sample feature information is input into the sample similarity feature information generation layer included in the above-mentioned real sample information generation model to generate sample similarity feature information. Among them, the sample similarity feature information generation layer can be a network layer that extracts similar feature semantic content between each sample feature information. Sample similarity feature information can be information in vector form that characterizes the semantic content of sample similarity features. In practice, the sample similarity feature information extraction layer can be selected according to the different anomaly detection fields corresponding to the samples. For example, the sample similarity feature information layer can be a 47-layer convolution layer connected in series. It can also be a network layer that extracts features by fusing the attention layer and the convolution layer.
[0066] In the third step, for each of the above sample feature information, the third generation step is performed:
[0067] Sub-step 1, input the above-mentioned sample feature information and the above-mentioned sample similarity feature information into the corresponding sample difference feature information generation layer included in the above-mentioned real sample information generation model to generate sample difference feature information corresponding to the above-mentioned sample feature information. Among them, the sample difference feature information generation layer can be a network layer that generates feature differences between sample feature information and sample similarity feature information. The sample difference feature information can be feature information in vector form. In practice, the sample difference feature information generation layer can be an attention mechanism layer, or it can be a multi-layer series of convolutional layers and fully connected layers.
[0068] Sub-step 2, input the above-mentioned sample difference feature information, the above-mentioned sample similarity feature information and the above-mentioned sample difference feature information into the corresponding attention layer included in the above-mentioned real sample information generation model to generate attention information. Among them, the attention layer can be a network layer based on the attention mechanism. The attention information can be fusion feature information in the form of a vector. The attention information can represent which aspects of the feature content are more concerned. That is, if in the subsequent process of generating real sample information, more attention is paid to the influence of the sample difference feature information on the generation of the real sample information, the generated attention information includes more feature content of the sample difference feature information.
[0069] In the fourth step, the attention information is input into the real sample information output layer included in the real sample information generation model to generate real sample information, wherein the real sample information includes: real sample probability. The real sample information output layer may be an output layer that outputs real sample related information. In practice, the real sample information output layer may be a fully connected layer.
[0070] Optionally, as one of the invention points, the problem that the conventional real sample information generation model is not accurate enough is solved. By setting the real sample information generation model to include a sample feature extraction layer, a sample similar feature information generation layer, a sample difference feature information generation layer, an attention layer, and a real sample information output layer, the similar features and different features between each real sample are obtained from multiple aspects, and based on the form of an attention mechanism, accurate recognition of real samples can be achieved.
[0071] In some optional implementations of some embodiments, the execution subject may obtain at least one combined sample set corresponding to the target time from the target sample database as at least one training sample set, including the following steps:
[0072] The first step is to obtain the slave device terminal distribution information corresponding to the slave device terminal set in the target area, wherein the target area has a corresponding resource computing device cluster. Each slave device terminal in the slave device terminal set is a device terminal deployed in the target area. The slave device terminal distribution information can be the location distribution of each slave device terminal in the target area.
[0073] In the second step, based on the slave terminal distribution information, terminal regions are divided for each slave terminal in the slave terminal set to obtain a slave terminal group set, wherein each slave terminal group has a corresponding sub-region.
[0074] In the third step, for each slave terminal group in the slave terminal group set, the following first determination step is performed:
[0075] Sub-step 1, determine the anomaly detection model usage information and anomaly detection model update method of the sub-area corresponding to the above-mentioned slave device terminal group, wherein the above-mentioned sub-area has a corresponding anomaly detection model processing system. Among them, the anomaly detection model usage information can represent whether each slave device terminal in the sub-area uses the usage information of the same model. The anomaly detection model update method can represent the update method of the anomaly detection model. The anomaly detection model update method can be one of the following: a first update method, a second update method. The first update method can represent a method for training based on the remaining computing resources of the above-mentioned resource computing device cluster. That is, the model update of the anomaly detection model is performed through the computing resources of the master device terminal. The second update method can represent a method for training based on the remaining computing resources of the slave device terminal itself. That is, the self-model update of the anomaly detection model is performed through the computing resources of the slave device terminal.
[0076] Sub-step 2, in response to determining that the above-mentioned anomaly detection model usage information is the same anomaly detection model usage information and the above-mentioned anomaly detection model update mode is the first update mode, obtain the slave device terminal training requirement information sent by the above-mentioned anomaly detection model processing system, wherein the above-mentioned slave device terminal training requirement information is obtained by summarizing the various training requirements corresponding to the above-mentioned slave device terminal group, wherein the above-mentioned first update mode represents the mode of training based on the remaining computing resources of the above-mentioned resource computing device cluster. The slave device terminal training requirement information may be the training requirement for the slave device terminal to perform anomaly detection model training. For example, the slave device terminal training requirement information may include: the reaction time is less than the target time, the model size is less than the target size, and the accuracy of the trained model is higher than the target accuracy. The anomaly detection model processing system may be a processing system that performs various processing on the anomaly detection model. Various processing may include: model parameter storage, model parameter transmission. The anomaly detection model processing system is an intermediary for transmitting model-related information between the master device terminal and the slave device terminal. Through the anomaly detection model processing system, the efficiency of information communication between the master device terminal and the slave device terminal can be greatly improved. There is a one-to-one correspondence between the slave device terminals in the slave device terminal group and the training requirements. The training requirements may represent the model training requirements of the anomaly detection model corresponding to the slave device terminal.
[0077] Sub-step 3: Generate a first sample query statement for the training requirement information of the slave device terminal and the target time, wherein the first sample query statement may be a sample query database statement.
[0078] Sub-step 4: using the first sample query statement, obtain a combined sample set corresponding to the target time from the target sample database as a training sample set.
[0079] Sub-step 5, in response to determining that the above-mentioned abnormality detection model usage information is different abnormality detection model usage information and the above-mentioned abnormality detection model update mode is the first update mode, obtaining a training requirement group. Different abnormality detection model usage information can represent that the usage models between each slave device terminal in the slave device terminal group can be the same or different.
[0080] Sub-step 6: for each training requirement in the above training requirement group, execute the fourth generation step:
[0081] The first sub-step is to generate a second sample query statement for the training requirement and the target time. The second sample query statement can be generated in a manner similar to the first sample query statement.
[0082] In the second sub-step, the second sample query statement is used to obtain the combined sample set corresponding to the target time from the target sample database as the training sample subset corresponding to the training requirement. The training sample subset is a sample set for training the anomaly detection model information model based on meeting the training requirement.
[0083] Sub-step 7: determining the obtained training sample subset group as the training sample set.
[0084] And the execution subject may update the anomaly detection model corresponding to the at least one slave device terminal according to the at least one training sample set, including the following steps:
[0085] In the first step, for each slave device terminal group in the above slave device terminal group set, the following second determination step is performed:
[0086] Sub-step 1, in response to determining that the corresponding anomaly detection model usage information is the same anomaly detection model usage information and the above-mentioned anomaly detection model update mode is the first update mode, obtain the pre-stored first anomaly detection model parameter information for the above-mentioned slave device terminal group. The first anomaly detection model parameter information can be the historically trained model parameter information corresponding to each anomaly detection model in the slave device terminal group.
[0087] Sub-step 2: generating a first historical anomaly detection model corresponding to the above-mentioned slave device terminal group according to the above-mentioned first anomaly detection model parameter information.
[0088] Sub-step 3: Based on the training sample set corresponding to the slave device terminal group, and using the remaining computing resources corresponding to the resource computing device cluster, the first historical anomaly detection model is trained to generate a first trained anomaly detection model.
[0089] Sub-step 4, sending the first updated anomaly detection model parameter information in the above-mentioned first trained anomaly detection model to the above-mentioned anomaly detection model processing system, so that the above-mentioned anomaly detection model processing system can send the above-mentioned first updated anomaly detection model parameter information to each slave device terminal in the above-mentioned slave device terminal group.
[0090] Sub-step 5: replacing the stored first anomaly detection model parameter information with the updated anomaly detection model parameter information.
[0091] In some optional implementations of some embodiments, after replacing the stored first anomaly detection model parameter information with the updated anomaly detection model parameter information, the method further includes:
[0092] In the first step, in response to determining that the corresponding anomaly detection model usage information is different anomaly detection model usage information and the above-mentioned anomaly detection model update mode is the first update mode, for each slave device terminal group in the above-mentioned slave device terminal group set, the following third determination step is performed:
[0093] Sub-step 1, obtaining a pre-stored anomaly detection model parameter information group for the above slave device terminal group, wherein there is a one-to-one correspondence between the slave device terminals in the slave device terminal group and the anomaly detection model parameters in the anomaly detection model parameter information group.
[0094] Sub-step 2: for each slave device terminal in the above slave device terminal group, perform the following fifth generation step:
[0095] The first sub-step is to determine the anomaly detection model parameter information in the anomaly detection model parameter information group corresponding to the slave device terminal as the second anomaly detection model parameter information.
[0096] The second sub-step is to determine the training sample subset corresponding to the slave device terminal as the first training sample subset.
[0097] The third sub-step is to generate the second historical anomaly detection model corresponding to the slave device terminal according to the second anomaly detection model parameter information.
[0098] The fourth sub-step is to perform model training on the second historical anomaly detection model based on the first training sample subset and using the remaining computing resources corresponding to the resource computing device cluster to generate a second trained anomaly detection model.
[0099] The fifth sub-step is to send the parameter information of the second updated anomaly detection model in the above-mentioned second trained anomaly detection model to the above-mentioned anomaly detection model processing system, so that the above-mentioned anomaly detection model processing system can send the above-mentioned second updated anomaly detection model parameter information to the above-mentioned slave device terminal.
[0100] The sixth sub-step is to replace the stored second anomaly detection model parameter information with the second updated anomaly detection model parameter information.
[0101] In the second step, in response to determining that the corresponding anomaly detection model usage information is different anomaly detection model usage information and the above-mentioned anomaly detection model update mode is the second update mode, for each slave device terminal group in the above-mentioned slave device terminal group set, the following fourth determination step is performed:
[0102] Sub-step 1: for each slave device terminal in the above slave device terminal group, perform the following sixth generation step:
[0103] The first sub-step is to determine the training sample subset corresponding to the slave device terminal as the second training sample subset, wherein the second updating method represents that the slave device terminal updates the anomaly detection model by itself.
[0104] The second sub-step is to send the second training sample subset and the terminal information corresponding to the slave device terminal to the anomaly detection model processing system, so that the anomaly detection model processing system can send the second training sample subset to the slave device terminal, wherein the slave device terminal updates the anomaly detection model by itself after receiving the second training sample subset.
[0105] Step 104 , combining the sampled positive sample set and the sampled negative sample set to generate a combined sample set.
[0106] In some embodiments, the execution entity may perform sample set combination on the sampled positive sample set and the sampled negative sample set to generate a combined sample set.
[0107] As an example, the execution entity may combine the sampled positive sample set and the sampled negative sample set in the order of time periods to generate a combined sample set.
[0108] Step 105: store the combined sample set and the target historical time period in the target sample database in a target storage manner.
[0109] In some embodiments, the execution entity may store the combined sample set and the target historical time period in the target sample database in a target storage manner, wherein the target storage manner may be a key-value pair storage manner.
[0110] Step 106, in response to receiving a model update request for at least one slave device terminal at the target time, obtaining at least one combined sample set corresponding to the target time from the target sample database as at least one training sample set.
[0111] In some embodiments, in response to receiving a model update request for at least one slave device terminal at a target time, the execution subject may obtain at least one combined sample set corresponding to the target time from the target sample database as at least one training sample set. Among them, at least one slave device terminal may be each slave device terminal to be trained with anomaly detection model. The model update request may be a request for updating anomaly detection model (i.e., model retraining). Among them, the target time may be the request time corresponding to the current model update request. In practice, there is a one-to-one correspondence between the slave device terminal in at least one slave device terminal and the combined sample in at least one combined sample set. Among them, the anomaly detection model may be an anomaly detection detection model. In practice, the anomaly detection model may be applied to various anomaly detection related tasks. For example, for the credit investigation field, the corresponding anomaly detection model may be an abnormal information generation model for user use of value products. For construction scenarios, the corresponding anomaly detection model may be an anomaly detection model for users during construction. For e-commerce scenarios, the corresponding anomaly detection model may be an order anomaly detection model.
[0112] Step 107: update the anomaly detection model corresponding to the at least one slave device terminal according to the at least one training sample set.
[0113] In some embodiments, the execution subject may update the anomaly detection model corresponding to the at least one slave device terminal, wherein the anomaly detection model may be a neural network model for detecting whether an anomaly exists.
[0114] As an example, the execution entity may update the anomaly detection model corresponding to the at least one slave device terminal using a gradient descent method based on the at least one training sample set.
[0115] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through the model training method of some embodiments of the present disclosure, on the basis of avoiding sample imbalance, the model of the anomaly detection model corresponding to at least one slave device terminal can be efficiently and accurately updated. Specifically, the reason for the inaccuracy of the model update is that in the field of anomaly detection, the data volume difference between positive training data and negative training data is large, resulting in poor model training effect of the initial anomaly detection model. It is not possible to select an adapted training data set for at least one slave device terminal to perform model training on the initial anomaly detection model of the slave device terminal. Based on this, the model training method of some embodiments of the present disclosure, first, obtains an anomaly detection sample set related to the target value field within the target historical time period from the target sample database, wherein the above-mentioned target database stores the anomaly detection sample set sent by each slave device terminal in the slave device terminal set, so as to generate a subsequent training sample set. Then, each anomaly detection sample in the above-mentioned anomaly detection sample set is subjected to sample preprocessing to generate a first preprocessed sample set, wherein the above-mentioned first preprocessed sample set includes: a positive sample set representing the presence of anomalies and a negative sample set representing the absence of anomalies, so as to ensure the high quality of the samples. Next, the positive sample set is oversampled to generate a sampled positive sample set, and the negative sample set is undersampled to generate a sampled negative sample set. Here, by oversampling the positive sample set and undersampling the negative sample set, the imbalance of positive and negative samples can be avoided, and the accuracy of subsequent model training can be improved. Then, the sampled positive sample set and the sampled negative sample set are combined to generate a combined sample set, so that the subsequent model can obtain a training sample set for training. Next, the combined sample set and the target historical time period are stored in the target sample database in a target storage manner. Furthermore, in response to receiving a model update request for at least one slave device terminal at the target time, at least one combined sample set corresponding to the target time is obtained from the target sample database as at least one training sample set. Here, after receiving a sample acquisition request from at least one slave device terminal, the corresponding training sample set is adaptively allocated to each slave device terminal to ensure the adaptive training of the model of the corresponding slave device terminal. Finally, based on the at least one training sample set, the anomaly detection model corresponding to the at least one slave device terminal can be accurately updated. In summary, by oversampling the positive sample set and undersampling the negative sample set, the problem of sample imbalance can be avoided. In addition, by matching the master device terminal and at least one slave device terminal, the corresponding training sample set can be accurately adapted for the slave device terminal to ensure accurate training of the anomaly detection model corresponding to the slave device terminal.
[0116] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.
[0117] refer to Figure 2 , shows a process 200 of some embodiments of the abnormal information processing method applied to a slave device terminal according to the present disclosure. The abnormal information processing method applied to the slave device terminal includes the following steps:
[0118] Step 201, in response to determining that the target slave device terminal corresponds to anomaly detection model usage information as different anomaly detection model usage information and the corresponding anomaly detection model update method is the second update method, obtain a training sample subset sent by the master device terminal corresponding to the target area as a third training sample subset.
[0119] In some embodiments, in response to determining that the target slave device terminal corresponds to the abnormality detection model usage information as different abnormality detection model usage information and the corresponding abnormality detection model update mode is the second update mode, the execution subject (i.e., the master device terminal) of the abnormality information processing method applied to the slave device terminal can obtain the training sample subset sent by the master device terminal corresponding to the target area as the third training sample subset. Among them, the target slave device terminal can be a slave device terminal for which the abnormality detection model is to be trained.
[0120] Step 202: Update the anomaly detection model corresponding to the target slave device terminal according to the third training sample subset to generate a trained anomaly detection model.
[0121] In some embodiments, the execution entity may update the anomaly detection model corresponding to the target slave device terminal according to the third training sample subset to generate a trained anomaly detection model.
[0122] Step 203, in response to receiving a request to generate abnormal information for a target user, inputting the abnormal detection data corresponding to the target user into the trained abnormal detection model to generate abnormal information.
[0123] In some embodiments, in response to receiving a request to generate abnormal information for a target user, the execution entity may input the abnormal detection data corresponding to the target user into the trained abnormal detection model to generate abnormal information. The abnormal information generation request may be a request to generate abnormal information.
[0124] Step 204: Generate multi-modal abnormal information corresponding to the abnormal information.
[0125] In some embodiments, the execution subject may generate multimodal abnormal information corresponding to the abnormal information. The multimodal abnormal information may include abnormal information in a multimodal form. For example, the multimodal abnormal information may include abnormal information in an audio mode, abnormal information in an image mode, and abnormal information in a text model.
[0126] Step 205, in response to determining that the above-mentioned target user outputs the abnormal information in the form of voice playback, performs language broadcast according to the voice abnormal information in the above-mentioned multimodal abnormal information, and performs text display of the abnormal information according to the text abnormal information in the above-mentioned multimodal abnormal information.
[0127] In some embodiments, in response to determining that the target user outputs the exception information in the form of voice playback, the execution entity may perform language broadcasting according to the voice exception information in the multimodal exception information, and perform text display of the exception information according to the text exception information in the multimodal exception information.
[0128] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: through the abnormal information processing method for a slave device terminal of some embodiments of the present disclosure, using a trained abnormality detection model, abnormal information can be accurately generated and displayed and output in the form of multi-modal abnormal information.
[0129] Further references Figure 3 As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a model training device applied to a master device terminal. These device embodiments are similar to Figure 1 Corresponding to the method embodiments shown, the model training device applied to the main device terminal can be specifically applied to various electronic devices.
[0130] like Figure 3As shown, a model training device 300 applied to a master device terminal includes: a first acquisition unit 301, a processing unit 302, a sampling unit 303, a combination unit 304, a storage unit 305, a second acquisition unit 306 and a first update unit 307. The first acquisition unit 301 is configured to acquire an anomaly detection sample set related to a target value field within a target historical time period from a target sample database, wherein the target database stores anomaly detection sample sets sent by each slave device terminal in a slave device terminal set; the processing unit 302 is configured to perform sample preprocessing on each anomaly detection sample in the anomaly detection sample set to generate a first preprocessed sample set, wherein the first preprocessed sample set includes: a positive sample set representing the presence of anomalies and a negative sample set representing the absence of anomalies; the sampling unit 303 is configured to perform oversampling processing on the positive sample set to generate a sampled positive sample set, and to perform undersampling processing on the negative sample set to generate a sampled negative sample set. The present invention relates to a sample set; a combining unit 304, configured to combine the above-mentioned post-sampling positive sample set and the above-mentioned post-sampling negative sample set to generate a combined sample set; a storage unit 305, configured to store the above-mentioned combined sample set and the above-mentioned target historical time period in the above-mentioned target sample database in a target storage manner; a second acquisition unit 306, configured to respond to receiving a model update request for at least one slave device terminal at a target time, and to acquire at least one combined sample set corresponding to the above-mentioned target time from the above-mentioned target sample database as at least one training sample set; a first updating unit 307, configured to update the model of the anomaly detection model corresponding to the above-mentioned at least one slave device terminal according to the above-mentioned at least one training sample set.
[0131] It can be understood that the units recorded in the model training device 300 applied to the main device terminal are similar to the reference Figure 1 Therefore, the operations, features and beneficial effects described above for the method are also applicable to the model training device 300 applied to the main device terminal and the units contained therein, and will not be repeated here.
[0132] Further references Figure 4 As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of an abnormal information processing device applied to a slave device terminal. These device embodiments are similar to Figure 1 Corresponding to the method embodiments shown, the abnormal information processing device applied to the slave device terminal can be specifically applied to various electronic devices.
[0133] like Figure 4As shown, an abnormal information processing device 300 applied to a slave device terminal includes: a third acquisition unit 401, a second update unit 402, a first generation unit 403, a second generation unit 404 and an execution unit 405. Among them, the third acquisition unit 401 is configured to, in response to determining that the target slave device terminal corresponds to the abnormal detection model usage information as different abnormal detection model usage information and the corresponding abnormal detection model update mode is the second update mode, obtain the training sample subset sent by the master device terminal corresponding to the target area as the third training sample subset; the second update unit 402 is configured to update the model of the abnormal detection model corresponding to the target slave device terminal according to the third training sample subset to generate a trained abnormal detection model; the first generation unit 403 is configured to, in response to receiving an abnormal information generation request for the target user, input the abnormal detection data corresponding to the target user into the trained abnormal detection model to generate abnormal information; the second generation unit 404 is configured to generate multimodal abnormal information corresponding to the abnormal information; the execution unit 405 is configured to, in response to determining that the target user outputs the abnormal information in the form of voice playback, perform language broadcast according to the voice abnormal information in the multimodal abnormal information, and perform text display of the abnormal information according to the text abnormal information in the multimodal abnormal information.
[0134] It can be understood that the units recorded in the abnormal information processing device 400 applied to the slave terminal are similar to those in the reference Figure 1 Therefore, the operations, features and beneficial effects described above for the method are also applicable to the abnormal information processing device 400 applied to the slave device terminal and the units contained therein, and will not be repeated here.
[0135] Reference below Figure 5 , which shows a structural schematic diagram of an electronic device (eg, an electronic device) 500 suitable for implementing some embodiments of the present disclosure. Figure 5 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.
[0136] like Figure 5As shown, the electronic device 500 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the electronic device 500 are also stored. The processing device 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.
[0137] Typically, the following devices may be connected to the I / O interface 505: an input device 506 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 507 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 508 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 509. The communication device 509 may allow the electronic device 500 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 5 The electronic device 500 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 5 Each block shown in the figure may represent one device, or may represent multiple devices as required.
[0138] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 509, or installed from the storage device 508, or installed from the ROM 502. When the computer program is executed by the processing device 501, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.
[0139] It should be noted that the computer-readable medium in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, the computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.
[0140] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.
[0141] The above-mentioned computer-readable medium may be included in the above-mentioned electronic device; or it may exist independently without being assembled into the electronic device. The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device: obtains an anomaly detection sample set related to the target value field within the target historical time period from the target sample database, wherein the above-mentioned target database stores the anomaly detection sample set sent by each slave device terminal in the slave device terminal set; performs sample preprocessing on each anomaly detection sample in the above-mentioned anomaly detection sample set to generate a first preprocessed sample set, wherein the above-mentioned first preprocessed sample set includes: a positive sample set representing the presence of anomalies and a negative sample set representing the absence of anomalies; performs oversampling processing on the above-mentioned positive sample set to generate a sample set. The method comprises the steps of: generating a post-sampling positive sample set, and performing under-sampling processing on the negative sample set to generate a post-sampling negative sample set; combining the post-sampling positive sample set and the post-sampling negative sample set to generate a combined sample set; storing the combined sample set and the target historical time period in the target sample database in a target storage manner; in response to receiving a model update request for at least one slave device terminal at a target time, obtaining at least one combined sample set corresponding to the target time from the target sample database as at least one training sample set; and updating the anomaly detection model corresponding to the at least one slave device terminal according to the at least one training sample set. In response to determining that the target slave device terminal corresponds to the abnormality detection model usage information as different abnormality detection model usage information and the corresponding abnormality detection model update method is the second update method, a training sample subset sent by the master device terminal corresponding to the target area is obtained as a third training sample subset; according to the third training sample subset, the abnormality detection model corresponding to the target slave device terminal is updated to generate a trained abnormality detection model; in response to receiving an abnormality information generation request for a target user, the abnormality detection data corresponding to the target user is input into the trained abnormality detection model to generate abnormality information; multimodal abnormality information corresponding to the abnormality information is generated; in response to determining that the target user outputs the abnormality information in the form of voice playback, language broadcast is performed according to the voice abnormality information in the multimodal abnormality information, and text display of the abnormality information is performed according to the text abnormality information in the multimodal abnormality information.
[0142] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0143] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0144] The units described in some embodiments of the present disclosure may be implemented by software or by hardware. The described units may also be set in a processor, for example, may be described as: a processor includes a first acquisition unit, a processing unit, a sampling unit, a combination unit, a storage unit, a second acquisition unit and a first update unit. Among them, the names of these units do not constitute a limitation on the unit itself under certain circumstances. For example, the first acquisition unit may also be described as "a unit for obtaining an anomaly detection sample set related to a target value field within a target historical time period from a target sample database."
[0145] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.
[0146] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) and the technical solutions formed.
Claims
1. A model training method applied to a master device terminal, comprising: Acquire an anomaly detection sample set related to a target value field within a target historical time period from a target sample database, wherein the target database stores an anomaly detection sample set sent by each slave device terminal in a slave device terminal set; Performing sample preprocessing on each anomaly detection sample in the anomaly detection sample set to generate a first preprocessed sample set, wherein the first preprocessed sample set includes: a positive sample set indicating the presence of anomalies and a negative sample set indicating the absence of anomalies; Performing oversampling processing on the positive sample set to generate a sampled positive sample set, and performing undersampling processing on the negative sample set to generate a sampled negative sample set; Combining the sampled positive sample set and the sampled negative sample set to generate a combined sample set; storing the combined sample set and the target historical time period in the target sample database in a target storage manner; In response to receiving a model update request for at least one slave device terminal at a target time, acquiring at least one combined sample set corresponding to the target time from the target sample database as at least one training sample set; According to the at least one training sample set, the anomaly detection model corresponding to the at least one slave device terminal is updated.
2. The method according to claim 1, wherein: The oversampling process is performed on the positive sample set to generate a sampled positive sample set, comprising: Determine a sample attribute set corresponding to the positive sample and a corresponding positive sample attribute range set; Obtain at least one anomaly rule requirement information corresponding to the anomaly detection sample; Generate an initial positive sample set using a pre-generated sample generation script according to the sample attribute set, the positive sample attribute range set and the at least one exception rule requirement information; Performing sample preprocessing on each initial positive sample in the initial positive sample set to generate a second preprocessed sample set; For each second preprocessed sample in the second preprocessed sample set, the following first generating step is performed: Determine whether there is a positive sample in the positive sample set that is the same as the second preprocessed sample; In response to determining that there is no such sample, a positive sample group having a sample similarity with the second preprocessed sample higher than a target value is screened out from the positive sample set; Adding the second preprocessed sample to the positive sample group to obtain an added positive sample group; Generating, according to the added positive sample group, a sample probability characterizing that the second preprocessed sample is a true sample; In response to determining that the sample probability is higher than a target probability value, determining the second preprocessed sample as a candidate positive sample; A sampled positive sample set is generated according to the obtained candidate positive sample subset and the positive sample set.
3. The method according to claim 2, wherein: The generating, according to the added positive sample group, a sample probability characterizing that the second preprocessed sample is a true sample comprises: Inputting all samples in the added positive sample group as a whole into a pre-trained real sample information generation model to generate each real sample probability; Filter out the true sample probability corresponding to the second preprocessed sample from the true sample probabilities as the target true sample probability; In response to determining that the target true sample probability is higher than the target probability value, obtaining at least one abnormal task information generation model corresponding to at least one abnormal task information; For each abnormal task information in the at least one abnormal task information, a second generating step is performed: Inputting the second preprocessed sample into an abnormal task information generation model corresponding to the abnormal task information to generate a task probability; Determine whether the task probability is within the probability interval corresponding to the positive sample; In response to determining that the task probability is, determining the task probability as a candidate abnormal task probability; In response to determining that it is not, setting a probability value of 0 as a sample probability characterizing that the second preprocessed sample is a true sample; Acquire at least one abnormal task weight corresponding to the at least one abnormal task information and at least one model accuracy corresponding to the at least one abnormal task information generation model; According to the at least one abnormal task weight and the at least one model accuracy, performing weighted summation processing on each task probability in the at least one task probability to generate a first weighted probability; According to the model accuracy corresponding to the real sample information generation model, the first weighted probability and the real sample probability are weightedly summed to generate a second weighted probability as the sample probability.
4. The method according to claim 1, wherein: The step of acquiring at least one combined sample set corresponding to the target time from the target sample database as at least one training sample set includes: Acquire slave device terminal distribution information corresponding to a slave device terminal set in a target area, wherein a corresponding resource computing device cluster exists in the target area; According to the slave terminal distribution information, terminal regions are divided for each slave terminal in the slave terminal set to obtain a slave terminal group set, wherein each slave terminal group has a corresponding sub-region; For each slave device terminal group in the slave device terminal group set, the following first determination step is performed: Determine the abnormality detection model usage information and abnormality detection model update method of the sub-area corresponding to the slave device terminal group, wherein the sub-area has a corresponding abnormality detection model processing system; In response to determining that the anomaly detection model usage information is the same anomaly detection model usage information and the anomaly detection model update mode is the first update mode, obtaining the slave device terminal training requirement information sent by the anomaly detection model processing system, wherein the slave device terminal training requirement information is obtained by summarizing the various training requirements corresponding to the slave device terminal group, wherein the first update mode represents a mode of performing training based on the remaining computing resources of the resource computing device cluster; Generate a first sample query statement for the training requirement information of the slave device terminal and the target time; Using the first sample query statement, obtaining a combined sample set corresponding to the target time from the target sample database as a training sample set; In response to determining that the anomaly detection model usage information is different anomaly detection model usage information and the anomaly detection model update mode is the first update mode, acquiring a training requirement group; For each training requirement in the training requirement group, executing the fourth generation step: Generate a second sample query statement for the training requirement and the target time; Using the second sample query statement, obtaining a combined sample set corresponding to the target time from the target sample database as a training sample subset corresponding to the training requirement; Determine the obtained training sample subset group as a training sample set; And the updating of the anomaly detection model corresponding to the at least one slave device terminal according to the at least one training sample set includes: For each slave device terminal group in the slave device terminal group set, the following second determination step is performed: In response to determining that the corresponding anomaly detection model usage information is the same anomaly detection model usage information and the anomaly detection model update mode is the first update mode, obtaining pre-stored first anomaly detection model parameter information for the slave device terminal group; generating a first historical anomaly detection model corresponding to the slave device terminal group according to the first anomaly detection model parameter information; According to the training sample set corresponding to the slave device terminal group, the first historical anomaly detection model is trained using the remaining computing resources corresponding to the resource computing device cluster to generate a first trained anomaly detection model; Sending parameter information of first updated anomaly detection model parameter information in the first trained anomaly detection model to the anomaly detection model processing system, so that the anomaly detection model processing system sends the first updated anomaly detection model parameter information to each slave device terminal in the slave device terminal group; The stored first anomaly detection model parameter information is replaced with the updated anomaly detection model parameter information.
5. The method according to claim 4, wherein: After replacing the stored first anomaly detection model parameter information with the updated anomaly detection model parameter information, the method further includes: In response to determining that the corresponding anomaly detection model usage information is different anomaly detection model usage information and the anomaly detection model update mode is the first update mode, for each slave device terminal group in the slave device terminal group set, the following third determination step is performed: Acquire a pre-stored abnormality detection model parameter information group for the slave device terminal group; For each slave device terminal in the slave device terminal group, the following fifth generating step is performed: Determine the anomaly detection model parameter information in the anomaly detection model parameter information group corresponding to the slave device terminal as the second anomaly detection model parameter information; Determine a training sample subset corresponding to the slave device terminal as a first training sample subset; generating a second historical anomaly detection model corresponding to the slave device terminal according to the second anomaly detection model parameter information; According to the first training sample subset, using the remaining computing resources corresponding to the resource computing device cluster, performing model training on the second historical anomaly detection model to generate a second trained anomaly detection model; Sending the second updated anomaly detection model parameter information in the second trained anomaly detection model to the anomaly detection model processing system, so that the anomaly detection model processing system sends the second updated anomaly detection model parameter information to the slave device terminal; Replacing the stored second anomaly detection model parameter information with the second updated anomaly detection model parameter information; In response to determining that the corresponding anomaly detection model usage information is different anomaly detection model usage information and the anomaly detection model update mode is the second update mode, for each slave device terminal group in the slave device terminal group set, the following fourth determination step is performed: For each slave device terminal in the slave device terminal group, the following sixth generating step is performed: Determine a training sample subset corresponding to the slave device terminal as a second training sample subset, wherein the second updating mode represents that the slave device terminal updates the anomaly detection model by itself; The second training sample subset and the corresponding terminal information of the slave device terminal are sent to the anomaly detection model processing system, so that the anomaly detection model processing system sends the second training sample subset to the slave device terminal, wherein the slave device terminal updates the anomaly detection model by itself after receiving the second training sample subset.
6. A method for processing abnormal information applied to a slave device terminal, comprising: In response to determining that the abnormality detection model usage information corresponding to the target slave device terminal is different abnormality detection model usage information and the corresponding abnormality detection model update mode is the second update mode, obtaining a training sample subset sent by the master device terminal corresponding to the target area as a third training sample subset; According to the third training sample subset, updating the anomaly detection model corresponding to the target slave device terminal to generate a trained anomaly detection model; In response to receiving a request for generating abnormal information for a target user, inputting the abnormal detection data corresponding to the target user into the trained abnormal detection model to generate abnormal information; Generate multimodal abnormal information corresponding to the abnormal information; In response to determining that the target user outputs the exception information in the form of voice playback, language broadcast is performed according to the voice exception information in the multimodal exception information, and text display of the exception information is performed according to the text exception information in the multimodal exception information.
7. A model training device applied to a main device terminal, comprising: A first acquisition unit is configured to acquire an anomaly detection sample set related to a target value field within a target historical time period from a target sample database, wherein the target database stores an anomaly detection sample set sent by each slave device terminal in a slave device terminal set; A processing unit is configured to perform sample preprocessing on each anomaly detection sample in the anomaly detection sample set to generate a first preprocessed sample set, wherein the first preprocessed sample set includes: a positive sample set representing the presence of anomalies and a negative sample set representing the absence of anomalies; A sampling unit, configured to perform an oversampling process on the positive sample set to generate a sampled positive sample set, and to perform an undersampling process on the negative sample set to generate a sampled negative sample set; a combining unit, configured to perform sample set combination on the sampled positive sample set and the sampled negative sample set to generate a combined sample set; a storage unit configured to store the combined sample set and the target historical time period in the target sample database in a target storage manner; A second acquisition unit is configured to, in response to receiving a model update request for at least one slave device terminal at a target time, acquire at least one combined sample set corresponding to the target time from the target sample database as at least one training sample set; The first updating unit is configured to update the anomaly detection model corresponding to the at least one slave device terminal according to the at least one training sample set.
8. An abnormal information processing device applied to a slave device terminal, comprising: A third acquisition unit is configured to, in response to determining that the abnormality detection model usage information corresponding to the target slave device terminal is different abnormality detection model usage information and the corresponding abnormality detection model update mode is the second update mode, acquire the training sample subset sent by the master device terminal corresponding to the target area as the third training sample subset; A second updating unit is configured to update the anomaly detection model corresponding to the target slave device terminal according to the third training sample subset to generate a trained anomaly detection model; A first generating unit is configured to, in response to receiving a request for generating abnormal information for a target user, input the abnormal detection data corresponding to the target user into the trained abnormal detection model to generate abnormal information; A second generating unit is configured to generate multimodal abnormal information corresponding to the abnormal information; The execution unit is configured to, in response to determining that the target user outputs the abnormal information in the form of voice playback, perform language broadcast according to the voice abnormal information in the multimodal abnormal information, and perform text display of the abnormal information according to the text abnormal information in the multimodal abnormal information.
9. An electronic device, comprising: one or more processors; a storage device having one or more programs stored thereon, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.
10. A computer readable medium having a computer program stored thereon, wherein: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.