Abnormality detection method and device, electronic equipment and storage medium

Abstract

The invention relates to the technical field of data processing, can be applied to various scenes such as cloud technology, artificial intelligence, intelligent traffic and auxiliary driving, and particularly relates to an anomaly detection method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining multimedia data and object description data associated with a target object; according to an input feature type corresponding to a current effective anomaly detection model, to-be-input features are extracted from the multimedia data and the object description data, the effective anomaly detection model is a historical anomaly detection model, or after the historical anomaly detection model reaches a performance anomaly condition, the to-be-input features are extracted from the multimedia data and the object description data; training the obtained target anomaly detection model; and adopting the effective anomaly detection model to obtain an anomaly detection result based on the to-be-input features. Therefore, the processing abnormality of the historical abnormality detection model can be timely detected, the situation that a model with poor performance is adopted for abnormality detection for a long time is avoided, and the abnormality detection accuracy can be improved.

Description

Background Art

[0002] In order to create a good Internet interaction environment, it is necessary to perform anomaly detection on the data content published by each business object in the business interaction scenario, so as to detect the anomalies of the business objects in time and reduce the risk of abnormal content being spread.

[0003] For example, in a live broadcast scenario, anomaly detection can be performed on the data stream of each live broadcast room to identify abnormal live broadcast rooms that use illegal means to make profits.

[0004] Currently, when performing anomaly detection on data content in a specified business scenario, a target anomaly detection model is usually obtained through periodic training, and corresponding detection results are obtained based on the data content.

[0005] However, since the types of data content in the Internet environment are changeable, the target anomaly detection model obtained through periodic training may not be able to adapt to the changes in the data content that needs to be detected. Therefore, when detecting data content based on the target anomaly detection model, it is difficult to effectively detect anomalies of business objects and the accuracy of anomaly detection cannot be guaranteed. Summary of the invention

[0006] The embodiments of the present application provide an anomaly detection method, an apparatus, an electronic device, and a storage medium to improve the accuracy of anomaly detection.

[0007] In the first aspect, an anomaly detection method is proposed, comprising:

[0008] Acquire multimedia data and object description data associated with a target object; the multimedia data is obtained from a multimedia data stream published by the target object;

[0009] According to the input feature type corresponding to the current effective anomaly detection model, the multimedia data and the object description data are screened and feature extracted to obtain the features to be input, wherein the effective anomaly detection model is: a model selected from a historical anomaly detection model and a target anomaly detection model trained by an initial anomaly detection model according to the acquisition timing of the multimedia data and the object description data; the training timing of the initial anomaly detection model is: based on the performance value determined by the detection result of the historical anomaly detection model in a specified historical period, it is determined that the performance anomaly condition is met;

[0010] The effective anomaly detection model is adopted to obtain an anomaly detection result of the target object based on the features to be input.

[0011] In a second aspect, an abnormality detection device is provided, comprising:

[0012] An acquisition unit, used to acquire multimedia data and object description data associated with a target object; the multimedia data is obtained from a multimedia data stream published by the target object;

[0013] An extraction unit is used to perform data screening and feature extraction on the multimedia data and object description data according to the input feature type corresponding to the current effective anomaly detection model to obtain the features to be input, wherein the effective anomaly detection model is: a model selected from a historical anomaly detection model and a target anomaly detection model obtained by training the initial anomaly detection model according to the acquisition timing of the multimedia data and the object description data; the training timing of the initial anomaly detection model is: based on the performance value determined by the detection result of the historical anomaly detection model in a specified historical period, it is determined that the performance anomaly condition is met;

[0014] The detection unit is used to adopt the effective anomaly detection model and obtain an anomaly detection result of the target object based on the feature to be input.

[0015] Optionally, if the multimedia data and the object description data are acquired after the target anomaly detection model and the historical anomaly detection model complete model evaluation based on real-time detection data within a set time period, the extraction unit is further used to:

[0016] In a case where the model evaluation value of the target anomaly detection model is higher than the model evaluation value of the historical anomaly detection model, determining the target anomaly detection model as the effective anomaly detection model;

[0017] In a case where the model evaluation value of the target anomaly detection model is lower than the model evaluation value of the historical anomaly detection model, the historical anomaly detection model is determined as the effective anomaly detection model.

[0018] Optionally, the extraction unit is further used for:

[0019] If the multimedia data and the object description data are obtained when the target anomaly detection model is in a model evaluation process, determining the historical anomaly detection model as the effective anomaly detection model;

[0020] If the multimedia data and the object description data are obtained during the process of training the target anomaly detection model, the historical anomaly detection model is determined as the effective anomaly detection model.

[0021] Optionally, the target anomaly detection model is obtained by training a training unit in the device in the following manner:

[0022] Acquire each training sample; each training sample includes: negative samples generated based on abnormal multimedia data successfully detected and not detected by the historical anomaly detection model, and positive samples generated based on normal multimedia data successfully detected and falsely detected as abnormal by the historical anomaly detection model; a training sample includes: sample input features obtained by sorting based on the selected input feature type, and corresponding anomaly labels;

[0023] The training samples are used to perform multiple rounds of iterative training on the constructed initial anomaly detection model to obtain a trained target anomaly detection model.

[0024] Optionally, when acquiring each training sample, the training unit performs any one of the following operations:

[0025] Polling the preset sample storage area to obtain each training sample;

[0026] Poll a preset sample storage area to obtain description information of each sample, and obtain each training sample based on the description information of each sample and the selected input feature type; the description information of each sample includes object identification information of each sample publishing object and abnormal identification information of each sample multimedia data.

[0027] Optionally, each training sample is generated by the training unit in the following manner:

[0028] In the historical data set generated within the specified historical period, respectively determine the abnormal identification information of each sample multimedia data and the object identification information of the corresponding sample publishing object, wherein each sample multimedia data includes: abnormal multimedia data successfully detected and not detected by the historical anomaly detection model, and normal multimedia data successfully detected and falsely detected as abnormal by the historical anomaly detection model;

[0029] An input feature type is selected from a preset full feature set, and for each sample publishing object, the following operations are performed: based on the input feature type, sample input features are extracted from sample multimedia data and object description data determined based on identification information of the sample publishing object and abnormal identification information of the corresponding sample multimedia data, and training samples are generated based on the sample input features and abnormal labels determined corresponding to the sample multimedia data.

[0030] Optionally, the input feature type is selected by the training unit in the following manner:

[0031] In the preset full feature set, determine the full features that match the current business scenario, and generate each screening sample based on the full features, each normal multimedia data and abnormal multimedia data, and the object description data of each sample publishing object, wherein a screening sample includes a full input feature and a corresponding abnormal label; the full features include candidate sub-features of various types;

[0032] The constructed initial feature screening model is trained for multiple rounds of iterations using the screening samples to obtain importance evaluation values ​​output for each type of candidate sub-features in the target feature screening model, and the input feature type is determined from each type of candidate sub-features based on each importance evaluation value.

[0033] Optionally, the input feature type is selected by the training unit in the following manner:

[0034] In the preset full feature set, each candidate sub-feature group is selected respectively, and for each candidate sub-feature group, based on each normal multimedia data and abnormal multimedia data, and object description data of each sample release object, a corresponding screening sample set is generated, wherein a screening sample includes: component input features constructed corresponding to a candidate sub-feature group, and a corresponding abnormal label;

[0035] Using each screening sample set, multiple rounds of iterative training are performed on each constructed initial feature screening model to obtain each trained target feature screening model, and the input feature type is screened out according to the importance evaluation value output for each type of candidate sub-feature in each target feature screening model.

[0036] Optionally, after obtaining the abnormality detection result of the target object, the detection unit is further used to:

[0037] When it is determined that the abnormality detection result of the target object is abnormal, the business service for the target object is stopped, and the multimedia data stream published by the target object is interrupted.

[0038] Optionally, when acquiring the multimedia data and object description data associated with the target object, the acquiring unit is used to:

[0039] In a multimedia data stream published by a target object, sampling an image frame at a preset first time interval, sampling an audio segment of a set duration at a preset second time interval, and determining the image frame and the audio segment as multimedia data associated with the target object currently obtained;

[0040] Acquire the publishing information of the multimedia data stream record published for the target object, the attribute information and the historical detection information associated with the target object, and determine the publishing information, the attribute information and the historical detection information as the object description data associated with the target object.

[0041] Optionally, when the multimedia data and the object description data are screened and feature extracted according to the input feature type to obtain the feature to be input, the extraction unit is used to:

[0042] Acquire the input feature type selected when constructing the initial anomaly detection model, and filter out matching input data from the multimedia data and the object description data according to the input feature type;

[0043] Feature extraction is performed on the input data to obtain features to be input.

[0044] In a third aspect, an electronic device is proposed, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the above-described methods when executing the computer program.

[0045] In a fourth aspect, a computer-readable storage medium is proposed, on which a computer program is stored, and when the computer program is executed by a processor, any of the methods described above is implemented.

[0046] In a fifth aspect, a computer program product is proposed, comprising a computer program, wherein when the computer program is executed by a processor, the method described in any one of the above items is implemented.

[0047] The beneficial effects of this application are as follows:

[0048] In an embodiment of the present application, an anomaly detection method, an apparatus, an electronic device and a storage medium are proposed to obtain multimedia data and object description data associated with a target object; the multimedia data is obtained from a multimedia data stream published by the target object; and then, according to the input feature type corresponding to the current effective anomaly detection model, data screening and feature extraction are performed on the multimedia data and object description data to obtain features to be input, wherein the effective anomaly detection model is: a model selected from a historical anomaly detection model and a target anomaly detection model trained by an initial anomaly detection model according to the acquisition timing of the multimedia data and the object description data; the training timing of the initial anomaly detection model is: based on the performance value determined by the detection result of the historical anomaly detection model in a specified historical period, it is determined that the performance anomaly condition is met; thereafter, the effective anomaly detection model is used to obtain the anomaly detection result of the target object based on the features to be input.

[0049] In this way, since the effective anomaly detection model used for anomaly detection in the anomaly detection process is selected from the historical anomaly detection model and the target anomaly detection model obtained by retraining after determining that the detection effect of the historical anomaly detection model is not good, and the training timing of the target anomaly detection model is: based on the performance value determined by the detection results of the historical anomaly detection model in the specified historical period, it is determined that the performance anomaly condition is met; this enables the training process of the target anomaly detection model to be started when it is determined that the model performance of the historical anomaly detection model is not sufficient to support the anomaly detection needs, so that the iterative training of the model can be triggered at the appropriate time. On the one hand, by judging the performance of the historical anomaly detection model, the processing performance detection of the historical anomaly detection model is realized, so as to timely detect the processing anomaly of the historical anomaly detection model; on the other hand, by starting the training of the target anomaly detection model when the training of the historical anomaly detection model reaches the performance anomaly detection condition, the flexible configuration of the model training timing is realized, and it is no longer limited to the periodic model training process, so that the variability of the data content in the Internet environment can be better coped with.

[0050] Furthermore, by timely training the target anomaly detection model, we can effectively avoid using models with poor performance for anomaly detection for a long time, which helps to efficiently detect abnormal data content and provide guarantees for the accuracy of anomaly detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 A schematic diagram of possible application scenarios in the embodiments of the present application;

[0052] Figure 2 This is a schematic diagram of the abnormality detection process in the embodiment of the present application;

[0053] Figure 3 A schematic diagram of a process of acquiring multimedia data in an embodiment of the present application;

[0054] Figure 4 A schematic diagram of the process of automatically iterating and optimizing to obtain a valid anomaly detection model in an embodiment of the present application;

[0055] Figure 5 A schematic diagram of the performance values ​​corresponding to each designated historical period in the embodiment of the present application;

[0056] Fig. 6A A schematic diagram of the process of training a target anomaly detection model in an embodiment of the present application;

[0057] Figure 6B Schematic diagram of the generation process of each training sample in the embodiment of the present application;

[0058] Fig. 7A A schematic diagram of the process of determining the input feature type in the implementation of this application;

[0059] Figure 7B A schematic diagram of the process of selecting an input feature type in an embodiment of the present application;

[0060] Fig. 8A A schematic diagram of the process of constructing each training sample in an embodiment of the present application;

[0061] Figure 8B This is a schematic diagram of the process of training a target anomaly detection model in an embodiment of the present application;

[0062] Figure 8C A schematic diagram of the process of comparing and determining the effective anomaly detection model in an embodiment of the present application;

[0063] Fig. 9 This is a schematic diagram of the logical structure of the abnormality detection device in the embodiment of the present application;

[0064] Fig.10 The present invention is a schematic diagram of the hardware structure of an electronic device to which the present invention is applied. DETAILED DESCRIPTION

[0065] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the technical solution of the present application, rather than all of the embodiments. Based on the embodiments recorded in the application documents, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the technical solution of the present application.

[0066] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the invention described herein can be implemented in sequences other than those illustrated or described herein.

[0067] Some of the terms used in the embodiments of the present application are explained below to facilitate understanding by those skilled in the art.

[0068] User Generated Content (UGC): refers to the content created by users on the Internet.

[0069] Automatic Speech Recognition (ASR) refers to the technology of recognizing speech implemented by a combination of processes and software. With the help of automatic speech recognition, human speech can be decoded and converted into digitized text.

[0070] Optical character recognition (Ocr): refers to the process of analyzing and identifying image files of text materials to obtain text and layout information.

[0071] Black industry: also known as black industry, refers to illegal activities that use the Internet as a medium and network technology as the main means to bring potential threats to the security of computer information systems and the management order of cyberspace.

[0072] The following is a brief introduction to the design concept of the embodiment of the present application:

[0073] In order to create a good Internet interaction environment, it is necessary to perform anomaly detection on the data content published by each business object in the business interaction scenario, so as to detect the anomalies of the business objects in time and reduce the risk of abnormal content being spread.

[0074] For example, in live broadcast scenarios, the confrontation between risk control strategies (or anomaly identification strategies) and black industries is very intense. Black industries will try various methods to circumvent and bypass risk control strategies, which poses a huge challenge to the accuracy of anomaly detection.

[0075] During the invention conception process, this application summarized the following characteristics of the black industry in the live broadcast scene:

[0076] 1) Profit-making motivation.

[0077] Specifically, the black industry usually obtains profits through fraud, violations and other means; moreover, in order to maintain this profit model, the black industry in the live broadcast scenario will try its best to bypass detection to avoid being discovered and banned.

[0078] In general, the black industry in the live broadcast scene usually relies on various gray behaviors to obtain direct monetary benefits or traffic benefits from the live broadcast platform.

[0079] 2) There are behaviors to avoid legal sanctions.

[0080] Specifically, black industries are usually accompanied by a series of subsequent illegal or irregular behaviors, such as fraudulent transactions and infringement of user privacy. Among them, fraudulent transactions refer to the use of items that do not conform to the agreement to complete transactions between users; a common behavior that infringes on user privacy is to guide users to add contact information.

[0081] At present, the black industry usually avoids legal prosecution and sanctions by bypassing detection. In the live broadcast scenario, the black industry usually uses anonymous identities, proxy servers and other means to hide their true identities and whereabouts in order to evade legal investigation.

[0082] 3) Maintain long-term existence with the goal of continuously gaining benefits.

[0083] Specifically, the black industry usually exists on live streaming platforms with the goal of continuously obtaining profits. Moreover, in order to achieve this goal, the black industry usually constantly changes its fraudulent methods and strategies to adapt to the risk control measures of the live streaming platforms and maintain its own existence and activities.

[0084] Currently, when performing anomaly detection on data content in a specified business scenario, a target anomaly detection model obtained through periodic training is usually used to obtain corresponding detection results based on the data content.

[0085] However, since the types of data content in the Internet environment are changeable, the target anomaly detection model obtained by periodic training may not be able to adapt to the changes in the data content that needs to be detected. Therefore, it is difficult to effectively detect abnormal data content based on the target anomaly detection model obtained by periodic training, and the accuracy of anomaly detection cannot be guaranteed.

[0086] In view of this, an anomaly detection method, device, electronic device and storage medium are proposed to obtain multimedia data and object description data associated with a target object; the multimedia data is obtained from a multimedia data stream released by the target object; then, according to the input feature type corresponding to the current effective anomaly detection model, data screening and feature extraction are performed on the multimedia data and object description data to obtain the features to be input, wherein the effective anomaly detection model is: according to the acquisition time of the multimedia data and the object description data, a model selected from a historical anomaly detection model and a target anomaly detection model trained by an initial anomaly detection model; the training time of the initial anomaly detection model is: based on the performance value determined by the detection result of the historical anomaly detection model in a specified historical period, it is determined that the performance anomaly condition is met; thereafter, the effective anomaly detection model is adopted to obtain the anomaly detection result of the target object based on the features to be input.

[0087] In this way, since the effective anomaly detection model used for anomaly detection in the anomaly detection process is selected from the historical anomaly detection model and the target anomaly detection model obtained by retraining after determining that the detection effect of the historical anomaly detection model is not good, and the training timing of the target anomaly detection model is: based on the performance value determined by the detection results of the historical anomaly detection model in the specified historical period, it is determined that the performance anomaly condition is met; this enables the training process of the target anomaly detection model to be started when it is determined that the model performance of the historical anomaly detection model is not sufficient to support the anomaly detection needs, so that the iterative training of the model can be triggered at the appropriate time. On the one hand, by judging the performance of the historical anomaly detection model, the processing performance detection of the historical anomaly detection model is realized, so as to timely detect the processing anomaly of the historical anomaly detection model; on the other hand, by starting the training of the target anomaly detection model when the training of the historical anomaly detection model reaches the performance anomaly detection condition, the flexible configuration of the model training timing is realized, and it is no longer limited to the periodic model training process, so that the variability of the data content in the Internet environment can be better coped with.

[0088] Furthermore, by timely training the target anomaly detection model, we can effectively avoid using models with poor performance for anomaly detection for a long time, which helps to efficiently detect abnormal data content and provide guarantees for the accuracy of anomaly detection.

[0089] The preferred embodiments of the present application are described below in conjunction with the drawings in the specification. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application. In addition, the embodiments of the present application and the features in the embodiments may be combined with each other if there is no conflict.

[0090] See also Figure 1 , which is a schematic diagram of a possible application scenario in an embodiment of the present application, includes a client device 110 and a processing device 120 .

[0091] In some feasible implementations of the present application, the processing device 120 may be a server capable of providing services for a designated business, obtaining UGCs published by business objects from each client device 110, and performing anomaly detection based on the data content published by each business object to detect whether the business object is abnormal;

[0092] In some other feasible implementations of the present application, the processing device 120 may be a device authorized to perform anomaly detection. The processing device 120 may obtain the UGC published by each business object from the server device, and perform anomaly detection based on the data content published by each business object to detect whether the business object is abnormal.

[0093] The client device 110 includes but is not limited to mobile phones, tablet computers, notebooks, e-book readers, intelligent voice interaction devices, smart home appliances, vehicle terminals, aircraft, etc. The embodiments of the present application can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, smart transportation, assisted driving, etc.

[0094] The processing device 120 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), as well as big data and artificial intelligence platforms.

[0095] In the embodiment of the present application, the client device 110 and the processing device 120 may communicate via a wired network or a wireless network. The following description only describes the anomaly detection process from the perspective of the processing device 120 .

[0096] The following describes the anomaly detection process in combination with possible application scenarios:

[0097] Application scenario: Detect abnormal live broadcast rooms in live broadcast scenarios.

[0098] Specifically, in this application scenario, the processing device obtains data streams from each live broadcast room, wherein the obtained data streams may be recorded data streams or live broadcast data streams; then, determines the effective anomaly detection model used for current anomaly detection, and the input feature type corresponding to the current effective anomaly detection model;

[0099] Furthermore, for the data stream of each live broadcast room, the following operations are performed respectively: according to the determined input feature type, the corresponding features to be input are extracted from the description information of the live broadcast room and the multimedia data obtained from the data stream; the features to be input are then input into the effective anomaly detection model to obtain the anomaly detection result corresponding to the live broadcast room.

[0100] In addition, it should be understood that in the specific implementation of the present application, it involves the acquisition and processing of multimedia data and object description data associated with the target object. When the embodiments recorded in the present application are applied to specific products or technologies, it is necessary to obtain the permission or consent of the relevant objects, and the collection, use and processing of relevant data need to comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0101] The following is an explanation of the anomaly detection process from the perspective of the processing device in conjunction with the accompanying drawings:

[0102] See also Figure 2As shown, it is a schematic diagram of the abnormality detection process in the embodiment of the present application. Figure 2 , the anomaly detection process is explained:

[0103] Step 201: The processing device obtains multimedia data and object description data associated with the target object.

[0104] In the embodiment of the present application, in order to perform anomaly detection on a target object, the processing device needs to obtain multimedia data and object description data associated with the target object, wherein the multimedia data is obtained from a multimedia data stream published by the target object.

[0105] According to actual processing needs, the multimedia data associated with the target object may include at least one or a combination of image content, audio content, and text content. The object description data is used to describe the multimedia data stream publishing status of the target object and the attribute information of the target object itself.

[0106] For example, when the target object is a live broadcast room in a live broadcast scenario, the acquired multimedia data includes image content and audio content.

[0107] In the case where the multimedia data associated with the target object includes image content and audio content, when executing step 201, the processing device samples image frames at a preset first time interval in the multimedia data stream published by the target object, and samples an audio segment of a set length at a preset second time interval, and determines the image frames and the audio segment as the multimedia data associated with the target object currently acquired; obtains the publishing information recorded in the multimedia data stream published for the target object, the attribute information and the historical detection information associated with the target object, and determines the publishing information, the attribute information and the historical detection information as the object description data associated with the target object.

[0108] Specifically, the processing device can configure the values ​​of the first time interval and the second time interval according to actual processing needs, wherein the values ​​of the first time interval and the second time interval can be the same or different; in the process of performing an anomaly detection on the target object, the sampled image frame is at least one and the sampled audio segment is at least one.

[0109] For example, see Figure 3 As shown, it is a schematic diagram of the process of obtaining multimedia data in an embodiment of the present application. Assume that the obtained multimedia data includes: obtaining an image frame every 10S from the multimedia data stream, and a 5S audio segment every 10S; then, according to Figure 3From the illustrated acquisition process, it can be seen that at 00:00:10 of the multimedia data stream, an image frame is acquired, and at 00:00:10, an audio segment lasting 5S begins to be acquired; furthermore, the acquired audio segment and image frame can be determined as the acquired multimedia data.

[0110] It should be noted that, in the embodiment of the present application, in the process of performing anomaly detection based on the multimedia data stream published by the target object, since the publication of the multimedia data stream is ongoing, the anomaly detection of the target object by the processing device can also be ongoing, that is, multiple anomaly detections are performed on the target object until it is determined that an anomaly exists in the target object, or the target object stops publishing the multimedia data stream.

[0111] In an embodiment of the present application, in order to combine more reference information to achieve auxiliary detection of the target object, the processing device also obtains the publishing information of the multimedia data stream record published for the target object, the attribute information of the target object, and the historical detection information recorded for the target object.

[0112] Among them, the release information includes the release time and release location of the multimedia data stream released by the target object; the attribute information includes the object identification of the target object, the service platform to which it belongs, etc.; the historical detection information refers to the detection results previously obtained based on the historical multimedia data streams released by the target object.

[0113] In this way, for the target object that currently needs to be detected for anomalies, the multimedia data based on anomaly detection can be obtained from the level of the data stream published by the target object, and the object description information that can characterize the current state of the target object, historical anomalies, and the publication status of the multimedia data stream can be obtained from the level of the target object itself, so that the data content based on anomaly detection can be enriched, which is conducive to improving the subsequent anomaly detection effect on the target object.

[0114] Step 202: The processing device performs data screening and feature extraction on the multimedia data and the object description data according to the input feature type corresponding to the current effective anomaly detection model to obtain the features to be input, wherein the effective anomaly detection model is: a model selected from the historical anomaly detection model and the target anomaly detection model trained by the initial anomaly detection model according to the acquisition timing of the multimedia data and the object description data.

[0115] In an embodiment of the present application, after the processing device obtains multimedia data and object description data for the target object, it performs data screening and feature extraction on the multimedia data and object description data based on the effective anomaly detection model used by the current anomaly detection to obtain input features that can be processed by the effective anomaly detection model. From the perspective of anomaly detection, the effective anomaly detection model based on which the current anomaly detection is based can be: a model selected from a historical anomaly detection model and a target anomaly detection model trained by an initial anomaly detection model according to the timing of obtaining the multimedia data and object description data; the training timing of the initial anomaly detection model is: determining that the performance anomaly condition is met based on a performance value determined based on the detection results of the historical anomaly detection model within a specified historical period.

[0116] The following discusses the different effective anomaly detection models selected at different acquisition times of multimedia data and object description data:

[0117] Case 1: When acquiring multimedia data and object description data, model evaluation has been completed for the historical anomaly detection model and the retrained target anomaly detection model.

[0118] At the data acquisition timing corresponding to situation one, if the multimedia data and object description data are acquired after the target anomaly detection model and the historical anomaly detection model complete model evaluation based on the real-time detection data within a set time period, then, when the model evaluation value of the target anomaly detection model is higher than the model evaluation value of the historical anomaly detection model, the target anomaly detection model is determined as the effective anomaly detection model; when the model evaluation value of the target anomaly detection model is lower than the model evaluation value of the historical anomaly detection model, the historical anomaly detection model is determined as the effective anomaly detection model.

[0119] In an embodiment of the present application, in order to ensure the processing effect of the new model, for the model used for anomaly detection, there is the following determination process. After determining the performance value of the historical anomaly detection model used in a specified historical period and determining that the performance anomaly condition is met, the training based on the initial anomaly detection model is triggered to obtain the target anomaly detection model; thereafter, after the target anomaly detection model is obtained through training, within a set time period, the historical anomaly detection model and the target anomaly detection model are used simultaneously to perform anomaly detection on the online real-time data, and the real-time detection results corresponding to the historical anomaly detection model and the target anomaly detection model within the set time period are obtained, and the model evaluation is completed based on the obtained real-time detection results.

[0120] Among them, in the model evaluation process, the steps executed include: calculating the model evaluation values ​​corresponding to the historical anomaly detection model and the target anomaly detection model respectively; then comparing the model evaluation values ​​of the two models, and determining a model with a better model evaluation value between the historical anomaly detection model and the target anomaly detection model; in addition, the training process of the target anomaly detection model and the training timing of the target anomaly detection model will be described in detail in the subsequent content.

[0121] Furthermore, a model with a better model evaluation value can be determined as an effective anomaly detection model.

[0122] Based on this, according to the difference in the model evaluation values ​​corresponding to the historical anomaly detection model and the target anomaly detection model, there is a situation where the model evaluation value of the target anomaly detection model is higher than the model evaluation value of the historical anomaly detection model, and there is a situation where the model evaluation value of the target anomaly detection model is lower than the model evaluation value of the historical anomaly detection model. Therefore, in the data acquisition timing corresponding to situation one, the processing device can determine the effective anomaly detection model based on the model evaluation value.

[0123] It should be noted that, in the embodiment of the present application, the model evaluation value can be specifically determined based on at least one or a combination of indicators such as the overall recall rate, the overall false alarm rate, the additional recall rate, the accuracy rate, and the additional false alarm rate, wherein the overall recall rate = (model hit anomaly number ∩ reported approval number) / reported approval number; the overall false alarm rate = the model wrong hit anomaly number determined by manual quality inspection / the model hit anomaly number; the additional recall rate = the accurate part of the new model recall but not the old model recall / the new model recall; the additional false alarm rate = the wrong part of the new model recall but not the old model recall / the new model recall, wherein the new model refers to the target anomaly detection model, and the old model refers to the historical anomaly detection model.

[0124] See also Figure 4 As shown, it is a schematic diagram of the process of automatically iterating and optimizing to obtain an effective anomaly detection model in an embodiment of the present application. In order to complete the training of the target anomaly detection model and the model evaluation between the target anomaly detection model and the historical anomaly detection model, the processing device can be divided into four functional modules, namely, a positive and negative sample pulling module, a model training module, a model effect comparison module, and a model replacement online module, wherein,

[0125] Positive and negative sample pulling model: It is used to trigger the sample pulling process and automatically obtain the historical data set when it is determined that the core indicator (i.e. performance value) of the online model (i.e. historical anomaly detection model) falls below the threshold. The historical data set includes manual review results. Then, various training samples are constructed for model training. The relevant sample construction process and training process will be described in detail later.

[0126] Model training module: Based on each training sample, train the initial anomaly detection model to obtain the target anomaly detection model.

[0127] Model effect comparison module: copy online data and use the target anomaly detection model to perform anomaly detection on the online data. At the same time, use the old model (i.e. the historical anomaly detection model) to perform anomaly detection. Combined with the results of manual review, compare the core indicators (such as overall recall rate, overall false positive rate, additional recall rate, accuracy rate, and additional false positive rate, etc.) between the new model (target anomaly detection model) and the old model.

[0128] Model replacement and online module: select a model with a higher model evaluation value from the new and old models as the effective anomaly detection model, connect the effective anomaly detection model to the online reasoning platform, and detect the real-time data stream. In particular, if the effective anomaly detection model is a new model, the old model will be offline to achieve the replacement of the new and old models.

[0129] In addition, the set time period considered in model evaluation can be set according to actual processing needs, such as setting it to 3 days; the importance of different indicators will be configured in combination with specific business requirements. For example, for recorded content in live broadcast scenarios, a higher weight will be assigned to the overall recall rate when determining the model evaluation value. For another example, for detecting vulgar content in live broadcast scenarios, a higher weight will be assigned to the accuracy rate when determining the model evaluation value.

[0130] In this way, after the initial anomaly detection model is trained to obtain the target anomaly detection model, the target anomaly detection model and the historical anomaly detection model will be compared. For the anomaly detection effect of real-time business data, the processing performance of the target anomaly detection model and the historical anomaly detection model can be intuitively compared through model evaluation, which is helpful in selecting an effective anomaly detection model that better meets the data processing needs during the anomaly detection process, so that the processing efficiency of the model can be improved to a certain extent;

[0131] Moreover, the processing process corresponding to situation one can indirectly reflect the model iteration and update logic of this application. After determining that the performance value of the historical anomaly detection model meets the performance anomaly condition, the training process of the target anomaly detection model obtained by the initial anomaly detection model training is automatically triggered; afterwards, the target anomaly detection model and the historical anomaly detection model are compared to determine the difference in processing capabilities in processing real-time data, and the effective anomaly detection model to be updated and launched is selected. This is equivalent to realizing the automation of the entire link of model training-comparison-online. On the one hand, it realizes the automation of model iteration, continuously optimizes model performance, improves the robustness of the risk control system, and achieves good results in actual business scenarios. On the other hand, it can avoid duplication of work caused by frequent model replacement and save labor costs.

[0132] Case 2: When acquiring multimedia data and object description data, the training timing of the target anomaly detection model has not been reached.

[0133] At the data acquisition timing corresponding to situation two, since the training timing of the target anomaly detection model has not been reached, for the historical anomaly detection model that has completed the most recent training and passed the model evaluation, the historical anomaly detection model has anomaly detection performance that meets the requirements, and the processing device can continue to use the historical anomaly detection model to perform anomaly detection based on the acquired multimedia data and object description data.

[0134] The model passing the model evaluation means: determining that the model has a better model evaluation value; the training timing of the target anomaly detection model means starting to train the initial anomaly detection model to obtain the target anomaly detection model.

[0135] Case 3: The timing for acquiring multimedia data and object description data is after the training timing of the target anomaly detection model is reached and before the target anomaly detection model completes model evaluation.

[0136] At the data acquisition timing corresponding to situation three, if the multimedia data and object description data are acquired when the target anomaly detection model is in the process of model evaluation, the historical anomaly detection model is determined as the effective anomaly detection model; and if the multimedia data and object description data are acquired during the process of training the target anomaly detection model, the historical anomaly detection model is determined as the effective anomaly detection model.

[0137] Specifically, when the target anomaly detection model is in the process of model evaluation and multimedia data and object description data are obtained, or when multimedia data and object description data are obtained during the process of training the target anomaly detection model, it can be determined that the target anomaly detection model cannot be put into use at this time, so the historical anomaly detection model can be directly used for anomaly detection.

[0138] It should be understood that in the embodiments of the present application, as the processing proceeds, the target anomaly detection model that currently passes the model evaluation may be used as a historical anomaly detection model for subsequent anomaly detection; for a target anomaly detection model, its corresponding performance evaluation results are pass or fail.

[0139] In this way, in the actual anomaly detection process before the target anomaly detection model passes the model evaluation, the historical anomaly detection model with performance anomalies is still used for processing, so as to avoid the impact of retraining the model on the anomaly detection business.

[0140] Furthermore, the processing device performs data screening and feature extraction on the multimedia data and object description data according to the input feature type. In the process of obtaining the features to be input, the processing device obtains the input feature type selected when building the initial anomaly detection model, and based on the input feature type, screens out matching input data in the multimedia data and object description data; then, feature extraction is performed on the input data to obtain the features to be input.

[0141] Specifically, the processing device may obtain data content matching the input feature type from the object description data and multimedia data associated with the target object according to the input feature type, and extract the feature to be input based on the matching data content.

[0142] It should be understood that in the embodiment of the present application, when extracting the features to be input, the processing device can combine ASR technology and conventional text feature extraction technology in the field to extract text features based on audio data, and combine OCR technology or conventional image feature extraction technology in the field to extract image features based on image data. At the same time, conventional text feature extraction technology under related technologies is used to extract text features for the matching data content in the object description data.

[0143] In this way, the object description data and multimedia data associated with the target object can be organized into a feature form that can be processed by the effective anomaly detection model according to the input feature type.

[0144] The following is an explanation of the training timing of the target anomaly detection model:

[0145] It should be noted that in an embodiment of the present application, when it is determined that the performance value of the historical anomaly detection model meets the performance anomaly condition, the initial anomaly detection model is trained based on the historical data of the most recent period of time to obtain the target anomaly detection model; in other words, the target anomaly detection model refers to: after determining through the most recent model performance judgment that the performance value of the historical anomaly detection model meets the performance anomaly condition, the model is re-trained based on the initial anomaly detection model. In this case, the historical anomaly detection model refers to the model determined when the performance anomaly condition was met most recently before acquiring the multimedia data and the object description data, wherein the initial anomaly detection model is unrelated to the historical anomaly detection model, but the initial anomaly detection model can be constructed using the same model architecture as the historical anomaly detection model.

[0146] In addition, in an embodiment of the present application, the processing device can obtain the detection results of the historical anomaly detection model in real time during the process of determining whether the performance value of the historical anomaly detection model meets the performance anomaly condition, and determine the performance value corresponding to the historical anomaly detection model based on the obtained detection results; specifically, it can be set to calculate the performance value once every period of time based on the detection results obtained in real time, and determine whether the historical anomaly detection model meets the performance anomaly condition based on the obtained performance value.

[0147] For example, you can set the specified historical period to a very short time, such as 1 hour, to calculate the performance value once. Therefore, you can calculate the performance value of the historical anomaly detection model based on the detection results obtained within 1 hour, and then determine whether the historical anomaly detection model meets the performance anomaly condition based on the matching of the obtained performance value with the performance anomaly condition.

[0148] It should be understood that when a historical anomaly detection model is judged to meet the performance anomaly condition, based on the performance value calculated next time, the historical anomaly detection model will most likely be judged to meet the performance anomaly condition. Based on this, in order to avoid repeatedly triggering the training process of the target anomaly detection model, the performance value calculation for the historical anomaly detection model can be stopped for a period of time after determining that the performance anomaly condition is met based on the performance value of the historical anomaly detection model.

[0149] Optionally, according to actual processing needs, in order to reduce the difficulty of model evaluation, it can be configured that the training process of the target anomaly detection model will not be re-triggered before the current target anomaly detection model has completed model evaluation, so as to avoid the existence of multiple historical anomaly detection models corresponding to one target anomaly detection model in the subsequent model evaluation process.

[0150] For example, assuming that the performance value is calculated once every half hour and whether the performance anomaly condition is met is determined, then after determining that the historical anomaly detection model meets the performance anomaly condition based on the performance value calculated in the current time, it can be configured not to calculate the performance value and not to determine whether the performance anomaly condition is met within the same day.

[0151] In an embodiment of the present application, the specific content corresponding to the performance value can be configured according to actual processing needs. In a feasible implementation method, at least one or a combination of omission rate, accuracy rate, omission rate fluctuation range, and accuracy rate fluctuation range can be determined as the performance value based on which the model performance is considered, wherein the omission rate is determined based on the ratio of the number of abnormal objects missed by the model to the total number of abnormal objects detected by the model; the accuracy rate is determined based on the ratio of the number of abnormal objects accurately detected by the model to the total number of abnormal objects detected by the model; the omission rate fluctuation range can be determined based on the difference between the omission rate currently calculated and the omission rate calculated last time; the accuracy rate fluctuation range can be determined based on the difference between the accuracy currently calculated and the accuracy calculated last time.

[0152] Based on this, in the performance abnormality condition, abnormal thresholds can be set for various performance values, and by defining the size relationship between the calculated performance value and the corresponding abnormal threshold, it is determined whether the performance abnormality condition is met.

[0153] For example, when the performance value is specifically a miss rate, the configured performance abnormality condition may be: the calculated miss rate is higher than a set miss rate threshold.

[0154] It should be noted that in the process of calculating the performance value, the corresponding performance value can be determined based on the detection results of the historical anomaly detection model within the specified historical time period, where, according to actual processing needs, the specified historical time period can refer to the historical time period from the current day to the performance value calculation time, or the specified historical time period can be the historical time period between two adjacent performance value calculation times. This application does not impose specific restrictions on this.

[0155] For example, assuming that the performance value is calculated every 30 minutes, then at any performance value calculation time, assuming it is 11:00:00 on March 26, the performance value of the historical anomaly detection model can be calculated based on the detection results generated between 00:00:00 and 11:00:00 on March 26.

[0156] For another example, assuming that it is determined to calculate the performance value every 30 minutes, then at any performance value calculation time, assuming it is 11:00:00 on March 26, the performance value of the historical anomaly detection model can be calculated based on the detection results generated between 00:10:30-11:00:00 on March 26.

[0157] For another example, when the performance value being considered is accuracy, an accuracy threshold can be set for the accuracy, so that when it is determined through real-time online processing of the model that the accuracy of the model is lower than the accuracy threshold, it can be determined that the performance abnormality condition is met.

[0158] For example, see Figure 5 As shown, it is a schematic diagram of the performance values ​​corresponding to each specified historical period in the embodiment of the present application. Assuming that the performance value is calculated once every hour, and the performance value is specifically the accuracy rate, then, combined with the attached Figure 5 It can be seen from the diagram that the accuracy calculated at 10:00:00 on March 26 is higher than the accuracy threshold, so the performance abnormality condition is not met. The accuracy calculated at 11:00:00 on March 26 is higher than the accuracy threshold, and it can be determined that the performance abnormality condition is met. At this time, the training of the initial anomaly detection model is triggered, and the accuracy calculation performed at other times on March 26 is stopped.

[0159] After clarifying the training timing of the target anomaly detection model, the following describes the process of training the initial anomaly detection model to obtain the target anomaly detection model in conjunction with the accompanying drawings:

[0160] See also Fig. 6A As shown in the figure, it is a schematic diagram of the process of training the target anomaly detection model in the embodiment of the present application. Fig. 6A , the process of training the target anomaly detection model is explained:

[0161] Step 601: Processing is performed to obtain each training sample; each training sample includes: negative samples generated based on abnormal multimedia data that are successfully detected and not detected by the historical anomaly detection model, and positive samples generated based on normal multimedia data that are successfully detected and falsely detected as abnormal by the historical anomaly detection model; a training sample includes: sample input features obtained by sorting based on the selected input feature type, and the corresponding anomaly label.

[0162] In the process of obtaining each training sample, the processing device can poll a preset sample storage area to obtain each training sample; or, poll a preset sample storage area to obtain each sample description information, and obtain each training sample based on each sample description information and a selected input feature type, wherein each sample description information includes object identification information of each sample publishing object and exception identification information of each sample multimedia data.

[0163] It should be noted that the abnormal identification information is used to indicate the multimedia data stream to which the sample multimedia data belongs, and the sampling position of the multimedia data in the multimedia data stream.

[0164] Specifically, in some feasible implementations, there may be a functional division of labor within the processing device. When it is determined that the performance value of the historical anomaly detection model reaches the performance anomaly condition, the processing device generates each training sample with the help of an internal sample generation function, and stores each generated training sample in a preset sample storage area; at the same time, the model training function within the processing device directly obtains each training sample by polling the sample storage area, and implements model training based on each training sample.

[0165] Optionally, in an embodiment of the present application, each training sample may be generated by other trusted devices and stored in a sample storage area. In this case, the processing device can obtain each training sample generated by other devices by polling a preset sample storage area.

[0166] In other feasible implementations, when the processing device polls a preset sample storage area to obtain each training sample, the processing device may poll the preset sample storage area to obtain each sample description information that can participate in the generation of training samples. Then, after generating each training sample based on each sample description information and the selected input feature type, the processing device obtains each training sample for training.

[0167] In this way, the method of obtaining training samples can be flexibly configured according to actual processing needs. When the method is configured to directly obtain each training sample from the sample storage area, it is equivalent to deploying the generation process of each training sample before the storage process, which can enhance the consistency of data processing; when the method is configured to obtain sample description data from the sample storage area, it is equivalent to deploying the generation process of each training sample after obtaining the stored data, which can reserve more processing time for the selection of input feature types, which is beneficial to reducing data processing time as a whole.

[0168] The following only takes the generation of training samples by the processing device as an example to illustrate the generation process of training samples. Figure 6B As shown, it is a schematic diagram of the generation process of each training sample in the embodiment of the present application. Figure 6B , the generation process of each training sample is explained:

[0169] Step 6011: The processing device determines the anomaly identification information of the sample multimedia data and the object identification information of the corresponding sample publishing object in the historical data set generated within the specified historical time period, wherein each sample multimedia data includes: abnormal multimedia data successfully detected and not detected by the historical anomaly detection model, and normal multimedia data successfully detected and falsely detected as abnormal by the historical anomaly detection model.

[0170] In an embodiment of the present application, abnormal multimedia data refers to multimedia data extracted from multimedia data streams published by abnormal objects, and normal multimedia data refers to multimedia data extracted from multimedia data streams published by normal objects; abnormal multimedia data includes two sources, one part comes from abnormal objects detected by historical anomaly detection models and manually reviewed, and the other part comes from abnormal objects determined by reports or inspections of related objects; normal multimedia data includes two sources, one part comes from normal objects that are not hit by any strategy (i.e., normal objects successfully detected and determined by historical anomaly detection models), and the other part comes from normal objects detected as abnormal by historical anomaly detection models but determined to be normal after manual review.

[0171] It should be noted that in the embodiment of the present application, since the object detected as abnormal will be severely dealt with, the processing device will manually re-check the abnormal object after the historical anomaly detection model detects an abnormal result to avoid the adverse consequences caused by model detection errors. Based on this, with the help of the processing of the historical anomaly detection model, the scope of manual detection can be greatly reduced and the efficiency of anomaly detection can be better improved.

[0172] The specified historical period may be N days before the training opportunity for the target anomaly detection model is determined to be reached, wherein the value of N is set according to actual processing needs, and this application does not impose any specific restrictions on this.

[0173] For example, assuming that 11:00:00 on March 26 is determined to be the training time for the target anomaly detection model, then the historical detection data of the past 7 days (March 19 00:00:00-March 26 00:00:00) can be obtained to generate various training samples.

[0174] Furthermore, after the processing device obtains the historical data set generated within the specified historical period, it can select each sample multimedia data including normal multimedia data and abnormal multimedia data according to the corresponding anomaly detection result; then, the anomaly identification information of each sample multimedia data and the object identification information of the corresponding sample release object can be determined respectively, wherein the corresponding anomaly detection result can be determined based on the anomaly identification information, and the corresponding multimedia data can be located in the corresponding multimedia data stream; based on the object identification information, various data associated with the sample release object including object description data can be obtained.

[0175] Step 6012: The processing device selects an input feature type from a preset full feature set, and performs the following operations for each sample publishing object: Based on the input feature type, extract the sample input features from the sample multimedia data and object description data determined based on the identification information of the sample publishing object and the abnormal identification information of the corresponding sample multimedia data, and generate a training sample based on the sample input features and the abnormal label determined for the corresponding sample multimedia data.

[0176] Specifically, when executing step 6012, the processing device selects the input feature type from the preset full feature set in the following ways:

[0177] It should be noted that the full feature set can be understood as including all feature types in various business scenarios.

[0178] Selection method 1: Filter the output results of the model based on a target feature obtained through training and select the input feature type.

[0179] Specifically, in the process of selecting the input feature type corresponding to the selection method 1, the processing device determines the full features that match the current business scenario in the preset full feature set, and generates each screening sample based on the full features, each normal multimedia data and abnormal multimedia data, and the object description data of each sample release object, wherein a screening sample includes a full input feature and a corresponding abnormal label; the full features include various types of candidate sub-features; and then each screening sample is used to perform multiple rounds of iterative training on the constructed initial feature screening model to obtain the importance evaluation value output for each type of candidate sub-feature in the trained target feature screening model, and based on each importance evaluation value, the input feature type is determined from each type of candidate sub-feature.

[0180] It should be noted that, in the embodiment of the present application, the full set of features includes all features applicable to the current business scenario (i.e., candidate sub-features of various types), and the sample content on which the construction of the screening sample is based can be the same as the sample content on which the training sample is based. In other words, suitable screening samples can also be generated based on the normal multimedia data and abnormal multimedia data determined from the historical data set generated within a specified historical period, as well as the object description data of each sample release object.

[0181] In addition, the model architecture of the initial feature screening model can be an extreme gradient boosting tree (eXtreme GradientBoosting, XGB) model; the importance evaluation value specifically refers to the contribution of various candidate sub-features to the model output results; the convergence condition of the initial feature screening model can be: the number of model training rounds reaches a first set value, wherein the value of the first set value is set according to actual processing needs.

[0182] For example, see Fig. 7A As shown, it is a schematic diagram of the process of determining the input feature type in the implementation of this application. Fig. 7A It can be seen from the illustrated processing procedure that the processing device determines the full set of features under the current business scenario from the full set of features, wherein the full set of features includes various candidate sub-features; then, based on the screened full set of features, each screening sample is constructed; thereafter, the initial feature screening model is iteratively trained for multiple rounds using each screening sample until the preset convergence condition is met, thereby obtaining a trained target feature screening model, and obtaining the importance evaluation value (i.e., the contribution to the model output result) output for each input candidate sub-feature in the target feature screening model; then, according to the importance evaluation value corresponding to each type of candidate sub-feature, M types of candidate sub-features with the largest importance evaluation value are screened out, and then the M types of candidate sub-features are used as the selected input feature type.

[0183] In this way, by means of the importance evaluation values ​​output for each type of candidate sub-features input in the trained target feature screening model, it is possible to intuitively characterize the importance of each type of candidate sub-feature included in the full set of features for anomaly detection, which helps to screen out candidate sub-features that have a strong correlation with the sample distribution as input feature types.

[0184] Selection method 2: Filter the output results of the model based on multiple target features obtained through training and select the input feature type.

[0185] Specifically, in the input feature type selection process corresponding to the second selection method, the processing device selects each candidate sub-feature group from the preset full feature set, and for each candidate sub-feature group, based on each normal multimedia data and abnormal multimedia data, and the object description data of each sample release object, generates a corresponding screening sample set, wherein a screening sample includes: component input features constructed corresponding to a candidate sub-feature group, and a corresponding abnormal label; then, using each screening sample set, the constructed initial feature screening models are respectively trained for multiple rounds of iterative training to obtain the trained target feature screening models, and the input feature type is screened out according to the importance evaluation value output for each type of candidate sub-feature in each target feature screening model.

[0186] It should be noted that, in the embodiments of the present application, there is at least one different candidate sub-feature in different candidate sub-feature groups; various types of candidate sub-features in different candidate sub-feature groups cover all feature types (or all feature types) in the current business scenario; constructing different candidate sub-feature groups is equivalent to configuring different input feature types for different initial feature screening models; the total number of constructed candidate sub-feature groups is the same as the total number of constructed initial feature screening models.

[0187] In the process of training the initial feature screening model to obtain the target feature screening model, the processing method can be similar to the processing method of the above-mentioned selection method one. The processing device performs the following operations for an initial feature screening model corresponding to any candidate sub-feature group: based on the candidate sub-feature group, each normal multimedia data and each abnormal multimedia data, and the object description data of each sample release object, a corresponding screening sample set is generated; then, based on the screening sample set, the constructed initial feature screening model is iteratively trained for multiple rounds until the number of training rounds reaches a second set value, so as to obtain the trained target feature screening model. At the same time, an importance evaluation value of the target feature screening model based on the input of each type of candidate sub-feature output is obtained.

[0188] Furthermore, after obtaining the importance evaluation values ​​output by each target feature screening model based on each type of input candidate sub-features, P candidate sub-features with the highest importance evaluation values ​​can be selected for each target feature screening model, and the union of the P candidate sub-features corresponding to each target feature screening model can be determined, and each type of candidate sub-features in the union can be determined as the input feature type.

[0189] It should be noted that when screening candidate sub-features based on the importance evaluation value, in a feasible implementation method, the screening conditions can be set only for the value of the importance evaluation value, so that the candidate sub-features whose corresponding importance evaluation values ​​are higher than the third set value can be screened; or, the screening conditions can be set only for the sorting result of the importance evaluation value, so that in the sorting result of the importance evaluation value, the candidate sub-features corresponding to the first P importance evaluation values ​​can be screened; or, the screening conditions can be set for both the value of the importance evaluation value and the sorting result of the importance evaluation value, so that the candidate sub-features whose importance evaluation values ​​are higher than the fourth set value and belong to the first H importance evaluation values ​​can be screened, wherein the values ​​of P, H, the second set value, the third set value, and the fourth set value are set according to actual processing needs, and the present application does not impose specific restrictions on this.

[0190] For example, see Figure 7B As shown, it is a schematic diagram of the process of selecting the input feature type in the embodiment of the present application. Figure 7BAs can be seen from the illustrated content, it is assumed that two candidate sub-feature sets are constructed, wherein candidate sub-feature set 1 includes candidate sub-features 1-4, and candidate sub-feature set 2 includes candidate sub-features x, y, z, and 3; then, according to the feature types in candidate sub-feature sets 1 and 2, each normal multimedia data and abnormal multimedia data, and the object description data of each sample release object, the corresponding screening sample sets 1 and 2 are generated respectively; then, the initial feature screening model 1 is trained based on the screening sample set 1 to obtain the trained target feature screening model 1, and the initial feature screening model 2 is trained based on the screening sample set 2 to obtain the trained target feature screening model 2.

[0191] Continue to combine Figure 6B The content is explained, after the processing device obtains the contribution of the target feature screening model 1 for each type of input candidate sub-feature output, and obtains the contribution of the target feature screening model 2 for each type of input candidate sub-feature output, the corresponding target feature screening model 1 and the target feature screening model 2 respectively select the two types of candidate sub-features with the highest contribution to form the input feature type.

[0192] In this way, with the help of the trained target feature screening models, it is possible to discover feature types that are highly important for anomaly detection from the perspective of the combination effects of different types of candidate sub-features, which helps to more deeply analyze the impact of candidate sub-features on the output results, and is conducive to better screening out feature types that have a greater impact on anomaly detection.

[0193] Furthermore, after selecting the input feature type, the processing device can obtain data content matching the input feature type in the object description data and sample multimedia data corresponding to each sample publishing object according to the input feature type, and extract the input features of each sample based on the matching data content, and then construct corresponding training samples based on the input features of each sample and the corresponding exception labels.

[0194] In this way, with the help of the processing of steps 6011-6012, it is possible to select the input feature type that meets the needs of anomaly detection, and then adapt to the input feature type to construct corresponding training samples, so that in the subsequent training process, it is possible to guide the initial anomaly detection model to learn anomaly detection capabilities based on the sample input features based on each training sample.

[0195] Step 602: The processing device uses each training sample to perform multiple rounds of iterative training on the constructed initial anomaly detection model to obtain a trained target anomaly detection model.

[0196] After the processing device obtains each training sample, it performs multiple rounds of iterative training on the constructed initial anomaly detection model with the help of each training sample to obtain a trained target anomaly detection model.

[0197] It should be noted that in the embodiment of the present application, the initial anomaly detection model is specifically constructed based on the (LightGradientBoostingMachine, LightGBM) model, wherein the storage format of the LightGBM model is .txt format. The convergence condition for training the initial anomaly detection model may be that the number of model training rounds reaches the fifth set value, wherein the value of the fifth set value is set according to the actual processing needs, and the present application does not make specific restrictions on this.

[0198] In this way, with the help of the training process of steps 601-602, it is possible to generate various training samples based on the anomaly detection data of the most recent period, and construct an initial anomaly detection model with the selected input feature type as input, and use each training sample to train the initial anomaly detection model, so that on the one hand, the timeliness of the training data can be guaranteed, and on the other hand, the feasibility of the input features for anomaly detection can be guaranteed, which helps to train a target anomaly detection model with better anomaly detection performance.

[0199] Step 203: The processing device uses the effective anomaly detection model to obtain an anomaly detection result of the target object based on the input features.

[0200] The processing device inputs the input features generated for the target object into the effective anomaly detection model to obtain the anomaly detection result output by the effective anomaly detection model for the target object.

[0201] Furthermore, when the processing device determines that the abnormality detection result of the target object is abnormal, the business service for the target object is stopped, and the multimedia data stream published by the target object is interrupted.

[0202] For example, in a live broadcast scenario, if the live broadcast room is determined to be abnormal, the live broadcast room can be directly banned.

[0203] Specifically, when the processing device determines that the abnormality detection result of the target object is abnormal, the business service for the target object can be directly stopped, and the multimedia data stream published by the target object can be interrupted.

[0204] In this way, target objects detected as abnormal can be processed in a timely manner to prevent the target objects from continuing to publish bad data content, reduce the impact of the target objects publishing bad content, and help maintain a compliant and legal Internet environment.

[0205] Optionally, when the processing device determines that the anomaly detection result of the target object is abnormal, it can submit for manual review based on the obtained multimedia data and object description data of the target object, and if the manual review is abnormal, continue to stop business services for the target object and continue to interrupt the multimedia data stream published by the target object; and, if the manual review is normal, start providing business services to the target object and display the multimedia data stream published by the target object.

[0206] On the contrary, when the abnormality detection result of the target object is normal, the service can continue to be provided to the target object, and the multimedia data stream published by the target object can be displayed.

[0207] In conjunction with the accompanying drawings, the following describes the process of training a target anomaly detection model and comparing and determining the effective anomaly detection model, taking anomaly detection in a live broadcast room under a live broadcast scenario as an example.

[0208] See also Fig. 8A As shown, it is a schematic diagram of the process of constructing each training sample in an embodiment of the present application.

[0209] It should be understood that the selection of training samples is directly related to the final quality of the model. As a pre-process of the model training process, the selection of positive and negative samples is the most critical link. In a feasible implementation, the processing can be realized with the help of the internal preset training sample management platform, realizing the two parts of data acquisition and data processing. The data acquisition process is used to pull and store the audit results, and the data processing process is used to select and design positive and negative samples.

[0210] During the data acquisition process, if the real-time online indicator detection system finds that the performance value of the historical anomaly detection model shows obvious fluctuations (for example, in the recording scenario, the omission rate fluctuates by more than 5%), the sample training management platform will be informed that new samples need to be pulled for training; then, the training sample management platform obtains the original data in a specific format from the historical data set including the review queue for subsequent processing, where the original data in a specific format may include one-dimensional text features and two-dimensional image features.

[0211] In the data processing process: The core of the data processing part is to divide the training samples. In order to obtain a model with better performance, the positive and negative samples can be further subdivided into the following categories:

[0212] A) Traditional negative samples: This part accounts for a large proportion of negative samples. It is the malicious live broadcast room that has been recalled by the existing historical anomaly detection model and confirmed by the auditors. Its main function is to ensure the basic recognition ability of the model and prevent the recognition ability from declining.

[0213] B) Difficult black samples: This part accounts for a small proportion of negative samples. They are malicious live broadcast rooms that have not been identified by the current historical anomaly detection model but have been recalled by other strategies (such as reporting, manual inspection, etc.). The main function is to add "positive" noise to the model and improve the coverage of the model in a direction, thereby achieving better robustness;

[0214] C) Traditional positive samples: This part accounts for a large proportion of positive samples and comes from random sampling of live broadcast rooms that do not have any strategy hits (by default, live broadcast rooms without violations). Its role is similar to that of traditional negative samples;

[0215] D) Difficult positive samples: This part accounts for a smaller proportion of the positive samples and comes from live broadcast rooms that are hit by the strategy but are judged as accidental injuries by auditors. It is similar to difficult black samples and is mainly used to improve the accuracy of the model and avoid accidental injuries.

[0216] After the samples are divided, the sample management platform will send the data to the distributed file system (HDFS), that is, the sample storage area for storage, so that the downstream training process can call it. The purpose of this design is to maintain the decoupling between sample acquisition and model training to avoid data conflicts.

[0217] See also Figure 8B As shown, it is a schematic diagram of the process of training the target anomaly detection model in an embodiment of the present application.

[0218] The model training module that implements model training in the processing device first obtains the latest training samples from the HDFS system, then selects features from the feature pool, decides whether to replace the features based on feature importance and relevance, and finally selects the input feature type to train the model.

[0219] Specifically, during the data acquisition process, the online model inference platform in the processing device will regularly poll the specified folder in the HDFS system every day to see if there are new training samples. If so, it will pull each training sample and start the model training process.

[0220] During the feature selection process, the online model inference platform in the processing device performs feature selection based on two main points: one is to calculate the importance evaluation values ​​of the features used by the existing model (historical anomaly detection model) and rank them, and delete the features with the importance evaluation values ​​at the bottom and whose importance evaluation values ​​are lower than a certain threshold; the second is to calculate the features not used by the historical anomaly detection model, calculate the correlation with the sample distribution, and select the features with correlation higher than a certain threshold to be added to the model training. Among them, the method for determining the feature importance evaluation value has been described in detail in the above description and will not be described in detail here.

[0221] During the model training process, after determining the types of training samples and input features, considering the training speed, resource usage, large-scale data processing capabilities, and the final business effect, the LightGBM model is selected as the prediction model, and the initial anomaly detection model is generated based on LightGBM. Moreover, the model file is stored in .txt format using the LightGBM customized method.

[0222] See also Figure 8C As shown, it is a schematic diagram of the process of comparing and determining the effective anomaly detection model in an embodiment of the present application.

[0223] In order to compare the advantages and disadvantages between online and offline models, the processing device copies the online data to the online model inference platform through the internal model performance comparison module to calculate the indicator differences between the models, and finally selects the optimal model for replacement and online.

[0224] Specifically, after the model training is completed, the online data is connected to the online model inference platform, and the target anomaly detection model is used for processing, and the prediction results are collected.

[0225] After accumulating a certain number of prediction results (usually three days), various business indicators are calculated. Among them, the most core indicators include overall recall rate, overall false alarm rate, additional recall accuracy rate and additional recall false alarm rate. The importance of different indicators will be combined with specific business demands. For example, the overall recall rate will be more considered for recorded broadcasts, but a higher accuracy rate is required for vulgarity.

[0226] Furthermore, after selecting the optimal model (the optimal model may be the current online historical anomaly detection model or the newly trained target anomaly detection model), the model is used as the effective anomaly detection model, replaced in the online environment, and the original model is taken offline to complete the model iteration.

[0227] In this way, this application realizes a set of automated processes covering sample acquisition, sample selection, model training, model comparison and model launch by manually reviewing data results in live broadcast scenarios, thereby improving the robustness of the model and strengthening the ability to fight against black industries.

[0228] In actual practice, the technical solution proposed in this application has also achieved good business benefits. In the management of malicious recorded broadcasts, from the perspective of work efficiency, the use of the technical solution proposed in this application can complete the model iteration work that used to take a week in one day, saving a lot of manpower costs while allowing the model to adapt to changes in the black industry more quickly. From the perspective of business indicators, the technical solution proposed in this application has increased the overall average monthly coverage of recorded broadcasts from the previous 89% to 95%+, and the accuracy rate from the previous 91% to 96%+, comprehensively improving the recognition ability of the recorded broadcast model.

[0229] In summary, in the technical solution proposed in this application, an automatic model iteration solution is proposed to improve the ability to fight against the black industry in the live broadcast scene. The processing device, as the risk control end, can automatically pull historical data sets including manual review results every day to obtain the latest positive and negative samples for model training. And compare the prediction performance of the new and old models according to the model evaluation values ​​of the new and old models (such as fluctuations in online accuracy and recall), which serves as the basis for replacing the model online. It is equivalent to realizing the automation of the entire link of model training-comparison-online, and has better business performance in fighting against strong malicious governance tasks. On the one hand, it realizes the automation of model iteration, continuously optimizes model performance, improves the robustness of the risk control system, and achieves good results in actual business scenarios. On the other hand, it can avoid the duplication of work caused by frequent model replacement and save labor costs.

[0230] Furthermore, based on the technical solution proposed in this application, it is possible to achieve real-time updating of the model. Specifically, it is possible to learn and update the model from new data in real time, so that the system can adapt to new risks and fraud patterns in a timely manner. In contrast, traditional batch learning requires manual periodic retraining of the model and cannot respond to changes in a timely manner. Moreover, the model update method proposed in this application can be dynamically adjusted and optimized according to the changing environment and data. It can adaptively learn new risk patterns and rules, and make decisions based on the latest data, thereby improving the accuracy and effectiveness of risk assessment.

[0231] Based on the same inventive concept, see Fig. 9 As shown, it is a schematic diagram of the logical structure of the abnormality detection device in the embodiment of the present application. The abnormality detection device 900 includes an acquisition unit 901, an extraction unit 902, and a detection unit 903, wherein:

[0232] The acquisition unit 901 is used to acquire multimedia data and object description data associated with the target object; the multimedia data is obtained from the multimedia data stream published by the target object;

[0233] The extraction unit 902 is used to perform data screening and feature extraction on the multimedia data and the object description data according to the input feature type corresponding to the current effective anomaly detection model to obtain the features to be input, wherein the effective anomaly detection model is: a model selected from the historical anomaly detection model and the target anomaly detection model obtained by training the initial anomaly detection model according to the acquisition timing of the multimedia data and the object description data; the training timing of the initial anomaly detection model is: based on the performance value determined by the detection result of the historical anomaly detection model in the specified historical period, it is determined that the performance anomaly condition is met;

[0234] The detection unit 903 is used to obtain an anomaly detection result of the target object based on the input feature by adopting the effective anomaly detection model.

[0235] Optionally, if the multimedia data and the object description data are obtained after the target anomaly detection model and the historical anomaly detection model complete the model evaluation based on the real-time detection data within a set time period, the extraction unit 902 is further used to:

[0236] When the model evaluation value of the target anomaly detection model is higher than the model evaluation value of the historical anomaly detection model, the target anomaly detection model is determined as the effective anomaly detection model;

[0237] When the model evaluation value of the target anomaly detection model is lower than the model evaluation value of the historical anomaly detection model, the historical anomaly detection model is determined as the effective anomaly detection model.

[0238] Optionally, the extraction unit 902 is further configured to:

[0239] If the multimedia data and the object description data are obtained when the target anomaly detection model is in the process of model evaluation, the historical anomaly detection model is determined as the effective anomaly detection model;

[0240] If the multimedia data and the object description data are obtained in the process of training the target anomaly detection model, the historical anomaly detection model is determined as the effective anomaly detection model.

[0241] Optionally, the target anomaly detection model is trained by the training unit 904 in the device in the following manner:

[0242] Acquire each training sample; each training sample includes: negative samples generated based on abnormal multimedia data successfully detected and not detected by the historical anomaly detection model, and positive samples generated based on normal multimedia data successfully detected and falsely detected as abnormal by the historical anomaly detection model; a training sample includes: sample input features obtained by sorting based on the selected input feature type, and corresponding anomaly labels;

[0243] Using each training sample, the constructed initial anomaly detection model is trained for multiple rounds of iterations to obtain a trained target anomaly detection model.

[0244] Optionally, when acquiring each training sample, the training unit 904 performs any one of the following operations:

[0245] Polling the preset sample storage area to obtain each training sample;

[0246] Poll the preset sample storage area to obtain the description information of each sample, and obtain each training sample based on the description information of each sample and the selected input feature type; each sample description information includes the object identification information of each sample publishing object and the abnormal identification information of each sample multimedia data.

[0247] Optionally, each training sample is generated by the training unit 904 in the following manner:

[0248] In the historical data set generated within the specified historical period, respectively determine the abnormal identification information of each sample multimedia data and the object identification information of the corresponding sample publishing object, wherein each sample multimedia data includes: abnormal multimedia data successfully detected and not detected by the historical anomaly detection model, and normal multimedia data successfully detected and falsely detected as abnormal by the historical anomaly detection model;

[0249] An input feature type is selected from a preset full feature set, and for each sample release object, the following operations are performed: based on the input feature type, sample input features are extracted from the sample multimedia data and object description data determined based on the identification information of the sample release object and the abnormal identification information of the corresponding sample multimedia data, and a training sample is generated based on the sample input features and the abnormal label determined for the corresponding sample multimedia data.

[0250] Optionally, the input feature type is selected by the training unit 904 in the following manner:

[0251] In the preset full feature set, determine the full features that match the current business scenario, and generate each screening sample based on the full features, each normal multimedia data and abnormal multimedia data, and the object description data of each sample release object, wherein a screening sample includes a full input feature and a corresponding abnormal label; the full features include candidate sub-features of various types;

[0252] The constructed initial feature screening model is trained for multiple rounds of iterations using various screening samples to obtain importance evaluation values ​​for each type of candidate sub-feature output in the target feature screening model, and the input feature type is determined from each type of candidate sub-feature based on each importance evaluation value.

[0253] Optionally, the input feature type is selected by the training unit 904 in the following manner:

[0254] In the preset full feature set, each candidate sub-feature group is selected respectively, and for each candidate sub-feature group, based on each normal multimedia data and abnormal multimedia data, and the object description data of each sample release object, a corresponding screening sample set is generated, wherein a screening sample includes: a component input feature constructed corresponding to a candidate sub-feature group, and a corresponding abnormal label;

[0255] Using each screening sample set, multiple rounds of iterative training are performed on each constructed initial feature screening model to obtain each trained target feature screening model, and the input feature type is screened out according to the importance evaluation value output for each type of candidate sub-feature in each target feature screening model.

[0256] Optionally, after obtaining the abnormality detection result of the target object, the detection unit 903 is further used to:

[0257] When it is determined that the abnormality detection result of the target object is abnormal, the business service for the target object is stopped, and the multimedia data stream published by the target object is interrupted.

[0258] Optionally, when acquiring multimedia data and object description data associated with the target object, the acquiring unit 901 is used to:

[0259] In a multimedia data stream published by a target object, sampling an image frame at a preset first time interval, sampling an audio segment of a set duration at a preset second time interval, and determining the image frame and the audio segment as multimedia data associated with the currently acquired target object;

[0260] Acquire the publishing information of the multimedia data stream record published for the target object, the attribute information and the historical detection information associated with the target object, and determine the publishing information, the attribute information and the historical detection information as the object description data associated with the target object.

[0261] Optionally, when data screening and feature extraction are performed on the multimedia data and the object description data according to the input feature type and the features to be input are obtained, the extraction unit 902 is used to:

[0262] Obtaining the input feature type selected when building the initial anomaly detection model, and filtering out matching input data from the multimedia data and the object description data according to the input feature type;

[0263] Perform feature extraction on the input data to obtain the features to be input.

[0264] After introducing the abnormality detection method and device according to the exemplary embodiment of the present application, next, an electronic device according to another exemplary embodiment of the present application is introduced.

[0265] Those skilled in the art will appreciate that various aspects of the present application may be implemented as a system, method or program product. Therefore, various aspects of the present application may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to as "circuit", "module" or "system" herein.

[0266] Based on the same inventive concept as the above method embodiment, when the electronic device in the embodiment of the present application corresponds to a processing device, refer to Fig.10 As shown, it is a schematic diagram of the hardware composition structure of an electronic device using an embodiment of the present application, and the electronic device 1000 may include at least a processor 1001 and a memory 1002. Among them, the memory 1002 stores a computer program, and when the computer program is executed by the processor 1001, the processor 1001 performs any of the above-mentioned abnormality detection steps.

[0267] In some possible implementations, the electronic device according to the present application may include at least one processor and at least one memory. The memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of abnormality detection according to various exemplary implementations of the present application described above in this specification. For example, the processor may perform the following steps: Figure 2 Follow the steps shown in .

[0268] Based on the same inventive concept as the above method embodiment, various aspects of the action recognition provided by the present application can also be implemented in the form of a program product, which includes a program code. When the program product is run on an electronic device, the program code is used to enable the electronic device to execute the steps of the abnormality detection method according to various exemplary embodiments of the present application described above in this specification. For example, the electronic device can execute the following steps: Figure 2 Follow the steps shown in .

[0269] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0270] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0271] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. An anomaly detection method, characterized in that: include: Acquire multimedia data and object description data associated with the target object; The multimedia data is obtained from the multimedia data stream published by the target object; According to the input feature type corresponding to the current effective anomaly detection model, the multimedia data and the object description data are screened and feature extracted to obtain the features to be input, wherein the effective anomaly detection model is: a model selected from a historical anomaly detection model and a target anomaly detection model trained by an initial anomaly detection model according to the acquisition timing of the multimedia data and the object description data; the training timing of the initial anomaly detection model is: based on the performance value determined by the detection result of the historical anomaly detection model in a specified historical period, it is determined that the performance anomaly condition is met; The effective anomaly detection model is adopted to obtain an anomaly detection result of the target object based on the features to be input.

2. The method according to claim 1, characterized in that If the multimedia data and the object description data are obtained after the target anomaly detection model and the historical anomaly detection model complete model evaluation based on real-time detection data within a set time period, the method further includes: In a case where the model evaluation value of the target anomaly detection model is higher than the model evaluation value of the historical anomaly detection model, determining the target anomaly detection model as the effective anomaly detection model; In a case where the model evaluation value of the target anomaly detection model is lower than the model evaluation value of the historical anomaly detection model, the historical anomaly detection model is determined as the effective anomaly detection model.

3. The method according to claim 1, characterized in that The method further comprises: If the multimedia data and the object description data are obtained when the target anomaly detection model is in a model evaluation process, determining the historical anomaly detection model as the effective anomaly detection model; If the multimedia data and the object description data are obtained during the process of training the target anomaly detection model, the historical anomaly detection model is determined as the effective anomaly detection model.

4. The method according to claim 1, characterized in that The target anomaly detection model is trained in the following way: Acquire each training sample; each training sample includes: negative samples generated based on abnormal multimedia data successfully detected and not detected by the historical anomaly detection model, and positive samples generated based on normal multimedia data successfully detected and falsely detected as abnormal by the historical anomaly detection model; a training sample includes: sample input features obtained by sorting based on the selected input feature type, and corresponding anomaly labels; The training samples are used to perform multiple rounds of iterative training on the constructed initial anomaly detection model to obtain a trained target anomaly detection model.

5. The method according to claim 4, characterized in that The obtaining of each training sample includes any one of the following operations: Polling the preset sample storage area to obtain each training sample; Poll a preset sample storage area to obtain description information of each sample, and obtain each training sample based on the description information of each sample and the selected input feature type; the description information of each sample includes object identification information of each sample publishing object and abnormal identification information of each sample multimedia data.

6. The method according to claim 4, characterized in that The training samples are generated in the following way: In the historical data set generated within the specified historical period, respectively determine the abnormal identification information of each sample multimedia data and the object identification information of the corresponding sample publishing object; Each sample multimedia data includes: abnormal multimedia data successfully detected and not detected by the historical anomaly detection model, and normal multimedia data successfully detected and falsely detected as abnormal by the historical anomaly detection model; An input feature type is selected from a preset full feature set, and for each sample publishing object, the following operations are performed: based on the input feature type, sample input features are extracted from sample multimedia data and object description data determined based on identification information of the sample publishing object and abnormal identification information of the corresponding sample multimedia data, and training samples are generated based on the sample input features and abnormal labels determined corresponding to the sample multimedia data.

7. The method according to claim 4, characterized in that The input feature type is selected in the following manner: In the preset full feature set, determine the full features that match the current business scenario, and generate each screening sample based on the full features, each normal multimedia data and abnormal multimedia data, and the object description data of each sample publishing object, wherein a screening sample includes a full input feature and a corresponding abnormal label; the full features include candidate sub-features of various types; The constructed initial feature screening model is trained for multiple rounds of iterations using the screening samples to obtain importance evaluation values ​​output for each type of candidate sub-features in the target feature screening model, and the input feature type is determined from each type of candidate sub-features based on each importance evaluation value.

8. The method according to claim 4, characterized in that The input feature type is selected in the following manner: In the preset full feature set, each candidate sub-feature group is selected respectively, and for each candidate sub-feature group, based on each normal multimedia data and abnormal multimedia data, and object description data of each sample release object, a corresponding screening sample set is generated, wherein a screening sample includes: component input features constructed corresponding to a candidate sub-feature group, and a corresponding abnormal label; Using each screening sample set, multiple rounds of iterative training are performed on each constructed initial feature screening model to obtain each trained target feature screening model, and the input feature type is screened out according to the importance evaluation value output for each type of candidate sub-feature in each target feature screening model.

9. The method according to any one of claims 1 to 8, characterized in that After obtaining the abnormality detection result of the target object, the method further includes: When it is determined that the abnormality detection result of the target object is abnormal, the business service for the target object is stopped, and the multimedia data stream published by the target object is interrupted.

10. The method according to any one of claims 1 to 8, characterized in that The step of obtaining multimedia data and object description data associated with the target object includes: In a multimedia data stream published by a target object, sampling an image frame at a preset first time interval, sampling an audio segment of a set duration at a preset second time interval, and determining the image frame and the audio segment as multimedia data associated with the target object currently obtained; Acquire the publishing information of the multimedia data stream record published for the target object, the attribute information and the historical detection information associated with the target object, and determine the publishing information, the attribute information and the historical detection information as the object description data associated with the target object.

11. The method according to any one of claims 1 to 8, characterized in that: The step of performing data screening and feature extraction on the multimedia data and the object description data according to the input feature type to obtain the features to be input includes: Acquire the input feature type selected when constructing the initial anomaly detection model, and filter out matching input data from the multimedia data and the object description data according to the input feature type; Feature extraction is performed on the input data to obtain features to be input.

12. An abnormality detection device, characterized in that: include: An acquisition unit, used for acquiring multimedia data and object description data associated with a target object; The multimedia data is obtained from the multimedia data stream published by the target object; An extraction unit is used to perform data screening and feature extraction on the multimedia data and object description data according to the input feature type corresponding to the current effective anomaly detection model to obtain the features to be input, wherein the effective anomaly detection model is: a model selected from a historical anomaly detection model and a target anomaly detection model trained by an initial anomaly detection model according to the acquisition timing of the multimedia data and the object description data; the training timing of the initial anomaly detection model is: based on the performance value determined by the detection result of the historical anomaly detection model in a specified historical period, it is determined that the performance anomaly condition is met; The detection unit is used to adopt the effective anomaly detection model and obtain an anomaly detection result of the target object based on the feature to be input.

13. The device according to claim 12, characterized in that If the multimedia data and the object description data are obtained after the target anomaly detection model and the historical anomaly detection model complete model evaluation based on real-time detection data within a set time period, the extraction unit is further used to: In a case where the model evaluation value of the target anomaly detection model is higher than the model evaluation value of the historical anomaly detection model, determining the target anomaly detection model as the effective anomaly detection model; In a case where the model evaluation value of the target anomaly detection model is lower than the model evaluation value of the historical anomaly detection model, the historical anomaly detection model is determined as the effective anomaly detection model.

14. The device according to claim 12, characterized in that The extraction unit is also used for: If the multimedia data and the object description data are obtained when the target anomaly detection model is in a model evaluation process, determining the historical anomaly detection model as the effective anomaly detection model; If the multimedia data and the object description data are obtained during the process of training the target anomaly detection model, the historical anomaly detection model is determined as the effective anomaly detection model.

15. The device according to claim 12, characterized in that The target anomaly detection model is obtained by training the training unit in the device in the following manner: Acquire each training sample; each training sample includes: negative samples generated based on abnormal multimedia data successfully detected and not detected by the historical anomaly detection model, and positive samples generated based on normal multimedia data successfully detected and falsely detected as abnormal by the historical anomaly detection model; a training sample includes: sample input features obtained by sorting based on the selected input feature type, and corresponding anomaly labels; The training samples are used to perform multiple rounds of iterative training on the constructed initial anomaly detection model to obtain a trained target anomaly detection model.

16. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 11 is implemented.

17. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 11 is implemented.

18. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 11 is implemented.

Images